ref-manual: New section and updates for runtime dep work

fixes [YOCTO #10248]

The bulk of this change is a new section called "Automatically
Added Runtime Dependencies".  Additionally, changes were made
to the RDEPENDS and DEPENDS variables. Some cross-referencing
to the new material was also added in the do_package task entry,
the do_packagedata task entry, the PKGDATA_DIR glossary entry,
and the PRIVATE_LIBS glossary entry.

(From yocto-docs rev: 16504cd0eb23e102322c6c6096c5b52500a04640)

Signed-off-by: Scott Rifenbark <srifenbark@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

documentation/ref-manual/ref-tasks.xml

@@ -334,6 +334,17 @@
             <link linkend='var-FILES'><filename>FILES</filename></link>
             variables.
         </para>
+
+        <para>
+            The <filename>do_package</filename> task, in conjunction with the
+            <link linkend='ref-tasks-packagedata'><filename>do_packagedata</filename></link>
+            task, also saves some important package metadata.
+            For additional information, see the
+            <link linkend='var-PKGDESTWORK'><filename>PKGDESTWORK</filename></link>
+            variable and the
+            "<link linkend='automatically-added-runtime-dependencies'>Automatically Added Runtime Dependencies</link>"
+            section.
+        </para>
     </section>
 
     <section id='ref-tasks-package_qa'>
@@ -408,6 +419,8 @@
         <para>
             Creates package metadata used by the build system to generate the
             final packages.
+            The data is saved in
+            <link linkend='var-PKGDATA_DIR'><filename>PKGDATA_DIR</filename></link>.
         </para>
     </section>
 

documentation/ref-manual/ref-variables.xml

@@ -2833,9 +2833,9 @@
                                 add any runtime dependencies between the
                                 packages produced by the two recipes.
                                 However, as explained in the
-                                <link linkend='var-RDEPENDS'><filename>RDEPENDS</filename></link>
+                                "<link linkend='automatically-added-runtime-dependencies'>Automatically Added Runtime Dependencies</link>"
-                                glossary entry, runtime dependencies will often
+                                section, runtime dependencies will often be
-                                be added automatically, meaning
+                                added automatically, meaning
                                 <filename>DEPENDS</filename> alone is
                                 sufficient for most recipes.
                                 </para></listitem>
@@ -9651,14 +9651,16 @@ recipes-graphics/xorg-font/font-alias_1.0.3.bb:PR = "${INC_PR}.3"
                     <link linkend='ref-tasks-packagedata'><filename>do_packagedata</filename></link>
                     task packages data for each recipe and installs it into
                     this temporary, shared area.
-                    This directory defaults to the following:
+                    This directory defaults to the following, which you should
+                    not change:
                     <literallayout class='monospaced'>
      ${STAGING_DIR_HOST}/pkgdata
                     </literallayout>
-                </para>
+                    For examples of how this data is used, see the
-
+                    "<link linkend='automatically-added-runtime-dependencies'>Automatically Added Runtime Dependencies</link>"
-                <para>
+                    section and the
-                    Do not change this default.
+                    "<link linkend='viewing-package-information-with-oe-pkgdata-util'>Viewing Package Information with <filename>oe-pkgdata-util</filename></link>"
+                    section.
                 </para>
             </glossdef>
         </glossentry>
@@ -10133,6 +10135,12 @@ recipes-graphics/xorg-font/font-alias_1.0.3.bb:PR = "${INC_PR}.3"
                      libplds4.so"
                     </literallayout>
                 </para>
+
+                <para>
+                    For more information, see the
+                    "<link linkend='automatically-added-runtime-dependencies'>Automatically Added Runtime Dependencies</link>"
+                    section.
+                </para>
             </glossdef>
         </glossentry>
 
@@ -10427,40 +10435,60 @@ recipes-graphics/xorg-font/font-alias_1.0.3.bb:PR = "${INC_PR}.3"
 
         <glossentry id='var-RDEPENDS'><glossterm>RDEPENDS</glossterm>
             <info>
-                RDEPENDS[doc] = "Lists a package's runtime dependencies (i.e. other packages) that must be installed for the package to be built. They must be the names of other packages as listed in the PACKAGES variable, not recipe names (PN)."
+                RDEPENDS[doc] = "Lists runtime dependencies of a package."
             </info>
             <glossdef>
                 <para role="glossdeffirst">
 <!--                <para role="glossdeffirst"><imagedata fileref="figures/define-generic.png" /> -->
-                    Lists a package's runtime dependencies (i.e. other packages)
+                    Lists runtime dependencies of a package.
-                    that must be installed in order for the built package to run
+                    These dependencies are other packages that must be
-                    correctly.
+                    installed in order for the package to function correctly.
-                    If a package in this list cannot be found during the build,
+                    As an example, the following assignment declares that the
-                    you will get a build error.
+                    package <filename>foo</filename> needs the packages
+                    <filename>bar</filename> and <filename>baz</filename> to
+                    be installed:
+                    <literallayout class='monospaced'>
+     RDEPENDS_foo = "bar baz"
+                    </literallayout>
+                    The most common types of package runtime dependencies are
+                    automatically detected and added.
+                    Therefore, most recipes do not need to set
+                    <filename>RDEPENDS</filename>.
+                    For more information, see the
+                    "<link linkend='automatically-added-runtime-dependencies'>Automatically Added Runtime Dependencies</link>"
+                    section.
                 </para>
 
                 <para>
-                    When you use the <filename>RDEPENDS</filename> variable
+                    The practical effect of the above
-                    in a recipe, you are essentially stating that the recipe's
+                    <filename>RDEPENDS</filename> assignment is that
+                    <filename>bar</filename> and <filename>baz</filename>
+                    will be declared as dependencies inside the package
+                    <filename>foo</filename> when it is written out by one of
+                    the
+                    <link linkend='ref-tasks-package_write_deb'><filename>do_package_write_*</filename></link>
+                    tasks.
+                    Exactly how this is done depends on the package format used
+                    as described in the
+                    <link linkend='var-PACKAGE_CLASSES'><filename>PACKAGE_CLASSES</filename></link>
+                    variable.
+                    When the corresponding package manager installs the
+                    package, it will know to also install the packages on
+                    which it depends.
+                </para>
+
+                <para>
+                    To ensure that the packages <filename>bar</filename> and
+                    <filename>baz</filename> get built, the previous
+                    <filename>RDEPENDS</filename> assignment also causes a task
+                    dependency to be added.
+                    This dependency is from the recipe's
                     <link linkend='ref-tasks-build'><filename>do_build</filename></link>
-                    task depends on the existence of a specific package.
+                    (not to be confused with the
-                    Consider this simple example for two recipes named "a" and
+                    <link linkend='ref-tasks-compile'><filename>do_compile</filename></link>)
-                    "b" that produce similarly named IPK packages.
+                    task to the to the <filename>do_package_write_*</filename>
-                    In this example, the <filename>RDEPENDS</filename>
+                    task of the recipes that build <filename>bar</filename> and
-                    statement appears in the "a" recipe:
+                    <filename>baz</filename>.
-                    <literallayout class='monospaced'>
-     RDEPENDS_${PN} = "b"
-                    </literallayout>
-                    Here, the dependency is such that the
-                    <filename>do_build</filename> task for recipe "a" depends
-                    on the
-                    <link linkend='ref-tasks-package_write_ipk'><filename>do_package_write_ipk</filename></link>
-                    task of recipe "b".
-                    This means the package file for "b" must be available when
-                    the output for recipe "a" has been completely built.
-                    More importantly, package "a" will be marked as depending
-                    on package "b" in a manner that is understood by the
-                    package manager.
                 </para>
 
                 <para>
@@ -10480,7 +10508,8 @@ recipes-graphics/xorg-font/font-alias_1.0.3.bb:PR = "${INC_PR}.3"
                 <para>
                     Because the <filename>RDEPENDS</filename> variable applies
                     to packages being built, you should always use the variable
-                    in a form with an attached package name.
+                    in a form with an attached package name (remember that a
+                    single recipe can build multiple packages).
                     For example, suppose you are building a development package
                     that depends on the <filename>perl</filename> package.
                     In this case, you would use the following
@@ -10508,41 +10537,18 @@ recipes-graphics/xorg-font/font-alias_1.0.3.bb:PR = "${INC_PR}.3"
                 </para>
 
                 <para>
-                    The package name you attach to the
+                    The package names you use with
-                    <filename>RDEPENDS</filename> variable must appear
+                    <filename>RDEPENDS</filename> must appear as they would in
-                    as it would in the <filename>PACKAGES</filename>
+                    the <filename>PACKAGES</filename> variable.
-                    namespace before any renaming of the output package by
+                    The
-                    classes like
+                    <link linkend='var-PKG'><filename>PKG</filename></link>
-                    <link linkend='ref-classes-debian'><filename>debian</filename></link>.
+                    variable allows a different name to be used for
-                </para>
+                    the final package (e.g. the
-
+                    <link linkend='ref-classes-debian'><filename>debian</filename></link>
-                <para>
+                    class uses this to rename packages), but this final package
-                    In many cases you do not need to explicitly add
+                    name cannot be used with <filename>RDEPENDS</filename>,
-                    runtime dependencies using
+                    which makes sense as <filename>RDEPENDS</filename> is meant
-                    <filename>RDEPENDS</filename> since some automatic
+                    to be independent of the package format used.
-                    handling occurs:
-                    <itemizedlist>
-                        <listitem><para><emphasis><filename>shlibdeps</filename></emphasis>:
-                            If a runtime package contains a compiled binary,
-                            the build processes the binary in order to
-                            determine any shared libraries
-                            (<filename>.so</filename> files) to which it is
-                            dynamically linked.
-                            The build process adds the packages containing
-                            these libraries to <filename>RDEPENDS</filename>
-                            when creating the runtime package.
-                            In addition, if the shared library is versioned,
-                            the dependency will have the version appended to it
-                            in order to force an upgrade to the appropriate
-                            version if needed.
-                            </para></listitem>
-                        <listitem><para><emphasis><filename>pcdeps</filename></emphasis>:  If
-                            the package ships a <filename>pkg-config</filename>
-                            information file, the build process uses this file
-                            to add items to the <filename>RDEPENDS</filename>
-                            variable to create the runtime packages.
-                            </para></listitem>
-                    </itemizedlist>
                 </para>
 
                 <para>

documentation/ref-manual/technical-details.xml

@@ -979,6 +979,113 @@
     </section>
 </section>
 
+<section id='automatically-added-runtime-dependencies'>
+    <title>Automatically Added Runtime Dependencies</title>
+
+    <para>
+        The OpenEmbedded build system automatically adds common types of
+        runtime dependencies between packages, which means you do not need to
+        explicitly declare the packages using
+        <link linkend='var-RDEPENDS'><filename>RDEPENDS</filename></link>.
+        Two automatic mechanisms exist (<filename>shlibdeps</filename> and
+        <filename>pcdeps</filename>) that handle shared libraries and
+        package configuration (pkg-config) modules, respectively.
+        For other types of runtime dependencies, you must manually declare
+        the dependencies.
+        <itemizedlist>
+            <listitem><para>
+                <filename>shlibdeps</filename>:
+                During the
+                <link linkend='ref-tasks-package'><filename>do_package</filename></link>
+                task of each recipe, all shared libraries installed by the
+                recipe are located.
+                For each shared library, the package that contains the shared
+                library is registered as providing the shared library.
+                More specifically, the package is registered as providing the
+                <ulink url='https://en.wikipedia.org/wiki/Soname'>soname</ulink>
+                of the library.
+                The resulting shared-library-to-package mapping
+                is saved globally in
+                <link linkend='var-PKGDATA_DIR'><filename>PKGDATA_DIR</filename></link>
+                by the
+                <link linkend='ref-tasks-packagedata'><filename>do_packagedata</filename></link>
+                task.</para>
+
+                <para>Simultaneously, all executables and shared libraries
+                installed by the recipe are inspected to see what shared
+                libraries they link against.
+                For each shared library dependency that is found,
+                <filename>PKGDATA_DIR</filename> is queried to
+                see if some package (likely from a different recipe) contains
+                the shared library.
+                If such a package is found, a runtime dependency is added from
+                the package that depends on the shared library to the package
+                that contains the library.</para>
+
+                <para>If the library being linked against is versioned
+                (i.e. includes a version number), then the automatically added
+                runtime dependency will specify at least that version, as if
+                "<replaceable>package</replaceable> (>= <replaceable>version</replaceable>)"
+                had been added to
+                <link linkend='var-RDEPENDS'><filename>RDEPENDS</filename></link>.
+                This dependency forces an upgrade of the package that provides
+                the shared library when installing the package that depends on
+                the library, if needed.</para>
+
+                <para>If you want to avoid a package being registered as
+                providing a particular shared library (e.g. because the library
+                is for internal use only), then add the library to
+                <link linkend='var-PRIVATE_LIBS'><filename>PRIVATE_LIBS</filename></link>
+                inside the package's recipe.
+                </para></listitem>
+            <listitem><para>
+                <filename>pcdeps</filename>:
+                During the
+                <link linkend='ref-tasks-package'><filename>do_package</filename></link>
+                task of each recipe, all pkg-config modules
+                (<filename>*.pc</filename> files) installed by the recipe are
+                located.
+                For each module, the package that contains the module is
+                registered as providing the module.
+                The resulting module-to-package mapping is saved globally in
+                <link linkend='var-PKGDATA_DIR'><filename>PKGDATA_DIR</filename></link>
+                by the
+                <link linkend='ref-tasks-packagedata'><filename>do_packagedata</filename></link>
+                task.</para>
+
+                <para>Simultaneously, all pkg-config modules installed by the
+                recipe are inspected to see what other modules on which the
+                pkg-config modules depend.
+                A module is seen as depending on another module if it contains
+                a "Requires:" line that specifies the other module.
+                For each module dependency,
+                <filename>PKGDATA_DIR</filename> is queried to see if some
+                package contains the module.
+                If such a package is found, a runtime dependency is added from
+                the package that depends on the module to the package that
+                contains the module.
+                <note>
+                    The <filename>pcdeps</filename> mechanism most often infers
+                    dependencies between <filename>-dev</filename> packages.
+                </note>
+                </para></listitem>
+        </itemizedlist>
+    </para>
+
+    <para>
+        The <filename>do_package</filename> task depends on the
+        <link linkend='ref-tasks-packagedata'><filename>do_packagedata</filename></link>
+        task of each recipe in
+        <link linkend='var-DEPENDS'><filename>DEPENDS</filename></link>
+        through use of a
+        <ulink url='&YOCTO_DOCS_BB_URL;#variable-flags'><filename>[vardeps]</filename></ulink>
+        declaration, which guarantees that the required
+        shared-library/module-to-package mapping information will be available
+        when needed as long as <filename>DEPENDS</filename> has been
+        correctly set.
+    </para>
+</section>
+
 <section id='fakeroot-and-pseudo'>
     <title>Fakeroot and Pseudo</title>