dev-manual: First draft of "multi-builds" section.
Fixes [YOCTO #5400] This is the first draft of a new section on multi-machine builds. (From yocto-docs rev: a47fd17295d943bff482a0747f3b2e69831aa985) Signed-off-by: Scott Rifenbark <scott.m.rifenbark@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This commit is contained in:
parent
146d4bb4dd
commit
5f4050f73d
|
@ -5973,6 +5973,180 @@ Gateways via their Web Interfaces</ulink>"</emphasis>
|
||||||
</section>
|
</section>
|
||||||
</section>
|
</section>
|
||||||
|
|
||||||
|
<section id='building-images-for-more-than-one-machine'>
|
||||||
|
<title>Building Images for More than One Machine</title>
|
||||||
|
|
||||||
|
<para>
|
||||||
|
A common scenario developers face is creating images for several
|
||||||
|
different machines that use the same software environment.
|
||||||
|
In this situation, it is tempting to set the
|
||||||
|
tunings and optimization flags for each build specifically for
|
||||||
|
the targeted hardware (i.e. "maxing out" the tunings).
|
||||||
|
Doing so can considerably add to build times and package feed
|
||||||
|
maintenance collectively for the machines.
|
||||||
|
For example, selecting tunes that are extremely specific to a
|
||||||
|
CPU core used in a system might enable some micro optimizations
|
||||||
|
in GCC for that particular system but would otherwise not gain
|
||||||
|
you much of a performance difference across the other systems
|
||||||
|
as compared to using a more general tuning across all the builds.
|
||||||
|
Another example is setting
|
||||||
|
<ulink url='var-DEFAULTTUNE'><filename>DEFAULTTUNE</filename></ulink>
|
||||||
|
specifically for each machine's build.
|
||||||
|
Rather than "max out" each build's tunings, you can take steps that
|
||||||
|
cause the OpenEmbedded build system to reuse software across the
|
||||||
|
various machines where it makes sense.
|
||||||
|
</para>
|
||||||
|
<para>
|
||||||
|
If build speed and package feed maintenance are considerations,
|
||||||
|
you should consider the points in this section that can help you
|
||||||
|
optimize your tunings to best consider build times and package
|
||||||
|
feed maintenance.
|
||||||
|
<itemizedlist>
|
||||||
|
<listitem><para><emphasis>Share the Build Directory:</emphasis>
|
||||||
|
If at all possible, share the
|
||||||
|
<ulink url='&YOCTO_DOCS_REF_URL;#var-TMPDIR'><filename>TMPDIR</filename></ulink>
|
||||||
|
across builds.
|
||||||
|
The Yocto Project supports switching between different
|
||||||
|
<ulink url='&YOCTO_DOCS_REF_URL;#var-MACHINE'><filename>MACHINE</filename></ulink>
|
||||||
|
values in the same <filename>TMPDIR</filename>.
|
||||||
|
This practice is well supported and regularly used by
|
||||||
|
developers when building for multiple machines.
|
||||||
|
When you use the same <filename>TMPDIR</filename> for
|
||||||
|
multiple machine builds, the OpenEmbedded build system can
|
||||||
|
reuse the existing native and often cross-recipes for
|
||||||
|
multiple machines.
|
||||||
|
Thus, build time decreases.
|
||||||
|
<note>
|
||||||
|
If
|
||||||
|
<ulink url='&YOCTO_DOCS_REF_URL;#var-DISTRO'><filename>DISTRO</filename></ulink>
|
||||||
|
settings change or fundamental configuration settings
|
||||||
|
such as the filesystem layout, you need to work with
|
||||||
|
a clean <filename>TMPDIR</filename>.
|
||||||
|
Sharing <filename>TMPDIR</filename> under these
|
||||||
|
circumstances might work but since it is not
|
||||||
|
guaranteed, you should use a clean
|
||||||
|
<filename>TMPDIR</filename>.
|
||||||
|
</note>
|
||||||
|
</para></listitem>
|
||||||
|
<listitem><para><emphasis>Enable the Appropriate Package Architecture:</emphasis>
|
||||||
|
By default, the OpenEmbedded build system enables three
|
||||||
|
levels of package architectures: "all", "tune" or "package",
|
||||||
|
and "machine".
|
||||||
|
Any given recipe usually selects one of these package
|
||||||
|
architectures (types) for its output.
|
||||||
|
Depending for what a given recipe creates packages, making
|
||||||
|
sure you enable the appropriate package architecture can
|
||||||
|
directly impact the build time.</para>
|
||||||
|
<para>A recipe that just generates scripts can enable
|
||||||
|
"all" architecture because there are no binaries to build.
|
||||||
|
To specifically enable "all" architecture, be sure your
|
||||||
|
recipe inherits the
|
||||||
|
<ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-allarch'><filename>allarch</filename></ulink>
|
||||||
|
class.
|
||||||
|
This class is useful for "all" architectures because it
|
||||||
|
contains many other variables that can be used across
|
||||||
|
multiple architectures.</para>
|
||||||
|
<para>If your recipe needs to generate packages that are
|
||||||
|
machine-specific or when one of the build or runtime
|
||||||
|
dependencies is already machine-architecture dependent,
|
||||||
|
which makes your recipe also machine-architecture dependent,
|
||||||
|
make sure your recipe enables the "machine" package
|
||||||
|
architecture through the
|
||||||
|
<ulink url='&YOCTO_DOCS_REF_URL;#var-MACHINE_ARCH'><filename>MACHINE_ARCH</filename></ulink>
|
||||||
|
variable:
|
||||||
|
<literallayout class='monospaced'>
|
||||||
|
PACKAGE_ARCH = "${MACHINE_ARCH}"
|
||||||
|
</literallayout>
|
||||||
|
If you select a specific tuning for the recipe,
|
||||||
|
the OpenEmbedded build system enables the package
|
||||||
|
architecture defined by that tuning file.</para>
|
||||||
|
<para>When you do not specifically enable a package
|
||||||
|
architecture through the
|
||||||
|
<ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGE_ARCH'><filename>PACKAGE_ARCH</filename></ulink>,
|
||||||
|
The OpenEmbedded build system defaults to the
|
||||||
|
<ulink url='&YOCTO_DOCS_REF_URL;#var-TUNE_ARCH'><filename>TUNE_ARCH</filename></ulink>
|
||||||
|
setting:
|
||||||
|
<literallayout class='monospaced'>
|
||||||
|
PACKAGE_ARCH == TUNE_ARCH
|
||||||
|
</literallayout>
|
||||||
|
</para></listitem>
|
||||||
|
<listitem><para><emphasis>Choose a Generic Tuning File if Possible:</emphasis>
|
||||||
|
Some tunes are more generic and can run on multiple targets
|
||||||
|
(e.g. an <filename>armv5</filename> set of packages could
|
||||||
|
run on <filename>armv6</filename> and
|
||||||
|
<filename>armv7</filename> processors in most cases).
|
||||||
|
Similarly, <filename>i486</filename> binaries could work
|
||||||
|
on <filename>i586</filename> and higher processors.</para>
|
||||||
|
<para>If you select the same tune for several different
|
||||||
|
machines, the OpenEmbedded build system reuses software
|
||||||
|
previously built, thus speeding up the overall build time.
|
||||||
|
Realize that even though a new sysroot for each machine is
|
||||||
|
generated, the software is not recompiled and only one
|
||||||
|
package feed exists.
|
||||||
|
</para></listitem>
|
||||||
|
<listitem><para><emphasis>Manage Granular Level Packaging:</emphasis>
|
||||||
|
Sometimes cases exist where where injecting another level
|
||||||
|
of package architecture beyond the three higher levels
|
||||||
|
noted earlier can be useful.
|
||||||
|
For example, consider the <filename>emgd</filename>
|
||||||
|
graphics stack in the
|
||||||
|
<filename>meta-intel</filename> layer.
|
||||||
|
In this layer, a subset of software exists that is
|
||||||
|
compiled against something different from the rest of the
|
||||||
|
generic packages.
|
||||||
|
You can examine the key code in the
|
||||||
|
<ulink url='http://git.yoctoproject.org/cgit/cgit.cgi'>Source Repositories</ulink>
|
||||||
|
in <filename>classes/emgd-gl.bbclass</filename>.
|
||||||
|
For a specific set of packages, the code redefines
|
||||||
|
<ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGE_ARCH'><filename>PACKAGE_ARCH</filename></ulink>.
|
||||||
|
<ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGE_EXTRA_ARCHS'><filename>PACKAGE_EXTRA_ARCHS</filename></ulink>
|
||||||
|
is then appended with this extra tune name in
|
||||||
|
<filename>meta-intel-emgd.inc</filename>.
|
||||||
|
The result is that when searching for packages, the
|
||||||
|
build system uses a four-level search and the packages
|
||||||
|
in this new level are preferred as compared to the standard
|
||||||
|
tune.
|
||||||
|
The overall result is that the build system reuses most
|
||||||
|
software from the common tune except for specific cases
|
||||||
|
as needed.
|
||||||
|
</para></listitem>
|
||||||
|
<listitem><para><emphasis>Use Tools to Debug Issues:</emphasis>
|
||||||
|
Sometimes you can run into situations where software is
|
||||||
|
being rebuilt when you think it should not be.
|
||||||
|
For example, the OpenEmbedded build system might not be
|
||||||
|
using shared state between machines when you think it
|
||||||
|
should be.
|
||||||
|
These types of situations are usually due to references
|
||||||
|
to variables such as <filename>MACHINE</filename>
|
||||||
|
in code that is supposed to only be "tune" specific.</para>
|
||||||
|
<note>
|
||||||
|
Patches to fix any issues identified are most welcome
|
||||||
|
as these issues occasionally do occur.
|
||||||
|
</note>
|
||||||
|
<para>For such cases, you can use some tools to help you
|
||||||
|
sort out the situation:
|
||||||
|
<itemizedlist>
|
||||||
|
<listitem><para><emphasis><filename>sstate-diff-machines.sh</filename>:</emphasis>
|
||||||
|
You can find this tool in the
|
||||||
|
<filename>scripts</filename> directory of the
|
||||||
|
Source Repositories.
|
||||||
|
See the comments in the script for information on
|
||||||
|
how to use the tool.
|
||||||
|
</para></listitem>
|
||||||
|
<listitem><para><emphasis>BitBake's "-S printdiff" Option:</emphasis>
|
||||||
|
Using this option causes BitBake to try to
|
||||||
|
establish the closest signature match it can
|
||||||
|
(e.g. in the shared state cache) and then run
|
||||||
|
<filename>bitbake-diffsigs</filename> over the
|
||||||
|
matches to determine the stamps and delta where
|
||||||
|
these two stamp trees diverge.
|
||||||
|
</para></listitem>
|
||||||
|
</itemizedlist>
|
||||||
|
</para></listitem>
|
||||||
|
</itemizedlist>
|
||||||
|
</para>
|
||||||
|
</section>
|
||||||
|
|
||||||
<section id='working-with-packages'>
|
<section id='working-with-packages'>
|
||||||
<title>Working with Packages</title>
|
<title>Working with Packages</title>
|
||||||
|
|
||||||
|
|
Loading…
Reference in New Issue