2148 lines
104 KiB
XML
2148 lines
104 KiB
XML
<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
|
|
"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd"
|
|
[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] >
|
|
|
|
<chapter id='technical-details'>
|
|
<title>Technical Details</title>
|
|
|
|
<para>
|
|
This chapter provides technical details for various parts of the Yocto Project.
|
|
Currently, topics include Yocto Project components,
|
|
shared state (sstate) cache, x32, and Licenses.
|
|
</para>
|
|
|
|
<section id='usingpoky-components'>
|
|
<title>Yocto Project Components</title>
|
|
|
|
<para>
|
|
The BitBake task executor together with various types of configuration files form the
|
|
OpenEmbedded Core.
|
|
This section overviews these by describing what they are used for
|
|
and how they interact.
|
|
</para>
|
|
|
|
<para>
|
|
BitBake handles the parsing and execution of the data files.
|
|
The data itself is of various types:
|
|
<itemizedlist>
|
|
<listitem><para><emphasis>Recipes:</emphasis> Provides details about particular
|
|
pieces of software.</para></listitem>
|
|
<listitem><para><emphasis>Class Data:</emphasis> Abstracts common build
|
|
information (e.g. how to build a Linux kernel).</para></listitem>
|
|
<listitem><para><emphasis>Configuration Data:</emphasis> Defines machine-specific settings,
|
|
policy decisions, and so forth.
|
|
Configuration data acts as the glue to bind everything together.</para></listitem>
|
|
</itemizedlist>
|
|
For more information on data, see the
|
|
"<ulink url='&YOCTO_DOCS_DEV_URL;#yocto-project-terms'>Yocto Project Terms</ulink>"
|
|
section in the Yocto Project Development Manual.
|
|
</para>
|
|
|
|
<para>
|
|
BitBake knows how to combine multiple data sources together and refers to each data source
|
|
as a layer.
|
|
For information on layers, see the
|
|
"<ulink url='&YOCTO_DOCS_DEV_URL;#understanding-and-creating-layers'>Understanding and
|
|
Creating Layers</ulink>" section of the Yocto Project Development Manual.
|
|
</para>
|
|
|
|
<para>
|
|
Following are some brief details on these core components.
|
|
For more detailed information on these components, see the
|
|
"<link linkend='ref-structure'>Source Directory Structure</link>" chapter.
|
|
</para>
|
|
|
|
<section id='usingpoky-components-bitbake'>
|
|
<title>BitBake</title>
|
|
|
|
<para>
|
|
BitBake is the tool at the heart of the OpenEmbedded build system
|
|
and is responsible for parsing the
|
|
<ulink url='&YOCTO_DOCS_DEV_URL;#metadata'>Metadata</ulink>,
|
|
generating a list of tasks from it, and then executing those tasks.
|
|
To see a list of the options BitBake supports, use the following
|
|
help command:
|
|
<literallayout class='monospaced'>
|
|
$ bitbake --help
|
|
</literallayout>
|
|
</para>
|
|
|
|
<para>
|
|
The most common usage for BitBake is <filename>bitbake <packagename></filename>, where
|
|
<filename>packagename</filename> is the name of the package you want to build
|
|
(referred to as the "target" in this manual).
|
|
The target often equates to the first part of a <filename>.bb</filename> filename.
|
|
So, to run the <filename>matchbox-desktop_1.2.3.bb</filename> file, you
|
|
might type the following:
|
|
<literallayout class='monospaced'>
|
|
$ bitbake matchbox-desktop
|
|
</literallayout>
|
|
Several different versions of <filename>matchbox-desktop</filename> might exist.
|
|
BitBake chooses the one selected by the distribution configuration.
|
|
You can get more details about how BitBake chooses between different
|
|
target versions and providers in the
|
|
"<link linkend='ref-bitbake-providers'>Preferences and Providers</link>" section.
|
|
</para>
|
|
|
|
<para>
|
|
BitBake also tries to execute any dependent tasks first.
|
|
So for example, before building <filename>matchbox-desktop</filename>, BitBake
|
|
would build a cross compiler and <filename>eglibc</filename> if they had not already
|
|
been built.
|
|
<note>This release of the Yocto Project does not support the <filename>glibc</filename>
|
|
GNU version of the Unix standard C library. By default, the OpenEmbedded build system
|
|
builds with <filename>eglibc</filename>.</note>
|
|
</para>
|
|
|
|
<para>
|
|
A useful BitBake option to consider is the <filename>-k</filename> or
|
|
<filename>--continue</filename> option.
|
|
This option instructs BitBake to try and continue processing the job as much
|
|
as possible even after encountering an error.
|
|
When an error occurs, the target that
|
|
failed and those that depend on it cannot be remade.
|
|
However, when you use this option other dependencies can still be processed.
|
|
</para>
|
|
</section>
|
|
|
|
<section id='usingpoky-components-metadata'>
|
|
<title>Metadata (Recipes)</title>
|
|
|
|
<para>
|
|
The <filename>.bb</filename> files are usually referred to as "recipes."
|
|
In general, a recipe contains information about a single piece of software.
|
|
The information includes the location from which to download the source patches
|
|
(if any are needed), which special configuration options to apply,
|
|
how to compile the source files, and how to package the compiled output.
|
|
</para>
|
|
|
|
<para>
|
|
The term "package" can also be used to describe recipes.
|
|
However, since the same word is used for the packaged output from the OpenEmbedded
|
|
build system (i.e. <filename>.ipk</filename> or <filename>.deb</filename> files),
|
|
this document avoids using the term "package" when referring to recipes.
|
|
</para>
|
|
</section>
|
|
|
|
<section id='usingpoky-components-classes'>
|
|
<title>Classes</title>
|
|
|
|
<para>
|
|
Class files (<filename>.bbclass</filename>) contain information that
|
|
is useful to share between
|
|
<ulink url='&YOCTO_DOCS_DEV_URL;#metadata'>Metadata</ulink> files.
|
|
An example is the Autotools class, which contains
|
|
common settings for any application that Autotools uses.
|
|
The "<link linkend='ref-classes'>Classes</link>" chapter provides details
|
|
about common classes and how to use them.
|
|
</para>
|
|
</section>
|
|
|
|
<section id='usingpoky-components-configuration'>
|
|
<title>Configuration</title>
|
|
|
|
<para>
|
|
The configuration files (<filename>.conf</filename>) define various configuration variables
|
|
that govern the OpenEmbedded build process.
|
|
These files fall into several areas that define machine configuration options,
|
|
distribution configuration options, compiler tuning options, general common configuration
|
|
options, and user configuration options in <filename>local.conf</filename>, which is found
|
|
in the
|
|
<ulink url='&YOCTO_DOCS_DEV_URL;#build-directory'>Build Directory</ulink>.
|
|
</para>
|
|
</section>
|
|
</section>
|
|
|
|
<section id="a-closer-look-at-the-yocto-project-development-environment">
|
|
<title>A Closer Look at the Yocto Project Development Environment</title>
|
|
|
|
<para>
|
|
This section takes a more detailed look at the Yocto Project
|
|
development environment.
|
|
The following diagram represents the development environment at a
|
|
high level.
|
|
The remainder of this section expands on the fundamental input, output,
|
|
process, and
|
|
<ulink url='&YOCTO_DOCS_DEV_URL;#metadata'>Metadata</ulink>) blocks
|
|
in the Yocto Project development environment.
|
|
</para>
|
|
|
|
<para>
|
|
<imagedata fileref="figures/yocto-environment-ref.png" align="center" width="8in" depth="4.25in" />
|
|
</para>
|
|
|
|
<para>
|
|
The generalized Yocto Project Development Environment consists of
|
|
several functional areas:
|
|
<itemizedlist>
|
|
<listitem><para><emphasis>User Configuration:</emphasis>
|
|
Metadata you can use to control the build process.
|
|
</para></listitem>
|
|
<listitem><para><emphasis>Metadata Layers:</emphasis>
|
|
Various layers that provide software, machine, and
|
|
distro Metadata.</para></listitem>
|
|
<listitem><para><emphasis>Source Files:</emphasis>
|
|
Upstream releases, local projects, and SCMs.</para></listitem>
|
|
<listitem><para><emphasis>Build System:</emphasis>
|
|
Processes under the control of BitBake.
|
|
This block expands on how BitBake fetches source, applies
|
|
patches, completes compilation, analyzes output for package
|
|
generation, creates and tests packages, generates images, and
|
|
generates cross-development tools.</para></listitem>
|
|
<listitem><para><emphasis>Package Feeds:</emphasis>
|
|
Directories containing output packages (rpm, deb or ipk),
|
|
which are subsequently used in the construction of an image or
|
|
SDK, produced by the build system.
|
|
These feeds can also be copied and shared using a web server or
|
|
other means to facilitate extending or updating existing
|
|
images on devices at runtime if runtime package management is
|
|
enabled.</para></listitem>
|
|
<listitem><para><emphasis>Images:</emphasis>
|
|
Images produced by the development process.
|
|
Where do they go?
|
|
Can you mess with them (i.e. freely delete them or move them?).
|
|
</para></listitem>
|
|
<listitem><para><emphasis>Application Development SDK:</emphasis>
|
|
Cross-development tools that are produced along with an image
|
|
or separately with BitBake.</para></listitem>
|
|
</itemizedlist>
|
|
</para>
|
|
|
|
<section id="user-configuration">
|
|
<title>User Configuration</title>
|
|
|
|
<para>
|
|
User configuration helps define the build.
|
|
Through user configuration, you can tell BitBake the
|
|
target architecture for which you are building the image,
|
|
where to store downloaded source, and other build properties.
|
|
The following figure shows an expanded representation of the
|
|
user configuration box of the Yocto Project development
|
|
environment:
|
|
</para>
|
|
|
|
<para>
|
|
<imagedata fileref="figures/user-configuration.png" align="center" width="6in" depth="3.5in" />
|
|
</para>
|
|
|
|
<para>
|
|
BitBake needs some basic configuration files in order to complete
|
|
a build.
|
|
These files are <filename>*.conf</filename> files.
|
|
The minimally necessary ones reside as example files in the
|
|
<ulink url='&YOCTO_DOCS_DEV_URL;#source-directory'>Source Directory</ulink>.
|
|
For simplicity, this section refers to the Source Directory as
|
|
the "Poky Directory."
|
|
</para>
|
|
|
|
<para>
|
|
When you clone the <filename>poky</filename> Git repository or you
|
|
download and unpack a Yocto Project release, you can set up the
|
|
Source Directory to be named anything you want.
|
|
For this discussion, the cloned repository uses the default
|
|
name <filename>poky</filename>.
|
|
Within the figure, layers appear inside the Source Directory using
|
|
a bold typeface.
|
|
<note>
|
|
The Poky repository is primarily an aggregation of existing
|
|
repositories.
|
|
It is not a canonical upstream source.
|
|
</note>
|
|
</para>
|
|
|
|
<para>
|
|
The <filename>meta-yocto</filename> layer inside Poky contains
|
|
a <filename>conf</filename> directory that has example
|
|
configuration files.
|
|
These example files are used as a basis for creating actual
|
|
configuration files when you source the build environment
|
|
script <filename>oe-init-build-env</filename>.
|
|
</para>
|
|
|
|
<para>
|
|
Sourcing the build environment script creates a
|
|
<ulink url='&YOCTO_DOCS_DEV_URL;#build-directory'>Build Directory</ulink>
|
|
if one does not already exist.
|
|
BitBake uses the Build Directory for all its work during builds.
|
|
The Build Directory has a <filename>conf</filename> directory that
|
|
contains default versions of your <filename>local.conf</filename>
|
|
and <filename>bblayers.conf</filename> configuration files.
|
|
These default configuration files are created only if versions
|
|
do not already exist in the Build Directory at the time you
|
|
source the <filename>oe-init-build-env</filename> script.
|
|
</para>
|
|
|
|
<para>
|
|
Because the Poky repository is fundamentally an aggregation of
|
|
existing repositories, some users might be familiar with running
|
|
the <filename>oe-init-build-env</filename> script in the context of
|
|
separate OpenEmbedded-Core and BitBake repositories rather than a
|
|
single Poky repository.
|
|
This discussion assumes the script is executed from within a cloned
|
|
or unpacked version of Poky.
|
|
</para>
|
|
|
|
<para>
|
|
Depending on where the script is sourced, different sub-scripts
|
|
are called to set up the Build Directory (Yocto or OpenEmbedded).
|
|
Specifically, the script
|
|
<filename>scripts/oe-setup-builddir</filename> inside the
|
|
poky directory sets up the Build Directory and seeds the directory
|
|
(if necessary) with configuration files appropriate for the
|
|
Yocto Project development environment.
|
|
<note>
|
|
The <filename>scripts/oe-setup-builddir</filename> script
|
|
uses the <filename>$TEMPLATECONF</filename> variable to
|
|
determine which sample configuration files to locate.
|
|
</note>
|
|
</para>
|
|
|
|
<para>
|
|
The <filename>local.conf</filename> file provides many
|
|
basic variables that define a build environment.
|
|
Here is a list of a few.
|
|
To see the default configurations in a <filename>local.conf</filename>
|
|
file created by the build environment script, see the
|
|
<filename>local.conf.sample</filename> in the
|
|
<filename>meta-yocto</filename> layer:
|
|
<itemizedlist>
|
|
<listitem><para><emphasis>Parallelism Options:</emphasis>
|
|
Controlled by the
|
|
<link linkend='var-BB_NUMBER_THREADS'><filename>BB_NUMBER_THREADS</filename></link>
|
|
and
|
|
<link linkend='var-PARALLEL_MAKE'><filename>PARALLEL_MAKE</filename></link>
|
|
variables.</para></listitem>
|
|
<listitem><para><emphasis>Target Machine Selection:</emphasis>
|
|
Controlled by the
|
|
<link linkend='var-MACHINE'><filename>MACHINE</filename></link>
|
|
variable.</para></listitem>
|
|
<listitem><para><emphasis>Download Directory:</emphasis>
|
|
Controlled by the
|
|
<link linkend='var-DL_DIR'><filename>DL_DIR</filename></link>
|
|
variable.</para></listitem>
|
|
<listitem><para><emphasis>Shared State Directory:</emphasis>
|
|
Controlled by the
|
|
<link linkend='var-SSTATE_DIR'><filename>SSTATE_DIR</filename></link>
|
|
variable.</para></listitem>
|
|
<listitem><para><emphasis>Build Output:</emphasis>
|
|
Controlled by the
|
|
<link linkend='var-TMPDIR'><filename>TMPDIR</filename></link>
|
|
variable.</para></listitem>
|
|
</itemizedlist>
|
|
<note>
|
|
Configurations set in the <filename>conf/local.conf</filename>
|
|
file can also be set in the
|
|
<filename>conf/site.conf</filename> and
|
|
<filename>conf/auto.conf</filename> configuration files.
|
|
</note>
|
|
</para>
|
|
|
|
<para>
|
|
The <filename>bblayers.conf</filename> file tells BitBake what
|
|
layers you want considered during the build.
|
|
By default, the layers listed in this file include layers
|
|
minimally needed by the build system.
|
|
However, you must manually add any custom layers you have created.
|
|
You can find more information on working with the
|
|
<filename>bblayers.conf</filename> file in the
|
|
"<ulink url='&YOCTO_DOCS_DEV_URL;#enabling-your-layer'>Enabling Your Layer</ulink>"
|
|
section in the Yocto Project Development Manual.
|
|
</para>
|
|
|
|
<para>
|
|
The files <filename>site.conf</filename> and
|
|
<filename>auto.conf</filename> are not created by the environment
|
|
initialization script.
|
|
If you want these configuration files, you must create them
|
|
yourself:
|
|
<itemizedlist>
|
|
<listitem><para><emphasis><filename>site.conf</filename>:</emphasis>
|
|
You can use the <filename>conf/site.conf</filename>
|
|
configuration file to configure multiple build directories.
|
|
For example, suppose you had several build environments and
|
|
they shared some common features.
|
|
You can set these default build properties here.
|
|
A good example is perhaps the level of parallelism you want
|
|
to use through the
|
|
<link linkend='var-BB_NUMBER_THREADS'><filename>BB_NUMBER_THREADS</filename></link>
|
|
and
|
|
<link linkend='var-PARALLEL_MAKE'><filename>PARALLEL_MAKE</filename></link>
|
|
variables.</para>
|
|
<para>One useful scenario for using the
|
|
<filename>conf/site.conf</filename> file is to extend your
|
|
<link linkend='var-BBPATH'><filename>BBPATH</filename></link>
|
|
variable to include the path to a
|
|
<filename>conf/site.conf</filename>.
|
|
Then, when BitBake looks for Metadata using
|
|
<filename>BBPATH</filename>, it finds the
|
|
<filename>conf/site.conf</filename> file and applies your
|
|
common configurations found in the file.
|
|
To override configurations in a particular build directory,
|
|
alter the similar configurations within that build
|
|
directory's <filename>conf/local.conf</filename> file.
|
|
</para></listitem>
|
|
<listitem><para><emphasis><filename>auto.conf</filename>:</emphasis>
|
|
This file is not hand-created.
|
|
Rather, the file is usually created and written to by
|
|
an autobuilder.
|
|
The settings put into the file are typically the same as
|
|
you would find in the <filename>conf/local.conf</filename>
|
|
or the <filename>conf/site.conf</filename> files.
|
|
</para></listitem>
|
|
</itemizedlist>
|
|
</para>
|
|
|
|
<para>
|
|
You can edit all configuration files to further define
|
|
any particular build environment.
|
|
This process is represented by the "User Configuration Edits"
|
|
box in the figure.
|
|
</para>
|
|
|
|
<para>
|
|
When you launch your build with the
|
|
<filename>bitbake <target></filename> command, BitBake
|
|
sorts out the configurations to ultimately define your build
|
|
environment.
|
|
</para>
|
|
</section>
|
|
|
|
<section id="metadata-machine-configuration-and-policy-configuration">
|
|
<title>Metadata, Machine Configuration, and Policy Configuration</title>
|
|
|
|
<para>
|
|
The previous section described the user configurations that
|
|
define the BitBake's global behavior.
|
|
This section takes a closer look at the layers the build system
|
|
uses to further control the build.
|
|
These layers provide Metadata for the software, machine, and
|
|
policy.
|
|
</para>
|
|
|
|
<para>
|
|
In general, three types of layer input exist:
|
|
<itemizedlist>
|
|
<listitem><para><emphasis>Policy Configuration:</emphasis>
|
|
Distribution Layers provide top-level or general
|
|
policies for the image or SDK being built.
|
|
For example, this layer would dictate whether BitBake
|
|
produces RPM or IPK packages.</para></listitem>
|
|
<listitem><para><emphasis>Machine Configuration:</emphasis>
|
|
Board Support Package (BSP) layers provide machine
|
|
configurations.
|
|
This type of information is specific to a particular
|
|
target architecture.</para></listitem>
|
|
<listitem><para><emphasis>Metadata:</emphasis>
|
|
Software layers contain user-supplied recipe files,
|
|
patches, and append files.
|
|
</para></listitem>
|
|
</itemizedlist>
|
|
</para>
|
|
|
|
<para>
|
|
The following figure shows an expanded representation of the
|
|
Metadata, Machine Configuration, and Policy Configuration input
|
|
(layers) boxes of the Yocto Project development environment:
|
|
</para>
|
|
|
|
<para>
|
|
<imagedata fileref="figures/layer-input.png" align="center" width="8in" depth="7.5in" />
|
|
</para>
|
|
|
|
<para>
|
|
In general, all layers have a similar structure.
|
|
They all contain a licensing file
|
|
(e.g. <filename>COPYING</filename>) if the layer is to be
|
|
distributed, a <filename>README</filename> file as good practice
|
|
and especially if the layer is to be distributed, a
|
|
configuration directory, and recipe directories.
|
|
</para>
|
|
|
|
<para>
|
|
The Yocto Project has many layers that can be used.
|
|
You can see a web-interface listing of them on the
|
|
<ulink url="http://git.yoctoproject.org/">Source Repositories</ulink>
|
|
page.
|
|
The layers are shown at the bottom categorized under
|
|
"Yocto Metadata Layers."
|
|
These layers are fundamentally a subset of the
|
|
<ulink url="http://layers.openembedded.org/layerindex/layers/">OpenEmbedded Metadata Index</ulink>,
|
|
which lists all layers provided by the OpenEmbedded community.
|
|
<note>
|
|
Layers exist in the Yocto Project Source Repositories that
|
|
cannot be found in the OpenEmbedded Metadata Index.
|
|
These layers are either deprecated or experimental in nature.
|
|
</note>
|
|
</para>
|
|
|
|
<para>
|
|
BitBake uses the <filename>conf/bblayers.conf</filename> file,
|
|
which is part of the user configuration, to find what layers it
|
|
should be using as part of the build.
|
|
</para>
|
|
|
|
<para>
|
|
For more information on layers, see the
|
|
"<ulink url='&YOCTO_DOCS_DEV_URL;#understanding-and-creating-layers'>Understanding and Creating Layers</ulink>"
|
|
section in the Yocto Project Development Manual.
|
|
</para>
|
|
|
|
<section id="distro-layer">
|
|
<title>Distro Layer</title>
|
|
|
|
<para>
|
|
The distribution layer provides policy configurations for your
|
|
distribution.
|
|
Best practices dictate that you isolate these types of
|
|
configurations into their own layer.
|
|
Settings you provide in
|
|
<filename>conf/<distro>.conf</filename> override similar
|
|
settings that BitBake finds in your
|
|
<filename>conf/local.conf</filename> file in the Build
|
|
Directory.
|
|
</para>
|
|
|
|
<para>
|
|
The following list provides some explanation and references
|
|
for what you typically find in the distribution layer:
|
|
<itemizedlist>
|
|
<listitem><para><emphasis>classes:</emphasis>
|
|
Class files (<filename>.bbclass</filename>) holds
|
|
common functionality that can be shared among
|
|
recipes in the distribution.
|
|
When your recipes inherit a class, they take on the
|
|
settings and functions for that class.
|
|
You can read more about class files in the
|
|
"<link linkend='ref-classes'>Classes</link>" section.
|
|
</para></listitem>
|
|
<listitem><para><emphasis>conf:</emphasis>
|
|
This area holds configuration files for the
|
|
layer (<filename>conf/layer.conf</filename>),
|
|
the distribution
|
|
(<filename>conf/distro/<distro>.conf</filename>),
|
|
and any distribution-wide include files.
|
|
</para></listitem>
|
|
<listitem><para><emphasis>recipes-*:</emphasis>
|
|
Recipes and append files that affect common
|
|
functionality across the distribution.
|
|
This area could include recipes and append files to
|
|
to add distribution-specific configuration,
|
|
initialization scripts, custom image recipes,
|
|
and so forth.</para></listitem>
|
|
</itemizedlist>
|
|
</para>
|
|
</section>
|
|
|
|
<section id="bsp-layer">
|
|
<title>BSP Layer</title>
|
|
|
|
<para>
|
|
The BSP Layer provides machine configurations.
|
|
Everything in this layer is specific to the machine for which
|
|
you are building the image or the SDK.
|
|
A common structure or form is defined for BSP layers.
|
|
You can learn more about this structure in the
|
|
<ulink url='&YOCTO_DOCS_BSP_URL;'>Yocto Project Board Support Package (BSP) Developer's Guide</ulink>.
|
|
<note>
|
|
In order for a BSP layer to be considered compliant with the
|
|
Yocto Project, it must meet some structural requirements.
|
|
</note>
|
|
</para>
|
|
|
|
<para>
|
|
The BSP Layer's configuration directory contains
|
|
configuration files for the machine
|
|
(<filename>conf/machine/<machine>.conf</filename>) and,
|
|
of course, the layer (<filename>conf/layer.conf</filename>).
|
|
</para>
|
|
|
|
<para>
|
|
The remainder of the layer is dedicated to specific recipes
|
|
by function: <filename>recipes-bsp</filename>,
|
|
<filename>recipes-core</filename>,
|
|
<filename>recipes-graphics</filename>, and
|
|
<filename>recipes-kernel</filename>.
|
|
Metadata can exist for multiple formfactors, graphics
|
|
support systems, and so forth.
|
|
<note>
|
|
While the figure shows several <filename>recipe-*</filename>
|
|
directories, not all these directories appear in all
|
|
BSP layers.
|
|
</note>
|
|
</para>
|
|
</section>
|
|
|
|
<section id="software-layer">
|
|
<title>Software Layer</title>
|
|
|
|
<para>
|
|
The software layer provides the Metadata for additional
|
|
software packages used during the build.
|
|
This layer does not include Metadata that is specific to the
|
|
distribution or the machine, which are found in their
|
|
respective layers.
|
|
</para>
|
|
|
|
<para>
|
|
This layer contains any new recipes that your project needs
|
|
in the form of recipe files.
|
|
</para>
|
|
</section>
|
|
</section>
|
|
|
|
<section id="sources-dev-environment">
|
|
<title>Sources</title>
|
|
|
|
<para>
|
|
In order for the OpenEmbedded build system to create an image or
|
|
any target, it must be able to access source files.
|
|
The main
|
|
<link linkend='a-closer-look-at-the-yocto-project-development-environment'>Yocto Project Development Environment figure</link>
|
|
represents source files using the "Upstream Project Releases",
|
|
"Local Projects", and "SCMs (optional)" boxes.
|
|
The figure represents mirrors, which also play a role in locating
|
|
source files, with the "Source Mirror(s)" box.
|
|
</para>
|
|
|
|
<para>
|
|
The method by which source files are ultimately organized is
|
|
a function of the project.
|
|
For example, for released software, projects tend to use tarballs
|
|
or other archived files that can capture the state of a release
|
|
guaranteeing that it is statically represented.
|
|
On the other hand, for a project that is more dynamic or
|
|
experimental in nature, a project might keep source files in a
|
|
repository controlled by a Source Control Manager (SCM) such as
|
|
Git.
|
|
Pulling source from a repository allows you to control
|
|
the point in the repository (the revision) from which you want to
|
|
build software.
|
|
Finally, a combination of the two might exist, which would give the
|
|
consumer a choice when deciding where to get source files.
|
|
</para>
|
|
|
|
<para>
|
|
BitBake uses the
|
|
<link linkend='var-SRC_URI'><filename>SRC_URI</filename></link>
|
|
variable to point to source files regardless of their location.
|
|
Each recipe must have a <filename>SRC_URI</filename> variable
|
|
that points to the source.
|
|
</para>
|
|
|
|
<para>
|
|
Another area that plays a significant role in where source files
|
|
comes from is pointed to by the
|
|
<link linkend='var-DL_DIR'><filename>DL_DIR</filename></link>
|
|
variable.
|
|
This area is a cache that can hold previously downloaded source.
|
|
Judicious use of a <filename>DL_DIR</filename> directory can
|
|
save the build system a trip across the Internet when looking
|
|
for files.
|
|
A good method for using a download directory is to have
|
|
<filename>DL_DIR</filename> point to an area outside of your
|
|
Build Directory.
|
|
Doing so allows you to safely delete the Build Directory
|
|
if needed without fear of removing any downloaded source file.
|
|
</para>
|
|
|
|
<para>
|
|
The remainder of this section provides a deeper look into the
|
|
source files and the mirrors.
|
|
Here is a more detailed look at the source file area of the
|
|
base figure:
|
|
<imagedata fileref="figures/source-input.png" align="center" width="7in" depth="7.5in" />
|
|
</para>
|
|
|
|
<section id='upstream-project-releases'>
|
|
<title>Upstream Project Releases</title>
|
|
|
|
<para>
|
|
Upstream project releases exist anywhere in the form of an
|
|
archived file (e.g. tarball or zip file).
|
|
These files correspond to individual recipes.
|
|
For example, the figure uses specific releases each for
|
|
BusyBox, Qt, and Dbus.
|
|
An archive file can be for any released product that can be
|
|
built using a recipe.
|
|
</para>
|
|
</section>
|
|
|
|
<section id='local-projects'>
|
|
<title>Local Projects</title>
|
|
|
|
<para>
|
|
Local projects are custom bits of software the user provides.
|
|
These bits reside somewhere local to a project - perhaps
|
|
a directory into which the user checks in items (e.g.
|
|
a local directory containing a development source tree
|
|
used by the group).
|
|
</para>
|
|
|
|
<para>
|
|
The canonical method through which to include a local project
|
|
is to use the
|
|
<link linkend='ref-classes-externalsrc'><filename>externalsrc.bbclass</filename></link>
|
|
class to include local project.
|
|
You use either the <filename>local.conf</filename> or a
|
|
recipe's append file to override or set the
|
|
recipe to point to the local directory on your disk to pull
|
|
in the whole source tree.
|
|
</para>
|
|
|
|
<para>
|
|
For information on how to use the
|
|
<filename>externalsrc.bbclass</filename>, see the
|
|
"<link linkend='ref-classes-externalsrc'>Using External Source - <filename>externalsrc.bbclass</filename></link>"
|
|
section.
|
|
</para>
|
|
</section>
|
|
|
|
<section id='scms'>
|
|
<title>Source Control Managers (Optional)</title>
|
|
|
|
<para>
|
|
Another place the build system can get source files from is
|
|
through an SCM such as Git or Subversion.
|
|
In this case, a repository is cloned or checked out.
|
|
The <filename>do_fetch</filename> task inside BitBake uses
|
|
the <link linkend='var-SRC_URI'><filename>SRC_URI</filename></link>
|
|
variable and the argument's prefix to determine the correct
|
|
fetcher module.
|
|
</para>
|
|
|
|
<para>
|
|
When fetching a repository, BitBake uses the
|
|
<link linkend='var-SRCREV'><filename>SRCREV</filename></link>
|
|
variable to determine the specific revision from which to
|
|
build.
|
|
</para>
|
|
</section>
|
|
|
|
<section id='source-mirrors'>
|
|
<title>Source Mirror(s)</title>
|
|
|
|
<para>
|
|
Two kinds of mirrors exist: pre-mirrors and regular mirrors.
|
|
The <link linkend='var-PREMIRRORS'><filename>PREMIRRORS</filename></link>
|
|
and
|
|
<link linkend='var-MIRRORS'><filename>MIRRORS</filename></link>
|
|
variables point to these, respectively.
|
|
BitBake checks pre-mirrors before looking upstream for any
|
|
source files.
|
|
Pre-mirrors are appropriate when you have a shared directory
|
|
that is not a directory defined by the
|
|
<link linkend='var-DL_DIR'><filename>DL_DIR</filename></link>
|
|
variable.
|
|
A Pre-mirror typically points to a shared directory that is
|
|
local to your organization.
|
|
</para>
|
|
|
|
<para>
|
|
Regular mirrors can be any site across the Internet that is
|
|
used as an alternative location for source code should the
|
|
primary site not be functioning for some reason or another.
|
|
</para>
|
|
</section>
|
|
</section>
|
|
|
|
<section id="package-feeds-dev-environment">
|
|
<title>Package Feeds</title>
|
|
|
|
<para>
|
|
When the OpenEmbedded build system generates an image or an SDK,
|
|
it gets the packages from a package feed area located in the
|
|
<ulink url='&YOCTO_DOCS_DEV_URL;#build-directory'>Build Directory</ulink>.
|
|
The main
|
|
<link linkend='a-closer-look-at-the-yocto-project-development-environment'>Yocto Project Development Environment</link>
|
|
figure shows this package feeds area in the upper-right corner.
|
|
</para>
|
|
|
|
<para>
|
|
This section looks a little closer into the package feeds area used
|
|
by the build system.
|
|
Here is a more detailed look at the area:
|
|
<imagedata fileref="figures/package-feeds.png" align="center" width="7in" depth="6in" />
|
|
</para>
|
|
|
|
<para>
|
|
Package feeds are an intermediary step in the build process.
|
|
BitBake generates packages whose type is defined by the
|
|
<link linkend='var-PACKAGE_CLASSES'><filename>PACKAGE_CLASSES</filename></link>
|
|
variable.
|
|
Before placing the packages into package feeds,
|
|
the build process validates them with generated output quality
|
|
assurance checks through the
|
|
<link linkend='ref-classes-insane'><filename>insane.bbclass</filename></link>
|
|
class.
|
|
</para>
|
|
|
|
<para>
|
|
The package feed area resides in
|
|
<filename>tmp/deploy</filename> of the Build Directory.
|
|
Folders are created that correspond to the package type
|
|
(IPK, DEB, or RPM) created.
|
|
Further organization is derived through the value of the
|
|
<link linkend='var-PACKAGE_ARCH'><filename>PACKAGE_ARCH</filename></link>
|
|
variable for each package.
|
|
For example, packages can exist for the i586 or qemux86
|
|
architectures.
|
|
The package files themselves reside within the appropriate
|
|
architecture folder.
|
|
</para>
|
|
|
|
<para>
|
|
BitBake uses the <filename>do_package_write_*</filename> task to
|
|
place generated packages into the package holding area (e.g.
|
|
<filename>do_package_write_ipk</filename> for IPK packages).
|
|
</para>
|
|
</section>
|
|
|
|
<section id='images-dev-environment'>
|
|
<title>Images</title>
|
|
|
|
<para>
|
|
The images produced by the OpenEmbedded build system
|
|
are compressed forms of the
|
|
root filesystems that are ready to boot on a target device.
|
|
You can see from the main
|
|
<link linkend='a-closer-look-at-the-yocto-project-development-environment'>Yocto Project Development Environment</link>
|
|
figure that BitBake output in part consists of images.
|
|
This section is going to look more closely at this output:
|
|
<imagedata fileref="figures/images.png" align="center" width="5in" depth="4in" />
|
|
</para>
|
|
|
|
<para>
|
|
For a list of example images that the Yocto Project provides,
|
|
the
|
|
"<link linkend='ref-images'>Images</link>" chapter.
|
|
</para>
|
|
|
|
<para>
|
|
Images are written out to the
|
|
<ulink url='&YOCTO_DOCS_DEV_URL;#build-directory'>Build Directory</ulink>
|
|
inside the <filename>deploy/images</filename> folder as shown
|
|
in the figure.
|
|
This folder contains any files expected to be loaded on the
|
|
target device.
|
|
The
|
|
<link linkend='var-DEPLOY_DIR'><filename>DEPLOY_DIR</filename></link>
|
|
variable points to the <filename>deploy</filename> directory.
|
|
<itemizedlist>
|
|
<listitem><para><filename><kernel-image></filename>:
|
|
A kernel binary file.
|
|
The <link linkend='var-KERNEL_IMAGETYPE'><filename>KERNEL_IMAGETYPE</filename></link>
|
|
variable setting determines the naming scheme for the
|
|
kernel image file.
|
|
Depending on that variable, the file could begin with
|
|
a variety of naming strings.
|
|
The <filename>deploy/images</filename> directory can
|
|
contain multiple image files.</para></listitem>
|
|
<listitem><para><filename><root-filesystem-image></filename>:
|
|
Root filesystems for the target device (e.g.
|
|
<filename>*.ext3</filename> or <filename>*.bz2</filename>
|
|
files).
|
|
The <link linkend='var-IMAGE_FSTYPES'><filename>IMAGE_FSTYPES</filename></link>
|
|
variable setting determines the root filesystem image
|
|
type.
|
|
The <filename>deploy/images</filename> directory can
|
|
contain multiple root filesystems.</para></listitem>
|
|
<listitem><para><filename><kernel-modules></filename>:
|
|
Tarballs that contain all the modules used by the
|
|
kernel.
|
|
Kernel module tarballs exist for legacy purposes and
|
|
can be suppressed by setting the
|
|
<link linkend='var-MODULE_TARBALL_DEPLOY'><filename>MODULE_TARBALL_DEPLOY</filename></link>
|
|
variable to "0".
|
|
The <filename>deploy/images</filename> directory can
|
|
contain multiple kernel module tarballs.
|
|
</para></listitem>
|
|
<listitem><para><filename><bootloaders></filename>:
|
|
Bootloaders supporting the image, if applicable to the
|
|
target machine.
|
|
The <filename>deploy/images</filename> directory can
|
|
contain multiple bootloaders.
|
|
</para></listitem>
|
|
<listitem><para><filename><symlinks></filename>:
|
|
The <filename>images/deploy</filename> folder contains
|
|
a symbolic link that points to the most recently built file
|
|
for each machine.
|
|
These links might be useful for external scripts that
|
|
need to obtain the latest version of each file.
|
|
</para></listitem>
|
|
</itemizedlist>
|
|
</para>
|
|
</section>
|
|
|
|
<section id='sdk-dev-environment'>
|
|
<title>Application Development SDK</title>
|
|
|
|
<para>
|
|
In the overview figure of the
|
|
<link linkend='a-closer-look-at-the-yocto-project-development-environment'>Yocto Project Development Environment</link>
|
|
the output labeled "Application Development SDK" represents an
|
|
SDK.
|
|
This section is going to take a closer look at this output:
|
|
<imagedata fileref="figures/sdk.png" align="center" width="5in" depth="4in" />
|
|
</para>
|
|
|
|
<para>
|
|
The specific form of this output is a self-extracting
|
|
SDK installer (<filename>*.sh</filename>) that, when run,
|
|
installs the SDK image, which consists of a cross-development
|
|
toolchain, a set of libraries and headers, and an SDK
|
|
environment setup script.
|
|
Running this installer essentially sets up your
|
|
cross-development environment.
|
|
You can think of the cross-toolchains as the "host" part
|
|
because they run on the SDK machine.
|
|
You can think of the libraries and headers as the "target"
|
|
part because they are built for the target hardware.
|
|
The setup script is added so that you can initialize the
|
|
environment before using the tools.
|
|
</para>
|
|
|
|
<note>
|
|
<para>
|
|
The Yocto Project supports several methods by which you can
|
|
set up this cross-development environment.
|
|
These methods include downloading pre-built SDK installers,
|
|
building and installing your own SDK installer, or running
|
|
an Application Development Toolkit (ADT) installer to
|
|
install not just cross-development toolchains
|
|
but also additional tools to help in this type of
|
|
development.
|
|
</para>
|
|
|
|
<para>
|
|
For background information on cross-development toolchains
|
|
in the Yocto Project development environment, see the
|
|
"<link linkend='cross-development-toolchain-generation'>Cross-Development Toolchain Generation</link>"
|
|
section.
|
|
For information on setting up a cross-development
|
|
environment, see the
|
|
"<ulink url='&YOCTO_DOCS_ADT_URL;#installing-the-adt'>Installing the ADT and Toolchains</ulink>"
|
|
section in the Yocto Project Application Developer's Guide.
|
|
</para>
|
|
</note>
|
|
|
|
<para>
|
|
Once built, the SDK installers are written out to the
|
|
<filename>deploy/sdk</filename> folder inside the
|
|
<ulink url='&YOCTO_DOCS_DEV_URL;#build-directory'>Build Directory</ulink>
|
|
as shown in the figure at the beginning of this section.
|
|
Several variables exist that help configure these files:
|
|
<itemizedlist>
|
|
<listitem><para><link linkend='var-DEPLOY_DIR'><filename>DEPLOY_DIR</filename></link>:
|
|
Points to the <filename>deploy</filename>
|
|
directory.</para></listitem>
|
|
<listitem><para><link linkend='var-SDKMACHINE'><filename>SDKMACHINE</filename></link>:
|
|
Specifies the architecture of the machine
|
|
on which the cross-development tools are run to
|
|
create packages for the target hardware.
|
|
</para></listitem>
|
|
<listitem><para><link linkend='var-SDKIMAGE_FEATURES'><filename>SDKIMAGE_FEATURES</filename></link>:
|
|
Lists the features to include in the "target" part
|
|
of the SDK.
|
|
</para></listitem>
|
|
<listitem><para><link linkend='var-TOOLCHAIN_HOST_TASK'><filename>TOOLCHAIN_HOST_TASK</filename></link>:
|
|
Lists packages that make up the host
|
|
part of the SDK installer (i.e. the part that runs on
|
|
the <filename>SDKMACHINE</filename>).
|
|
When you use
|
|
<filename>bitbake -c populate_sdk <imagename></filename>
|
|
to create the SDK installer, a set of default packages
|
|
apply.
|
|
This variable allows you to add more packages.
|
|
</para></listitem>
|
|
<listitem><para><link linkend='var-TOOLCHAIN_TARGET_TASK'><filename>TOOLCHAIN_TARGET_TASK</filename></link>:
|
|
Lists packages that make up the target part
|
|
of the SDK installer (i.e. the part built for the
|
|
target hardware).
|
|
</para></listitem>
|
|
</itemizedlist>
|
|
</para>
|
|
</section>
|
|
</section>
|
|
|
|
<section id="cross-development-toolchain-generation">
|
|
<title>Cross-Development Toolchain Generation</title>
|
|
|
|
<para>
|
|
The Yocto Project does most of the work for you when it comes to
|
|
creating
|
|
<ulink url='&YOCTO_DOCS_DEV_URL;#cross-development-toolchain'>cross-development toolchains</ulink>.
|
|
This section provides some technical background information on how
|
|
cross-development toolchains are created and used.
|
|
For more information on these toolchain, you can also see the
|
|
<ulink url='&YOCTO_DOCS_ADT_URL;'>the Yocto Project Application Developer's Guide</ulink>.
|
|
</para>
|
|
|
|
<para>
|
|
In the Yocto Project development environment, cross-development
|
|
toolchains are used to build the image and applications that run on the
|
|
target hardware.
|
|
With just a few commands, the OpenEmbedded build system creates
|
|
these necessary toolchains for you.
|
|
</para>
|
|
|
|
<para>
|
|
The following figure shows a high-level build environment regarding
|
|
toolchain construction and use.
|
|
</para>
|
|
|
|
<para>
|
|
<imagedata fileref="figures/cross-development-toolchains.png" width="8in" depth="6in" align="center" />
|
|
</para>
|
|
|
|
<para>
|
|
Most of the work occurs on the Build Host.
|
|
This is the machine used to build images and generally work within the
|
|
the Yocto Project environment.
|
|
When you run BitBake to create an image, the OpenEmbedded build system
|
|
uses the host <filename>gcc</filename> compiler to bootstrap a
|
|
cross-compiler named <filename>gcc-cross</filename>.
|
|
The <filename>gcc-cross</filename> compiler is what BitBake uses to
|
|
compile source files when creating the target image.
|
|
You can think of <filename>gcc-cross</filename> simply as an
|
|
automatically generated cross-compiler that is used internally within
|
|
BitBake only.
|
|
</para>
|
|
|
|
<para>
|
|
The chain of events that occurs when <filename>gcc-cross</filename> is
|
|
bootstrapped is as follows:
|
|
<literallayout class='monospaced'>
|
|
gcc -> binutils-cross -> gcc-cross-initial -> linux_libc-headers -> eglibc-initial -> eglibc -> gcc-cross -> gcc-runtime
|
|
</literallayout>
|
|
<itemizedlist>
|
|
<listitem><para><filename>gcc</filename>:
|
|
The build host's GNU Compiler Collection (GCC).
|
|
</para></listitem>
|
|
<listitem><para><filename>binutils-cross</filename>:
|
|
The bare minimum binary utilities needed in order to run
|
|
the <filename>gcc-cross-initial</filename> phase of the
|
|
bootstrap operation.
|
|
</para></listitem>
|
|
<listitem><para><filename>gcc-cross-initial</filename>:
|
|
An early stage of the bootstrap process for creating
|
|
the cross-compiler.
|
|
This stage builds enough of the <filename>gcc-cross</filename>,
|
|
the C library, and other pieces needed to finish building the
|
|
final cross-compiler in later stages.
|
|
This tool is a "native" package (i.e. it is designed to run on
|
|
the build host).
|
|
</para></listitem>
|
|
<listitem><para><filename>linux_libc-headers</filename>:
|
|
Headers needed for the cross-compiler.
|
|
</para></listitem>
|
|
<listitem><para><filename>eglibc-initial</filename>:
|
|
An initial version of the Embedded GLIBC needed to bootstrap
|
|
<filename>eglibc</filename>.
|
|
</para></listitem>
|
|
<listitem><para><filename>gcc-cross</filename>:
|
|
The final stage of the bootstrap process for the
|
|
cross-compiler.
|
|
This stage results in the actual cross-compiler that
|
|
BitBake uses when it builds an image for a targeted
|
|
device.
|
|
<note>
|
|
If you are replacing this cross compiler toolchain
|
|
with a custom version, you must replace
|
|
<filename>gcc-cross</filename>.
|
|
</note>
|
|
This tool is also a "native" package (i.e. it is
|
|
designed to run on the build host).
|
|
</para></listitem>
|
|
<listitem><para><filename>gcc-runtime</filename>:
|
|
Runtime libraries resulting from the toolchain bootstrapping
|
|
process.
|
|
This tool produces a binary that consists of the
|
|
runtime libraries need for the targeted device.
|
|
</para></listitem>
|
|
</itemizedlist>
|
|
</para>
|
|
|
|
<para>
|
|
You can use the OpenEmbedded build system to build an installer for
|
|
the relocatable SDK used to develop applications.
|
|
When you run the installer, it installs the toolchain, which contains
|
|
the development tools (e.g., the
|
|
<filename>gcc-cross-canadian</filename>),
|
|
<filename>binutils-cross-canadian</filename>, and other
|
|
<filename>nativesdk-*</filename> tools you need to cross-compile and
|
|
test your software.
|
|
The figure shows the commands you use to easily build out this
|
|
toolchain.
|
|
This cross-development toolchain is built to execute on the
|
|
<link linkend='var-SDKMACHINE'><filename>SDKMACHINE</filename></link>,
|
|
which might or might not be the same
|
|
machine as the Build Host.
|
|
<note>
|
|
If your target architecture is supported by the Yocto Project,
|
|
you can take advantage of pre-built images that ship with the
|
|
Yocto Project and already contain cross-development toolchain
|
|
installers.
|
|
</note>
|
|
</para>
|
|
|
|
<para>
|
|
Here is the bootstrap process for the relocatable toolchain:
|
|
<literallayout class='monospaced'>
|
|
gcc -> binutils-crosssdk -> gcc-crosssdk-initial -> linux_libc-headers -> eglibc-initial -> nativesdk-eglibc -> gcc-crosssdk -> gcc-cross-canadian
|
|
</literallayout>
|
|
<itemizedlist>
|
|
<listitem><para><filename>gcc</filename>:
|
|
The build host's GNU Compiler Collection (GCC).
|
|
</para></listitem>
|
|
<listitem><para><filename>binutils-crosssdk</filename>:
|
|
The bare minimum binary utilities needed in order to run
|
|
the <filename>gcc-crosssdk-initial</filename> phase of the
|
|
bootstrap operation.
|
|
</para></listitem>
|
|
<listitem><para><filename>gcc-crosssdk-initial</filename>:
|
|
An early stage of the bootstrap process for creating
|
|
the cross-compiler.
|
|
This stage builds enough of the
|
|
<filename>gcc-crosssdk</filename> and supporting pieces so that
|
|
the final stage of the bootstrap process can produce the
|
|
finished cross-compiler.
|
|
This tool is a "native" binary that runs on the build host.
|
|
</para></listitem>
|
|
<listitem><para><filename>linux_libc-headers</filename>:
|
|
Headers needed for the cross-compiler.
|
|
</para></listitem>
|
|
<listitem><para><filename>eglibc-initial</filename>:
|
|
An initial version of the Embedded GLIBC needed to bootstrap
|
|
<filename>nativesdk-eglibc</filename>.
|
|
</para></listitem>
|
|
<listitem><para><filename>nativesdk-eglibc</filename>:
|
|
The Embedded GLIBC needed to bootstrap the
|
|
<filename>gcc-crosssdk</filename>.
|
|
</para></listitem>
|
|
<listitem><para><filename>gcc-crosssdk</filename>:
|
|
The final stage of the bootstrap process for the
|
|
relocatable cross-compiler.
|
|
The <filename>gcc-crosssdk</filename> is a transitory compiler
|
|
and never leaves the build host.
|
|
Its purpose is to help in the bootstrap process to create the
|
|
eventual relocatable <filename>gcc-cross-canadian</filename>
|
|
compiler, which is relocatable.
|
|
This tool is also a "native" package (i.e. it is
|
|
designed to run on the build host).
|
|
</para></listitem>
|
|
<listitem><para><filename>gcc-cross-canadian</filename>:
|
|
The final relocatable cross-compiler.
|
|
When run on the
|
|
<link linkend='var-SDKMACHINE'><filename>SDKMACHINE</filename></link>,
|
|
this tool
|
|
produces executable code that runs on the target device.
|
|
</para></listitem>
|
|
</itemizedlist>
|
|
</para>
|
|
</section>
|
|
|
|
<section id="shared-state-cache">
|
|
<title>Shared State Cache</title>
|
|
|
|
<para>
|
|
By design, the OpenEmbedded build system builds everything from scratch unless
|
|
BitBake can determine that parts do not need to be rebuilt.
|
|
Fundamentally, building from scratch is attractive as it means all parts are
|
|
built fresh and there is no possibility of stale data causing problems.
|
|
When developers hit problems, they typically default back to building from scratch
|
|
so they know the state of things from the start.
|
|
</para>
|
|
|
|
<para>
|
|
Building an image from scratch is both an advantage and a disadvantage to the process.
|
|
As mentioned in the previous paragraph, building from scratch ensures that
|
|
everything is current and starts from a known state.
|
|
However, building from scratch also takes much longer as it generally means
|
|
rebuilding things that do not necessarily need rebuilt.
|
|
</para>
|
|
|
|
<para>
|
|
The Yocto Project implements shared state code that supports incremental builds.
|
|
The implementation of the shared state code answers the following questions that
|
|
were fundamental roadblocks within the OpenEmbedded incremental build support system:
|
|
<itemizedlist>
|
|
<listitem>What pieces of the system have changed and what pieces have not changed?</listitem>
|
|
<listitem>How are changed pieces of software removed and replaced?</listitem>
|
|
<listitem>How are pre-built components that do not need to be rebuilt from scratch
|
|
used when they are available?</listitem>
|
|
</itemizedlist>
|
|
</para>
|
|
|
|
<para>
|
|
For the first question, the build system detects changes in the "inputs" to a given task by
|
|
creating a checksum (or signature) of the task's inputs.
|
|
If the checksum changes, the system assumes the inputs have changed and the task needs to be
|
|
rerun.
|
|
For the second question, the shared state (sstate) code tracks which tasks add which output
|
|
to the build process.
|
|
This means the output from a given task can be removed, upgraded or otherwise manipulated.
|
|
The third question is partly addressed by the solution for the second question
|
|
assuming the build system can fetch the sstate objects from remote locations and
|
|
install them if they are deemed to be valid.
|
|
</para>
|
|
|
|
<note>
|
|
The OpenEmbedded build system does not maintain
|
|
<link linkend='var-PR'><filename>PR</filename></link> information
|
|
as part of the shared state packages.
|
|
Consequently, considerations exist that affect maintaining shared
|
|
state feeds.
|
|
For information on how the OpenEmbedded works with packages and can
|
|
track incrementing <filename>PR</filename> information, see the
|
|
"<ulink url='&YOCTO_DOCS_DEV_URL;#incrementing-a-package-revision-number'>Incrementing a Package Revision Number</ulink>"
|
|
section.
|
|
</note>
|
|
|
|
<para>
|
|
The rest of this section goes into detail about the overall incremental build
|
|
architecture, the checksums (signatures), shared state, and some tips and tricks.
|
|
</para>
|
|
|
|
<section id='overall-architecture'>
|
|
<title>Overall Architecture</title>
|
|
|
|
<para>
|
|
When determining what parts of the system need to be built, BitBake
|
|
uses a per-task basis and does not use a per-recipe basis.
|
|
You might wonder why using a per-task basis is preferred over a per-recipe basis.
|
|
To help explain, consider having the IPK packaging backend enabled and then switching to DEB.
|
|
In this case, <filename>do_install</filename> and <filename>do_package</filename>
|
|
output are still valid.
|
|
However, with a per-recipe approach, the build would not include the
|
|
<filename>.deb</filename> files.
|
|
Consequently, you would have to invalidate the whole build and rerun it.
|
|
Rerunning everything is not the best situation.
|
|
Also in this case, the core must be "taught" much about specific tasks.
|
|
This methodology does not scale well and does not allow users to easily add new tasks
|
|
in layers or as external recipes without touching the packaged-staging core.
|
|
</para>
|
|
</section>
|
|
|
|
<section id='checksums'>
|
|
<title>Checksums (Signatures)</title>
|
|
|
|
<para>
|
|
The shared state code uses a checksum, which is a unique signature of a task's
|
|
inputs, to determine if a task needs to be run again.
|
|
Because it is a change in a task's inputs that triggers a rerun, the process
|
|
needs to detect all the inputs to a given task.
|
|
For shell tasks, this turns out to be fairly easy because
|
|
the build process generates a "run" shell script for each task and
|
|
it is possible to create a checksum that gives you a good idea of when
|
|
the task's data changes.
|
|
</para>
|
|
|
|
<para>
|
|
To complicate the problem, there are things that should not be included in
|
|
the checksum.
|
|
First, there is the actual specific build path of a given task -
|
|
the <link linkend='var-WORKDIR'><filename>WORKDIR</filename></link>.
|
|
It does not matter if the working directory changes because it should not
|
|
affect the output for target packages.
|
|
Also, the build process has the objective of making native or cross packages relocatable.
|
|
The checksum therefore needs to exclude <filename>WORKDIR</filename>.
|
|
The simplistic approach for excluding the working directory is to set
|
|
<filename>WORKDIR</filename> to some fixed value and create the checksum
|
|
for the "run" script.
|
|
</para>
|
|
|
|
<para>
|
|
Another problem results from the "run" scripts containing functions that
|
|
might or might not get called.
|
|
The incremental build solution contains code that figures out dependencies
|
|
between shell functions.
|
|
This code is used to prune the "run" scripts down to the minimum set,
|
|
thereby alleviating this problem and making the "run" scripts much more
|
|
readable as a bonus.
|
|
</para>
|
|
|
|
<para>
|
|
So far we have solutions for shell scripts.
|
|
What about Python tasks?
|
|
The same approach applies even though these tasks are more difficult.
|
|
The process needs to figure out what variables a Python function accesses
|
|
and what functions it calls.
|
|
Again, the incremental build solution contains code that first figures out
|
|
the variable and function dependencies, and then creates a checksum for the data
|
|
used as the input to the task.
|
|
</para>
|
|
|
|
<para>
|
|
Like the <filename>WORKDIR</filename> case, situations exist where dependencies
|
|
should be ignored.
|
|
For these cases, you can instruct the build process to ignore a dependency
|
|
by using a line like the following:
|
|
<literallayout class='monospaced'>
|
|
PACKAGE_ARCHS[vardepsexclude] = "MACHINE"
|
|
</literallayout>
|
|
This example ensures that the <filename>PACKAGE_ARCHS</filename> variable does not
|
|
depend on the value of <filename>MACHINE</filename>, even if it does reference it.
|
|
</para>
|
|
|
|
<para>
|
|
Equally, there are cases where we need to add dependencies BitBake is not able to find.
|
|
You can accomplish this by using a line like the following:
|
|
<literallayout class='monospaced'>
|
|
PACKAGE_ARCHS[vardeps] = "MACHINE"
|
|
</literallayout>
|
|
This example explicitly adds the <filename>MACHINE</filename> variable as a
|
|
dependency for <filename>PACKAGE_ARCHS</filename>.
|
|
</para>
|
|
|
|
<para>
|
|
Consider a case with in-line Python, for example, where BitBake is not
|
|
able to figure out dependencies.
|
|
When running in debug mode (i.e. using <filename>-DDD</filename>), BitBake
|
|
produces output when it discovers something for which it cannot figure out
|
|
dependencies.
|
|
The Yocto Project team has currently not managed to cover those dependencies
|
|
in detail and is aware of the need to fix this situation.
|
|
</para>
|
|
|
|
<para>
|
|
Thus far, this section has limited discussion to the direct inputs into a task.
|
|
Information based on direct inputs is referred to as the "basehash" in the
|
|
code.
|
|
However, there is still the question of a task's indirect inputs - the
|
|
things that were already built and present in the
|
|
<ulink url='&YOCTO_DOCS_DEV_URL;#build-directory'>Build Directory</ulink>.
|
|
The checksum (or signature) for a particular task needs to add the hashes
|
|
of all the tasks on which the particular task depends.
|
|
Choosing which dependencies to add is a policy decision.
|
|
However, the effect is to generate a master checksum that combines the basehash
|
|
and the hashes of the task's dependencies.
|
|
</para>
|
|
|
|
<para>
|
|
At the code level, there are a variety of ways both the basehash and the
|
|
dependent task hashes can be influenced.
|
|
Within the BitBake configuration file, we can give BitBake some extra information
|
|
to help it construct the basehash.
|
|
The following statements effectively result in a list of global variable
|
|
dependency excludes - variables never included in any checksum:
|
|
<literallayout class='monospaced'>
|
|
BB_HASHBASE_WHITELIST ?= "TMPDIR FILE PATH PWD BB_TASKHASH BBPATH"
|
|
BB_HASHBASE_WHITELIST += "DL_DIR SSTATE_DIR THISDIR FILESEXTRAPATHS"
|
|
BB_HASHBASE_WHITELIST += "FILE_DIRNAME HOME LOGNAME SHELL TERM USER"
|
|
BB_HASHBASE_WHITELIST += "FILESPATH USERNAME STAGING_DIR_HOST STAGING_DIR_TARGET"
|
|
</literallayout>
|
|
The previous example actually excludes
|
|
<link linkend='var-WORKDIR'><filename>WORKDIR</filename></link>
|
|
since it is actually constructed as a path within
|
|
<link linkend='var-TMPDIR'><filename>TMPDIR</filename></link>, which is on
|
|
the whitelist.
|
|
</para>
|
|
|
|
<para>
|
|
The rules for deciding which hashes of dependent tasks to include through
|
|
dependency chains are more complex and are generally accomplished with a
|
|
Python function.
|
|
The code in <filename>meta/lib/oe/sstatesig.py</filename> shows two examples
|
|
of this and also illustrates how you can insert your own policy into the system
|
|
if so desired.
|
|
This file defines the two basic signature generators <filename>OE-Core</filename>
|
|
uses: "OEBasic" and "OEBasicHash".
|
|
By default, there is a dummy "noop" signature handler enabled in BitBake.
|
|
This means that behavior is unchanged from previous versions.
|
|
<filename>OE-Core</filename> uses the "OEBasicHash" signature handler by default
|
|
through this setting in the <filename>bitbake.conf</filename> file:
|
|
<literallayout class='monospaced'>
|
|
BB_SIGNATURE_HANDLER ?= "OEBasicHash"
|
|
</literallayout>
|
|
The "OEBasicHash" <filename>BB_SIGNATURE_HANDLER</filename> is the same as the
|
|
"OEBasic" version but adds the task hash to the stamp files.
|
|
This results in any
|
|
<ulink url='&YOCTO_DOCS_DEV_URL;#metadata'>Metadata</ulink>
|
|
change that changes the task hash, automatically
|
|
causing the task to be run again.
|
|
This removes the need to bump <link linkend='var-PR'><filename>PR</filename></link>
|
|
values and changes to Metadata automatically ripple across the build.
|
|
</para>
|
|
|
|
<para>
|
|
It is also worth noting that the end result of these signature generators is to
|
|
make some dependency and hash information available to the build.
|
|
This information includes:
|
|
<literallayout class='monospaced'>
|
|
BB_BASEHASH_task-<taskname> - the base hashes for each task in the recipe
|
|
BB_BASEHASH_<filename:taskname> - the base hashes for each dependent task
|
|
BBHASHDEPS_<filename:taskname> - The task dependencies for each task
|
|
BB_TASKHASH - the hash of the currently running task
|
|
</literallayout>
|
|
</para>
|
|
</section>
|
|
|
|
<section id='shared-state'>
|
|
<title>Shared State</title>
|
|
|
|
<para>
|
|
Checksums and dependencies, as discussed in the previous section, solve half the
|
|
problem.
|
|
The other part of the problem is being able to use checksum information during the build
|
|
and being able to reuse or rebuild specific components.
|
|
</para>
|
|
|
|
<para>
|
|
The shared state class (<filename>sstate.bbclass</filename>)
|
|
is a relatively generic implementation of how to "capture" a snapshot of a given task.
|
|
The idea is that the build process does not care about the source of a task's output.
|
|
Output could be freshly built or it could be downloaded and unpacked from
|
|
somewhere - the build process does not need to worry about its source.
|
|
</para>
|
|
|
|
<para>
|
|
There are two types of output, one is just about creating a directory
|
|
in <link linkend='var-WORKDIR'><filename>WORKDIR</filename></link>.
|
|
A good example is the output of either <filename>do_install</filename> or
|
|
<filename>do_package</filename>.
|
|
The other type of output occurs when a set of data is merged into a shared directory
|
|
tree such as the sysroot.
|
|
</para>
|
|
|
|
<para>
|
|
The Yocto Project team has tried to keep the details of the implementation hidden in
|
|
<filename>sstate.bbclass</filename>.
|
|
From a user's perspective, adding shared state wrapping to a task
|
|
is as simple as this <filename>do_deploy</filename> example taken from
|
|
<filename>do_deploy.bbclass</filename>:
|
|
<literallayout class='monospaced'>
|
|
DEPLOYDIR = "${WORKDIR}/deploy-${PN}"
|
|
SSTATETASKS += "do_deploy"
|
|
do_deploy[sstate-name] = "deploy"
|
|
do_deploy[sstate-inputdirs] = "${DEPLOYDIR}"
|
|
do_deploy[sstate-outputdirs] = "${DEPLOY_DIR_IMAGE}"
|
|
|
|
python do_deploy_setscene () {
|
|
sstate_setscene(d)
|
|
}
|
|
addtask do_deploy_setscene
|
|
</literallayout>
|
|
In the example, we add some extra flags to the task, a name field ("deploy"), an
|
|
input directory where the task sends data, and the output
|
|
directory where the data from the task should eventually be copied.
|
|
We also add a <filename>_setscene</filename> variant of the task and add the task
|
|
name to the <filename>SSTATETASKS</filename> list.
|
|
</para>
|
|
|
|
<para>
|
|
If you have a directory whose contents you need to preserve, you can do this with
|
|
a line like the following:
|
|
<literallayout class='monospaced'>
|
|
do_package[sstate-plaindirs] = "${PKGD} ${PKGDEST}"
|
|
</literallayout>
|
|
This method, as well as the following example, also works for multiple directories.
|
|
<literallayout class='monospaced'>
|
|
do_package[sstate-inputdirs] = "${PKGDESTWORK} ${SHLIBSWORKDIR}"
|
|
do_package[sstate-outputdirs] = "${PKGDATA_DIR} ${SHLIBSDIR}"
|
|
do_package[sstate-lockfile] = "${PACKAGELOCK}"
|
|
</literallayout>
|
|
These methods also include the ability to take a lockfile when manipulating
|
|
shared state directory structures since some cases are sensitive to file
|
|
additions or removals.
|
|
</para>
|
|
|
|
<para>
|
|
Behind the scenes, the shared state code works by looking in
|
|
<link linkend='var-SSTATE_DIR'><filename>SSTATE_DIR</filename></link> and
|
|
<link linkend='var-SSTATE_MIRRORS'><filename>SSTATE_MIRRORS</filename></link>
|
|
for shared state files.
|
|
Here is an example:
|
|
<literallayout class='monospaced'>
|
|
SSTATE_MIRRORS ?= "\
|
|
file://.* http://someserver.tld/share/sstate/PATH \n \
|
|
file://.* file:///some/local/dir/sstate/PATH"
|
|
</literallayout>
|
|
<note>
|
|
The shared state directory (<filename>SSTATE_DIR</filename>) is
|
|
organized into two-character subdirectories, where the subdirectory
|
|
names are based on the first two characters of the hash.
|
|
If the shared state directory structure for a mirror has the
|
|
same structure as <filename>SSTATE_DIR</filename>, you must
|
|
specify "PATH" as part of the URI to enable the build system
|
|
to map to the appropriate subdirectory.
|
|
</note>
|
|
</para>
|
|
|
|
<para>
|
|
The shared state package validity can be detected just by looking at the
|
|
filename since the filename contains the task checksum (or signature) as
|
|
described earlier in this section.
|
|
If a valid shared state package is found, the build process downloads it
|
|
and uses it to accelerate the task.
|
|
</para>
|
|
|
|
<para>
|
|
The build processes use the <filename>*_setscene</filename> tasks
|
|
for the task acceleration phase.
|
|
BitBake goes through this phase before the main execution code and tries
|
|
to accelerate any tasks for which it can find shared state packages.
|
|
If a shared state package for a task is available, the shared state
|
|
package is used.
|
|
This means the task and any tasks on which it is dependent are not
|
|
executed.
|
|
</para>
|
|
|
|
<para>
|
|
As a real world example, the aim is when building an IPK-based image,
|
|
only the <filename>do_package_write_ipk</filename> tasks would have their
|
|
shared state packages fetched and extracted.
|
|
Since the sysroot is not used, it would never get extracted.
|
|
This is another reason why a task-based approach is preferred over a
|
|
recipe-based approach, which would have to install the output from every task.
|
|
</para>
|
|
</section>
|
|
|
|
<section id='tips-and-tricks'>
|
|
<title>Tips and Tricks</title>
|
|
|
|
<para>
|
|
The code in the build system that supports incremental builds is not
|
|
simple code.
|
|
This section presents some tips and tricks that help you work around
|
|
issues related to shared state code.
|
|
</para>
|
|
|
|
<section id='debugging'>
|
|
<title>Debugging</title>
|
|
|
|
<para>
|
|
When things go wrong, debugging needs to be straightforward.
|
|
Because of this, the Yocto Project team included strong debugging
|
|
tools:
|
|
<itemizedlist>
|
|
<listitem><para>Whenever a shared state package is written out, so is a
|
|
corresponding <filename>.siginfo</filename> file.
|
|
This practice results in a pickled Python database of all
|
|
the metadata that went into creating the hash for a given shared state
|
|
package.</para></listitem>
|
|
<listitem><para>If you run BitBake with the <filename>--dump-signatures</filename>
|
|
(or <filename>-S</filename>) option, BitBake dumps out
|
|
<filename>.siginfo</filename> files in
|
|
the stamp directory for every task it would have executed instead of
|
|
building the specified target package.</para></listitem>
|
|
<listitem><para>There is a <filename>bitbake-diffsigs</filename> command that
|
|
can process <filename>.siginfo</filename> files.
|
|
If you specify one of these files, BitBake dumps out the dependency
|
|
information in the file.
|
|
If you specify two files, BitBake compares the two files and dumps out
|
|
the differences between the two.
|
|
This more easily helps answer the question of "What
|
|
changed between X and Y?"</para></listitem>
|
|
</itemizedlist>
|
|
</para>
|
|
</section>
|
|
|
|
<section id='invalidating-shared-state'>
|
|
<title>Invalidating Shared State</title>
|
|
|
|
<para>
|
|
The shared state code uses checksums and shared state
|
|
cache to avoid unnecessarily rebuilding tasks.
|
|
As with all schemes, this one has some drawbacks.
|
|
It is possible that you could make implicit changes that are not factored
|
|
into the checksum calculation, but do affect a task's output.
|
|
A good example is perhaps when a tool changes its output.
|
|
Assume that the output of <filename>rpmdeps</filename> needed to change.
|
|
The result of the change should be that all the
|
|
<filename>package</filename>, <filename>package_write_rpm</filename>,
|
|
and <filename>package_deploy-rpm</filename> shared state cache
|
|
items would become invalid.
|
|
But, because this is a change that is external to the code and therefore implicit,
|
|
the associated shared state cache items do not become invalidated.
|
|
In this case, the build process uses the cached items rather than running the
|
|
task again.
|
|
Obviously, these types of implicit changes can cause problems.
|
|
</para>
|
|
|
|
<para>
|
|
To avoid these problems during the build, you need to understand the effects of any
|
|
change you make.
|
|
Note that any changes you make directly to a function automatically are factored into
|
|
the checksum calculation and thus, will invalidate the associated area of sstate cache.
|
|
You need to be aware of any implicit changes that are not obvious changes to the
|
|
code and could affect the output of a given task.
|
|
Once you are aware of such changes, you can take steps to invalidate the cache
|
|
and force the tasks to run.
|
|
The steps to take are as simple as changing function's comments in the source code.
|
|
For example, to invalidate package shared state files, change the comment statements
|
|
of <filename>do_package</filename> or the comments of one of the functions it calls.
|
|
The change is purely cosmetic, but it causes the checksum to be recalculated and
|
|
forces the task to be run again.
|
|
</para>
|
|
|
|
<note>
|
|
For an example of a commit that makes a cosmetic change to invalidate
|
|
a shared state, see this
|
|
<ulink url='&YOCTO_GIT_URL;/cgit.cgi/poky/commit/meta/classes/package.bbclass?id=737f8bbb4f27b4837047cb9b4fbfe01dfde36d54'>commit</ulink>.
|
|
</note>
|
|
</section>
|
|
</section>
|
|
</section>
|
|
|
|
<section id='x32'>
|
|
<title>x32</title>
|
|
|
|
<para>
|
|
x32 is a processor-specific Application Binary Interface (psABI) for x86_64.
|
|
An ABI defines the calling conventions between functions in a processing environment.
|
|
The interface determines what registers are used and what the sizes are for various C data types.
|
|
</para>
|
|
|
|
<para>
|
|
Some processing environments prefer using 32-bit applications even when running
|
|
on Intel 64-bit platforms.
|
|
Consider the i386 psABI, which is a very old 32-bit ABI for Intel 64-bit platforms.
|
|
The i386 psABI does not provide efficient use and access of the Intel 64-bit processor resources,
|
|
leaving the system underutilized.
|
|
Now consider the x86_64 psABI.
|
|
This ABI is newer and uses 64-bits for data sizes and program pointers.
|
|
The extra bits increase the footprint size of the programs, libraries,
|
|
and also increases the memory and file system size requirements.
|
|
Executing under the x32 psABI enables user programs to utilize CPU and system resources
|
|
more efficiently while keeping the memory footprint of the applications low.
|
|
Extra bits are used for registers but not for addressing mechanisms.
|
|
</para>
|
|
|
|
<section id='support'>
|
|
<title>Support</title>
|
|
|
|
<para>
|
|
While the x32 psABI specifications are not fully finalized, this Yocto Project
|
|
release supports current development specifications of x32 psABI.
|
|
As of this release of the Yocto Project, x32 psABI support exists as follows:
|
|
<itemizedlist>
|
|
<listitem><para>You can create packages and images in x32 psABI format on x86_64 architecture targets.
|
|
</para></listitem>
|
|
<listitem><para>You can successfully build many recipes with the x32 toolchain.</para></listitem>
|
|
<listitem><para>You can create and boot <filename>core-image-minimal</filename> and
|
|
<filename>core-image-sato</filename> images.</para></listitem>
|
|
</itemizedlist>
|
|
</para>
|
|
</section>
|
|
|
|
<section id='stabilizing-and-completing-x32'>
|
|
<title>Stabilizing and Completing x32</title>
|
|
|
|
<para>
|
|
As of this Yocto Project release, the x32 psABI kernel and library
|
|
interfaces specifications are not finalized.
|
|
</para>
|
|
|
|
<para>
|
|
Future Plans for the x32 psABI in the Yocto Project include the following:
|
|
<itemizedlist>
|
|
<listitem><para>Enhance and fix the few remaining recipes so they
|
|
work with and support x32 toolchains.</para></listitem>
|
|
<listitem><para>Enhance RPM Package Manager (RPM) support for x32 binaries.</para></listitem>
|
|
<listitem><para>Support larger images.</para></listitem>
|
|
</itemizedlist>
|
|
</para>
|
|
</section>
|
|
|
|
<section id='using-x32-right-now'>
|
|
<title>Using x32 Right Now</title>
|
|
|
|
<para>
|
|
Follow these steps to use the x32 spABI:
|
|
<itemizedlist>
|
|
<listitem><para>Enable the x32 psABI tuning file for <filename>x86_64</filename>
|
|
machines by editing the <filename>conf/local.conf</filename> like this:
|
|
<literallayout class='monospaced'>
|
|
MACHINE = "qemux86-64"
|
|
DEFAULTTUNE = "x86-64-x32"
|
|
baselib = "${@d.getVar('BASE_LIB_tune-' + (d.getVar('DEFAULTTUNE', True) \
|
|
or 'INVALID'), True) or 'lib'}"
|
|
#MACHINE = "atom-pc"
|
|
#DEFAULTTUNE = "core2-64-x32"
|
|
</literallayout></para></listitem>
|
|
<listitem><para>As usual, use BitBake to build an image that supports the x32 psABI.
|
|
Here is an example:
|
|
<literallayout class='monospaced'>
|
|
$ bitbake core-image-sato
|
|
</literallayout></para></listitem>
|
|
<listitem><para>As usual, run your image using QEMU:
|
|
<literallayout class='monospaced'>
|
|
$ runqemu qemux86-64 core-image-sato
|
|
</literallayout></para></listitem>
|
|
</itemizedlist>
|
|
</para>
|
|
</section>
|
|
</section>
|
|
|
|
<section id="wayland">
|
|
<title>Wayland</title>
|
|
|
|
<para>
|
|
<ulink url='http://en.wikipedia.org/wiki/Wayland_(display_server_protocol)#Weston'>Wayland</ulink>
|
|
is a computer display server protocol that when implemented
|
|
provides a method for compositing window managers to communicate
|
|
directly with applications and video hardware and expects them to
|
|
communicate with input hardware using other libraries.
|
|
Using Wayland with supporting targets can result in better control
|
|
over graphics frame rendering than an application might otherwise
|
|
achieve.
|
|
</para>
|
|
|
|
<para>
|
|
The Yocto Project provides the Wayland protocol libraries and the
|
|
reference Weston compositor as part of it release.
|
|
This section describes what you need to do to implement Wayland and
|
|
use the compositor when building an image for a supporting target.
|
|
</para>
|
|
|
|
<section id="wayland-support">
|
|
<title>Support</title>
|
|
|
|
<para>
|
|
The Wayland protocol libraries and the reference Weston compositor
|
|
ship as integrated packages in the <filename>meta</filename> layer
|
|
of the
|
|
<ulink url='&YOCTO_DOCS_DEV_URL;#source-directory'>Source Directory</ulink>.
|
|
Specifically, you can find the recipes that build both Wayland
|
|
and Weston at <filename>meta/recipes-graphics/wayland</filename>.
|
|
</para>
|
|
|
|
<para>
|
|
You can build both the Wayland and Weston packages for use only
|
|
with targets that accept the
|
|
<ulink url='http://dri.freedesktop.org/wiki/'>Mesa 3D and Direct Rendering Infrastructure</ulink>,
|
|
which is also known as Mesa DRI.
|
|
This implies that you cannot build and use the packages if your
|
|
target uses, for example, the
|
|
<trademark class='registered'>Intel</trademark> Embedded Media and
|
|
Graphics Driver (<trademark class='registered'>Intel</trademark>
|
|
EMGD) that overrides Mesa DRI.
|
|
</para>
|
|
|
|
<note>
|
|
Due to lack of EGL support, Weston 1.0.3 will not run directly on
|
|
the emulated QEMU hardware.
|
|
However, this version of Weston will run under X emulation without
|
|
issues.
|
|
</note>
|
|
</section>
|
|
|
|
<section id="enabling-wayland-in-an-image">
|
|
<title>Enabling Wayland in an Image</title>
|
|
|
|
<para>
|
|
To enable Wayland, you need to enable it to be built and enable
|
|
it to be included in the image.
|
|
</para>
|
|
|
|
<section id="enable-building">
|
|
<title>Building</title>
|
|
|
|
<para>
|
|
To cause Mesa to build the <filename>wayland-egl</filename>
|
|
platform and Weston to build Wayland with Kernel Mode
|
|
Setting
|
|
(<ulink url='https://wiki.archlinux.org/index.php/Kernel_Mode_Setting'>KMS</ulink>)
|
|
support, include the "wayland" flag in the
|
|
<link linkend="var-DISTRO_FEATURES"><filename>DISTRO_FEATURES</filename></link>
|
|
statement in your <filename>local.conf</filename> file:
|
|
<literallayout class='monospaced'>
|
|
DISTRO_FEATURES_append = " wayland"
|
|
</literallayout>
|
|
</para>
|
|
|
|
<note>
|
|
If X11 has been enabled elsewhere, Weston will build Wayland
|
|
with X11 support
|
|
</note>
|
|
</section>
|
|
|
|
<section id="enable-installation-in-an-image">
|
|
<title>Installing</title>
|
|
|
|
<para>
|
|
To install the Wayland feature into an image, you must
|
|
include the following
|
|
<link linkend='var-CORE_IMAGE_EXTRA_INSTALL'><filename>CORE_IMAGE_EXTRA_INSTALL</filename></link>
|
|
statement in your <filename>local.conf</filename> file:
|
|
<literallayout class='monospaced'>
|
|
CORE_IMAGE_EXTRA_INSTALL += "wayland weston"
|
|
</literallayout>
|
|
</para>
|
|
</section>
|
|
</section>
|
|
|
|
<section id="running-weston">
|
|
<title>Running Weston</title>
|
|
|
|
<para>
|
|
To run Weston inside X11, enabling it as described earlier and
|
|
building a Sato image is sufficient.
|
|
If you are running your image under Sato, a Weston Launcher appears
|
|
in the "Utility" category.
|
|
</para>
|
|
|
|
<para>
|
|
Alternatively, you can run Weston through the command-line
|
|
interpretor (CLI), which is better suited for development work.
|
|
To run Weston under the CLI you need to do the following after
|
|
your image is built:
|
|
<orderedlist>
|
|
<listitem><para>Run these commands to export
|
|
<filename>XDG_RUNTIME_DIR</filename>:
|
|
<literallayout class='monospaced'>
|
|
mkdir -p /tmp/$USER-weston
|
|
chmod 0700 /tmp/$USER-weston
|
|
export XDG_RUNTIME_DIR=/tmp/$USER=weston
|
|
</literallayout></para></listitem>
|
|
<listitem><para>Launch Weston in the shell:
|
|
<literallayout class='monospaced'>
|
|
weston
|
|
</literallayout></para></listitem>
|
|
</orderedlist>
|
|
</para>
|
|
</section>
|
|
</section>
|
|
|
|
<section id="licenses">
|
|
<title>Licenses</title>
|
|
|
|
<para>
|
|
This section describes the mechanism by which the OpenEmbedded build system
|
|
tracks changes to licensing text.
|
|
The section also describes how to enable commercially licensed recipes,
|
|
which by default are disabled.
|
|
</para>
|
|
|
|
<para>
|
|
For information that can help you maintain compliance with various open
|
|
source licensing during the lifecycle of the product, see the
|
|
"<ulink url='&YOCTO_DOCS_DEV_URL;#maintaining-open-source-license-compliance-during-your-products-lifecycle'>Maintaining Open Source License Compliance During Your Project's Lifecycle</ulink>" section
|
|
in the Yocto Project Development Manual.
|
|
</para>
|
|
|
|
<section id="usingpoky-configuring-LIC_FILES_CHKSUM">
|
|
<title>Tracking License Changes</title>
|
|
|
|
<para>
|
|
The license of an upstream project might change in the future.
|
|
In order to prevent these changes going unnoticed, the
|
|
<filename><link linkend='var-LIC_FILES_CHKSUM'>LIC_FILES_CHKSUM</link></filename>
|
|
variable tracks changes to the license text. The checksums are validated at the end of the
|
|
configure step, and if the checksums do not match, the build will fail.
|
|
</para>
|
|
|
|
<section id="usingpoky-specifying-LIC_FILES_CHKSUM">
|
|
<title>Specifying the <filename>LIC_FILES_CHKSUM</filename> Variable</title>
|
|
|
|
<para>
|
|
The <filename>LIC_FILES_CHKSUM</filename>
|
|
variable contains checksums of the license text in the source code for the recipe.
|
|
Following is an example of how to specify <filename>LIC_FILES_CHKSUM</filename>:
|
|
<literallayout class='monospaced'>
|
|
LIC_FILES_CHKSUM = "file://COPYING;md5=xxxx \
|
|
file://licfile1.txt;beginline=5;endline=29;md5=yyyy \
|
|
file://licfile2.txt;endline=50;md5=zzzz \
|
|
..."
|
|
</literallayout>
|
|
</para>
|
|
|
|
<para>
|
|
The build system uses the
|
|
<filename><link linkend='var-S'>S</link></filename> variable as the
|
|
default directory used when searching files listed in
|
|
<filename>LIC_FILES_CHKSUM</filename>.
|
|
The previous example employs the default directory.
|
|
</para>
|
|
|
|
<para>
|
|
You can also use relative paths as shown in the following example:
|
|
<literallayout class='monospaced'>
|
|
LIC_FILES_CHKSUM = "file://src/ls.c;beginline=5;endline=16;\
|
|
md5=bb14ed3c4cda583abc85401304b5cd4e"
|
|
LIC_FILES_CHKSUM = "file://../license.html;md5=5c94767cedb5d6987c902ac850ded2c6"
|
|
</literallayout>
|
|
</para>
|
|
|
|
<para>
|
|
In this example, the first line locates a file in
|
|
<filename>${S}/src/ls.c</filename>.
|
|
The second line refers to a file in
|
|
<filename><link linkend='var-WORKDIR'>WORKDIR</link></filename>, which is the parent
|
|
of <filename><link linkend='var-S'>S</link></filename>.
|
|
</para>
|
|
<para>
|
|
Note that <filename>LIC_FILES_CHKSUM</filename> variable is
|
|
mandatory for all recipes, unless the
|
|
<filename>LICENSE</filename> variable is set to "CLOSED".
|
|
</para>
|
|
</section>
|
|
|
|
<section id="usingpoky-LIC_FILES_CHKSUM-explanation-of-syntax">
|
|
<title>Explanation of Syntax</title>
|
|
<para>
|
|
As mentioned in the previous section, the
|
|
<filename>LIC_FILES_CHKSUM</filename> variable lists all the
|
|
important files that contain the license text for the source code.
|
|
It is possible to specify a checksum for an entire file, or a specific section of a
|
|
file (specified by beginning and ending line numbers with the "beginline" and "endline"
|
|
parameters, respectively).
|
|
The latter is useful for source files with a license notice header,
|
|
README documents, and so forth.
|
|
If you do not use the "beginline" parameter, then it is assumed that the text begins on the
|
|
first line of the file.
|
|
Similarly, if you do not use the "endline" parameter, it is assumed that the license text
|
|
ends with the last line of the file.
|
|
</para>
|
|
|
|
<para>
|
|
The "md5" parameter stores the md5 checksum of the license text.
|
|
If the license text changes in any way as compared to this parameter
|
|
then a mismatch occurs.
|
|
This mismatch triggers a build failure and notifies the developer.
|
|
Notification allows the developer to review and address the license text changes.
|
|
Also note that if a mismatch occurs during the build, the correct md5
|
|
checksum is placed in the build log and can be easily copied to the recipe.
|
|
</para>
|
|
|
|
<para>
|
|
There is no limit to how many files you can specify using the
|
|
<filename>LIC_FILES_CHKSUM</filename> variable.
|
|
Generally, however, every project requires a few specifications for license tracking.
|
|
Many projects have a "COPYING" file that stores the license information for all the source
|
|
code files.
|
|
This practice allows you to just track the "COPYING" file as long as it is kept up to date.
|
|
</para>
|
|
|
|
<tip>
|
|
If you specify an empty or invalid "md5" parameter, BitBake returns an md5 mis-match
|
|
error and displays the correct "md5" parameter value during the build.
|
|
The correct parameter is also captured in the build log.
|
|
</tip>
|
|
|
|
<tip>
|
|
If the whole file contains only license text, you do not need to use the "beginline" and
|
|
"endline" parameters.
|
|
</tip>
|
|
</section>
|
|
</section>
|
|
|
|
<section id="enabling-commercially-licensed-recipes">
|
|
<title>Enabling Commercially Licensed Recipes</title>
|
|
|
|
<para>
|
|
By default, the OpenEmbedded build system disables
|
|
components that have commercial or other special licensing
|
|
requirements.
|
|
Such requirements are defined on a
|
|
recipe-by-recipe basis through the <filename>LICENSE_FLAGS</filename> variable
|
|
definition in the affected recipe.
|
|
For instance, the
|
|
<filename>$HOME/poky/meta/recipes-multimedia/gstreamer/gst-plugins-ugly</filename>
|
|
recipe contains the following statement:
|
|
<literallayout class='monospaced'>
|
|
LICENSE_FLAGS = "commercial"
|
|
</literallayout>
|
|
Here is a slightly more complicated example that contains both an
|
|
explicit recipe name and version (after variable expansion):
|
|
<literallayout class='monospaced'>
|
|
LICENSE_FLAGS = "license_${PN}_${PV}"
|
|
</literallayout>
|
|
In order for a component restricted by a <filename>LICENSE_FLAGS</filename>
|
|
definition to be enabled and included in an image, it
|
|
needs to have a matching entry in the global
|
|
<filename>LICENSE_FLAGS_WHITELIST</filename> variable, which is a variable
|
|
typically defined in your <filename>local.conf</filename> file.
|
|
For example, to enable
|
|
the <filename>$HOME/poky/meta/recipes-multimedia/gstreamer/gst-plugins-ugly</filename>
|
|
package, you could add either the string
|
|
"commercial_gst-plugins-ugly" or the more general string
|
|
"commercial" to <filename>LICENSE_FLAGS_WHITELIST</filename>.
|
|
See the
|
|
"<link linkend='license-flag-matching'>License Flag Matching</link>" section
|
|
for a full explanation of how <filename>LICENSE_FLAGS</filename> matching works.
|
|
Here is the example:
|
|
<literallayout class='monospaced'>
|
|
LICENSE_FLAGS_WHITELIST = "commercial_gst-plugins-ugly"
|
|
</literallayout>
|
|
Likewise, to additionally enable the package built from the recipe containing
|
|
<filename>LICENSE_FLAGS = "license_${PN}_${PV}"</filename>, and assuming
|
|
that the actual recipe name was <filename>emgd_1.10.bb</filename>,
|
|
the following string would enable that package as well as
|
|
the original <filename>gst-plugins-ugly</filename> package:
|
|
<literallayout class='monospaced'>
|
|
LICENSE_FLAGS_WHITELIST = "commercial_gst-plugins-ugly license_emgd_1.10"
|
|
</literallayout>
|
|
As a convenience, you do not need to specify the complete license string
|
|
in the whitelist for every package.
|
|
you can use an abbreviated form, which consists
|
|
of just the first portion or portions of the license string before
|
|
the initial underscore character or characters.
|
|
A partial string will match
|
|
any license that contains the given string as the first
|
|
portion of its license.
|
|
For example, the following
|
|
whitelist string will also match both of the packages
|
|
previously mentioned as well as any other packages that have
|
|
licenses starting with "commercial" or "license".
|
|
<literallayout class='monospaced'>
|
|
LICENSE_FLAGS_WHITELIST = "commercial license"
|
|
</literallayout>
|
|
</para>
|
|
|
|
<section id="license-flag-matching">
|
|
<title>License Flag Matching</title>
|
|
|
|
<para>
|
|
License flag matching allows you to control what recipes the
|
|
OpenEmbedded build system includes in the build.
|
|
Fundamentally, the build system attempts to match
|
|
<filename>LICENSE_FLAG</filename> strings found in
|
|
recipes against <filename>LICENSE_FLAGS_WHITELIST</filename>
|
|
strings found in the whitelist.
|
|
A match, causes the build system to include a recipe in the
|
|
build, while failure to find a match causes the build system to
|
|
exclude a recipe.
|
|
</para>
|
|
|
|
<para>
|
|
In general, license flag matching is simple.
|
|
However, understanding some concepts will help you
|
|
correctly and effectively use matching.
|
|
</para>
|
|
|
|
<para>
|
|
Before a flag
|
|
defined by a particular recipe is tested against the
|
|
contents of the whitelist, the expanded string
|
|
<filename>_${PN}</filename> is appended to the flag.
|
|
This expansion makes each <filename>LICENSE_FLAGS</filename>
|
|
value recipe-specific.
|
|
After expansion, the string is then matched against the
|
|
whitelist.
|
|
Thus, specifying
|
|
<filename>LICENSE_FLAGS = "commercial"</filename>
|
|
in recipe "foo", for example, results in the string
|
|
<filename>"commercial_foo"</filename>.
|
|
And, to create a match, that string must appear in the
|
|
whitelist.
|
|
</para>
|
|
|
|
<para>
|
|
Judicious use of the <filename>LICENSE_FLAGS</filename>
|
|
strings and the contents of the
|
|
<filename>LICENSE_FLAGS_WHITELIST</filename> variable
|
|
allows you a lot of flexibility for including or excluding
|
|
recipes based on licensing.
|
|
For example, you can broaden the matching capabilities by
|
|
using license flags string subsets in the whitelist.
|
|
<note>When using a string subset, be sure to use the part of
|
|
the expanded string that precedes the appended underscore
|
|
character (e.g. <filename>usethispart_1.3</filename>,
|
|
<filename>usethispart_1.4</filename>, and so forth).
|
|
</note>
|
|
For example, simply specifying the string "commercial" in
|
|
the whitelist matches any expanded
|
|
<filename>LICENSE_FLAGS</filename> definition that starts with
|
|
the string "commercial" such as "commercial_foo" and
|
|
"commercial_bar", which are the strings the build system
|
|
automatically generates for hypothetical recipes named
|
|
"foo" and "bar" assuming those recipes simply specify the
|
|
following:
|
|
<literallayout class='monospaced'>
|
|
LICENSE_FLAGS = "commercial"
|
|
</literallayout>
|
|
Thus, you can choose to exhaustively
|
|
enumerate each license flag in the whitelist and
|
|
allow only specific recipes into the image, or
|
|
you can use a string subset that causes a broader range of
|
|
matches to allow a range of recipes into the image.
|
|
</para>
|
|
|
|
<para>
|
|
This scheme works even if the
|
|
<filename>LICENSE_FLAG</filename> string already
|
|
has <filename>_${PN}</filename> appended.
|
|
For example, the build system turns the license flag
|
|
"commercial_1.2_foo" into "commercial_1.2_foo_foo" and would
|
|
match both the general "commercial" and the specific
|
|
"commercial_1.2_foo" strings found in the whitelist, as
|
|
expected.
|
|
</para>
|
|
|
|
<para>
|
|
Here are some other scenarios:
|
|
<itemizedlist>
|
|
<listitem><para>You can specify a versioned string in the
|
|
recipe such as "commercial_foo_1.2" in a "foo" recipe.
|
|
The build system expands this string to
|
|
"commercial_foo_1.2_foo".
|
|
Combine this license flag with a whitelist that has
|
|
the string "commercial" and you match the flag along
|
|
with any other flag that starts with the string
|
|
"commercial".</para></listitem>
|
|
<listitem><para>Under the same circumstances, you can
|
|
use "commercial_foo" in the whitelist and the
|
|
build system not only matches "commercial_foo_1.2" but
|
|
also matches any license flag with the string
|
|
"commercial_foo", regardless of the version.
|
|
</para></listitem>
|
|
<listitem><para>You can be very specific and use both the
|
|
package and version parts in the whitelist (e.g.
|
|
"commercial_foo_1.2") to specifically match a
|
|
versioned recipe.</para></listitem>
|
|
</itemizedlist>
|
|
</para>
|
|
</section>
|
|
|
|
<section id="other-variables-related-to-commercial-licenses">
|
|
<title>Other Variables Related to Commercial Licenses</title>
|
|
|
|
<para>
|
|
Other helpful variables related to commercial
|
|
license handling exist and are defined in the
|
|
<filename>$HOME/poky/meta/conf/distro/include/default-distrovars.inc</filename> file:
|
|
<literallayout class='monospaced'>
|
|
COMMERCIAL_AUDIO_PLUGINS ?= ""
|
|
COMMERCIAL_VIDEO_PLUGINS ?= ""
|
|
COMMERCIAL_QT = ""
|
|
</literallayout>
|
|
If you want to enable these components, you can do so by making sure you have
|
|
statements similar to the following
|
|
in your <filename>local.conf</filename> configuration file:
|
|
<literallayout class='monospaced'>
|
|
COMMERCIAL_AUDIO_PLUGINS = "gst-plugins-ugly-mad \
|
|
gst-plugins-ugly-mpegaudioparse"
|
|
COMMERCIAL_VIDEO_PLUGINS = "gst-plugins-ugly-mpeg2dec \
|
|
gst-plugins-ugly-mpegstream gst-plugins-bad-mpegvideoparse"
|
|
COMMERCIAL_QT ?= "qmmp"
|
|
LICENSE_FLAGS_WHITELIST = "commercial_gst-plugins-ugly commercial_gst-plugins-bad commercial_qmmp"
|
|
</literallayout>
|
|
Of course, you could also create a matching whitelist
|
|
for those components using the more general "commercial"
|
|
in the whitelist, but that would also enable all the
|
|
other packages with <filename>LICENSE_FLAGS</filename> containing
|
|
"commercial", which you may or may not want:
|
|
<literallayout class='monospaced'>
|
|
LICENSE_FLAGS_WHITELIST = "commercial"
|
|
</literallayout>
|
|
</para>
|
|
|
|
<para>
|
|
Specifying audio and video plug-ins as part of the
|
|
<filename>COMMERCIAL_AUDIO_PLUGINS</filename> and
|
|
<filename>COMMERCIAL_VIDEO_PLUGINS</filename> statements
|
|
or commercial Qt components as part of
|
|
the <filename>COMMERCIAL_QT</filename> statement (along
|
|
with the enabling <filename>LICENSE_FLAGS_WHITELIST</filename>) includes the
|
|
plug-ins or components into built images, thus adding
|
|
support for media formats or components.
|
|
</para>
|
|
</section>
|
|
</section>
|
|
</section>
|
|
</chapter>
|
|
<!--
|
|
vim: expandtab tw=80 ts=4
|
|
-->
|