230 lines
11 KiB
XML
230 lines
11 KiB
XML
<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
|
|
"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd">
|
|
<chapter id='technical-details'>
|
|
<title>Technical Details</title>
|
|
|
|
<para>
|
|
This chapter provides technical details for various parts of the Yocto Project.
|
|
Currently, topics include Yocto Project components and shared state (sstate) cache.
|
|
</para>
|
|
|
|
<section id='usingpoky-components'>
|
|
<title>Yocto Project Components</title>
|
|
|
|
<para>
|
|
The BitBake task executor together with various types of configuration files form the
|
|
Yocto Project core.
|
|
This section overviews the BitBake task executor and the
|
|
configuration files by describing what they are used for and how they interact.
|
|
</para>
|
|
|
|
<para>
|
|
BitBake handles the parsing and execution of the data files.
|
|
The data itself is of various types:
|
|
<itemizedlist>
|
|
<listitem><para><emphasis>Recipes:</emphasis> Provides details about particular
|
|
pieces of software</para></listitem>
|
|
<listitem><para><emphasis>Class Data:</emphasis> An abstraction of common build
|
|
information (e.g. how to build a Linux kernel).</para></listitem>
|
|
<listitem><para><emphasis>Configuration Data:</emphasis> Defines machine-specific settings,
|
|
policy decisions, etc.
|
|
Configuration data acts as the glue to bind everything together.</para></listitem>
|
|
</itemizedlist>
|
|
For more information on data, see the
|
|
<ulink url='http://www.yoctoproject.org/docs/latest/dev-manual/dev-manual.html#yocto-project-terms'>
|
|
Yocto Project Terms</ulink> section in
|
|
<ulink url='http://www.yoctoproject.org/docs/latest/dev-manual/dev-manual.html'>
|
|
The Yocto Project Development Manual</ulink>.
|
|
</para>
|
|
|
|
<para>
|
|
BitBake knows how to combine multiple data sources together and refers to each data source
|
|
as a "<link linkend='usingpoky-changes-layers'>layer</link>".
|
|
</para>
|
|
|
|
<para>
|
|
Following are some brief details on these core components.
|
|
For more detailed information on these components see the
|
|
<link linkend='ref-structure'>'Reference: Directory Structure'</link>
|
|
appendix.
|
|
</para>
|
|
|
|
<section id='usingpoky-components-bitbake'>
|
|
<title>BitBake</title>
|
|
|
|
<para>
|
|
BitBake is the tool at the heart of the Yocto Project and is responsible
|
|
for parsing the metadata, generating a list of tasks from it,
|
|
and then executing those tasks.
|
|
To see a list of the options BitBake supports, use the following help command:
|
|
<literallayout class='monospaced'>
|
|
$ bitbake --help
|
|
</literallayout>
|
|
</para>
|
|
|
|
<para>
|
|
The most common usage for BitBake is <filename>bitbake <packagename></filename>, where
|
|
<filename>packagename</filename> is the name of the package you want to build
|
|
(referred to as the "target" in this manual).
|
|
The target often equates to the first part of a <filename>.bb</filename> filename.
|
|
So, to run the <filename>matchbox-desktop_1.2.3.bb</filename> file, you
|
|
might type the following:
|
|
<literallayout class='monospaced'>
|
|
$ bitbake matchbox-desktop
|
|
</literallayout>
|
|
Several different versions of <filename>matchbox-desktop</filename> might exist.
|
|
BitBake chooses the one selected by the distribution configuration.
|
|
You can get more details about how BitBake chooses between different
|
|
target versions and providers in the
|
|
<link linkend='ref-bitbake-providers'>Preferences and Providers</link> section.
|
|
</para>
|
|
|
|
<para>
|
|
BitBake also tries to execute any dependent tasks first.
|
|
So for example, before building <filename>matchbox-desktop</filename>, BitBake
|
|
would build a cross compiler and <filename>eglibc</filename> if they had not already
|
|
been built.
|
|
<note>This release of the Yocto Project does not support the <filename>glibc</filename>
|
|
GNU version of the Unix standard C library. By default, the Yocto Project builds with
|
|
<filename>eglibc</filename>.</note>
|
|
</para>
|
|
|
|
<para>
|
|
A useful BitBake option to consider is the <filename>-k</filename> or
|
|
<filename>--continue</filename> option.
|
|
This option instructs BitBake to try and continue processing the job as much
|
|
as possible even after encountering an error.
|
|
When an error occurs, the target that
|
|
failed and those that depend on it cannot be remade.
|
|
However, when you use this option other dependencies can still be processed.
|
|
</para>
|
|
</section>
|
|
|
|
<section id='usingpoky-components-metadata'>
|
|
<title>Metadata (Recipes)</title>
|
|
|
|
<para>
|
|
The <filename>.bb</filename> files are usually referred to as "recipes."
|
|
In general, a recipe contains information about a single piece of software.
|
|
The information includes the location from which to download the source patches
|
|
(if any are needed), which special configuration options to apply,
|
|
how to compile the source files, and how to package the compiled output.
|
|
</para>
|
|
|
|
<para>
|
|
The term "package" can also be used to describe recipes.
|
|
However, since the same word is used for the packaged output from the Yocto
|
|
Project (i.e. <filename>.ipk</filename> or <filename>.deb</filename> files),
|
|
this document avoids using the term "package" when refering to recipes.
|
|
</para>
|
|
</section>
|
|
|
|
<section id='usingpoky-components-classes'>
|
|
<title>Classes</title>
|
|
|
|
<para>
|
|
Class files (<filename>.bbclass</filename>) contain information that is useful to share
|
|
between metadata files.
|
|
An example is the Autotools class, which contains
|
|
common settings for any application that Autotools uses.
|
|
The <link linkend='ref-classes'>Reference: Classes</link> appendix provides details
|
|
about common classes and how to use them.
|
|
</para>
|
|
</section>
|
|
|
|
<section id='usingpoky-components-configuration'>
|
|
<title>Configuration</title>
|
|
|
|
<para>
|
|
The configuration files (<filename>.conf</filename>) define various configuration variables
|
|
that govern the Yocto Project build process.
|
|
These files fall into several areas that define machine configuration options,
|
|
distribution configuration options, compiler tuning options, general common configuration
|
|
options and user configuration options (<filename>local.conf</filename>, which is found
|
|
in the Yocto Project files build directory).
|
|
</para>
|
|
</section>
|
|
</section>
|
|
|
|
<section id="shared-state-cache">
|
|
<title>Shared State Cache</title>
|
|
|
|
<para>
|
|
By design, the Yocto Project builds everything from scratch unless it can determine that
|
|
a given task's inputs have not changed.
|
|
While building from scratch ensures that everything is current, it does also
|
|
mean that a lot of time could be spent rebuiding things that don't necessarily need built.
|
|
</para>
|
|
|
|
<para>
|
|
The Yocto Project build process uses a shared state caching scheme to avoid having to
|
|
rebuild software when it is not necessary.
|
|
Because the build time for a Yocto image can be significant, it is helpful to try and
|
|
determine what really needs built and what can be skipped given a particular project's
|
|
build process.
|
|
</para>
|
|
|
|
<para>
|
|
The scheme that the Yocto Project uses involves checksum generation and comparison for
|
|
a task's inputs.
|
|
The scheme also employs an area of memory called the shared state cache that is
|
|
pointed to by the <filename>SSTATE_DIR</filename> variable.
|
|
This area contains task output generated from a previous build.
|
|
If a given task's checksum matches the checksum of a previous build for the same
|
|
task, the build process uses the state of the cache rather than rerunning that
|
|
task.
|
|
</para>
|
|
|
|
<para>
|
|
The previous paragraph is a simplistic explanation of how the build process
|
|
uses checksums and shared state memory cache to avoide building tasks that
|
|
don't need built.
|
|
If you want a bit more explanation on the topic,
|
|
see "<ulink url='https://lists.yoctoproject.org/pipermail/yocto/2011-March/003366.html'>Shared
|
|
State - What does it mean and why should I care?</ulink>" from the Yocto
|
|
Project discussion archives.
|
|
</para>
|
|
|
|
<para>
|
|
As with all schemes, this one has some drawbacks.
|
|
It is possible that you could make implicit changes that are not factored into the checksum
|
|
calculation, but do affect a task's output.
|
|
A good example is perhaps when a tool changes its output.
|
|
Let's say that the output of <filename>rpmdeps</filename> needed to change.
|
|
The result of the change should be that all the "package", "package_write_rpm",
|
|
and "package_deploy-rpm" sstate-cache items would become invalid.
|
|
But, because this is a change that is external to the code and therefore implicit,
|
|
the associated sstate-cache items do not become invalidated.
|
|
In this case, the build process would use the cache items rather than running the
|
|
task again.
|
|
Obviously, these types of implicit changes can cause problems.
|
|
</para>
|
|
|
|
<para>
|
|
To avoid these problems during the build, you need to understand the effects of any
|
|
change you make.
|
|
Note that any changes you make directly to a function automatically are factored into
|
|
the checksum calculation and thus, will invalidate the associated area of sstate cache.
|
|
You need to be aware of any implicit changes that are not obvious changes to the
|
|
code and could affect the output of a given task.
|
|
Once you are aware of such a change, you can take steps to invalidate the cache
|
|
and force the task to run.
|
|
The step to take is as simple as changing a function's comments in the source code.
|
|
For example, to invalidate package sstate files, change the comment statments
|
|
of <filename>do_package</filename> or one of the functions it calls.
|
|
The change is purely cosmetic, but it causes the checksum to be recalculated and
|
|
forces the task to be run again.
|
|
</para>
|
|
|
|
<note>
|
|
For an example of a commit that makes a cosmetic change to invalidate an sstate,
|
|
see this
|
|
<ulink url='http://git.yoctoproject.org/cgit.cgi/poky/commit/meta/classes/package.bbclass?id=737f8bbb4f27b4837047cb9b4fbfe01dfde36d54'>commit</ulink>.
|
|
</note>
|
|
</section>
|
|
|
|
</chapter>
|
|
<!--
|
|
vim: expandtab tw=80 ts=4
|
|
-->
|