Collections and overlays are deprecated in favour of layers, a much cleaner and non-intrusive way of extending Poky with custom recipes, classes and configuration. This patch updates the Extending Poky section of the handbook to show how to use layers to tailor Poky while dropping the existing section on using collections. Signed-off-by: Joshua Lock <josh@linux.intel.com>
342 lines
18 KiB
XML
342 lines
18 KiB
XML
<!DOCTYPE appendix PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
|
|
"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd">
|
|
|
|
<appendix id='ref-bitbake'>
|
|
|
|
<title>Reference: Bitbake</title>
|
|
|
|
<para>
|
|
Bitbake a program written in Python which interprets the metadata
|
|
that makes up Poky. At some point, people wonder what actually happens
|
|
when you type <command>bitbake poky-image-sato</command>. This section
|
|
aims to give an overview of what happens behind the scenes from a
|
|
BitBake perspective.
|
|
</para>
|
|
|
|
<para>
|
|
It is worth noting that bitbake aims to be a generic "task" executor
|
|
capable of handling complex dependency relationships. As such it has no
|
|
real knowledge of what the tasks its executing actually do. It just
|
|
considers a list of tasks with dependencies and handles metadata
|
|
consisting of variables in a certain format which get passed to the
|
|
tasks.
|
|
</para>
|
|
|
|
<section id='ref-bitbake-parsing'>
|
|
<title>Parsing</title>
|
|
|
|
<para>
|
|
The first thing BitBake does is work out its configuration by
|
|
looking for a file called <filename>bitbake.conf</filename>.
|
|
Bitbake searches through the <varname>BBPATH</varname> environment
|
|
variable looking for a <filename class="directory">conf/</filename>
|
|
directory containing a <filename>bitbake.conf</filename> file and
|
|
adds the first <filename>bitbake.conf</filename> file found in
|
|
<varname>BBPATH</varname> (similar to the PATH environment variable).
|
|
For Poky, <filename>bitbake.conf</filename> is found in <filename
|
|
class="directory">meta/conf/</filename>.
|
|
</para>
|
|
|
|
<para>
|
|
In Poky, <filename>bitbake.conf</filename> lists other configuration
|
|
files to include from a <filename class="directory">conf/</filename>
|
|
directory below the directories listed in <varname>BBPATH</varname>.
|
|
In general the most important configuration file from a user's perspective
|
|
is <filename>local.conf</filename>, which contains a users customized
|
|
settings for Poky. Other notable configuration files are the distribution
|
|
configuration file (set by the <glossterm><link linkend='var-DISTRO'>
|
|
DISTRO</link></glossterm> variable) and the machine configuration file
|
|
(set by the <glossterm><link linkend='var-MACHINE'>MACHINE</link>
|
|
</glossterm> variable). The <glossterm><link linkend='var-DISTRO'>
|
|
DISTRO</link></glossterm> and <glossterm><link linkend='var-MACHINE'>
|
|
MACHINE</link></glossterm> environment variables are both usually set in
|
|
the <filename>local.conf</filename> file. Valid distribution
|
|
configuration files are available in the <filename class="directory">
|
|
meta/conf/distro/</filename> directory and valid machine configuration
|
|
files in the <filename class="directory">meta/conf/machine/</filename>
|
|
directory. Within the <filename class="directory">
|
|
meta/conf/machine/include/</filename> directory are various <filename>
|
|
tune-*.inc</filename> configuration files which provide common
|
|
"tuning" settings specific to and shared between particular
|
|
architectures and machines.
|
|
</para>
|
|
|
|
<para>
|
|
After the parsing of the configuration files some standard classes
|
|
are included. In particular, <filename>base.bbclass</filename> is
|
|
always included, as will any other classes
|
|
specified in the configuration using the <glossterm><link
|
|
linkend='var-INHERIT'>INHERIT</link></glossterm>
|
|
variable. Class files are searched for in a classes subdirectory
|
|
under the paths in <varname>BBPATH</varname> in the same way as
|
|
configuration files.
|
|
</para>
|
|
|
|
<para>
|
|
After the parsing of the configuration files is complete, the
|
|
variable <glossterm><link linkend='var-BBFILES'>BBFILES</link></glossterm>
|
|
is set, usually in
|
|
<filename>local.conf</filename>, and defines the list of places to search for
|
|
<filename class="extension">.bb</filename> files. By
|
|
default this specifies the <filename class="directory">meta/packages/
|
|
</filename> directory within Poky, but other directories such as
|
|
<filename class="directory">meta-extras/</filename> can be included
|
|
too. Adding extra content to
|
|
<glossterm><link linkend='var-BBFILES'>BBFILES</link></glossterm> is best
|
|
acheived through the use of Bitbake
|
|
<link linkend='usingpoky-changes-layers'>"layers"</link>.
|
|
</para>
|
|
|
|
<para>
|
|
Bitbake parses each <filename class="extension">.bb</filename> file in
|
|
<glossterm><link linkend='var-BBFILES'>BBFILES</link></glossterm> and
|
|
stores the values of various variables. In summary, for each
|
|
<filename class="extension">.bb</filename>
|
|
file the configuration + base class of variables are set, followed
|
|
by the data in the <filename class="extension">.bb</filename> file
|
|
itself, followed by any inherit commands that
|
|
<filename class="extension">.bb</filename> file might contain.
|
|
</para>
|
|
|
|
<para>
|
|
Parsing <filename class="extension">.bb</filename> files is a time
|
|
consuming process, so a cache is kept to speed up subsequent parsing.
|
|
This cache is invalid if the timestamp of the <filename class="extension">.bb</filename>
|
|
file itself has changed, or if the timestamps of any of the include,
|
|
configuration or class files the <filename class="extension">.bb</filename>
|
|
file depends on have changed.
|
|
</para>
|
|
</section>
|
|
|
|
<section id='ref-bitbake-providers'>
|
|
<title>Preferences and Providers</title>
|
|
|
|
<para>
|
|
Once all the <filename class="extension">.bb</filename> files have been
|
|
parsed, BitBake will proceed to build "poky-image-sato" (or whatever was
|
|
specified on the commandline) and looks for providers of that target.
|
|
Once a provider is selected, BitBake resolves all the dependencies for
|
|
the target. In the case of "poky-image-sato", it would lead to
|
|
<filename>task-oh.bb</filename> and <filename>task-base.bb</filename>
|
|
which in turn would lead to packages like <application>Contacts</application>,
|
|
<application>Dates</application>, <application>BusyBox</application>
|
|
and these in turn depend on glibc and the toolchain.
|
|
</para>
|
|
|
|
<para>
|
|
Sometimes a target might have multiple providers and a common example
|
|
is "virtual/kernel" that is provided by each kernel package. Each machine
|
|
will often elect the best provider of its kernel with a line like the
|
|
following in the machine configuration file:
|
|
</para>
|
|
<programlisting><glossterm><link linkend='var-PREFERRED_PROVIDER'>PREFERRED_PROVIDER</link></glossterm>_virtual/kernel = "linux-rp"</programlisting>
|
|
<para>
|
|
The default <glossterm><link linkend='var-PREFERRED_PROVIDER'>
|
|
PREFERRED_PROVIDER</link></glossterm> is the provider with the same name as
|
|
the target.
|
|
</para>
|
|
|
|
<para>
|
|
Understanding how providers are chosen is complicated by the fact
|
|
multiple versions might be present. Bitbake defaults to the highest
|
|
version of a provider by default. Version comparisons are made using
|
|
the same method as Debian. The <glossterm><link
|
|
linkend='var-PREFERRED_VERSION'>PREFERRED_VERSION</link></glossterm>
|
|
variable can be used to specify a particular version
|
|
(usually in the distro configuration) but the order can
|
|
also be influenced by the <glossterm><link
|
|
linkend='var-DEFAULT_PREFERENCE'>DEFAULT_PREFERENCE</link></glossterm>
|
|
variable. By default files
|
|
have a preference of "0". Setting the
|
|
<glossterm><link
|
|
linkend='var-DEFAULT_PREFERENCE'>DEFAULT_PREFERENCE</link></glossterm> to "-1" will
|
|
make a package unlikely to be used unless it was explicitly referenced and
|
|
"1" makes it likely the package will be used.
|
|
<glossterm><link
|
|
linkend='var-PREFERRED_VERSION'>PREFERRED_VERSION</link></glossterm> overrides
|
|
any default preference. <glossterm><link
|
|
linkend='var-DEFAULT_PREFERENCE'>DEFAULT_PREFERENCE</link></glossterm>
|
|
is often used to mark more
|
|
experimental new versions of packages until they've undergone sufficient
|
|
testing to be considered stable.
|
|
</para>
|
|
|
|
<para>
|
|
The end result is that internally, BitBake has now built a list of
|
|
providers for each target it needs in order of priority.
|
|
</para>
|
|
</section>
|
|
|
|
<section id='ref-bitbake-dependencies'>
|
|
<title>Dependencies</title>
|
|
|
|
<para>
|
|
Each target BitBake builds consists of multiple tasks (e.g. fetch,
|
|
unpack, patch, configure, compile etc.). For best performance on
|
|
multi-core systems, BitBake considers each task as an independent
|
|
entity with a set of dependencies. There are many variables that
|
|
are used to signify these dependencies and more information can be found
|
|
found about these in the <ulink url='http://bitbake.berlios.de/manual/'>
|
|
BitBake manual</ulink>. At a basic level it is sufficient to know
|
|
that BitBake uses the <glossterm><link
|
|
linkend='var-DEPENDS'>DEPENDS</link></glossterm> and
|
|
<glossterm><link linkend='var-RDEPENDS'>RDEPENDS</link></glossterm> variables when
|
|
calculating dependencies and descriptions of these variables are
|
|
available through the links.
|
|
</para>
|
|
|
|
</section>
|
|
|
|
<section id='ref-bitbake-tasklist'>
|
|
<title>The Task List</title>
|
|
|
|
<para>
|
|
Based on the generated list of providers and the dependency information,
|
|
BitBake can now calculate exactly which tasks it needs to run and in what
|
|
order. The build now starts with BitBake forking off threads up to
|
|
the limit set in the <glossterm><link
|
|
linkend='var-BB_NUMBER_THREADS'>BB_NUMBER_THREADS</link></glossterm> variable
|
|
as long there are tasks ready to run, i.e. tasks with all their
|
|
dependencies met.
|
|
</para>
|
|
|
|
<para>
|
|
As each task completes, a timestamp is written to the directory
|
|
specified by the <glossterm><link
|
|
linkend='var-STAMPS'>STAMPS</link></glossterm> variable (usually
|
|
<filename class="directory">build/tmp/stamps/*/</filename>). On
|
|
subsequent runs, BitBake looks at the <glossterm><link
|
|
linkend='var-STAMPS'>STAMPS</link></glossterm>
|
|
directory and will not rerun
|
|
tasks its already completed unless a timestamp is found to be invalid.
|
|
Currently, invalid timestamps are only considered on a per <filename
|
|
class="extension">.bb</filename> file basis so if for example the configure stamp has a timestamp greater than the
|
|
compile timestamp for a given target the compile task would rerun but this
|
|
has no effect on other providers depending on that target. This could
|
|
change or become configurable in future versions of BitBake. Some tasks
|
|
are marked as "nostamp" tasks which means no timestamp file will be written
|
|
and the task will always rerun.
|
|
</para>
|
|
|
|
<para>Once all the tasks have been completed BitBake exits.</para>
|
|
|
|
</section>
|
|
|
|
<section id='ref-bitbake-runtask'>
|
|
<title>Running a Task</title>
|
|
|
|
<para>
|
|
It's worth noting what BitBake does to run a task. A task can either
|
|
be a shell task or a python task. For shell tasks, BitBake writes a
|
|
shell script to <filename>${WORKDIR}/temp/run.do_taskname.pid</filename>
|
|
and then executes the script. The generated
|
|
shell script contains all the exported variables, and the shell functions
|
|
with all variables expanded. Output from the shell script is
|
|
sent to the file <filename>${WORKDIR}/temp/log.do_taskname.pid</filename>.
|
|
Looking at the
|
|
expanded shell functions in the run file and the output in the log files
|
|
is a useful debugging technique.
|
|
</para>
|
|
|
|
<para>
|
|
Python functions are executed internally to BitBake itself and
|
|
logging goes to the controlling terminal. Future versions of BitBake will
|
|
write the functions to files in a similar way to shell functions and
|
|
logging will also go to the log files in a similar way.
|
|
</para>
|
|
</section>
|
|
|
|
|
|
<section id='ref-bitbake-commandline'>
|
|
<title>Commandline</title>
|
|
|
|
<para>
|
|
To quote from "bitbake --help":
|
|
</para>
|
|
|
|
<screen>Usage: bitbake [options] [package ...]
|
|
|
|
Executes the specified task (default is 'build') for a given set of BitBake files.
|
|
It expects that BBFILES is defined, which is a space separated list of files to
|
|
be executed. BBFILES does support wildcards.
|
|
Default BBFILES are the .bb files in the current directory.
|
|
|
|
Options:
|
|
--version show program's version number and exit
|
|
-h, --help show this help message and exit
|
|
-b BUILDFILE, --buildfile=BUILDFILE
|
|
execute the task against this .bb file, rather than a
|
|
package from BBFILES.
|
|
-k, --continue continue as much as possible after an error. While the
|
|
target that failed, and those that depend on it,
|
|
cannot be remade, the other dependencies of these
|
|
targets can be processed all the same.
|
|
-f, --force force run of specified cmd, regardless of stamp status
|
|
-i, --interactive drop into the interactive mode also called the BitBake
|
|
shell.
|
|
-c CMD, --cmd=CMD Specify task to execute. Note that this only executes
|
|
the specified task for the providee and the packages
|
|
it depends on, i.e. 'compile' does not implicitly call
|
|
stage for the dependencies (IOW: use only if you know
|
|
what you are doing). Depending on the base.bbclass a
|
|
listtasks tasks is defined and will show available
|
|
tasks
|
|
-r FILE, --read=FILE read the specified file before bitbake.conf
|
|
-v, --verbose output more chit-chat to the terminal
|
|
-D, --debug Increase the debug level. You can specify this more
|
|
than once.
|
|
-n, --dry-run don't execute, just go through the motions
|
|
-p, --parse-only quit after parsing the BB files (developers only)
|
|
-d, --disable-psyco disable using the psyco just-in-time compiler (not
|
|
recommended)
|
|
-s, --show-versions show current and preferred versions of all packages
|
|
-e, --environment show the global or per-package environment (this is
|
|
what used to be bbread)
|
|
-g, --graphviz emit the dependency trees of the specified packages in
|
|
the dot syntax
|
|
-I IGNORED_DOT_DEPS, --ignore-deps=IGNORED_DOT_DEPS
|
|
Stop processing at the given list of dependencies when
|
|
generating dependency graphs. This can help to make
|
|
the graph more appealing
|
|
-l DEBUG_DOMAINS, --log-domains=DEBUG_DOMAINS
|
|
Show debug logging for the specified logging domains
|
|
-P, --profile profile the command and print a report</screen>
|
|
|
|
</section>
|
|
|
|
<section id='ref-bitbake-fetchers'>
|
|
<title>Fetchers</title>
|
|
|
|
<para>
|
|
As well as the containing the parsing and task/dependency handling
|
|
code, bitbake also contains a set of "fetcher" modules which allow
|
|
fetching of source code from various types of sources. Example
|
|
sources might be from disk with the metadata, from websites, from
|
|
remote shell accounts or from SCM systems like cvs/subversion/git.
|
|
</para>
|
|
|
|
<para>
|
|
The fetchers are usually triggered by entries in
|
|
<glossterm><link linkend='var-SRC_URI'>SRC_URI</link></glossterm>. Information about the
|
|
options and formats of entries for specific fetchers can be found in the
|
|
<ulink url='http://bitbake.berlios.de/manual/'>BitBake manual</ulink>.
|
|
</para>
|
|
|
|
<para>
|
|
One useful feature for certain SCM fetchers is the ability to
|
|
"auto-update" when the upstream SCM changes version. Since this
|
|
requires certain functionality from the SCM only certain systems
|
|
support it, currently Subversion, Bazaar and to a limited extent, Git. It
|
|
works using the <glossterm><link linkend='var-SRCREV'>SRCREV</link>
|
|
</glossterm> variable. See the <link linkend='platdev-appdev-srcrev'>
|
|
developing with an external SCM based project</link> section for more
|
|
information.
|
|
</para>
|
|
|
|
</section>
|
|
|
|
</appendix>
|
|
<!--
|
|
vim: expandtab tw=80 ts=4 spell spelllang=en_gb
|
|
-->
|