sysmo-bts
/
generic-poky
9
0
Fork 0
Code Pull Requests Releases Activity

documentation/poky-ref-manual/ref-bitbake.xml: scrubbed for Poky 

I swapped the Yocto Project for Poky throughout.  Some other general
edits were done.  I put in a reference to a spot to learn how to best
set the BB_NUMBER_THREADS variable.

(From yocto-docs rev: 2543a915957014a59e6c49a1cdde270725d47af3)

Signed-off-by: Scott Rifenbark <scott.m.rifenbark@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This commit is contained in:
Scott Rifenbark 2011-08-19 10:11:41 -07:00 committed by Richard Purdie
parent 39de08c105
commit 32a860851c
1 changed files with 97 additions and 73 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

170
documentation/poky-ref-manual/ref-bitbake.xml
View File

@ -6,23 +6,23 @@
<title>Reference: BitBake</title> <title>Reference: BitBake</title>
<para> <para>
BitBake is a program written in Python that interprets the metadata that makes up Poky. BitBake is a program written in Python that interprets the metadata that makes up the Yocto Project.
At some point, people wonder what actually happens when you enter: At some point, developers wonder what actually happens when you enter:
<literallayout class='monospaced'> <literallayout class='monospaced'>
$ bitbake core-image-sato $ bitbake core-image-sato
</literallayout> </literallayout>
</para> </para>
<para> <para>
This appendix provides an overview of what happens behind the scenes from BitBake's perspective. This appendix provides an overview of what happens behind the scenes from BitBake's perspective.
</para> </para>
<note><para> <note>
BitBake strives to be a generic "task" executor that is capable of handling complex dependency relationships. BitBake strives to be a generic "task" executor that is capable of handling complex dependency relationships.
As such, it has no real knowledge of what the tasks being executed actually do. As such, it has no real knowledge of what the tasks being executed actually do.
BitBake just considers a list of tasks with dependencies and handles metadata BitBake just considers a list of tasks with dependencies and handles metadata
that consists of variables in a certain format that get passed to the tasks. that consists of variables in a certain format that get passed to the tasks.
</para></note> </note>
<section id='ref-bitbake-parsing'> <section id='ref-bitbake-parsing'>
<title>Parsing</title> <title>Parsing</title>
@ -33,61 +33,64 @@
<para> <para>
The first thing BitBake does is look for the <filename>bitbake.conf</filename> file. The first thing BitBake does is look for the <filename>bitbake.conf</filename> file.
Poky keeps this file in <filename>meta/conf/</filename>. The Yocto Project keeps this file in the Yocto Project file's <filename>meta/conf/</filename>
directory.
BitBake finds it by examining the <filename>BBPATH</filename> environment BitBake finds it by examining the <filename>BBPATH</filename> environment
variable and looking for the <filename>meta/conf/</filename> variable and looking for the <filename>meta/conf/</filename>
directory. directory.
</para> </para>
<para> <para>
In Poky, <filename>bitbake.conf</filename> lists other configuration In the Yocto Project, <filename>bitbake.conf</filename> lists other configuration
files to include from a <filename>conf/</filename> files to include from a <filename>conf/</filename>
directory below the directories listed in <filename>BBPATH</filename>. directory below the directories listed in <filename>BBPATH</filename>.
In general the most important configuration file from a user's perspective In general, the most important configuration file from a user's perspective
is <filename>local.conf</filename>, which contains a user's customized is <filename>local.conf</filename>, which contains a user's customized
settings for Poky. settings for the Yocto Project build environment.
Other notable configuration files are the distribution Other notable configuration files are the distribution
configuration file (set by the <glossterm><link linkend='var-DISTRO'> configuration file (set by the
DISTRO</link></glossterm> variable) and the machine configuration file <filename><link linkend='var-DISTRO'>DISTRO</link></filename> variable)
(set by the <glossterm><link linkend='var-MACHINE'>MACHINE</link> and the machine configuration file
</glossterm> variable). (set by the
The DISTRO and MACHINE environment variables are both usually set in <filename><link linkend='var-MACHINE'>MACHINE</link></filename> variable).
The <filename>DISTRO</filename> and <filename>MACHINE</filename> environment
variables are both usually set in
the <filename>local.conf</filename> file. the <filename>local.conf</filename> file.
Valid distribution Valid distribution
configuration files are available in the <filename> configuration files are available in the <filename>meta/conf/distro/</filename> directory
meta/conf/distro/</filename> directory and valid machine configuration and valid machine configuration
files in the <filename>meta/conf/machine/</filename> files in the <filename>meta/conf/machine/</filename> directory.
directory.
Within the <filename>meta/conf/machine/include/</filename> Within the <filename>meta/conf/machine/include/</filename>
directory are various <filename>tune-*.inc</filename> configuration files that provide common directory are various <filename>tune-*.inc</filename> configuration files that provide common
"tuning" settings specific to and shared between particular architectures and machines. "tuning" settings specific to and shared between particular architectures and machines.
</para> </para>
<para> <para>
After the parsing of the configuration files some standard classes are included. After the parsing of the configuration files, some standard classes are included.
The <filename>base.bbclass</filename> file is always included. The <filename>base.bbclass</filename> file is always included.
Other classes that are specified in the configuration using the Other classes that are specified in the configuration using the
<glossterm><link linkend='var-INHERIT'>INHERIT</link></glossterm> <filename><link linkend='var-INHERIT'>INHERIT</link></filename>
variable are also inculded. variable are also inculded.
Class files are searched for in a classes subdirectory Class files are searched for in a <filename>classes</filename> subdirectory
under the paths in <filename>BBPATH</filename> in the same way as under the paths in <filename>BBPATH</filename> in the same way as
configuration files. configuration files.
</para> </para>
<para> <para>
After classes are included, the After classes are included, the variable
variable <glossterm><link linkend='var-BBFILES'>BBFILES</link></glossterm> <filename><link linkend='var-BBFILES'>BBFILES</link></filename>
is set, usually in is set, usually in
<filename>local.conf</filename>, and defines the list of places to search for <filename>local.conf</filename>, and defines the list of places to search for
<filename>.bb</filename> files. <filename>.bb</filename> files.
By default, the BBFILES variable specifies the <filename>meta/recipes-*/ By default, the <filename>BBFILES</filename> variable specifies the
</filename> directory within Poky. <filename>meta/recipes-*/</filename> directory within Poky.
Adding extra content to BBFILES is best achieved through the use of BitBake Adding extra content to <filename>BBFILES</filename> is best achieved through the use of
<link linkend='usingpoky-changes-layers'>"layers"</link>. BitBake layers as described in the
<link linkend='usingpoky-changes-layers'>BitBake Layers</link> section.
</para> </para>
<para> <para>
BitBake parses each <filename>.bb</filename> file in BBFILES and BitBake parses each <filename>.bb</filename> file in <filename>BBFILES</filename> and
stores the values of various variables. stores the values of various variables.
In summary, for each <filename>.bb</filename> In summary, for each <filename>.bb</filename>
file the configuration plus the base class of variables are set, followed file the configuration plus the base class of variables are set, followed
@ -111,13 +114,14 @@
<para> <para>
Once all the <filename>.bb</filename> files have been Once all the <filename>.bb</filename> files have been
parsed, BitBake starts to build the target (core-image-sato in the previous section's parsed, BitBake starts to build the target (<filename>core-image-sato</filename>
example) and looks for providers of that target. in the previous section's example) and looks for providers of that target.
Once a provider is selected, BitBake resolves all the dependencies for Once a provider is selected, BitBake resolves all the dependencies for
the target. the target.
In the case of "core-image-sato", it would lead to <filename>task-base.bb</filename>, In the case of <filename>core-image-sato</filename>, it would lead to
which in turn leads to packages like <application>Contacts</application>, <filename>task-base.bb</filename>,
<application>Dates</application> and <application>BusyBox</application>. which in turn leads to packages like <filename>Contacts</filename>,
<filename>Dates</filename> and <filename>BusyBox</filename>.
These packages in turn depend on glibc and the toolchain. These packages in turn depend on glibc and the toolchain.
</para> </para>
@ -128,12 +132,12 @@
following in the machine configuration file: following in the machine configuration file:
</para> </para>
<programlisting> <literallayout class='monospaced'>
PREFERRED_PROVIDER_virtual/kernel = "linux-rp" PREFERRED_PROVIDER_virtual/kernel = "linux-yocto"
</programlisting> </literallayout>
<para> <para>
The default <glossterm><link linkend='var-PREFERRED_PROVIDER'>PREFERRED_PROVIDER</link></glossterm> The default <filename><link linkend='var-PREFERRED_PROVIDER'>PREFERRED_PROVIDER</link></filename>
is the provider with the same name as the target. is the provider with the same name as the target.
</para> </para>
@ -142,17 +146,18 @@ PREFERRED_PROVIDER_virtual/kernel = "linux-rp"
that multiple versions might exist. that multiple versions might exist.
BitBake defaults to the highest version of a provider. BitBake defaults to the highest version of a provider.
Version comparisons are made using the same method as Debian. Version comparisons are made using the same method as Debian.
You can use the <glossterm><link linkend='var-PREFERRED_VERSION'>PREFERRED_VERSION</link></glossterm> You can use the
<filename><link linkend='var-PREFERRED_VERSION'>PREFERRED_VERSION</link></filename>
variable to specify a particular version (usually in the distro configuration). variable to specify a particular version (usually in the distro configuration).
You can influence the order by using the You can influence the order by using the
<glossterm><link linkend='var-DEFAULT_PREFERENCE'>DEFAULT_PREFERENCE</link></glossterm> <filename><link linkend='var-DEFAULT_PREFERENCE'>DEFAULT_PREFERENCE</link></filename>
variable. variable.
By default, files have a preference of "0". By default, files have a preference of "0".
Setting the DEFAULT_PREFERENCE to "-1" makes the package unlikely to be used unless it is Setting the <filename>DEFAULT_PREFERENCE</filename> to "-1" makes the
explicitly referenced. package unlikely to be used unless it is explicitly referenced.
Setting the DEFAULT_PREFERENCE to "1" makes it likely the package is used. Setting the <filename>DEFAULT_PREFERENCE</filename> to "1" makes it likely the package is used.
PREFERRED_VERSION overrides any DEFAULT_PREFERENCE setting. <filename>PREFERRED_VERSION</filename> overrides any <filename>DEFAULT_PREFERENCE</filename> setting.
DEFAULT_PREFERENCE is often used to mark newer and more experimental package <filename>DEFAULT_PREFERENCE</filename> is often used to mark newer and more experimental package
versions until they have undergone sufficient testing to be considered stable. versions until they have undergone sufficient testing to be considered stable.
</para> </para>
@ -165,8 +170,10 @@ PREFERRED_PROVIDER_virtual/kernel = "linux-rp"
<title>Dependencies</title> <title>Dependencies</title>
<para> <para>
Each target BitBake builds consists of multiple tasks such as fetch, unpack, patch, configure, Each target BitBake builds consists of multiple tasks such as
and compile. <filename>fetch</filename>, <filename>unpack</filename>,
<filename>patch</filename>, <filename>configure</filename>,
and <filename>compile</filename>.
For best performance on multi-core systems, BitBake considers each task as an independent For best performance on multi-core systems, BitBake considers each task as an independent
entity with its own set of dependencies. entity with its own set of dependencies.
</para> </para>
@ -175,12 +182,11 @@ PREFERRED_PROVIDER_virtual/kernel = "linux-rp"
Dependencies are defined through several variables. Dependencies are defined through several variables.
You can find information about variables BitBake uses in the You can find information about variables BitBake uses in the
<ulink url='http://bitbake.berlios.de/manual/'>BitBake manual</ulink>. <ulink url='http://bitbake.berlios.de/manual/'>BitBake manual</ulink>.
At a basic level it is sufficient to know that BitBake uses the At a basic level, it is sufficient to know that BitBake uses the
<glossterm><link linkend='var-DEPENDS'>DEPENDS</link></glossterm> and <filename><link linkend='var-DEPENDS'>DEPENDS</link></filename> and
<glossterm><link linkend='var-RDEPENDS'>RDEPENDS</link></glossterm> variables when <filename><link linkend='var-RDEPENDS'>RDEPENDS</link></filename> variables when
calculating dependencies. calculating dependencies.
</para> </para>
</section> </section>
<section id='ref-bitbake-tasklist'> <section id='ref-bitbake-tasklist'>
@ -191,39 +197,49 @@ PREFERRED_PROVIDER_virtual/kernel = "linux-rp"
BitBake can now calculate exactly what tasks it needs to run and in what BitBake can now calculate exactly what tasks it needs to run and in what
order it needs to run them. order it needs to run them.
The build now starts with BitBake forking off threads up to the limit set in the The build now starts with BitBake forking off threads up to the limit set in the
<glossterm><link linkend='var-BB_NUMBER_THREADS'>BB_NUMBER_THREADS</link></glossterm> variable. <filename><link linkend='var-BB_NUMBER_THREADS'>BB_NUMBER_THREADS</link></filename> variable.
BitBake continues to fork threads as long as there are tasks ready to run, BitBake continues to fork threads as long as there are tasks ready to run,
those tasks have all their dependencies met, and the thread threshold has not been those tasks have all their dependencies met, and the thread threshold has not been
exceeded. exceeded.
</para> </para>
<para>
It is worth noting that you can greatly speed up the build time by properly setting
the <filename>BB_NUMBER_THREADS</filename> variable.
See the
<ulink url='http://www.yoctoproject.org/docs/1.1/yocto-project-qs/yocto-project-qs.html#building-image'>
Building an Image</ulink> section in the
<ulink url='http://www.yoctoproject.org/docs/1.1/yocto-project-qs/yocto-project-qs.html'>
Yocto Project Quick Start</ulink> for more information.
</para>
<para> <para>
As each task completes, a timestamp is written to the directory specified by the As each task completes, a timestamp is written to the directory specified by the
<glossterm><link linkend='var-STAMPS'>STAMPS</link></glossterm> variable (usually <filename><link linkend='var-STAMPS'>STAMPS</link></filename> variable (usually
<filename>build/tmp/stamps/*/</filename>). <filename>build/tmp/stamps/*/</filename>).
On subsequent runs, BitBake looks at the STAMPS directory and does not rerun On subsequent runs, BitBake looks at the <filename>STAMPS</filename> directory and does not rerun
tasks that are already completed unless a timestamp is found to be invalid. tasks that are already completed unless a timestamp is found to be invalid.
Currently, invalid timestamps are only considered on a per Currently, invalid timestamps are only considered on a per
<filename>.bb</filename> file basis. <filename>.bb</filename> file basis.
So, for example, if the configure stamp has a timestamp greater than the So, for example, if the configure stamp has a timestamp greater than the
compile timestamp for a given target then the compile task would rerun. compile timestamp for a given target, then the compile task would rerun.
Running the compile task again, however, has no effect on other providers Running the compile task again, however, has no effect on other providers
that depend on that target. that depend on that target.
This behavior could change or become configurable in future versions of BitBake. This behavior could change or become configurable in future versions of BitBake.
</para> </para>
<note><para> <note>
Some tasks are marked as "nostamp" tasks. Some tasks are marked as "nostamp" tasks.
No timestamp file is created when these tasks are run. No timestamp file is created when these tasks are run.
Consequently, "nostamp" tasks are always rerun. Consequently, "nostamp" tasks are always rerun.
</para></note> </note>
</section> </section>
<section id='ref-bitbake-runtask'> <section id='ref-bitbake-runtask'>
<title>Running a Task</title> <title>Running a Task</title>
<para> <para>
Tasks can either be a shell task or a python task. Tasks can either be a shell task or a Python task.
For shell tasks, BitBake writes a shell script to For shell tasks, BitBake writes a shell script to
<filename>${WORKDIR}/temp/run.do_taskname.pid</filename> and then executes the script. <filename>${WORKDIR}/temp/run.do_taskname.pid</filename> and then executes the script.
The generated shell script contains all the exported variables, and the shell functions The generated shell script contains all the exported variables, and the shell functions
@ -268,7 +284,8 @@ Options:
-h, --help show this help message and exit -h, --help show this help message and exit
-b BUILDFILE, --buildfile=BUILDFILE -b BUILDFILE, --buildfile=BUILDFILE
execute the task against this .bb file, rather than a execute the task against this .bb file, rather than a
package from BBFILES. package from BBFILES. Does not handle any
dependencies.
-k, --continue continue as much as possible after an error. While the -k, --continue continue as much as possible after an error. While the
target that failed, and those that depend on it, target that failed, and those that depend on it,
cannot be remade, the other dependencies of these cannot be remade, the other dependencies of these
@ -276,8 +293,6 @@ Options:
-a, --tryaltconfigs continue with builds by trying to use alternative -a, --tryaltconfigs continue with builds by trying to use alternative
providers where possible. providers where possible.
-f, --force force run of specified cmd, regardless of stamp status -f, --force force run of specified cmd, regardless of stamp status
-i, --interactive drop into the interactive mode also called the BitBake
shell.
-c CMD, --cmd=CMD Specify task to execute. Note that this only executes -c CMD, --cmd=CMD Specify task to execute. Note that this only executes
the specified task for the providee and the packages the specified task for the providee and the packages
it depends on, i.e. 'compile' does not implicitly call it depends on, i.e. 'compile' does not implicitly call
@ -285,26 +300,35 @@ Options:
what you are doing). Depending on the base.bbclass a what you are doing). Depending on the base.bbclass a
listtasks tasks is defined and will show available listtasks tasks is defined and will show available
tasks tasks
-r FILE, --read=FILE read the specified file before bitbake.conf -r PREFILE, --read=PREFILE
read the specified file before bitbake.conf
-R POSTFILE, --postread=POSTFILE
read the specified file after bitbake.conf
-v, --verbose output more chit-chat to the terminal -v, --verbose output more chit-chat to the terminal
-D, --debug Increase the debug level. You can specify this more -D, --debug Increase the debug level. You can specify this more
than once. than once.
-n, --dry-run don't execute, just go through the motions -n, --dry-run don't execute, just go through the motions
-S, --dump-signatures
don't execute, just dump out the signature
construction information
-p, --parse-only quit after parsing the BB files (developers only) -p, --parse-only quit after parsing the BB files (developers only)
-d, --disable-psyco disable using the psyco just-in-time compiler (not
recommended)
-s, --show-versions show current and preferred versions of all packages -s, --show-versions show current and preferred versions of all packages
-e, --environment show the global or per-package environment (this is -e, --environment show the global or per-package environment (this is
what used to be bbread) what used to be bbread)
-g, --graphviz emit the dependency trees of the specified packages in -g, --graphviz emit the dependency trees of the specified packages in
the dot syntax the dot syntax
-I IGNORED_DOT_DEPS, --ignore-deps=IGNORED_DOT_DEPS -I EXTRA_ASSUME_PROVIDED, --ignore-deps=EXTRA_ASSUME_PROVIDED
Stop processing at the given list of dependencies when Assume these dependencies don't exist and are already
generating dependency graphs. This can help to make provided (equivalent to ASSUME_PROVIDED). Useful to
the graph more appealing make dependency graphs more appealing
-l DEBUG_DOMAINS, --log-domains=DEBUG_DOMAINS -l DEBUG_DOMAINS, --log-domains=DEBUG_DOMAINS
Show debug logging for the specified logging domains Show debug logging for the specified logging domains
-P, --profile profile the command and print a report -P, --profile profile the command and print a report
-u UI, --ui=UI userinterface to use
-t SERVERTYPE, --servertype=SERVERTYPE
Choose which server to use, none, process or xmlrpc
--revisions-changed Set the exit code depending on whether upstream
floating revisions have changed or not
</screen> </screen>
</section> </section>
@ -321,22 +345,22 @@ Options:
<para> <para>
Fetchers are usually triggered by entries in Fetchers are usually triggered by entries in
<glossterm><link linkend='var-SRC_URI'>SRC_URI</link></glossterm>. <filename><link linkend='var-SRC_URI'>SRC_URI</link></filename>.
You can find information about the options and formats of entries for specific You can find information about the options and formats of entries for specific
fetchers in the <ulink url='http://bitbake.berlios.de/manual/'>BitBake manual</ulink>. fetchers in the <ulink url='http://bitbake.berlios.de/manual/'>BitBake manual</ulink>.
</para> </para>
<para> <para>
One useful feature for certain SCM fetchers is the ability to One useful feature for certain Source Code Manager (SCM) fetchers is the ability to
"auto-update" when the upstream SCM changes version. "auto-update" when the upstream SCM changes version.
Since this ability requires certain functionality from the SCM, not all Since this ability requires certain functionality from the SCM, not all
systems support it. systems support it.
Currently Subversion, Bazaar and to a limited extent, Git support the ability to "auto-update". Currently Subversion, Bazaar and to a limited extent, Git support the ability to "auto-update".
This feature works using the <glossterm><link linkend='var-SRCREV'>SRCREV</link></glossterm> This feature works using the <filename><link linkend='var-SRCREV'>SRCREV</link></filename>
variable. variable.
See the See the
<link linkend='platdev-appdev-srcrev'>Developing within Poky with an External SCM-based Package</link> <link linkend='platdev-appdev-srcrev'>Development Within Yocto Project for a Package that Uses
section for more information. an External SCM</link> section for more information.
</para> </para>
</section> </section>