sysmo-bts
/
generic-poky
9
0
Fork 0
Code Pull Requests Releases Activity

bitbake: user-manual-execution.xml: Draft of "Execution" chapter 

This is a rough draft of the chapter.  Major sections are in
place.  We need to scrub content now.

Seems like the discussion of BBFILES fit better in this
execution flow chapter.  So, I moved it above the parsing
section.

(Bitbake rev: 6c4c6fb689b88dbefe63f0062b78d8403fb80d41)

Signed-off-by: Scott Rifenbark <scott.m.rifenbark@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This commit is contained in:
Scott Rifenbark 2014-02-14 13:55:42 -06:00 committed by Richard Purdie
parent 013c9d9181
commit 0c5bf41968
2 changed files with 669 additions and 732 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

152
bitbake/doc/user-manual/user-manual-bitbakecommand.xml
View File

@ -4,17 +4,22 @@
<chapter id="user-manual-command">
<title>The BitBake Command</title>
<section id='bitbake-command-introduction'>
<title>Introduction</title>
<para>
BitBake is the underlying piece of the build system.
Two excellent examples are the Yocto Project and the OpenEmbedded
build systems.
Each provide an environment in which to develop embedded Linux
images, and each use BitBake as their underlying build engine.
</para>
<para>
Bitbake is the primary command in the system.
It facilitates executing tasks in a single <filename>.bb</filename>
BitBake facilitates executing tasks in a single <filename>.bb</filename>
file, or executing a given task on a set of multiple
<filename>.bb</filename> files, accounting for interdependencies
amongst them.
This chapter presents the BitBake syntax, provides some execution
examples, and shows you how to control BitBake with key metadata.
</para>
</section>
<section id='usage-and-syntax'>
<title>Usage and syntax</title>
@ -110,7 +115,9 @@ Options:
<para>
Executing tasks for a single recipe file is relatively simple.
You specify the file in question, and BitBake parses
it and executes the specified task (or “build” by default).
it and executes the specified task.
If you do not specify a task, BitBake executes the default
task, which is "build”.
BitBake obeys inter-task dependencies when doing
so.
</para>
@ -138,38 +145,36 @@ Options:
when one wants to manage multiple <filename>.bb</filename>
files.
Clearly there needs to be a way to tell BitBake what
files are available, and of those, which we
want to execute at this time.
There also needs to be a way for each <filename>.bb</filename>
files are available, and of those, which you
want to execute.
There also needs to be a way for each recipe
to express its dependencies, both for build-time and
runtime.
There must be a way for the user to express their preferences
There must be a way for you to express recipe preferences
when multiple recipes provide the same functionality, or when
there are multiple versions of a <filename>.bb</filename> file.
</para>
<para>
The next section, Metadata, outlines how to specify such things.
there are multiple versions of a recipe.
</para>
<para>
The <filename>bitbake</filename> command, when not using
"--buildfile", accepts a PROVIDER, not a filename or
anything else.
By default, a <filename>.bb</filename> generally PROVIDES its
packagename, packagename-version, and packagename-version-revision.
"--buildfile" or "-b" only accepts a "PROVIDER".
You cannot provide anything else.
By default, a recipe file generally "PROVIDES" its
"packagename", "packagename-version", and
"packagename-version-revision" as shown in the following
example:
<literallayout class='monospaced'>
$ bitbake foo
$ bitbake foo-1.0
$ bitbake foo-1.0-r0
</literallayout>
This next example "PROVIDES" the package name and also uses
the "-c" option to tell BitBake to just excute the
<filename>do_clean</filename> task:
<literallayout class='monospaced'>
$ bitbake -c clean foo
$ bitbake virtual/whatever
$ bitbake -c clean virtual/whatever
</literallayout>
</para>
</section>
@ -180,19 +185,31 @@ Options:
<para>
BitBake is able to generate dependency graphs using
the dot syntax.
These graphs can be converted to images using the dot
You can convert these graphs into images using the dot
application from
<ulink url='http://www.graphviz.org'>Graphviz</ulink>.
Two files will be written into the current working directory:
<filename>depends.dot</filename> containing dependency information
at the package level and <filename>task-depends.dot</filename>
containing a breakdown of the dependencies at the task level.
To stop depending on common depends, one can use the "-I" depend
option to omit these from the graph.
This can lead to more readable graphs.
This way, <filename>DEPENDS</filename> from inherited classes
such as <filename>base.bbclass</filename> can be removed from the
graph.
</para>
<para>
When you generate a dependency graph, BitBake writes two files
to the current working directory:
<filename>depends.dot</filename>, which contains dependency information
at the package level, and <filename>task-depends.dot</filename>,
which contains a breakdown of the dependencies at the task level.
</para>
<para>
To stop depending on common depends, use use the "-I" depend
option and BitBake omits them from the graph.
Leaving this information out can produce more readable graphs.
This way, you can remove from the graph
<filename>DEPENDS</filename> from inherited classes
such as <filename>base.bbclass</filename>.
</para>
<para>
Here are two exmples that create dependency graphs.
The second example omits common depends from the graph:
<literallayout class='monospaced'>
$ bitbake -g foo
@ -202,66 +219,23 @@ Options:
</section>
</section>
<section id='special-variables'>
<title>Special Variables</title>
<section id='controlling-bitbake'>
<title>Controlling BitBake</title>
<para>
Certain variables affect BitBake operation:
Including variables in your recipe and class files help control
how BitBake operates.
</para>
<section id='bb-number-threads'>
<title><filename>BB_NUMBER_THREADS</filename></title>
<section id='execution-threads'>
<title>Execution Threads</title>
<para>
The number of threads BitBake should run at once (default: 1).
</para>
</section>
</section>
<section id='bitbake-command-metadata'>
<title>Metadata</title>
<para>
As you may have seen in the usage information, or in the
information about <filename>.bb</filename> files, the
<filename>BBFILES</filename> variable is how the BitBake
tool locates its files.
This variable is a space-separated list of files
that are available, and supports wildcards.
</para>
<section id='setting-bbfiles'>
<title>Setting <filename>BBFILES</filename></title>
<para>
<literallayout class='monospaced'>
BBFILES = "/path/to/bbfiles/*.bb"
</literallayout>
With regard to dependencies, it expects the
<filename>.bb</filename> to define a
<filename>DEPENDS</filename> variable, which contains a
space separated list of “package names”, which themselves
are the <filename>PN</filename> variable. The
<filename>PN</filename> variable is, in general,
set to a component of the <filename>.bb</filename>
filename by default.
</para>
</section>
<section id='depending-on-another-recipe-file'>
<title>Depending on Another Recipe File</title>
<para>
<literallayout class='monospaced'>
a.bb:
PN = "package-a"
DEPENDS += "package-b"
b.bb:
PN = "package-b"
</literallayout>
You can control how many thread BitBake supports by using the
<link linkend='var-BB_NUMBER_THREADS'><filename>BB_NUMBER_THREADS</filename></link>
variable.
You would set this in your <filename>local.conf</filename>
configuration file.
</para>
</section>

365
bitbake/doc/user-manual/user-manual-execution.xml
View File

@ -5,54 +5,49 @@
<title>Execution</title>
<para>
Tasks can either be a shell task or a Python task.
For shell tasks, BitBake writes a shell script to
<filename>${WORKDIR}/temp/run.do_taskname.pid</filename>
and then executes the script.
The generated shell script contains all the exported variables,
and the shell functions with all variables expanded.
Output from the shell script goes to the file
<filename>${WORKDIR}/temp/log.do_taskname.pid</filename>.
Looking at the expanded shell functions in the run file and
the output in the log files is a useful debugging technique.
The primary purpose for running BitBake is to produce an
image, which can be a kernel or a software development kit (SDK).
Of course, you can execute the <filename>bitbake</filename>
command with options that cause it to execute single tasks,
compile single recipe files, capture or clear data, or simply
return information about the execution environment.
</para>
<para>
For Python tasks, BitBake executes the task internally and logs
information to the controlling terminal.
Future versions of BitBake will write the functions to files
similar to the way shell tasks are handled.
Logging will be handled in a way similar to shell tasks as well.
This chapter describes BitBake's execution process from start
to finish when you use it to create an image.
The execution process is launched using the following command
form:
<literallayout class='monospaced'>
$ bitbake &lt;target&gt;
</literallayout>
For information on the BitBake command and its options,
see the
"<link linkend='user-manual-command'>BitBake Command</link>
chapter.
</para>
<section id='parsing-the-base-configuration-metadata'>
<title>Parsing the Base Configuration Metadata</title>
<para>
The first thing BitBake does is parse base configuration
metadata.
Base configuration metadata consists of the
<filename>bblayers.conf</filename> file to determine what
layers BitBake needs to recognize, all necessary
<filename>layer.conf</filename> files (one from each layer),
and <filename>bitbake.conf</filename>.
</para>
<para>
Once all the tasks have been completed BitBake exits.
</para>
<section id='parsing-and-execution'>
<title>Parsing and Execution</title>
<section id='parsing-overview'>
<title>Parsing Overview</title>
<para>
BitBake parses configuration files, classes, recipes, and append
files.
</para>
<para>
The first thing BitBake does is look for the
<filename>bitbake.conf</filename> file.
This file resides in the <filename>conf</filename>
directory, which must be listed in
The <filename>bitbake.conf</filename> file resides in the
<filename>conf</filename> directory, which must be listed in
<link linkend='var-BBPATH'><filename>BBPATH</filename></link>.
</para>
<para>
The <filename>bitbake.conf</filename> file lists other configuration
files to include from the <filename>conf</filename> directory below the
This configuratoin file lists and includes other configuration
files from the <filename>conf</filename> directory below the
directories listed in <filename>BBPATH</filename>.
In general, the most important of these
In general, the most important of these included
configuration files from a user's perspective
is <filename>local.conf</filename>, which contains the user's
customized settings for the build environment.
@ -68,51 +63,6 @@
variables, respectively.
</para>
<para>
After parsing of the configuration files, some standard classes are
included.
The <filename>base.bbclass</filename> file
is always included.
Other classes that are specified in the configuration using the
<link linkend='var-INHERIT'><filename>INHERIT</filename></link>
variable are also included.
BitBake searches for class files in a "classes" subdirectory under
the paths in <filename>BBPATH</filename> in the same way as
configuration files.
</para>
<para>
After classes are included, the variable
<filename>BBFILES</filename> is set, usually in
<filename>local.conf</filename>, and defines the list of
places to search for recipe and append files.
Adding extra content to <filename>BBFILES</filename> is best
achieved through the use of BitBake layers.
</para>
<para>
BitBake parses each recipe and append file located with
<filename>BBFILES</filename> and stores the values of various
variables into the datastore.
In summary, for each recipe and append file pairing, the configuration
plus the base class of variables are set, followed by the data in the
recipe file itself, followed by any inherit commands
that the recipe file might contain.
</para>
<para>
Because parsing recipe and append files is a time consuming
process, a cache, referred to as the "setscene"
is kept to speed up subsequent parsing.
The setscene is invalid if the timestamps of a recipe changes,
any of the include files change, configuration files change,
or class files on which the recipe file depends change.
</para>
</section>
<section id='parsing-configuration-files'>
<title>Configuration files</title>
<para>
Prior to parsing configuration files, Bitbake looks
at certain variables, including:
@ -127,33 +77,34 @@
</para>
<para>
The first kind of metadata in BitBake is configuration metadata.
This metadata is global, and therefore affects all packages and
tasks that are executed.
The base configuration metadata is global
and therefore affects all packages and tasks that are executed.
</para>
<para>
BitBake will first search the current working directory for an
BitBake first searches the current working directory for an
optional <filename>conf/bblayers.conf</filename> configuration file.
This file is expected to contain a
<link linkend='var-BBLAYERS'><filename>BBLAYERS</filename></link>
variable that is a space delimited list of 'layer' directories.
For each directory in this list, a <filename>conf/layer.conf</filename>
file will be searched for and parsed with the
<link linkend='var-LAYERDIR'><filename>LAYERDIR</filename></link>
variable being set to the directory where the layer was found.
The idea is these files will setup
<link linkend='var-BBPATH'><filename>BBPATH</filename></link>
and other variables correctly for a given build directory automatically
for the user.
</para>
<para>
BitBake will then expect to find <filename>conf/bitbake.conf</filename>
file somewhere in the user specified <filename>BBPATH</filename>.
For each directory (layer) in this list, a <filename>conf/layer.conf</filename>
file is searched for and parsed with the
<link linkend='var-LAYERDIR'><filename>LAYERDIR</filename></link>
variable being set to the directory where the layer was found.
The idea is these files automatically setup
<link linkend='var-BBPATH'><filename>BBPATH</filename></link>
and other variables correctly for a given build directory.
</para>
<para>
BitBake then expects to find the <filename>conf/bitbake.conf</filename>
file somewhere in the user-specified <filename>BBPATH</filename>.
That configuration file generally has include directives to pull
in any other metadata (generally files specific to architecture,
machine, local and so on).
in any other metadata such as files specific to the architecture,
the machine, the local environment, and so forth.
</para>
<para>
@ -212,37 +163,92 @@
</itemizedlist>
</para>
<section id='layers'>
<title>Layers</title>
<para>
Layers allow you to isolate different types of
customizations from each other.
While you might find it tempting to keep everything in one layer
when working on a single project, the more modular you organize
your metadata, the easier it is to cope with future changes.
After parsing configuration files, BitBake uses its rudimentary
inheritance mechanism, which is through class files, to inherit
some standard classes.
BitBake parses a class when the inherit directive responsible
for getting that class is encountered.
</para>
<para>
To illustrate how you can use layers to keep things modular,
consider machine customizations.
These types of customizations typically reside in a special layer,
rather than a general layer, called a Board Specific Package (BSP) Layer.
Furthermore, the machine customizations should be isolated from
recipes and metadata that support a new GUI environment, for
example.
This situation gives you a couple of layers: one for the machine
configurations and one for the GUI environment.
It is important to understand, however, that the BSP layer can still
make machine-specific additions to recipes within
the GUI environment layer without polluting the GUI layer itself
with those machine-specific changes.
You can accomplish this through a recipe that is a BitBake append
(<filename>.bbappend</filename>) file.
The <filename>base.bbclass</filename> file is always included.
Other classes that are specified in the configuration using the
<link linkend='var-INHERIT'><filename>INHERIT</filename></link>
variable are also included.
BitBake searches for class files in a "classes" subdirectory under
the paths in <filename>BBPATH</filename> in the same way as
configuration files.
</para>
<para>
There are certain variables specific to layers:
A good way to get an idea of the configuration files and
the class files used in your execution environment is to
run the following BitBake command:
<literallayout class='monospaced'>
$ bitbake -e > mybb.log
</literallayout>
Examining the top of the <filename>mybb.log</filename>
shows you the many configuration files and class files
used in your execution environment.
</para>
</section>
<section id='locating-recipes'>
<title>Locating Recipes</title>
<para>
The <filename>BBFILES</filename> variable is how BitBake
locates files.
This variable is a space-separated list of files
that are available, supports wildcards, and is set shortly
after the parsing phase of BitBake's execution.
</para>
<para>
<literallayout class='monospaced'>
BBFILES = "/path/to/bbfiles/*.bb"
</literallayout>
With regard to dependencies, it expects the
<filename>.bb</filename> to define a
<filename>DEPENDS</filename> variable, which contains a
space separated list of “package names”, which themselves
are the <filename>PN</filename> variable. The
<filename>PN</filename> variable is, in general,
set to a component of the <filename>.bb</filename>
filename by default.
</para>
</section>
<section id='parsing-recipes'>
<title>Parsing Recipes</title>
<para>
After classes are included, the variable
<filename>BBFILES</filename> is set, usually in
<filename>local.conf</filename>, and defines the list of
places to search for recipe and append files.
Adding extra content to <filename>BBFILES</filename> is best
achieved through the use of BitBake layers.
</para>
<para>
BitBake parses each recipe and append file located with
<filename>BBFILES</filename> and stores the values of various
variables into the datastore.
In summary, for each recipe and append file pairing, the configuration
plus the base class of variables are set, followed by the data in the
recipe file itself, followed by any inherit commands
that the recipe file might contain.
</para>
<para>
Part of parsing a recipe is making sure that all the recipes
that the recipe being parsed depends on are understood.
These other recipes could be located in other layers or in
a specific layer version.
These two variables in a recipe can help with with these
cases:
<itemizedlist>
<listitem><para>
<link linkend='var-LAYERDEPENDS'><filename>LAYERDEPENDS</filename></link>
@ -252,10 +258,40 @@
</para></listitem>
</itemizedlist>
</para>
<para>
Because parsing recipe and append files is a time consuming
process, a cache, referred to as the "setscene"
is kept to speed up subsequent parsing.
The setscene is invalid if the timestamps of a recipe changes,
any of the include files change, configuration files change,
or class files on which the recipe file depends change.
</para>
</section>
<section id='schedulers'>
<title>Schedulers</title>
<section id='executing-tasks'>
<title>Executing Tasks</title>
<para>
Tasks can either be a shell task or a Python task.
For shell tasks, BitBake writes a shell script to
<filename>${WORKDIR}/temp/run.do_taskname.pid</filename>
and then executes the script.
The generated shell script contains all the exported variables,
and the shell functions with all variables expanded.
Output from the shell script goes to the file
<filename>${WORKDIR}/temp/log.do_taskname.pid</filename>.
Looking at the expanded shell functions in the run file and
the output in the log files is a useful debugging technique.
</para>
<para>
For Python tasks, BitBake executes the task internally and logs
information to the controlling terminal.
Future versions of BitBake will write the functions to files
similar to the way shell tasks are handled.
Logging will be handled in a way similar to shell tasks as well.
</para>
<para>
Variables specific to scheduling functionality exist:
@ -269,78 +305,6 @@
</itemizedlist>
</para>
</section>
</section>
<section id='metadata-classes'>
<title>Classes</title>
<para>
BitBake's rudimentary inheritance mechanism is accomplished using
classes.
As briefly mentioned in the metadata introduction, BitBake
parses a class when an inherit directive is encountered, and it
is located in the <filename>classes</filename> directory
relative to the directories in
<link linkend='var-BBPATH'><filename>BBPATH</filename></link>.
</para>
</section>
<section id='recipe-bb-files'>
<title>Recipe (<filename>.bb</filename>) Files</title>
<para>
Recipe files, which are files that have the
<filename>.bb</filename> file extension, are logical units of
tasks for execution.
Normally, that logical unit is a package that needs to be
built.
</para>
<para>
BitBake obeys all inter-recipe dependencies.
</para>
<para>
Recipe files must reside in locations found in the
<link linkend='var-BBFILES'><filename>BBFILES</filename></link>
variable.
</para>
</section>
<section id='append-bbappend-files'>
<title>Append (<filename>.bbappend</filename>) Files</title>
<para>
Append files, which are files that have the
<filename>.bbappend</filename> file extension, add or
extend build information to an existing
<link linkend='recipe-bb-files'>recipe file</link>.
</para>
<para>
BitBake expects every append file to have a corresponding recipe file.
Furthermore, the append file and corresponding recipe file
must use the same root filename.
The filenames can differ only in the file type suffix used
(e.g. <filename>formfactor_0.0.bb</filename> and
<filename>formfactor_0.0.bbappend</filename>).
</para>
<para>
Information in append files overrides the information in the
similarly-named recipe file.
</para>
</section>
</section>
<section id='bitbake-dev-environment'>
<title>BitBake</title>
<para>
The OpenEmbedded build system uses BitBake to produce images.
BitBake consists of several functional areas.
This section takes a closer look at each of those areas.
</para>
<section id='source-fetching-dev-environment'>
<title>Source Fetching</title>
@ -350,7 +314,7 @@
the source code:
<imagedata fileref="figures/source-fetching.png" align="center" width="6.5in" depth="5in" />
</para>
figures/
<para>
The <filename>do_fetch</filename> and
<filename>do_unpack</filename> tasks fetch the source files
@ -686,5 +650,4 @@ figures/
which includes the environment setup script.
</para>
</section>
</section>
</chapter>