diff --git a/bitbake/doc/user-manual/user-manual-execution.xml b/bitbake/doc/user-manual/user-manual-execution.xml
new file mode 100644
index 0000000000..365392dbff
--- /dev/null
+++ b/bitbake/doc/user-manual/user-manual-execution.xml
@@ -0,0 +1,690 @@
+
+
+
+ Execution
+
+
+ Tasks can either be a shell task or a Python task.
+ For shell tasks, BitBake writes a shell script to
+ ${WORKDIR}/temp/run.do_taskname.pid
+ and then executes the script.
+ The generated shell script contains all the exported variables,
+ and the shell functions with all variables expanded.
+ Output from the shell script goes to the file
+ ${WORKDIR}/temp/log.do_taskname.pid.
+ Looking at the expanded shell functions in the run file and
+ the output in the log files is a useful debugging technique.
+
+
+
+ For Python tasks, BitBake executes the task internally and logs
+ information to the controlling terminal.
+ Future versions of BitBake will write the functions to files
+ similar to the way shell tasks are handled.
+ Logging will be handled in a way similar to shell tasks as well.
+
+
+
+ Once all the tasks have been completed BitBake exits.
+
+
+
+ Parsing and Execution
+
+
+ Parsing Overview
+
+
+ BitBake parses configuration files, classes, recipes, and append
+ files.
+
+
+
+ The first thing BitBake does is look for the
+ bitbake.conf file.
+ This file resides in the conf
+ directory, which must be listed in
+ BBPATH.
+
+
+
+ The bitbake.conf file lists other configuration
+ files to include from the conf directory below the
+ directories listed in BBPATH.
+ In general, the most important of these
+ configuration files from a user's perspective
+ is local.conf, which contains the user's
+ customized settings for the build environment.
+
+
+
+ Other notable configuration files are the distribution configuration
+ file and the machine configuration file.
+ These configuration files are normally identified by
+ variables unique to the build systems using BitBake.
+ For example, the Yocto Project uses the
+ DISTRO and MACHINE
+ variables, respectively.
+
+
+
+ After parsing of the configuration files, some standard classes are
+ included.
+ The base.bbclass file
+ is always included.
+ Other classes that are specified in the configuration using the
+ INHERIT
+ variable are also included.
+ BitBake searches for class files in a "classes" subdirectory under
+ the paths in BBPATH in the same way as
+ configuration files.
+
+
+
+ After classes are included, the variable
+ BBFILES is set, usually in
+ local.conf, and defines the list of
+ places to search for recipe and append files.
+ Adding extra content to BBFILES is best
+ achieved through the use of BitBake layers.
+
+
+
+ BitBake parses each recipe and append file located with
+ BBFILES and stores the values of various
+ variables into the datastore.
+ In summary, for each recipe and append file pairing, the configuration
+ plus the base class of variables are set, followed by the data in the
+ recipe file itself, followed by any inherit commands
+ that the recipe file might contain.
+
+
+
+ Because parsing recipe and append files is a time consuming
+ process, a cache, referred to as the "setscene"
+ is kept to speed up subsequent parsing.
+ The setscene is invalid if the timestamps of a recipe changes,
+ any of the include files change, configuration files change,
+ or class files on which the recipe file depends change.
+
+
+
+
+ Configuration files
+
+
+ Prior to parsing configuration files, Bitbake looks
+ at certain variables, including:
+
+ BB_ENV_WHITELIST
+ BB_PRESERVE_ENV
+ BB_ENV_EXTRAWHITE
+ BB_ORIGENV
+ PREFERRED_VERSION
+ PREFERRED_PROVIDERS
+
+
+
+
+ The first kind of metadata in BitBake is configuration metadata.
+ This metadata is global, and therefore affects all packages and
+ tasks that are executed.
+
+
+
+ BitBake will first search the current working directory for an
+ optional conf/bblayers.conf configuration file.
+ This file is expected to contain a
+ BBLAYERS
+ variable that is a space delimited list of 'layer' directories.
+ For each directory in this list, a conf/layer.conf
+ file will be searched for and parsed with the
+ LAYERDIR
+ variable being set to the directory where the layer was found.
+ The idea is these files will setup
+ BBPATH
+ and other variables correctly for a given build directory automatically
+ for the user.
+
+
+
+ BitBake will then expect to find conf/bitbake.conf
+ file somewhere in the user specified BBPATH.
+ That configuration file generally has include directives to pull
+ in any other metadata (generally files specific to architecture,
+ machine, local and so on).
+
+
+
+ Only variable definitions and include directives are allowed
+ in .conf files.
+ The following variables include:
+
+
+ BITBAKE_UI
+
+
+ BBDEBUG
+
+
+ MULTI_PROVIDER_WHITELIST
+
+
+ BB_NUMBER_PARSE_THREADS
+
+
+ BBPKGS
+
+
+ BB_DEFAULT_TASK
+
+
+ TOPDIR
+
+
+ BB_VERBOSE_LOGS
+
+
+ BB_NICE_LEVEL
+
+
+ BBFILE_COLLECTIONS
+
+
+ ASSUME_PROVIDED
+
+
+ BB_DANGLINGAPPENDS_WARNONLY
+
+
+ BBINCLUDED
+
+
+ BBFILE_PRIORITY
+
+
+ BUILDNAME
+
+
+ BBMASK
+
+
+
+
+
+ Layers
+
+
+ Layers allow you to isolate different types of
+ customizations from each other.
+ While you might find it tempting to keep everything in one layer
+ when working on a single project, the more modular you organize
+ your metadata, the easier it is to cope with future changes.
+
+
+
+ To illustrate how you can use layers to keep things modular,
+ consider machine customizations.
+ These types of customizations typically reside in a special layer,
+ rather than a general layer, called a Board Specific Package (BSP) Layer.
+ Furthermore, the machine customizations should be isolated from
+ recipes and metadata that support a new GUI environment, for
+ example.
+ This situation gives you a couple of layers: one for the machine
+ configurations and one for the GUI environment.
+ It is important to understand, however, that the BSP layer can still
+ make machine-specific additions to recipes within
+ the GUI environment layer without polluting the GUI layer itself
+ with those machine-specific changes.
+ You can accomplish this through a recipe that is a BitBake append
+ (.bbappend) file.
+
+
+
+ There are certain variables specific to layers:
+
+
+ LAYERDEPENDS
+
+
+ LAYERVERSION
+
+
+
+
+
+
+ Schedulers
+
+
+ Variables specific to scheduling functionality exist:
+
+
+ BB_SCHEDULER
+
+
+ BB_SCHEDULERS
+
+
+
+
+
+
+
+ Classes
+
+
+ BitBake's rudimentary inheritance mechanism is accomplished using
+ classes.
+ As briefly mentioned in the metadata introduction, BitBake
+ parses a class when an inherit directive is encountered, and it
+ is located in the classes directory
+ relative to the directories in
+ BBPATH.
+
+
+
+
+ Recipe (.bb) Files
+
+
+ Recipe files, which are files that have the
+ .bb file extension, are logical units of
+ tasks for execution.
+ Normally, that logical unit is a package that needs to be
+ built.
+
+
+
+ BitBake obeys all inter-recipe dependencies.
+
+
+
+ Recipe files must reside in locations found in the
+ BBFILES
+ variable.
+
+
+
+
+ Append (.bbappend) Files
+
+
+ Append files, which are files that have the
+ .bbappend file extension, add or
+ extend build information to an existing
+ recipe file.
+
+
+
+ BitBake expects every append file to have a corresponding recipe file.
+ Furthermore, the append file and corresponding recipe file
+ must use the same root filename.
+ The filenames can differ only in the file type suffix used
+ (e.g. formfactor_0.0.bb and
+ formfactor_0.0.bbappend).
+
+
+
+ Information in append files overrides the information in the
+ similarly-named recipe file.
+
+
+
+
+
+ BitBake
+
+
+ The OpenEmbedded build system uses BitBake to produce images.
+ BitBake consists of several functional areas.
+ This section takes a closer look at each of those areas.
+
+
+
+ Source Fetching
+
+
+ The first stages of building a recipe are to fetch and unpack
+ the source code:
+
+
+figures/
+
+ The do_fetch and
+ do_unpack tasks fetch the source files
+ and unpack them into the work directory.
+ By default, everything is accomplished in the
+ build directory,
+ which has a defined structure.
+
+
+
+ Unpacked source files are pointed to by a variable.
+ For example, in the Yocto Project and OpenEmbedded build systems,
+ the S variable points to these source files.
+ Each recipe has an area in the Build Directory where the
+ unpacked source code resides.
+ The name of that directory for any given recipe is defined from
+ several different variables.
+ You can see the variables that define these directories
+ by looking at the figure that shows the structure and variables
+ used in the Yocto Project:
+
+ TMPDIR
+
+ PACKAGE_ARCH
+
+ TARGET_OS
+
+ PN
+
+ PV
+
+ PR
+
+ WORKDIR
+
+ S
+
+
+
+
+
+ Briefly, the S directory contains the
+ unpacked source files for a recipe.
+ The WORKDIR directory is where all the
+ building goes on for a given recipe.
+
+
+
+
+ Patching
+
+
+ Once source code is fetched and unpacked, BitBake locates
+ patch files and applies them to the source files:
+
+
+
+
+ The do_patch task processes recipes by
+ using the
+ SRC_URI
+ variable to locate applicable patch files, which by default
+ are *.patch or
+ *.diff files, or any file if
+ "apply=yes" is specified for the file in
+ SRC_URI.
+
+
+
+ BitBake finds and applies multiple patches for a single recipe
+ in the order in which it finds the patches.
+ Patches are applied to the recipe's source files located in the
+ S directory.
+
+
+
+ For more information on how the source directories are
+ created, see the
+ "Source Fetching"
+ section.
+
+
+
+
+ Configuration and Compilation
+
+
+ After source code is patched, BitBake executes tasks that
+ configure and compile the source code:
+
+
+
+
+ This step in the build process consists of three tasks:
+
+ do_configure:
+ This task configures the source by enabling and
+ disabling any build-time and configuration options for
+ the software being built.
+ Configurations can come from the recipe itself as well
+ as from an inherited class.
+ Additionally, the software itself might configure itself
+ depending on the target for which it is being built.
+
+
+ The configurations handled by the
+ do_configure task are specific
+ to source code configuration for the source code
+ being built by the recipe.
+
+ If you are using the Autotools class
+ (autotools.bbclass),
+ you can add additional configuration options by using
+ the EXTRA_OECONF
+ variable.
+ For information on how this variable works within
+ that class, see the
+ meta/classes/autotools.bbclass file.
+
+ do_compile:
+ Once a configuration task has been satisfied, BitBake
+ compiles the source using the
+ do_compile task.
+ Compilation occurs in the directory pointed to by the
+ B
+ variable.
+ Realize that the B directory is, by
+ default, the same as the
+ S
+ directory.
+ do_install:
+ Once compilation is done, BitBake executes the
+ do_install task.
+ This task copies files from the B
+ directory and places them in a holding area pointed to
+ by the D variable.
+
+
+
+
+
+ Package Splitting
+
+
+ After source code is configured and compiled, the
+ OpenEmbedded build system analyzes
+ the results and splits the output into packages:
+
+
+
+
+ The do_package and
+ do_packagedata tasks combine to analyze
+ the files found in the D directory
+ and split them into subsets based on available packages and
+ files.
+ The analyzing process involves the following as well as other
+ items: splitting out debugging symbols,
+ looking at shared library dependencies between packages,
+ and looking at package relationships.
+ The do_packagedata task creates package
+ metadata based on the analysis such that the
+ OpenEmbedded build system can generate the final packages.
+ Working, staged, and intermediate results of the analysis
+ and package splitting process use these areas:
+
+ PKGD
+
+ PKGDATA_DIR
+
+ PKGDESTWORK
+
+ PKGDEST
+
+
+ The FILES
+ variable defines the files that go into each package in
+ PACKAGES.
+ If you want details on how this is accomplished in the Yocto Project
+ for example, you can look at the package.bbclass
+ file in a Yocto tree.
+
+
+
+ Depending on the type of packages being created (RPM, DEB, or
+ IPK), the do_package_write_* task
+ creates the actual packages and places them in the
+ Package Feed area, which is
+ ${TMPDIR}/deploy.
+
+ Support for creating feeds directly from the
+ deploy/* directories does not exist.
+ Creating such feeds usually requires some kind of feed
+ maintenance mechanism that would upload the new packages
+ into an official package feed (e.g. the
+ Ångström distribution).
+ This functionality is highly distribution-specific
+ and thus is not provided out of the box.
+
+
+
+
+
+ Image Generation
+
+
+ Once packages are split and stored in the Package Feeds area,
+ the OpenEmbedded build system uses BitBake to generate the
+ root filesystem image:
+
+
+
+
+ The image generation process consists of several stages and
+ depends on many variables.
+ The do_rootfs task uses these key variables
+ to help create the list of packages to actually install:
+
+ IMAGE_INSTALL:
+ Lists out the base set of packages to install from
+ the Package Feeds area.
+ PACKAGE_EXCLUDE:
+ Specifies packages that should not be installed.
+
+ IMAGE_FEATURES:
+ Specifies features to include in the image.
+ Most of these features map to additional packages for
+ installation.
+ PACKAGE_CLASSES:
+ Specifies the package backend to use and consequently
+ helps determine where to locate packages within the
+ Package Feeds area.
+ IMAGE_LINGUAS:
+ Determines the language(s) for which additional
+ language support packages are installed.
+
+
+
+
+
+ Package installation is under control of the package manager
+ (e.g. smart/rpm, opkg, or apt/dpkg) regardless of whether or
+ not package management is enabled for the target.
+ At the end of the process, if package management is not
+ enabled for the target, the package manager's data files
+ are deleted from the root filesystem.
+
+
+
+ During image generation, the build system attempts to run
+ all post-installation scripts.
+ Any that fail to run on the build host are run on the
+ target when the target system is first booted.
+ If you are using a
+ read-only root filesystem,
+ all the post installation scripts must succeed during the
+ package installation phase since the root filesystem cannot be
+ written into.
+
+
+
+ During Optimization, optimizing processes are run across
+ the image.
+ These processes include mklibs and
+ prelink.
+ The mklibs process optimizes the size
+ of the libraries.
+ A prelink process optimizes the dynamic
+ linking of shared libraries to reduce start up time of
+ executables.
+
+
+
+ Part of the image generation process includes compressing the
+ root filesystem image.
+ Compression is accomplished through several optimization
+ routines designed to reduce the overall size of the image.
+
+
+
+ After the root filesystem has been constructed, the image
+ generation process turns everything into an image file or
+ a set of image files.
+ The formats used for the root filesystem depend on the
+ IMAGE_FSTYPES variable.
+
+
+
+ The entire image generation process is run under Pseudo.
+ Running under Pseudo ensures that the files in the root
+ filesystem have correct ownership.
+
+
+
+
+ SDK Generation
+
+
+ The OpenEmbedded build system uses BitBake to generate the
+ Software Development Kit (SDK) installer script:
+
+
+
+
+ Like image generation, the SDK script process consists of
+ several stages and depends on many variables.
+ The do_populate_sdk task uses these
+ key variables to help create the list of packages to actually
+ install.
+
+
+
+ The do_populate_sdk task handles two
+ parts: a target part and a host part.
+ The target part is the part built for the target hardware and
+ includes libraries and headers.
+ The host part is the part of the SDK that runs on the
+ SDKMACHINE.
+
+
+
+ Once both parts are constructed, the
+ do_populate_sdk task performs some cleanup
+ on both parts.
+ After the cleanup, the task creates a cross-development
+ environment setup script and any configuration files that
+ might be needed.
+
+
+
+ The final output of the task is the Cross-development
+ toolchain installation script (.sh file),
+ which includes the environment setup script.
+
+
+
+
diff --git a/bitbake/doc/user-manual/user-manual-metadata.xml b/bitbake/doc/user-manual/user-manual-metadata.xml
index 796d27b151..8d37f6bee1 100644
--- a/bitbake/doc/user-manual/user-manual-metadata.xml
+++ b/bitbake/doc/user-manual/user-manual-metadata.xml
@@ -895,35 +895,6 @@
-
- Running a Task
-
-
- Tasks can either be a shell task or a Python task.
- For shell tasks, BitBake writes a shell script to
- ${WORKDIR}/temp/run.do_taskname.pid
- and then executes the script.
- The generated shell script contains all the exported variables,
- and the shell functions with all variables expanded.
- Output from the shell script goes to the file
- ${WORKDIR}/temp/log.do_taskname.pid.
- Looking at the expanded shell functions in the run file and
- the output in the log files is a useful debugging technique.
-
-
-
- For Python tasks, BitBake executes the task internally and logs
- information to the controlling terminal.
- Future versions of BitBake will write the functions to files
- similar to the way shell tasks are handled.
- Logging will be handled in a way similar to shell tasks as well.
-
-
-
- Once all the tasks have been completed BitBake exits.
-
-
-
Variable Flags
@@ -1033,310 +1004,6 @@
-
- Parsing and Execution
-
-
- Parsing Overview
-
-
- BitBake parses configuration files, classes, recipes, and append
- files.
-
-
-
- The first thing BitBake does is look for the
- bitbake.conf file.
- This file resides in the conf
- directory, which must be listed in
- BBPATH.
-
-
-
- The bitbake.conf file lists other configuration
- files to include from the conf directory below the
- directories listed in BBPATH.
- In general, the most important of these
- configuration files from a user's perspective
- is local.conf, which contains the user's
- customized settings for the build environment.
-
-
-
- Other notable configuration files are the distribution configuration
- file and the machine configuration file.
- These configuration files are normally identified by
- variables unique to the build systems using BitBake.
- For example, the Yocto Project uses the
- DISTRO and MACHINE
- variables, respectively.
-
-
-
- After parsing of the configuration files, some standard classes are
- included.
- The base.bbclass file
- is always included.
- Other classes that are specified in the configuration using the
- INHERIT
- variable are also included.
- BitBake searches for class files in a "classes" subdirectory under
- the paths in BBPATH in the same way as
- configuration files.
-
-
-
- After classes are included, the variable
- BBFILES is set, usually in
- local.conf, and defines the list of
- places to search for recipe and append files.
- Adding extra content to BBFILES is best
- achieved through the use of BitBake layers.
-
-
-
- BitBake parses each recipe and append file located with
- BBFILES and stores the values of various
- variables into the datastore.
- In summary, for each recipe and append file pairing, the configuration
- plus the base class of variables are set, followed by the data in the
- recipe file itself, followed by any inherit commands
- that the recipe file might contain.
-
-
-
- Because parsing recipe and append files is a time consuming
- process, a cache, referred to as the "setscene"
- is kept to speed up subsequent parsing.
- The setscene is invalid if the timestamps of a recipe changes,
- any of the include files change, configuration files change,
- or class files on which the recipe file depends change.
-
-
-
-
- Configuration files
-
-
- Prior to parsing configuration files, Bitbake looks
- at certain variables, including:
-
- BB_ENV_WHITELIST
- BB_PRESERVE_ENV
- BB_ENV_EXTRAWHITE
- BB_ORIGENV
- PREFERRED_VERSION
- PREFERRED_PROVIDERS
-
-
-
-
- The first kind of metadata in BitBake is configuration metadata.
- This metadata is global, and therefore affects all packages and
- tasks that are executed.
-
-
-
- BitBake will first search the current working directory for an
- optional conf/bblayers.conf configuration file.
- This file is expected to contain a
- BBLAYERS
- variable that is a space delimited list of 'layer' directories.
- For each directory in this list, a conf/layer.conf
- file will be searched for and parsed with the
- LAYERDIR
- variable being set to the directory where the layer was found.
- The idea is these files will setup
- BBPATH
- and other variables correctly for a given build directory automatically
- for the user.
-
-
-
- BitBake will then expect to find conf/bitbake.conf
- file somewhere in the user specified BBPATH.
- That configuration file generally has include directives to pull
- in any other metadata (generally files specific to architecture,
- machine, local and so on).
-
-
-
- Only variable definitions and include directives are allowed
- in .conf files.
- The following variables include:
-
-
- BITBAKE_UI
-
-
- BBDEBUG
-
-
- MULTI_PROVIDER_WHITELIST
-
-
- BB_NUMBER_PARSE_THREADS
-
-
- BBPKGS
-
-
- BB_DEFAULT_TASK
-
-
- TOPDIR
-
-
- BB_VERBOSE_LOGS
-
-
- BB_NICE_LEVEL
-
-
- BBFILE_COLLECTIONS
-
-
- ASSUME_PROVIDED
-
-
- BB_DANGLINGAPPENDS_WARNONLY
-
-
- BBINCLUDED
-
-
- BBFILE_PRIORITY
-
-
- BUILDNAME
-
-
- BBMASK
-
-
-
-
-
- Layers
-
-
- Layers allow you to isolate different types of
- customizations from each other.
- While you might find it tempting to keep everything in one layer
- when working on a single project, the more modular you organize
- your metadata, the easier it is to cope with future changes.
-
-
-
- To illustrate how you can use layers to keep things modular,
- consider machine customizations.
- These types of customizations typically reside in a special layer,
- rather than a general layer, called a Board Specific Package (BSP) Layer.
- Furthermore, the machine customizations should be isolated from
- recipes and metadata that support a new GUI environment, for
- example.
- This situation gives you a couple of layers: one for the machine
- configurations and one for the GUI environment.
- It is important to understand, however, that the BSP layer can still
- make machine-specific additions to recipes within
- the GUI environment layer without polluting the GUI layer itself
- with those machine-specific changes.
- You can accomplish this through a recipe that is a BitBake append
- (.bbappend) file.
-
-
-
- There are certain variables specific to layers:
-
-
- LAYERDEPENDS
-
-
- LAYERVERSION
-
-
-
-
-
-
- Schedulers
-
-
- Variables specific to scheduling functionality exist:
-
-
- BB_SCHEDULER
-
-
- BB_SCHEDULERS
-
-
-
-
-
-
-
- Classes
-
-
- BitBake's rudimentary inheritance mechanism is accomplished using
- classes.
- As briefly mentioned in the metadata introduction, BitBake
- parses a class when an inherit directive is encountered, and it
- is located in the classes directory
- relative to the directories in
- BBPATH.
-
-
-
-
- Recipe (.bb) Files
-
-
- Recipe files, which are files that have the
- .bb file extension, are logical units of
- tasks for execution.
- Normally, that logical unit is a package that needs to be
- built.
-
-
-
- BitBake obeys all inter-recipe dependencies.
-
-
-
- Recipe files must reside in locations found in the
- BBFILES
- variable.
-
-
-
-
- Append (.bbappend) Files
-
-
- Append files, which are files that have the
- .bbappend file extension, add or
- extend build information to an existing
- recipe file.
-
-
-
- BitBake expects every append file to have a corresponding recipe file.
- Furthermore, the append file and corresponding recipe file
- must use the same root filename.
- The filenames can differ only in the file type suffix used
- (e.g. formfactor_0.0.bb and
- formfactor_0.0.bbappend).
-
-
-
- Information in append files overrides the information in the
- similarly-named recipe file.
-
-
-
-
Events
diff --git a/bitbake/doc/user-manual/user-manual.xml b/bitbake/doc/user-manual/user-manual.xml
index 1e8777a4b3..ba690ab243 100644
--- a/bitbake/doc/user-manual/user-manual.xml
+++ b/bitbake/doc/user-manual/user-manual.xml
@@ -75,6 +75,8 @@
+
+