diff --git a/documentation/ref-manual/closer-look.xml b/documentation/ref-manual/closer-look.xml
new file mode 100644
index 0000000000..8c3ceb7210
--- /dev/null
+++ b/documentation/ref-manual/closer-look.xml
@@ -0,0 +1,1050 @@
+ %poky; ] >
+
+
+A Closer Look at the Yocto Project Development Environment
+
+
+ This chapter takes a more detailed look at the Yocto Project
+ development environment.
+ The following diagram represents the development environment at a
+ high level.
+ The remainder of this chapter expands on the fundamental input, output,
+ process, and
+ Metadata) blocks
+ in the Yocto Project development environment.
+
+
+
+
+
+
+
+ The generalized Yocto Project Development Environment consists of
+ several functional areas:
+
+ User Configuration:
+ Metadata you can use to control the build process.
+
+ Metadata Layers:
+ Various layers that provide software, machine, and
+ distro Metadata.
+ Source Files:
+ Upstream releases, local projects, and SCMs.
+ Build System:
+ Processes under the control of BitBake.
+ This block expands on how BitBake fetches source, applies
+ patches, completes compilation, analyzes output for package
+ generation, creates and tests packages, generates images, and
+ generates cross-development tools.
+ Package Feeds:
+ Directories containing output packages (rpm, deb or ipk),
+ which are subsequently used in the construction of an image or
+ SDK, produced by the build system.
+ These feeds can also be copied and shared using a web server or
+ other means to facilitate extending or updating existing
+ images on devices at runtime if runtime package management is
+ enabled.
+ Images:
+ Images produced by the development process.
+ Where do they go?
+ Can you mess with them (i.e. freely delete them or move them?).
+
+ Application Development SDK:
+ Cross-development tools that are produced along with an image
+ or separately with BitBake.
+
+
+
+
+ User Configuration
+
+
+ User configuration helps define the build.
+ Through user configuration, you can tell BitBake the
+ target architecture for which you are building the image,
+ where to store downloaded source, and other build properties.
+
+
+
+ The following figure shows an expanded representation of the
+ "User Configuration" box of the
+ general Yocto Project Development Environment figure:
+
+
+
+
+
+
+
+ BitBake needs some basic configuration files in order to complete
+ a build.
+ These files are *.conf files.
+ The minimally necessary ones reside as example files in the
+ Source Directory.
+ For simplicity, this section refers to the Source Directory as
+ the "Poky Directory."
+
+
+
+ When you clone the poky Git repository or you
+ download and unpack a Yocto Project release, you can set up the
+ Source Directory to be named anything you want.
+ For this discussion, the cloned repository uses the default
+ name poky.
+
+ The Poky repository is primarily an aggregation of existing
+ repositories.
+ It is not a canonical upstream source.
+
+
+
+
+ The meta-yocto layer inside Poky contains
+ a conf directory that has example
+ configuration files.
+ These example files are used as a basis for creating actual
+ configuration files when you source the build environment
+ script oe-init-build-env.
+
+
+
+ Sourcing the build environment script creates a
+ Build Directory
+ if one does not already exist.
+ BitBake uses the Build Directory for all its work during builds.
+ The Build Directory has a conf directory that
+ contains default versions of your local.conf
+ and bblayers.conf configuration files.
+ These default configuration files are created only if versions
+ do not already exist in the Build Directory at the time you
+ source the oe-init-build-env script.
+
+
+
+ Because the Poky repository is fundamentally an aggregation of
+ existing repositories, some users might be familiar with running
+ the oe-init-build-env script in the context of
+ separate OpenEmbedded-Core and BitBake repositories rather than a
+ single Poky repository.
+ This discussion assumes the script is executed from within a cloned
+ or unpacked version of Poky.
+
+
+
+ Depending on where the script is sourced, different sub-scripts
+ are called to set up the Build Directory (Yocto or OpenEmbedded).
+ Specifically, the script
+ scripts/oe-setup-builddir inside the
+ poky directory sets up the Build Directory and seeds the directory
+ (if necessary) with configuration files appropriate for the
+ Yocto Project development environment.
+
+ The scripts/oe-setup-builddir script
+ uses the $TEMPLATECONF variable to
+ determine which sample configuration files to locate.
+
+
+
+
+ The local.conf file provides many
+ basic variables that define a build environment.
+ Here is a list of a few.
+ To see the default configurations in a local.conf
+ file created by the build environment script, see the
+ local.conf.sample in the
+ meta-yocto layer:
+
+ Parallelism Options:
+ Controlled by the
+ BB_NUMBER_THREADS
+ and
+ PARALLEL_MAKE
+ variables.
+ Target Machine Selection:
+ Controlled by the
+ MACHINE
+ variable.
+ Download Directory:
+ Controlled by the
+ DL_DIR
+ variable.
+ Shared State Directory:
+ Controlled by the
+ SSTATE_DIR
+ variable.
+ Build Output:
+ Controlled by the
+ TMPDIR
+ variable.
+
+
+ Configurations set in the conf/local.conf
+ file can also be set in the
+ conf/site.conf and
+ conf/auto.conf configuration files.
+
+
+
+
+ The bblayers.conf file tells BitBake what
+ layers you want considered during the build.
+ By default, the layers listed in this file include layers
+ minimally needed by the build system.
+ However, you must manually add any custom layers you have created.
+ You can find more information on working with the
+ bblayers.conf file in the
+ "Enabling Your Layer"
+ section in the Yocto Project Development Manual.
+
+
+
+ The files site.conf and
+ auto.conf are not created by the environment
+ initialization script.
+ If you want these configuration files, you must create them
+ yourself:
+
+ site.conf:
+ You can use the conf/site.conf
+ configuration file to configure multiple build directories.
+ For example, suppose you had several build environments and
+ they shared some common features.
+ You can set these default build properties here.
+ A good example is perhaps the level of parallelism you want
+ to use through the
+ BB_NUMBER_THREADS
+ and
+ PARALLEL_MAKE
+ variables.
+ One useful scenario for using the
+ conf/site.conf file is to extend your
+ BBPATH
+ variable to include the path to a
+ conf/site.conf.
+ Then, when BitBake looks for Metadata using
+ BBPATH, it finds the
+ conf/site.conf file and applies your
+ common configurations found in the file.
+ To override configurations in a particular build directory,
+ alter the similar configurations within that build
+ directory's conf/local.conf file.
+
+ auto.conf:
+ This file is not hand-created.
+ Rather, the file is usually created and written to by
+ an autobuilder.
+ The settings put into the file are typically the same as
+ you would find in the conf/local.conf
+ or the conf/site.conf files.
+
+
+
+
+
+ You can edit all configuration files to further define
+ any particular build environment.
+ This process is represented by the "User Configuration Edits"
+ box in the figure.
+
+
+
+ When you launch your build with the
+ bitbake <target> command, BitBake
+ sorts out the configurations to ultimately define your build
+ environment.
+
+
+
+
+ Metadata, Machine Configuration, and Policy Configuration
+
+
+ The previous section described the user configurations that
+ define the BitBake's global behavior.
+ This section takes a closer look at the layers the build system
+ uses to further control the build.
+ These layers provide Metadata for the software, machine, and
+ policy.
+
+
+
+ In general, three types of layer input exist:
+
+ Policy Configuration:
+ Distribution Layers provide top-level or general
+ policies for the image or SDK being built.
+ For example, this layer would dictate whether BitBake
+ produces RPM or IPK packages.
+ Machine Configuration:
+ Board Support Package (BSP) layers provide machine
+ configurations.
+ This type of information is specific to a particular
+ target architecture.
+ Metadata:
+ Software layers contain user-supplied recipe files,
+ patches, and append files.
+
+
+
+
+
+ The following figure shows an expanded representation of the
+ Metadata, Machine Configuration, and Policy Configuration input
+ (layers) boxes of the
+ general Yocto Project Development Environment figure:
+
+
+
+
+
+
+
+ In general, all layers have a similar structure.
+ They all contain a licensing file
+ (e.g. COPYING) if the layer is to be
+ distributed, a README file as good practice
+ and especially if the layer is to be distributed, a
+ configuration directory, and recipe directories.
+
+
+
+ The Yocto Project has many layers that can be used.
+ You can see a web-interface listing of them on the
+ Source Repositories
+ page.
+ The layers are shown at the bottom categorized under
+ "Yocto Metadata Layers."
+ These layers are fundamentally a subset of the
+ OpenEmbedded Metadata Index,
+ which lists all layers provided by the OpenEmbedded community.
+
+ Layers exist in the Yocto Project Source Repositories that
+ cannot be found in the OpenEmbedded Metadata Index.
+ These layers are either deprecated or experimental in nature.
+
+
+
+
+ BitBake uses the conf/bblayers.conf file,
+ which is part of the user configuration, to find what layers it
+ should be using as part of the build.
+
+
+
+ For more information on layers, see the
+ "Understanding and Creating Layers"
+ section in the Yocto Project Development Manual.
+
+
+
+ Distro Layer
+
+
+ The distribution layer provides policy configurations for your
+ distribution.
+ Best practices dictate that you isolate these types of
+ configurations into their own layer.
+ Settings you provide in
+ conf/<distro>.conf override similar
+ settings that BitBake finds in your
+ conf/local.conf file in the Build
+ Directory.
+
+
+
+ The following list provides some explanation and references
+ for what you typically find in the distribution layer:
+
+ classes:
+ Class files (.bbclass) holds
+ common functionality that can be shared among
+ recipes in the distribution.
+ When your recipes inherit a class, they take on the
+ settings and functions for that class.
+ You can read more about class files in the
+ "Classes" section.
+
+ conf:
+ This area holds configuration files for the
+ layer (conf/layer.conf),
+ the distribution
+ (conf/distro/<distro>.conf),
+ and any distribution-wide include files.
+
+ recipes-*:
+ Recipes and append files that affect common
+ functionality across the distribution.
+ This area could include recipes and append files to
+ to add distribution-specific configuration,
+ initialization scripts, custom image recipes,
+ and so forth.
+
+
+
+
+
+ BSP Layer
+
+
+ The BSP Layer provides machine configurations.
+ Everything in this layer is specific to the machine for which
+ you are building the image or the SDK.
+ A common structure or form is defined for BSP layers.
+ You can learn more about this structure in the
+ Yocto Project Board Support Package (BSP) Developer's Guide.
+
+ In order for a BSP layer to be considered compliant with the
+ Yocto Project, it must meet some structural requirements.
+
+
+
+
+ The BSP Layer's configuration directory contains
+ configuration files for the machine
+ (conf/machine/<machine>.conf) and,
+ of course, the layer (conf/layer.conf).
+
+
+
+ The remainder of the layer is dedicated to specific recipes
+ by function: recipes-bsp,
+ recipes-core,
+ recipes-graphics, and
+ recipes-kernel.
+ Metadata can exist for multiple formfactors, graphics
+ support systems, and so forth.
+
+ While the figure shows several recipe-*
+ directories, not all these directories appear in all
+ BSP layers.
+
+
+
+
+
+ Software Layer
+
+
+ The software layer provides the Metadata for additional
+ software packages used during the build.
+ This layer does not include Metadata that is specific to the
+ distribution or the machine, which are found in their
+ respective layers.
+
+
+
+ This layer contains any new recipes that your project needs
+ in the form of recipe files.
+
+
+
+
+
+ Sources
+
+
+ In order for the OpenEmbedded build system to create an image or
+ any target, it must be able to access source files.
+ The
+ general Yocto Project Development Environment figure
+ represents source files using the "Upstream Project Releases",
+ "Local Projects", and "SCMs (optional)" boxes.
+ The figure represents mirrors, which also play a role in locating
+ source files, with the "Source Mirror(s)" box.
+
+
+
+ The method by which source files are ultimately organized is
+ a function of the project.
+ For example, for released software, projects tend to use tarballs
+ or other archived files that can capture the state of a release
+ guaranteeing that it is statically represented.
+ On the other hand, for a project that is more dynamic or
+ experimental in nature, a project might keep source files in a
+ repository controlled by a Source Control Manager (SCM) such as
+ Git.
+ Pulling source from a repository allows you to control
+ the point in the repository (the revision) from which you want to
+ build software.
+ Finally, a combination of the two might exist, which would give the
+ consumer a choice when deciding where to get source files.
+
+
+
+ BitBake uses the
+ SRC_URI
+ variable to point to source files regardless of their location.
+ Each recipe must have a SRC_URI variable
+ that points to the source.
+
+
+
+ Another area that plays a significant role in where source files
+ comes from is pointed to by the
+ DL_DIR
+ variable.
+ This area is a cache that can hold previously downloaded source.
+ Judicious use of a DL_DIR directory can
+ save the build system a trip across the Internet when looking
+ for files.
+ A good method for using a download directory is to have
+ DL_DIR point to an area outside of your
+ Build Directory.
+ Doing so allows you to safely delete the Build Directory
+ if needed without fear of removing any downloaded source file.
+
+
+
+ The remainder of this section provides a deeper look into the
+ source files and the mirrors.
+ Here is a more detailed look at the source file area of the
+ base figure:
+
+
+
+
+ Upstream Project Releases
+
+
+ Upstream project releases exist anywhere in the form of an
+ archived file (e.g. tarball or zip file).
+ These files correspond to individual recipes.
+ For example, the figure uses specific releases each for
+ BusyBox, Qt, and Dbus.
+ An archive file can be for any released product that can be
+ built using a recipe.
+
+
+
+
+ Local Projects
+
+
+ Local projects are custom bits of software the user provides.
+ These bits reside somewhere local to a project - perhaps
+ a directory into which the user checks in items (e.g.
+ a local directory containing a development source tree
+ used by the group).
+
+
+
+ The canonical method through which to include a local project
+ is to use the
+ externalsrc.bbclass
+ class to include local project.
+ You use either the local.conf or a
+ recipe's append file to override or set the
+ recipe to point to the local directory on your disk to pull
+ in the whole source tree.
+
+
+
+ For information on how to use the
+ externalsrc.bbclass, see the
+ "externalsrc.bbclass"
+ section.
+
+
+
+
+ Source Control Managers (Optional)
+
+
+ Another place the build system can get source files from is
+ through an SCM such as Git or Subversion.
+ In this case, a repository is cloned or checked out.
+ The do_fetch task inside BitBake uses
+ the SRC_URI
+ variable and the argument's prefix to determine the correct
+ fetcher module.
+
+
+
+ When fetching a repository, BitBake uses the
+ SRCREV
+ variable to determine the specific revision from which to
+ build.
+
+
+
+
+ Source Mirror(s)
+
+
+ Two kinds of mirrors exist: pre-mirrors and regular mirrors.
+ The PREMIRRORS
+ and
+ MIRRORS
+ variables point to these, respectively.
+ BitBake checks pre-mirrors before looking upstream for any
+ source files.
+ Pre-mirrors are appropriate when you have a shared directory
+ that is not a directory defined by the
+ DL_DIR
+ variable.
+ A Pre-mirror typically points to a shared directory that is
+ local to your organization.
+
+
+
+ Regular mirrors can be any site across the Internet that is
+ used as an alternative location for source code should the
+ primary site not be functioning for some reason or another.
+
+
+
+
+
+ BitBake
+
+
+ The OpenEmbedded build system uses BitBake to produce images.
+ You can see from the
+ general Yocto Project Development Environment figure,
+ the BitBake area consists of several functional areas.
+ This section takes a closer look at each of those areas.
+
+
+
+ Source Fetching
+
+
+ The first stages of building a recipe are to fetch and unpack
+ the source code:
+
+
+
+
+ The do_fetch and
+ do_unpack tasks fetch the source files
+ and unpack them into a working directory.
+ By default, everything is accomplished in the
+ Build Directory,
+ which has a defined structure.
+ For additional general information on the Build Directory,
+ see the
+ "build/"
+ section.
+
+
+
+ Unpacked source source files are pointed to by the
+ S variable.
+ Each recipe has an area in the Build Directory where the
+ unpacked source code resides.
+ The name of directory for any given recipe is defined from
+ several different variables.
+ You can see the variables that define these directories
+ by looking at the figure:
+
+ TMPDIR
+
+ PACKAGE_ARCH
+
+ TARGET_OS
+
+ PN
+
+ PV
+
+ PR
+
+ WORKDIR
+
+ S
+
+
+
+
+
+ Briefly, the S directory contains the
+ unpacked source files for a recipe.
+ The WORKDIR directory is where all the
+ building goes on for a given recipe.
+
+
+
+
+ Patching
+
+
+ Once source code is fetched and unpacked, BitBake locates
+ patch files and applies them to the source files:
+
+
+
+
+ The do_patch task processes recipes by
+ using the
+ SRC_URI
+ variable to locate applicable patch files, which by default
+ are *.patch or
+ *.diff files, or any file if
+ "apply=yes" is specified for the file in
+ SRC_URI.
+
+
+
+ BitBake finds and applies multiple patches for a single recipe
+ in the order in which it finds the patches.
+ Patches are applied to the recipe's source files located in the
+ S directory.
+
+
+
+ For more information on how the source directories are
+ created, see the
+ "Source Fetching"
+ section.
+
+
+
+
+ Configuration and Compilation
+
+
+ After source code is patched, BitBake executes tasks that
+ configure and compile the source code:
+
+
+
+
+ This step in the build process consists of three tasks:
+
+ do_configure:
+ This task configures the source by enabling and
+ disabling any build-time and configuration options for
+ the software being built.
+ Configurations can come from the recipe itself as well
+ as from an inherited class.
+ Additionally, the software itself might configure itself
+ depending on the target for which it is being built.
+
+
+ The configurations handled by the
+ do_configure task are specific
+ to source code configuration for the source code
+ being built by the recipe.
+
+ If you are using
+ autotools.bbclass,
+ you can add additional configuration options by using
+ the EXTRA_OECONF
+ variable.
+ For information on how this variable works within
+ that class, see the
+ meta/classes/autotools.bbclass.
+
+ do_compile:
+ Once a configuration task has been satisfied, BitBake
+ compiles the source using the
+ do_compile task.
+ Compilation occurs in the directory pointed to by the
+ B
+ variable.
+ Realize that the B directory, by
+ default, is the same as the
+ S
+ directory.
+ do_install:
+ Once compilation is done, BitBake executes the
+ do_install task.
+ This task copies files from the B
+ directory and places them in a holding area pointed to
+ by the
+ D
+ variable.
+
+
+
+
+
+ Package Splitting
+
+
+ After source code is configured and compiled, the
+ OpenEmbedded build system analyzes
+ the results and splits the output into packages:
+
+
+
+
+ The do_package and
+ do_packagedata tasks combine to analyze
+ the files found in the
+ D directory
+ and split them into subsets based on available packages and
+ files.
+ The analyzing process involves the following as well as other
+ items: splitting out debugging symbols,
+ looking at shared library dependencies between packages,
+ and looking at package relationships.
+ The do_packagedata task creates package
+ metadata based on the analysis such that the
+ OpenEmbedded build system can generate the final packages.
+ Working, staged, and intermediate results of the analysis
+ and package splitting process use these areas:
+
+ PKGD
+
+ PKGDATA_DIR
+
+ PKGDESTWORK
+
+ PKGDEST
+
+
+ The FILES
+ variable defines the files that go into each package in
+ PACKAGES.
+ If you want details on how this is accomplished, you can
+ look at
+ package.bbclass.
+
+
+
+ Depending on the type of packages being created (RPM, DEB, or
+ IPK), the do_package_write_* task
+ creates the actual packages and places them in the
+ Package Feed area, which is
+ ${TMPDIR}/deploy.
+ You can see the
+ "Package Feeds"
+ section for more detail on that part of the build process.
+
+ Support for creating feeds directly from the
+ deploy/* directories does not exist.
+ Creating such feeds usually requires some kind of feed
+ maintenance mechanism that would upload the new packages
+ into an official package feed (e.g. the
+ Ångström distribution).
+ This functionality is highly distribution-specific
+ and thus is not provided out of the box.
+
+
+
+
+
+
+ Package Feeds
+
+
+ When the OpenEmbedded build system generates an image or an SDK,
+ it gets the packages from a package feed area located in the
+ Build Directory.
+ The
+ general Yocto Project Development Environment figure
+ shows this package feeds area in the upper-right corner.
+
+
+
+ This section looks a little closer into the package feeds area used
+ by the build system.
+ Here is a more detailed look at the area:
+
+
+
+
+ Package feeds are an intermediary step in the build process.
+ BitBake generates packages whose type is defined by the
+ PACKAGE_CLASSES
+ variable.
+ Before placing the packages into package feeds,
+ the build process validates them with generated output quality
+ assurance checks through the
+ insane.bbclass
+ class.
+
+
+
+ The package feed area resides in
+ tmp/deploy of the Build Directory.
+ Folders are created that correspond to the package type
+ (IPK, DEB, or RPM) created.
+ Further organization is derived through the value of the
+ PACKAGE_ARCH
+ variable for each package.
+ For example, packages can exist for the i586 or qemux86
+ architectures.
+ The package files themselves reside within the appropriate
+ architecture folder.
+
+
+
+ BitBake uses the do_package_write_* task to
+ place generated packages into the package holding area (e.g.
+ do_package_write_ipk for IPK packages).
+
+
+
+
+ Images
+
+
+ The images produced by the OpenEmbedded build system
+ are compressed forms of the
+ root filesystems that are ready to boot on a target device.
+ You can see from the
+ general Yocto Project Development Environment figure
+ that BitBake output in part consists of images.
+ This section is going to look more closely at this output:
+
+
+
+
+ For a list of example images that the Yocto Project provides,
+ the
+ "Images" chapter.
+
+
+
+ Images are written out to the
+ Build Directory
+ inside the deploy/images folder as shown
+ in the figure.
+ This folder contains any files expected to be loaded on the
+ target device.
+ The
+ DEPLOY_DIR
+ variable points to the deploy directory.
+
+ <kernel-image>:
+ A kernel binary file.
+ The KERNEL_IMAGETYPE
+ variable setting determines the naming scheme for the
+ kernel image file.
+ Depending on that variable, the file could begin with
+ a variety of naming strings.
+ The deploy/images directory can
+ contain multiple image files.
+ <root-filesystem-image>:
+ Root filesystems for the target device (e.g.
+ *.ext3 or *.bz2
+ files).
+ The IMAGE_FSTYPES
+ variable setting determines the root filesystem image
+ type.
+ The deploy/images directory can
+ contain multiple root filesystems.
+ <kernel-modules>:
+ Tarballs that contain all the modules built for the kernel.
+ Kernel module tarballs exist for legacy purposes and
+ can be suppressed by setting the
+ MODULE_TARBALL_DEPLOY
+ variable to "0".
+ The deploy/images directory can
+ contain multiple kernel module tarballs.
+
+ <bootloaders>:
+ Bootloaders supporting the image, if applicable to the
+ target machine.
+ The deploy/images directory can
+ contain multiple bootloaders.
+
+ <symlinks>:
+ The deploy/images folder contains
+ a symbolic link that points to the most recently built file
+ for each machine.
+ These links might be useful for external scripts that
+ need to obtain the latest version of each file.
+
+
+
+
+
+
+ Application Development SDK
+
+
+ In the
+ general Yocto Project Development Environment figure,
+ the output labeled "Application Development SDK" represents an
+ SDK.
+ This section is going to take a closer look at this output:
+
+
+
+
+ The specific form of this output is a self-extracting
+ SDK installer (*.sh) that, when run,
+ installs the SDK, which consists of a cross-development
+ toolchain, a set of libraries and headers, and an SDK
+ environment setup script.
+ Running this installer essentially sets up your
+ cross-development environment.
+ You can think of the cross-toolchain as the "host"
+ part because it runs on the SDK machine.
+ You can think of the libraries and headers as the "target"
+ part because they are built for the target hardware.
+ The setup script is added so that you can initialize the
+ environment before using the tools.
+
+
+
+
+ The Yocto Project supports several methods by which you can
+ set up this cross-development environment.
+ These methods include downloading pre-built SDK installers,
+ building and installing your own SDK installer, or running
+ an Application Development Toolkit (ADT) installer to
+ install not just cross-development toolchains
+ but also additional tools to help in this type of
+ development.
+
+
+
+ For background information on cross-development toolchains
+ in the Yocto Project development environment, see the
+ "Cross-Development Toolchain Generation"
+ section.
+ For information on setting up a cross-development
+ environment, see the
+ "Installing the ADT and Toolchains"
+ section in the Yocto Project Application Developer's Guide.
+
+
+
+
+ Once built, the SDK installers are written out to the
+ deploy/sdk folder inside the
+ Build Directory
+ as shown in the figure at the beginning of this section.
+ Several variables exist that help configure these files:
+
+ DEPLOY_DIR:
+ Points to the deploy
+ directory.
+ SDKMACHINE:
+ Specifies the architecture of the machine
+ on which the cross-development tools are run to
+ create packages for the target hardware.
+
+ SDKIMAGE_FEATURES:
+ Lists the features to include in the "target" part
+ of the SDK.
+
+ TOOLCHAIN_HOST_TASK:
+ Lists packages that make up the host
+ part of the SDK (i.e. the part that runs on
+ the SDKMACHINE).
+ When you use
+ bitbake -c populate_sdk <imagename>
+ to create the SDK, a set of default packages
+ apply.
+ This variable allows you to add more packages.
+
+ TOOLCHAIN_TARGET_TASK:
+ Lists packages that make up the target part
+ of the SDK (i.e. the part built for the
+ target hardware).
+
+
+
+
+
+
+
diff --git a/documentation/ref-manual/ref-manual.xml b/documentation/ref-manual/ref-manual.xml
index 78a0aae888..cbc598c10b 100644
--- a/documentation/ref-manual/ref-manual.xml
+++ b/documentation/ref-manual/ref-manual.xml
@@ -99,6 +99,8 @@
+
+
diff --git a/documentation/ref-manual/technical-details.xml b/documentation/ref-manual/technical-details.xml
index bfab8a6c68..40733fae6a 100644
--- a/documentation/ref-manual/technical-details.xml
+++ b/documentation/ref-manual/technical-details.xml
@@ -153,1045 +153,6 @@
-
- A Closer Look at the Yocto Project Development Environment
-
-
- This section takes a more detailed look at the Yocto Project
- development environment.
- The following diagram represents the development environment at a
- high level.
- The remainder of this section expands on the fundamental input, output,
- process, and
- Metadata) blocks
- in the Yocto Project development environment.
-
-
-
-
-
-
-
- The generalized Yocto Project Development Environment consists of
- several functional areas:
-
- User Configuration:
- Metadata you can use to control the build process.
-
- Metadata Layers:
- Various layers that provide software, machine, and
- distro Metadata.
- Source Files:
- Upstream releases, local projects, and SCMs.
- Build System:
- Processes under the control of BitBake.
- This block expands on how BitBake fetches source, applies
- patches, completes compilation, analyzes output for package
- generation, creates and tests packages, generates images, and
- generates cross-development tools.
- Package Feeds:
- Directories containing output packages (rpm, deb or ipk),
- which are subsequently used in the construction of an image or
- SDK, produced by the build system.
- These feeds can also be copied and shared using a web server or
- other means to facilitate extending or updating existing
- images on devices at runtime if runtime package management is
- enabled.
- Images:
- Images produced by the development process.
- Where do they go?
- Can you mess with them (i.e. freely delete them or move them?).
-
- Application Development SDK:
- Cross-development tools that are produced along with an image
- or separately with BitBake.
-
-
-
-
- User Configuration
-
-
- User configuration helps define the build.
- Through user configuration, you can tell BitBake the
- target architecture for which you are building the image,
- where to store downloaded source, and other build properties.
- The following figure shows an expanded representation of the
- user configuration box of the Yocto Project development
- environment:
-
-
-
-
-
-
-
- BitBake needs some basic configuration files in order to complete
- a build.
- These files are *.conf files.
- The minimally necessary ones reside as example files in the
- Source Directory.
- For simplicity, this section refers to the Source Directory as
- the "Poky Directory."
-
-
-
- When you clone the poky Git repository or you
- download and unpack a Yocto Project release, you can set up the
- Source Directory to be named anything you want.
- For this discussion, the cloned repository uses the default
- name poky.
-
- The Poky repository is primarily an aggregation of existing
- repositories.
- It is not a canonical upstream source.
-
-
-
-
- The meta-yocto layer inside Poky contains
- a conf directory that has example
- configuration files.
- These example files are used as a basis for creating actual
- configuration files when you source the build environment
- script oe-init-build-env.
-
-
-
- Sourcing the build environment script creates a
- Build Directory
- if one does not already exist.
- BitBake uses the Build Directory for all its work during builds.
- The Build Directory has a conf directory that
- contains default versions of your local.conf
- and bblayers.conf configuration files.
- These default configuration files are created only if versions
- do not already exist in the Build Directory at the time you
- source the oe-init-build-env script.
-
-
-
- Because the Poky repository is fundamentally an aggregation of
- existing repositories, some users might be familiar with running
- the oe-init-build-env script in the context of
- separate OpenEmbedded-Core and BitBake repositories rather than a
- single Poky repository.
- This discussion assumes the script is executed from within a cloned
- or unpacked version of Poky.
-
-
-
- Depending on where the script is sourced, different sub-scripts
- are called to set up the Build Directory (Yocto or OpenEmbedded).
- Specifically, the script
- scripts/oe-setup-builddir inside the
- poky directory sets up the Build Directory and seeds the directory
- (if necessary) with configuration files appropriate for the
- Yocto Project development environment.
-
- The scripts/oe-setup-builddir script
- uses the $TEMPLATECONF variable to
- determine which sample configuration files to locate.
-
-
-
-
- The local.conf file provides many
- basic variables that define a build environment.
- Here is a list of a few.
- To see the default configurations in a local.conf
- file created by the build environment script, see the
- local.conf.sample in the
- meta-yocto layer:
-
- Parallelism Options:
- Controlled by the
- BB_NUMBER_THREADS
- and
- PARALLEL_MAKE
- variables.
- Target Machine Selection:
- Controlled by the
- MACHINE
- variable.
- Download Directory:
- Controlled by the
- DL_DIR
- variable.
- Shared State Directory:
- Controlled by the
- SSTATE_DIR
- variable.
- Build Output:
- Controlled by the
- TMPDIR
- variable.
-
-
- Configurations set in the conf/local.conf
- file can also be set in the
- conf/site.conf and
- conf/auto.conf configuration files.
-
-
-
-
- The bblayers.conf file tells BitBake what
- layers you want considered during the build.
- By default, the layers listed in this file include layers
- minimally needed by the build system.
- However, you must manually add any custom layers you have created.
- You can find more information on working with the
- bblayers.conf file in the
- "Enabling Your Layer"
- section in the Yocto Project Development Manual.
-
-
-
- The files site.conf and
- auto.conf are not created by the environment
- initialization script.
- If you want these configuration files, you must create them
- yourself:
-
- site.conf:
- You can use the conf/site.conf
- configuration file to configure multiple build directories.
- For example, suppose you had several build environments and
- they shared some common features.
- You can set these default build properties here.
- A good example is perhaps the level of parallelism you want
- to use through the
- BB_NUMBER_THREADS
- and
- PARALLEL_MAKE
- variables.
- One useful scenario for using the
- conf/site.conf file is to extend your
- BBPATH
- variable to include the path to a
- conf/site.conf.
- Then, when BitBake looks for Metadata using
- BBPATH, it finds the
- conf/site.conf file and applies your
- common configurations found in the file.
- To override configurations in a particular build directory,
- alter the similar configurations within that build
- directory's conf/local.conf file.
-
- auto.conf:
- This file is not hand-created.
- Rather, the file is usually created and written to by
- an autobuilder.
- The settings put into the file are typically the same as
- you would find in the conf/local.conf
- or the conf/site.conf files.
-
-
-
-
-
- You can edit all configuration files to further define
- any particular build environment.
- This process is represented by the "User Configuration Edits"
- box in the figure.
-
-
-
- When you launch your build with the
- bitbake <target> command, BitBake
- sorts out the configurations to ultimately define your build
- environment.
-
-
-
-
- Metadata, Machine Configuration, and Policy Configuration
-
-
- The previous section described the user configurations that
- define the BitBake's global behavior.
- This section takes a closer look at the layers the build system
- uses to further control the build.
- These layers provide Metadata for the software, machine, and
- policy.
-
-
-
- In general, three types of layer input exist:
-
- Policy Configuration:
- Distribution Layers provide top-level or general
- policies for the image or SDK being built.
- For example, this layer would dictate whether BitBake
- produces RPM or IPK packages.
- Machine Configuration:
- Board Support Package (BSP) layers provide machine
- configurations.
- This type of information is specific to a particular
- target architecture.
- Metadata:
- Software layers contain user-supplied recipe files,
- patches, and append files.
-
-
-
-
-
- The following figure shows an expanded representation of the
- Metadata, Machine Configuration, and Policy Configuration input
- (layers) boxes of the Yocto Project development environment:
-
-
-
-
-
-
-
- In general, all layers have a similar structure.
- They all contain a licensing file
- (e.g. COPYING) if the layer is to be
- distributed, a README file as good practice
- and especially if the layer is to be distributed, a
- configuration directory, and recipe directories.
-
-
-
- The Yocto Project has many layers that can be used.
- You can see a web-interface listing of them on the
- Source Repositories
- page.
- The layers are shown at the bottom categorized under
- "Yocto Metadata Layers."
- These layers are fundamentally a subset of the
- OpenEmbedded Metadata Index,
- which lists all layers provided by the OpenEmbedded community.
-
- Layers exist in the Yocto Project Source Repositories that
- cannot be found in the OpenEmbedded Metadata Index.
- These layers are either deprecated or experimental in nature.
-
-
-
-
- BitBake uses the conf/bblayers.conf file,
- which is part of the user configuration, to find what layers it
- should be using as part of the build.
-
-
-
- For more information on layers, see the
- "Understanding and Creating Layers"
- section in the Yocto Project Development Manual.
-
-
-
- Distro Layer
-
-
- The distribution layer provides policy configurations for your
- distribution.
- Best practices dictate that you isolate these types of
- configurations into their own layer.
- Settings you provide in
- conf/<distro>.conf override similar
- settings that BitBake finds in your
- conf/local.conf file in the Build
- Directory.
-
-
-
- The following list provides some explanation and references
- for what you typically find in the distribution layer:
-
- classes:
- Class files (.bbclass) holds
- common functionality that can be shared among
- recipes in the distribution.
- When your recipes inherit a class, they take on the
- settings and functions for that class.
- You can read more about class files in the
- "Classes" section.
-
- conf:
- This area holds configuration files for the
- layer (conf/layer.conf),
- the distribution
- (conf/distro/<distro>.conf),
- and any distribution-wide include files.
-
- recipes-*:
- Recipes and append files that affect common
- functionality across the distribution.
- This area could include recipes and append files to
- to add distribution-specific configuration,
- initialization scripts, custom image recipes,
- and so forth.
-
-
-
-
-
- BSP Layer
-
-
- The BSP Layer provides machine configurations.
- Everything in this layer is specific to the machine for which
- you are building the image or the SDK.
- A common structure or form is defined for BSP layers.
- You can learn more about this structure in the
- Yocto Project Board Support Package (BSP) Developer's Guide.
-
- In order for a BSP layer to be considered compliant with the
- Yocto Project, it must meet some structural requirements.
-
-
-
-
- The BSP Layer's configuration directory contains
- configuration files for the machine
- (conf/machine/<machine>.conf) and,
- of course, the layer (conf/layer.conf).
-
-
-
- The remainder of the layer is dedicated to specific recipes
- by function: recipes-bsp,
- recipes-core,
- recipes-graphics, and
- recipes-kernel.
- Metadata can exist for multiple formfactors, graphics
- support systems, and so forth.
-
- While the figure shows several recipe-*
- directories, not all these directories appear in all
- BSP layers.
-
-
-
-
-
- Software Layer
-
-
- The software layer provides the Metadata for additional
- software packages used during the build.
- This layer does not include Metadata that is specific to the
- distribution or the machine, which are found in their
- respective layers.
-
-
-
- This layer contains any new recipes that your project needs
- in the form of recipe files.
-
-
-
-
-
- Sources
-
-
- In order for the OpenEmbedded build system to create an image or
- any target, it must be able to access source files.
- The main
- Yocto Project Development Environment figure
- represents source files using the "Upstream Project Releases",
- "Local Projects", and "SCMs (optional)" boxes.
- The figure represents mirrors, which also play a role in locating
- source files, with the "Source Mirror(s)" box.
-
-
-
- The method by which source files are ultimately organized is
- a function of the project.
- For example, for released software, projects tend to use tarballs
- or other archived files that can capture the state of a release
- guaranteeing that it is statically represented.
- On the other hand, for a project that is more dynamic or
- experimental in nature, a project might keep source files in a
- repository controlled by a Source Control Manager (SCM) such as
- Git.
- Pulling source from a repository allows you to control
- the point in the repository (the revision) from which you want to
- build software.
- Finally, a combination of the two might exist, which would give the
- consumer a choice when deciding where to get source files.
-
-
-
- BitBake uses the
- SRC_URI
- variable to point to source files regardless of their location.
- Each recipe must have a SRC_URI variable
- that points to the source.
-
-
-
- Another area that plays a significant role in where source files
- comes from is pointed to by the
- DL_DIR
- variable.
- This area is a cache that can hold previously downloaded source.
- Judicious use of a DL_DIR directory can
- save the build system a trip across the Internet when looking
- for files.
- A good method for using a download directory is to have
- DL_DIR point to an area outside of your
- Build Directory.
- Doing so allows you to safely delete the Build Directory
- if needed without fear of removing any downloaded source file.
-
-
-
- The remainder of this section provides a deeper look into the
- source files and the mirrors.
- Here is a more detailed look at the source file area of the
- base figure:
-
-
-
-
- Upstream Project Releases
-
-
- Upstream project releases exist anywhere in the form of an
- archived file (e.g. tarball or zip file).
- These files correspond to individual recipes.
- For example, the figure uses specific releases each for
- BusyBox, Qt, and Dbus.
- An archive file can be for any released product that can be
- built using a recipe.
-
-
-
-
- Local Projects
-
-
- Local projects are custom bits of software the user provides.
- These bits reside somewhere local to a project - perhaps
- a directory into which the user checks in items (e.g.
- a local directory containing a development source tree
- used by the group).
-
-
-
- The canonical method through which to include a local project
- is to use the
- externalsrc.bbclass
- class to include local project.
- You use either the local.conf or a
- recipe's append file to override or set the
- recipe to point to the local directory on your disk to pull
- in the whole source tree.
-
-
-
- For information on how to use the
- externalsrc.bbclass, see the
- "externalsrc.bbclass"
- section.
-
-
-
-
- Source Control Managers (Optional)
-
-
- Another place the build system can get source files from is
- through an SCM such as Git or Subversion.
- In this case, a repository is cloned or checked out.
- The do_fetch task inside BitBake uses
- the SRC_URI
- variable and the argument's prefix to determine the correct
- fetcher module.
-
-
-
- When fetching a repository, BitBake uses the
- SRCREV
- variable to determine the specific revision from which to
- build.
-
-
-
-
- Source Mirror(s)
-
-
- Two kinds of mirrors exist: pre-mirrors and regular mirrors.
- The PREMIRRORS
- and
- MIRRORS
- variables point to these, respectively.
- BitBake checks pre-mirrors before looking upstream for any
- source files.
- Pre-mirrors are appropriate when you have a shared directory
- that is not a directory defined by the
- DL_DIR
- variable.
- A Pre-mirror typically points to a shared directory that is
- local to your organization.
-
-
-
- Regular mirrors can be any site across the Internet that is
- used as an alternative location for source code should the
- primary site not be functioning for some reason or another.
-
-
-
-
-
- BitBake
-
-
- The OpenEmbedded build system uses BitBake to produce images.
- You can see from the
- Yocto Project Development Environment
- figure, the BitBake area consists of several functional areas.
- This section takes a closer look at each of those areas.
-
-
-
- Source Fetching
-
-
- The first stages of building a recipe are to fetch and unpack
- the source code:
-
-
-
-
- The do_fetch and
- do_unpack tasks fetch the source files
- and unpack them into a working directory.
- By default, everything is accomplished in the
- Build Directory,
- which has a defined structure.
- For additional general information on the Build Directory,
- see the
- "build/"
- section.
-
-
-
- Unpacked source source files are pointed to by the
- S variable.
- Each recipe has an area in the Build Directory where the
- unpacked source code resides.
- The name of directory for any given recipe is defined from
- several different variables.
- You can see the variables that define these directories
- by looking at the figure:
-
- TMPDIR
-
- PACKAGE_ARCH
-
- TARGET_OS
-
- PN
-
- PV
-
- PR
-
- WORKDIR
-
- S
-
-
-
-
-
- Briefly, the S directory contains the
- unpacked source files for a recipe.
- The WORKDIR directory is where all the
- building goes on for a given recipe.
-
-
-
-
- Patching
-
-
- Once source code is fetched and unpacked, BitBake locates
- patch files and applies them to the source files:
-
-
-
-
- The do_patch task processes recipes by
- using the
- SRC_URI
- variable to locate applicable patch files, which by default
- are *.patch or
- *.diff files, or any file if
- "apply=yes" is specified for the file in
- SRC_URI.
-
-
-
- BitBake finds and applies multiple patches for a single recipe
- in the order in which it finds the patches.
- Patches are applied to the recipe's source files located in the
- S directory.
-
-
-
- For more information on how the source directories are
- created, see the
- "Source Fetching"
- section.
-
-
-
-
- Configuration and Compilation
-
-
- After source code is patched, BitBake executes tasks that
- configure and compile the source code:
-
-
-
-
- This step in the build process consists of three tasks:
-
- do_configure:
- This task configures the source by enabling and
- disabling any build-time and configuration options for
- the software being built.
- Configurations can come from the recipe itself as well
- as from an inherited class.
- Additionally, the software itself might configure itself
- depending on the target for which it is being built.
-
-
- The configurations handled by the
- do_configure task are specific
- to source code configuration for the source code
- being built by the recipe.
-
- If you are using
- autotools.bbclass,
- you can add additional configuration options by using
- the EXTRA_OECONF
- variable.
- For information on how this variable works within
- that class, see the
- meta/classes/autotools.bbclass.
-
- do_compile:
- Once a configuration task has been satisfied, BitBake
- compiles the source using the
- do_compile task.
- Compilation occurs in the directory pointed to by the
- B
- variable.
- Realize that the B directory, by
- default, is the same as the
- S
- directory.
- do_install:
- Once compilation is done, BitBake executes the
- do_install task.
- This task copies files from the B
- directory and places them in a holding area pointed to
- by the
- D
- variable.
-
-
-
-
-
- Package Splitting
-
-
- After source code is configured and compiled, the
- OpenEmbedded build system analyzes
- the results and splits the output into packages:
-
-
-
-
- The do_package and
- do_packagedata tasks combine to analyze
- the files found in the
- D directory
- and split them into subsets based on available packages and
- files.
- The analyzing process involves the following as well as other
- items: splitting out debugging symbols,
- looking at shared library dependencies between packages,
- and looking at package relationships.
- The do_packagedata task creates package
- metadata based on the analysis such that the
- OpenEmbedded build system can generate the final packages.
- Working, staged, and intermediate results of the analysis
- and package splitting process use these areas:
-
- PKGD
-
- PKGDATA_DIR
-
- PKGDESTWORK
-
- PKGDEST
-
-
- The FILES
- variable defines the files that go into each package in
- PACKAGES.
- If you want details on how this is accomplished, you can
- look at
- package.bbclass.
-
-
-
- Depending on the type of packages being created (RPM, DEB, or
- IPK), the do_package_write_* task
- creates the actual packages and places them in the
- Package Feed area, which is
- ${TMPDIR}/deploy.
- You can see the
- "Package Feeds"
- section for more detail on that part of the build process.
-
- Support for creating feeds directly from the
- deploy/* directories does not exist.
- Creating such feeds usually requires some kind of feed
- maintenance mechanism that would upload the new packages
- into an official package feed (e.g. the
- Ångström distribution).
- This functionality is highly distribution-specific
- and thus is not provided out of the box.
-
-
-
-
-
-
- Package Feeds
-
-
- When the OpenEmbedded build system generates an image or an SDK,
- it gets the packages from a package feed area located in the
- Build Directory.
- The main
- Yocto Project Development Environment
- figure shows this package feeds area in the upper-right corner.
-
-
-
- This section looks a little closer into the package feeds area used
- by the build system.
- Here is a more detailed look at the area:
-
-
-
-
- Package feeds are an intermediary step in the build process.
- BitBake generates packages whose type is defined by the
- PACKAGE_CLASSES
- variable.
- Before placing the packages into package feeds,
- the build process validates them with generated output quality
- assurance checks through the
- insane.bbclass
- class.
-
-
-
- The package feed area resides in
- tmp/deploy of the Build Directory.
- Folders are created that correspond to the package type
- (IPK, DEB, or RPM) created.
- Further organization is derived through the value of the
- PACKAGE_ARCH
- variable for each package.
- For example, packages can exist for the i586 or qemux86
- architectures.
- The package files themselves reside within the appropriate
- architecture folder.
-
-
-
- BitBake uses the do_package_write_* task to
- place generated packages into the package holding area (e.g.
- do_package_write_ipk for IPK packages).
-
-
-
-
- Images
-
-
- The images produced by the OpenEmbedded build system
- are compressed forms of the
- root filesystems that are ready to boot on a target device.
- You can see from the main
- Yocto Project Development Environment
- figure that BitBake output in part consists of images.
- This section is going to look more closely at this output:
-
-
-
-
- For a list of example images that the Yocto Project provides,
- the
- "Images" chapter.
-
-
-
- Images are written out to the
- Build Directory
- inside the deploy/images folder as shown
- in the figure.
- This folder contains any files expected to be loaded on the
- target device.
- The
- DEPLOY_DIR
- variable points to the deploy directory.
-
- <kernel-image>:
- A kernel binary file.
- The KERNEL_IMAGETYPE
- variable setting determines the naming scheme for the
- kernel image file.
- Depending on that variable, the file could begin with
- a variety of naming strings.
- The deploy/images directory can
- contain multiple image files.
- <root-filesystem-image>:
- Root filesystems for the target device (e.g.
- *.ext3 or *.bz2
- files).
- The IMAGE_FSTYPES
- variable setting determines the root filesystem image
- type.
- The deploy/images directory can
- contain multiple root filesystems.
- <kernel-modules>:
- Tarballs that contain all the modules built for the kernel.
- Kernel module tarballs exist for legacy purposes and
- can be suppressed by setting the
- MODULE_TARBALL_DEPLOY
- variable to "0".
- The deploy/images directory can
- contain multiple kernel module tarballs.
-
- <bootloaders>:
- Bootloaders supporting the image, if applicable to the
- target machine.
- The deploy/images directory can
- contain multiple bootloaders.
-
- <symlinks>:
- The deploy/images folder contains
- a symbolic link that points to the most recently built file
- for each machine.
- These links might be useful for external scripts that
- need to obtain the latest version of each file.
-
-
-
-
-
-
- Application Development SDK
-
-
- In the overview figure of the
- Yocto Project Development Environment
- the output labeled "Application Development SDK" represents an
- SDK.
- This section is going to take a closer look at this output:
-
-
-
-
- The specific form of this output is a self-extracting
- SDK installer (*.sh) that, when run,
- installs the SDK, which consists of a cross-development
- toolchain, a set of libraries and headers, and an SDK
- environment setup script.
- Running this installer essentially sets up your
- cross-development environment.
- You can think of the cross-toolchain as the "host"
- part because it runs on the SDK machine.
- You can think of the libraries and headers as the "target"
- part because they are built for the target hardware.
- The setup script is added so that you can initialize the
- environment before using the tools.
-
-
-
-
- The Yocto Project supports several methods by which you can
- set up this cross-development environment.
- These methods include downloading pre-built SDK installers,
- building and installing your own SDK installer, or running
- an Application Development Toolkit (ADT) installer to
- install not just cross-development toolchains
- but also additional tools to help in this type of
- development.
-
-
-
- For background information on cross-development toolchains
- in the Yocto Project development environment, see the
- "Cross-Development Toolchain Generation"
- section.
- For information on setting up a cross-development
- environment, see the
- "Installing the ADT and Toolchains"
- section in the Yocto Project Application Developer's Guide.
-
-
-
-
- Once built, the SDK installers are written out to the
- deploy/sdk folder inside the
- Build Directory
- as shown in the figure at the beginning of this section.
- Several variables exist that help configure these files:
-
- DEPLOY_DIR:
- Points to the deploy
- directory.
- SDKMACHINE:
- Specifies the architecture of the machine
- on which the cross-development tools are run to
- create packages for the target hardware.
-
- SDKIMAGE_FEATURES:
- Lists the features to include in the "target" part
- of the SDK.
-
- TOOLCHAIN_HOST_TASK:
- Lists packages that make up the host
- part of the SDK (i.e. the part that runs on
- the SDKMACHINE).
- When you use
- bitbake -c populate_sdk <imagename>
- to create the SDK, a set of default packages
- apply.
- This variable allows you to add more packages.
-
- TOOLCHAIN_TARGET_TASK:
- Lists packages that make up the target part
- of the SDK (i.e. the part built for the
- target hardware).
-
-
-
-
-
-
Cross-Development Toolchain Generation