diff --git a/documentation/poky-ref-manual/poky-ref-manual.xml b/documentation/poky-ref-manual/poky-ref-manual.xml
index 37059096e3..e9f6b5eba4 100644
--- a/documentation/poky-ref-manual/poky-ref-manual.xml
+++ b/documentation/poky-ref-manual/poky-ref-manual.xml
@@ -92,6 +92,8 @@
+
+
diff --git a/documentation/poky-ref-manual/technical-details.xml b/documentation/poky-ref-manual/technical-details.xml
new file mode 100644
index 0000000000..8f83ecb3a1
--- /dev/null
+++ b/documentation/poky-ref-manual/technical-details.xml
@@ -0,0 +1,229 @@
+
+
+Technical Details
+
+
+ This chapter provides technical details for various parts of the Yocto Project.
+ Currently, topics include Yocto Project components and shared state (sstate) cache.
+
+
+
+ Yocto Project Components
+
+
+ The BitBake task executor together with various types of configuration files form the
+ Yocto Project core.
+ This section overviews the BitBake task executor and the
+ configuration files by describing what they are used for and how they interact.
+
+
+
+ BitBake handles the parsing and execution of the data files.
+ The data itself is of various types:
+
+ Recipes: Provides details about particular
+ pieces of software
+ Class Data: An abstraction of common build
+ information (e.g. how to build a Linux kernel).
+ Configuration Data: Defines machine-specific settings,
+ policy decisions, etc.
+ Configuration data acts as the glue to bind everything together.
+
+ For more information on data, see the
+
+ Yocto Project Terms section in
+
+ The Yocto Project Development Manual.
+
+
+
+ BitBake knows how to combine multiple data sources together and refers to each data source
+ as a 'layer'.
+
+
+
+ Following are some brief details on these core components.
+ For more detailed information on these components see the
+ 'Reference: Directory Structure'
+ appendix.
+
+
+
+ BitBake
+
+
+ BitBake is the tool at the heart of the Yocto Project and is responsible
+ for parsing the metadata, generating a list of tasks from it,
+ and then executing those tasks.
+ To see a list of the options BitBake supports, use the following help command:
+
+ $ bitbake --help
+
+
+
+
+ The most common usage for BitBake is bitbake <packagename>, where
+ packagename is the name of the package you want to build
+ (referred to as the "target" in this manual).
+ The target often equates to the first part of a .bb filename.
+ So, to run the matchbox-desktop_1.2.3.bb file, you
+ might type the following:
+
+ $ bitbake matchbox-desktop
+
+ Several different versions of matchbox-desktop might exist.
+ BitBake chooses the one selected by the distribution configuration.
+ You can get more details about how BitBake chooses between different
+ target versions and providers in the
+ Preferences and Providers section.
+
+
+
+ BitBake also tries to execute any dependent tasks first.
+ So for example, before building matchbox-desktop, BitBake
+ would build a cross compiler and eglibc if they had not already
+ been built.
+ This release of the Yocto Project does not support the glibc
+ GNU version of the Unix standard C library. By default, the Yocto Project builds with
+ eglibc.
+
+
+
+ A useful BitBake option to consider is the -k or
+ --continue option.
+ This option instructs BitBake to try and continue processing the job as much
+ as possible even after encountering an error.
+ When an error occurs, the target that
+ failed and those that depend on it cannot be remade.
+ However, when you use this option other dependencies can still be processed.
+
+
+
+
+ Metadata (Recipes)
+
+
+ The .bb files are usually referred to as "recipes."
+ In general, a recipe contains information about a single piece of software.
+ The information includes the location from which to download the source patches
+ (if any are needed), which special configuration options to apply,
+ how to compile the source files, and how to package the compiled output.
+
+
+
+ The term "package" can also be used to describe recipes.
+ However, since the same word is used for the packaged output from the Yocto
+ Project (i.e. .ipk or .deb files),
+ this document avoids using the term "package" to refer to recipes.
+
+
+
+
+ Classes
+
+
+ Class files (.bbclass) contain information that is useful to share
+ between metadata files.
+ An example is the Autotools class, which contains
+ common settings for any application that Autotools uses.
+ The Reference: Classes appendix provides details
+ about common classes and how to use them.
+
+
+
+
+ Configuration
+
+
+ The configuration files (.conf) define various configuration variables
+ that govern the Yocto Project build process.
+ These files fall into several areas that define machine configuration options,
+ distribution configuration options, compiler tuning options, general common configuration
+ options and user configuration options (local.conf, which is found
+ in the Yocto Project files build directory).
+
+
+
+
+
+ Considering Shared State Cache
+
+
+ By design, the Yocto Project builds everything from scratch unless it can determine that
+ a given task's inputs have not changed.
+ While building from scratch ensures that everything is current, it does also
+ mean that a lot of time could be spent rebuiding things that don't necessarily need built.
+
+
+
+ The Yocto Project build process uses a shared state caching scheme to avoid having to
+ rebuild software when it is not necessary.
+ Because the build time for a Yocto image can be significant, it is helpful to try and
+ determine what really needs built and what can be skipped given a particular project's
+ build process.
+
+
+
+ The scheme that the Yocto Project uses involves checksum generation and comparison for
+ a task's inputs.
+ The scheme also employs an area of memory called the shared state cache that is
+ pointed to by the SSTATE_DIR variable.
+ This area contains task output generated from a previous build.
+ If a given task's checksum matches the checksum of a previous build for the same
+ task, the build process uses the state of the cache rather than rerunning that
+ task.
+
+
+
+ The previous paragraph is a simplistic explanation of how the build process
+ uses checksums and shared state memory cache to avoide building tasks that
+ don't need built.
+ If you want a bit more explanation on the topic,
+ see "Shared
+ State - What does it mean and why should I care?" from the Yocto
+ Project discussion archives.
+
+
+
+ As with all schemes, this one has some drawbacks.
+ It is possible that you could make implicit changes that are not factored into the checksum
+ calculation, but do affect a task's output.
+ A good example is perhaps when a tool changes its output.
+ Let's say that the output of rpmdeps needed to change.
+ The result of the change should be that all the "package", "package_write_rpm",
+ and "package_deploy-rpm" sstate-cache items would become invalid.
+ But, because this is a change that is external to the code and therefore implicit,
+ the associated sstate-cache items do not become invalidated.
+ In this case, the build process would use the cache items rather than running the
+ task again.
+ Obviously, these types of implicit changes can cause problems.
+
+
+
+ To avoid these problems during the build, you need to understand the effects of any
+ change you make.
+ Note that any changes you make directly to a function automatically are factored into
+ the checksum calculation and thus, will invalidate the associated area of sstate cache.
+ You need to be aware of any implicit changes that are not obvious changes to the
+ code and could affect the output of a given task.
+ Once you are aware of such a change, you can take steps to invalidate the cache
+ and force the task to run.
+ The step to take is as simple as changing a function's comments in the source code.
+ For example, to invalidate package sstate files, change the comment statments
+ of do_package or one of the functions it calls.
+ The change is purely cosmetic, but it causes the checksum to be recalculated and
+ forces the task to be run again.
+
+
+
+ For an example of a commit that makes a cosmetic change to invalidate an sstate,
+ see this
+ commit.
+
+
+
+
+
diff --git a/documentation/poky-ref-manual/usingpoky.xml b/documentation/poky-ref-manual/usingpoky.xml
index 1e21c93778..e741e23d93 100644
--- a/documentation/poky-ref-manual/usingpoky.xml
+++ b/documentation/poky-ref-manual/usingpoky.xml
@@ -4,150 +4,9 @@
Using the Yocto Project
- This section gives an overview of the components that make up the Yocto Project
- followed by information about Yocto Project builds and dealing with any
- problems that might arise.
+ This section describes the common usage for the Yocto Project.
-
- Yocto Project Components
-
-
- The BitBake task executor together with various types of configuration files form the
- Yocto Project core.
- This section overviews the BitBake task executor and the
- configuration files by describing what they are used for and they they interact.
-
-
-
- BitBake handles the parsing and execution of the data files.
- The data itself is of various types:
-
- Recipes: Provides details about particular
- pieces of software
- Class Data: An abstraction of common build
- information (e.g. how to build a Linux kernel).
- Configuration Data: Defines machine-specific settings,
- policy decisions, etc.
- Configuration data acts a the glue to bind everything together.
-
- For more information on data, see the
-
- Yocto Project Terms section in
-
- The Yocto Project Development Manual.
-
-
-
- BitBake knows how to combine multiple data sources together and refers to each data source
- as a 'layer'.
-
-
-
- Following are some brief details on these core components.
- For more detailed information on these components see the
- 'Reference: Directory Structure'
- appendix.
-
-
-
- BitBake
-
-
- BitBake is the tool at the heart of the Yocto Project and is responsible
- for parsing the metadata, generating a list of tasks from it,
- and then executing those tasks.
- To see a list of the options BitBake supports, use the following help command:
-
- $ bitbake --help
-
-
-
-
- The most common usage for BitBake is bitbake <packagename>, where
- packagename is the name of the package you want to build
- (referred to as the "target" in this manual).
- The target often equates to the first part of a .bb filename.
- So, to run the matchbox-desktop_1.2.3.bb file, you
- might type the following:
-
- $ bitbake matchbox-desktop
-
- Several different versions of matchbox-desktop might exist.
- BitBake chooses the one selected by the distribution configuration.
- You can get more details about how BitBake chooses between different
- target versions and providers in the
- Preferences and Providers section.
-
-
-
- BitBake also tries to execute any dependent tasks first.
- So for example, before building matchbox-desktop, BitBake
- would build a cross compiler and eglibc if they had not already
- been built.
- This release of the Yocto Project does not support the glibc
- GNU version of the Unix standard C library. By default, the Yocto Project builds with
- eglibc.
-
-
-
- A useful BitBake option to consider is the -k or
- --continue option.
- This option instructs BitBake to try and continue processing the job as much
- as possible even after encountering an error.
- When an error occurs, the target that
- failed and those that depend on it cannot be remade.
- However, when you use this option other dependencies can still be processed.
-
-
-
-
- Metadata (Recipes)
-
-
- The .bb files are usually referred to as "recipes."
- In general, a recipe contains information about a single piece of software.
- The information includes the location from which to download the source patches
- (if any are needed), which special configuration options to apply,
- how to compile the source files, and how to package the compiled output.
-
-
-
- The term "package" can also be used to describe recipes.
- However, since the same word is used for the packaged output from the Yocto
- Project (i.e. .ipk or .deb files),
- this document avoids using the term "package" to refer to recipes.
-
-
-
-
- Classes
-
-
- Class files (.bbclass) contain information that is useful to share
- between metadata files.
- An example is the Autotools class, which contains
- common settings for any application that Autotools uses.
- The Reference: Classes appendix provides details
- about common classes and how to use them.
-
-
-
-
- Configuration
-
-
- The configuration files (.conf) define various configuration variables
- that govern the Yocto Project build process.
- These files fall into several areas that define machine configuration options,
- distribution configuration options, compiler tuning options, general common configuration
- options and user configuration options (local.conf, which is found
- in the Yocto Project files build directory).
-
-
-
-
-
Running a Build
@@ -221,83 +80,6 @@
due to some components not being rebuilt.
-
-
- Considering Shared State Cache
-
-
- By design, the Yocto Project builds everything from scratch unless it can determine that
- a given task's inputs have not changed.
- While building from scratch ensures that everything is current, it does also
- mean that a lot of time could be spent rebuiding things that don't necessarily need built.
-
-
-
- The Yocto Project build process uses a shared state caching scheme to avoid having to
- rebuild software when it is not necessary.
- Because the build time for a Yocto image can be significant, it is helpful to try and
- determine what really needs built and what can be skipped given a particular project's
- build process.
-
-
-
- The scheme that the Yocto Project uses involves checksum generation and comparison for
- a task's inputs.
- The scheme also employs an area of memory called the shared state cache that is
- pointed to by the SSTATE_DIR variable.
- This area contains task output generated from a previous build.
- If a given task's checksum matches the checksum of a previous build for the same
- task, the build process uses the state of the cache rather than rerunning that
- task.
-
-
-
- The previous paragraph is a simplistic explanation of how the build process
- uses checksums and shared state memory cache to avoide building tasks that
- don't need built.
- If you want a bit more explanation on the topic,
- see "Shared
- State - What does it mean and why should I care?" from the Yocto
- Project discussion archives.
-
-
-
- As with all schemes, this one has some drawbacks.
- It is possible that you could make implicit changes that are not factored into the checksum
- calculation, but do affect a task's output.
- A good example is perhaps when a tool changes its output.
- Let's say that the output of rpmdeps needed to change.
- The result of the change should be that all the "package", "package_write_rpm",
- and "package_deploy-rpm" sstate-cache items would become invalid.
- But, because this is a change that is external to the code and therefore implicit,
- the associated sstate-cache items do not become invalidated.
- In this case, the build process would use the cache items rather than running the
- task again.
- Obviously, these types of implicit changes can cause problems.
-
-
-
- To avoid these problems during the build, you need to understand the effects of any
- change you make.
- Note that any changes you make directly to a function automatically are factored into
- the checksum calculation and thus, will invalidate the associated area of sstate cache.
- You need to be aware of any implicit changes that are not obvious changes to the
- code and could affect the output of a given task.
- Once you are aware of such a change, you can take steps to invalidate the cache
- and force the task to run.
- The step to take is as simple as changing a function's comments in the source code.
- For example, to invalidate package sstate files, change the comment statments
- of do_package or one of the functions it calls.
- The change is purely cosmetic, but it causes the checksum to be recalculated and
- forces the task to be run again.
-
-
-
- For an example of a commit that makes a cosmetic change to invalidate an sstate,
- see this
- commit.
-
-