diff --git a/documentation/poky-ref-manual/extendpoky.xml b/documentation/poky-ref-manual/extendpoky.xml index 30fce1459f..85366aecfc 100644 --- a/documentation/poky-ref-manual/extendpoky.xml +++ b/documentation/poky-ref-manual/extendpoky.xml @@ -3,13 +3,14 @@ -Extending the Yocto Project +Common Tasks - This chapter provides information about how to extend the functionality - already present in the Yocto Project. - The chapter also documents standard tasks such as adding new + This chapter describes standard tasks such as adding new software packages, extending or customizing images or porting the Yocto Project to new hardware (adding a new machine). + The chapter also describes ways to modify package source code, combine multiple + versions of library files into a single image, track license changes, and handle + a package name alias. Finally, the chapter contains advice about how to make changes to the Yocto Project to achieve the best results. @@ -658,324 +659,6 @@ -
- Making and Maintaining Changes - - Because the Yocto Project is extremely configurable and flexible, - we recognize that developers will want - to extend, configure or optimize it for their specific uses. - To best keep pace with future Yocto Project changes, - we recommend you make controlled changes to the Yocto Project. - - - - The Yocto Project supports a "layers" concept. - If you use layers properly, you can ease future upgrades and allow segregation - between the Yocto Project core and a given developer's changes. - The following section provides more advice on managing changes to the Yocto Project. - - -
- BitBake Layers - - Often, developers want to extend the Yocto Project either by adding packages - or by overriding files contained within the Yocto Project to add their own - functionality. - BitBake has a powerful mechanism called - "layers", which provides a way to handle this extension in a fully - supported and non-invasive fashion. - - - - The Yocto Project files include several additional layers such as - meta-rt and meta-yocto - that demonstrate this functionality. - The meta-rt layer is not enabled by default. - However, the meta-yocto layer is. - - - - To enable a layer, you simply add the layer's path to the - BBLAYERS variable in your - bblayers.conf file, which is found in the Yocto Project file's - build directory. - The following example shows how to enable the meta-rt: - - LCONF_VERSION = "1" - - BBFILES ?= "" - BBLAYERS = " \ - /path/to/poky/meta \ - /path/to/poky/meta-yocto \ - /path/to/poky/meta-rt \ - " - - - - - BitBake parses each conf/layer.conf file for each layer in - BBLAYERS - and adds the recipes, classes and configurations contained within the layer to - the Yocto Project. - To create your own layer, independent of the Yocto Project files, - simply create a directory with a conf/layer.conf file and - add the directory to your bblayers.conf file. - - - - The meta-yocto/conf/layer.conf file demonstrates the - required syntax: - - # We have a conf and classes directory, add to BBPATH - BBPATH := "${BBPATH}:${LAYERDIR}" - - # We have a packages directory, add to BBFILES - BBFILES := "${BBFILES} ${LAYERDIR}/recipes-*/*/*.bb \ - ${LAYERDIR}/recipes-*/*/*.bbappend" - - BBFILE_COLLECTIONS += "yocto" - BBFILE_PATTERN_yocto := "^${LAYERDIR}/" - BBFILE_PRIORITY_yocto = "5" - - - - - In the previous example, the recipes for the layers are added to - BBFILES. - The BBFILE_COLLECTIONS - variable is then appended with the layer name. - The BBFILE_PATTERN variable - immediately expands with a regular expression used to match files from - BBFILES into - a particular layer, in this case by using the base pathname. - The BBFILE_PRIORITY variable - then assigns different priorities to the files in different layers. - Applying priorities is useful in situations where the same package might appear in multiple - layers and allows you to choose what layer should take precedence. - - - - Note the use of the LAYERDIR - variable with the immediate expansion operator. - The LAYERDIR variable expands to the directory of the current layer and - requires the immediate expansion operator so that BitBake does not wait to expand the variable - when it's parsing a different directory. - - - - BitBake can locate where other .bbclass and configuration files - are applied through the BBPATH environment variable. - For these cases, BitBake uses the first file with the matching name found in - BBPATH. - This is similar to the way the PATH variable is used for binaries. - We recommend, therefore, that you use unique .bbclass - and configuration file names in your custom layer. - - - - We also recommend the following: - - Store custom layers in a Git repository that uses the - meta-prvt-XXXX format. - Clone the repository alongside other meta - directories in the Yocto Project source files area. - - Following these recommendations keeps your Yocto Project files area and - its configuration entirely inside the Yocto Project's core base. - -
- -
- Committing Changes - - - Modifications to the Yocto Project are often managed under some kind of source - revision control system. - Because some simple practices can significantly improve usability, policy for committing changes - is important. - It helps to use a consistent documentation style when committing changes. - The Yocto Project development team has found the following practices work well: - - The first line of the commit summarizes the change and begins with the - name of the affected package or packages. - However, not all changes apply to specific packages. - Consequently, the prefix could also be a machine name or class name. - The second part of the commit (if needed) is a longer more detailed - description of the changes. - Placing a blank line between the first and second parts helps with - readability. - - - - - Following is an example commit: - - bitbake/data.py: Add emit_func() and generate_dependencies() functions - - These functions allow generation of dependency data between functions and - variables allowing moves to be made towards generating checksums and allowing - use of the dependency information in other parts of BitBake. - - Signed-off-by: Richard Purdie richard.purdie@linuxfoundation.org - - - - - All commits should be self-contained such that they leave the - metadata in a consistent state that builds both before and after the - commit is made. - Besides being a good practice to follow, it helps ensure autobuilder test results - are valid. - -
- -
- Package Revision Incrementing - - - If a committed change results in changing the package output, - then the value of the - PR variable needs to be increased - (or "bumped") as part of that commit. - This means that for new recipes you must be sure to add the PR - variable and set its initial value equal to "r0". - Failing to define PR makes it easy to miss when you bump a package. - Note that you can only use integer values following the "r" in the - PR variable. - - - - If you are sharing a common .inc file with multiple recipes, - you can also use the - INC_PR variable to ensure that - the recipes sharing the .inc file are rebuilt when the - .inc file itself is changed. - The .inc file must set INC_PR - (initially to "r0"), and all recipes referring to it should set PR - to "$(INC_PR).0" initially, incrementing the last number when the recipe is changed. - If the .inc file is changed then its - INC_PR should be incremented. - - - - When upgrading the version of a package, assuming the - PV changes, - the PR variable should be reset to "r0" - (or "$(INC_PR).0" if you are using INC_PR). - - - - Usually, version increases occur only to packages. - However, if for some reason PV changes but does not - increase, you can increase the - PE variable (Package Epoch). - The PE variable defaults to "0". - - - - Version numbering strives to follow the - - Debian Version Field Policy Guidelines. - These guidelines define how versions are compared and what "increasing" a version means. - - - - There are two reasons for following the previously mentioned guidelines. - First, to ensure that when a developer updates and rebuilds, they get all the changes to - the repository and do not have to remember to rebuild any sections. - Second, to ensure that target users are able to upgrade their - devices using package manager commands such as opkg upgrade - (or similar commands for dpkg/apt or rpm-based systems). - - - - The goal is to ensure the Yocto Project has packages that can be upgraded in all cases. - -
- -
- Using The Yocto Project in a Team Environment - - - It might not be immediately clear how you can use the Yocto Project in a team environment, - or scale it for a large team of developers. - The specifics of any situation determine the best solution. - Granted that the Yocto Project offers immense flexibility regarding this, practices do exist - that experience has shown work well. - - - - The core component of any development effort with the Yocto Project is often an - automated build and testing framework along with an image generation process. - You can use these core components to check that the metadata can be built, - highlight when commits break the build, and provide up-to-date images that - allow developers to test the end result and use it as a base platform for further - development. - Experience shows that buildbot is a good fit for this role. - What works well is to configure buildbot to make two types of builds: - incremental and full (from scratch). - See the buildbot for the - Yocto Project for an example implementation that uses buildbot. - - - - You can tie incremental builds to a commit hook that triggers the build - each time a commit is made to the metadata. - This practice results in useful acid tests that determine whether a given commit - breaks the build in some serious way. - Associating a build to a commit can catch a lot of simple errors. - Furthermore, the tests are fast so developers can get quick feedback on changes. - - - - Full builds build and test everything from the ground up. - These types of builds usually happen at predetermined times like during the - night when the machine load is low. - - - - Most teams have many pieces of software undergoing active development at any given time. - You can derive large benefits by putting these pieces under the control of a source - control system that is compatible with the Yocto Project (i.e. Git or Subversion (SVN). - You can then set the autobuilder to pull the latest revisions of the packages - and test the latest commits by the builds. - This practice quickly highlights issues. - The Yocto Project easily supports testing configurations that use both a - stable known good revision and a floating revision. - The Yocto Project can also take just the changes from specific source control branches. - This capability allows you to track and test specific changes. - - - - Perhaps the hardest part of setting this up is defining the software project or - the Yocto Project metadata policies that surround the different source control systems. - Of course circumstances will be different in each case. - However, this situation reveals one of the Yocto Project's advantages - - the system itself does not - force any particular policy on users, unlike a lot of build systems. - The system allows the best policies to be chosen for the given circumstances. - -
- -
- Updating Existing Images - - - Often, rather than re-flashing a new image, you might wish to install updated - packages into an existing running system. - You can do this by first sharing the tmp/deploy/ipk/ directory - through a web server and then by changing /etc/opkg/base-feeds.conf - to point at the shared server. - Following is an example: - - $ src/gz all http://www.mysite.com/somedir/deploy/ipk/all - $ src/gz armv7a http://www.mysite.com/somedir/deploy/ipk/armv7a - $ src/gz beagleboard http://www.mysite.com/somedir/deploy/ipk/beagleboard - - -
-
-
Modifying Package Source Code @@ -1300,7 +983,7 @@ important files that contain the license text for the source code. It is possible to specify a checksum for an entire file, or a specific section of a file (specified by beginning and ending line numbers with the "beginline" and "endline" - parameters respectively). + parameters, respectively). The latter is useful for source files with a license notice header, README documents, and so forth. If you do not use the "beginline" parameter, then it is assumed that the text begins on the @@ -1381,6 +1064,325 @@ file found in the Yocto Project files area.
+ +
+ Making and Maintaining Changes + + Because the Yocto Project is extremely configurable and flexible, + we recognize that developers will want + to extend, configure or optimize it for their specific uses. + To best keep pace with future Yocto Project changes, + we recommend you make controlled changes to the Yocto Project. + + + + The Yocto Project supports a "layers" concept. + If you use layers properly, you can ease future upgrades and allow segregation + between the Yocto Project core and a given developer's changes. + The following section provides more advice on managing changes to the Yocto Project. + + +
+ BitBake Layers + + Often, developers want to extend the Yocto Project either by adding packages + or by overriding files contained within the Yocto Project to add their own + functionality. + BitBake has a powerful mechanism called + "layers", which provides a way to handle this extension in a fully + supported and non-invasive fashion. + + + + The Yocto Project files include several additional layers such as + meta-rt and meta-yocto + that demonstrate this functionality. + The meta-rt layer is not enabled by default. + However, the meta-yocto layer is. + + + + To enable a layer, you simply add the layer's path to the + BBLAYERS variable in your + bblayers.conf file, which is found in the Yocto Project file's + build directory. + The following example shows how to enable the meta-rt: + + LCONF_VERSION = "1" + + BBFILES ?= "" + BBLAYERS = " \ + /path/to/poky/meta \ + /path/to/poky/meta-yocto \ + /path/to/poky/meta-rt \ + " + + + + + BitBake parses each conf/layer.conf file for each layer in + BBLAYERS + and adds the recipes, classes and configurations contained within the layer to + the Yocto Project. + To create your own layer, independent of the Yocto Project files, + simply create a directory with a conf/layer.conf file and + add the directory to your bblayers.conf file. + + + + The meta-yocto/conf/layer.conf file demonstrates the + required syntax: + + # We have a conf and classes directory, add to BBPATH + BBPATH := "${BBPATH}:${LAYERDIR}" + + # We have a packages directory, add to BBFILES + BBFILES := "${BBFILES} ${LAYERDIR}/recipes-*/*/*.bb \ + ${LAYERDIR}/recipes-*/*/*.bbappend" + + BBFILE_COLLECTIONS += "yocto" + BBFILE_PATTERN_yocto := "^${LAYERDIR}/" + BBFILE_PRIORITY_yocto = "5" + + + + + In the previous example, the recipes for the layers are added to + BBFILES. + The BBFILE_COLLECTIONS + variable is then appended with the layer name. + The BBFILE_PATTERN variable + immediately expands with a regular expression used to match files from + BBFILES into + a particular layer, in this case by using the base pathname. + The BBFILE_PRIORITY variable + then assigns different priorities to the files in different layers. + Applying priorities is useful in situations where the same package might appear in multiple + layers and allows you to choose what layer should take precedence. + + + + Note the use of the LAYERDIR + variable with the immediate expansion operator. + The LAYERDIR variable expands to the directory of the current layer and + requires the immediate expansion operator so that BitBake does not wait to expand the variable + when it's parsing a different directory. + + + + BitBake can locate where other .bbclass and configuration files + are applied through the BBPATH environment variable. + For these cases, BitBake uses the first file with the matching name found in + BBPATH. + This is similar to the way the PATH variable is used for binaries. + We recommend, therefore, that you use unique .bbclass + and configuration file names in your custom layer. + + + + We also recommend the following: + + Store custom layers in a Git repository that uses the + meta-prvt-XXXX format. + Clone the repository alongside other meta + directories in the Yocto Project source files area. + + Following these recommendations keeps your Yocto Project files area and + its configuration entirely inside the Yocto Project's core base. + +
+ +
+ Committing Changes + + + Modifications to the Yocto Project are often managed under some kind of source + revision control system. + Because some simple practices can significantly improve usability, policy for committing changes + is important. + It helps to use a consistent documentation style when committing changes. + The Yocto Project development team has found the following practices work well: + + The first line of the commit summarizes the change and begins with the + name of the affected package or packages. + However, not all changes apply to specific packages. + Consequently, the prefix could also be a machine name or class name. + The second part of the commit (if needed) is a longer more detailed + description of the changes. + Placing a blank line between the first and second parts helps with + readability. + + + + + Following is an example commit: + + bitbake/data.py: Add emit_func() and generate_dependencies() functions + + These functions allow generation of dependency data between functions and + variables allowing moves to be made towards generating checksums and allowing + use of the dependency information in other parts of BitBake. + + Signed-off-by: Richard Purdie richard.purdie@linuxfoundation.org + + + + + All commits should be self-contained such that they leave the + metadata in a consistent state that builds both before and after the + commit is made. + Besides being a good practice to follow, it helps ensure autobuilder test results + are valid. + +
+ +
+ Package Revision Incrementing + + + If a committed change results in changing the package output, + then the value of the + PR variable needs to be increased + (or "bumped") as part of that commit. + This means that for new recipes you must be sure to add the PR + variable and set its initial value equal to "r0". + Failing to define PR makes it easy to miss when you bump a package. + Note that you can only use integer values following the "r" in the + PR variable. + + + + If you are sharing a common .inc file with multiple recipes, + you can also use the + INC_PR variable to ensure that + the recipes sharing the .inc file are rebuilt when the + .inc file itself is changed. + The .inc file must set INC_PR + (initially to "r0"), and all recipes referring to it should set PR + to "$(INC_PR).0" initially, incrementing the last number when the recipe is changed. + If the .inc file is changed then its + INC_PR should be incremented. + + + + When upgrading the version of a package, assuming the + PV changes, + the PR variable should be reset to "r0" + (or "$(INC_PR).0" if you are using INC_PR). + + + + Usually, version increases occur only to packages. + However, if for some reason PV changes but does not + increase, you can increase the + PE variable (Package Epoch). + The PE variable defaults to "0". + + + + Version numbering strives to follow the + + Debian Version Field Policy Guidelines. + These guidelines define how versions are compared and what "increasing" a version means. + + + + There are two reasons for following the previously mentioned guidelines. + First, to ensure that when a developer updates and rebuilds, they get all the changes to + the repository and do not have to remember to rebuild any sections. + Second, to ensure that target users are able to upgrade their + devices using package manager commands such as opkg upgrade + (or similar commands for dpkg/apt or rpm-based systems). + + + + The goal is to ensure the Yocto Project has packages that can be upgraded in all cases. + +
+ +
+ Using The Yocto Project in a Team Environment + + + It might not be immediately clear how you can use the Yocto Project in a team environment, + or scale it for a large team of developers. + The specifics of any situation determine the best solution. + Granted that the Yocto Project offers immense flexibility regarding this, practices do exist + that experience has shown work well. + + + + The core component of any development effort with the Yocto Project is often an + automated build and testing framework along with an image generation process. + You can use these core components to check that the metadata can be built, + highlight when commits break the build, and provide up-to-date images that + allow developers to test the end result and use it as a base platform for further + development. + Experience shows that buildbot is a good fit for this role. + What works well is to configure buildbot to make two types of builds: + incremental and full (from scratch). + See the buildbot for the + Yocto Project for an example implementation that uses buildbot. + + + + You can tie incremental builds to a commit hook that triggers the build + each time a commit is made to the metadata. + This practice results in useful acid tests that determine whether a given commit + breaks the build in some serious way. + Associating a build to a commit can catch a lot of simple errors. + Furthermore, the tests are fast so developers can get quick feedback on changes. + + + + Full builds build and test everything from the ground up. + These types of builds usually happen at predetermined times like during the + night when the machine load is low. + + + + Most teams have many pieces of software undergoing active development at any given time. + You can derive large benefits by putting these pieces under the control of a source + control system that is compatible with the Yocto Project (i.e. Git or Subversion (SVN). + You can then set the autobuilder to pull the latest revisions of the packages + and test the latest commits by the builds. + This practice quickly highlights issues. + The Yocto Project easily supports testing configurations that use both a + stable known good revision and a floating revision. + The Yocto Project can also take just the changes from specific source control branches. + This capability allows you to track and test specific changes. + + + + Perhaps the hardest part of setting this up is defining the software project or + the Yocto Project metadata policies that surround the different source control systems. + Of course circumstances will be different in each case. + However, this situation reveals one of the Yocto Project's advantages - + the system itself does not + force any particular policy on users, unlike a lot of build systems. + The system allows the best policies to be chosen for the given circumstances. + +
+ +
+ Updating Existing Images + + + Often, rather than re-flashing a new image, you might wish to install updated + packages into an existing running system. + You can do this by first sharing the tmp/deploy/ipk/ directory + through a web server and then by changing /etc/opkg/base-feeds.conf + to point at the shared server. + Following is an example: + + $ src/gz all http://www.mysite.com/somedir/deploy/ipk/all + $ src/gz armv7a http://www.mysite.com/somedir/deploy/ipk/armv7a + $ src/gz beagleboard http://www.mysite.com/somedir/deploy/ipk/beagleboard + + +
+
+