2013-01-11 17:45:15 +00:00
|
|
|
<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
|
|
|
|
"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd">
|
2014-01-17 15:00:48 +00:00
|
|
|
|
|
|
|
<chapter id="user-manual-intro">
|
2014-02-03 13:29:54 +00:00
|
|
|
<title>Overview</title>
|
2014-01-17 14:58:20 +00:00
|
|
|
|
2014-02-11 13:21:58 +00:00
|
|
|
<para>
|
|
|
|
Welcome to the BitBake User Manual.
|
|
|
|
This manual provides information on BitBake.
|
|
|
|
The information attempts to be as independent as possible regarding
|
|
|
|
systems that use BitBake, such as the Yocto Project and
|
|
|
|
OpenEmbeddeded.
|
|
|
|
In some cases, scenarios or examples that within the context of
|
|
|
|
a build system are used in the manual to help with understanding.
|
|
|
|
For these cases, the manual clearly states the context.
|
|
|
|
</para>
|
|
|
|
|
2014-01-17 14:58:20 +00:00
|
|
|
<section id="intro">
|
2013-01-11 17:45:15 +00:00
|
|
|
<title>Introduction</title>
|
2014-01-17 15:00:48 +00:00
|
|
|
|
|
|
|
<para>
|
2014-02-18 13:49:12 +00:00
|
|
|
Fundamentally, BitBake is a generic task execution
|
2014-01-14 00:34:16 +00:00
|
|
|
engine that allows shell and Python tasks to be run
|
|
|
|
efficiently and in parallel while working within
|
|
|
|
complex inter-task dependency constraints.
|
|
|
|
One of BitBake's main users, OpenEmbedded, takes this core
|
|
|
|
and builds embedded Linux software stacks using
|
|
|
|
a task-oriented approach.
|
|
|
|
</para>
|
|
|
|
|
|
|
|
<para>
|
|
|
|
Conceptually, BitBake is similar to GNU Make in
|
|
|
|
some regards but has significant differences:
|
|
|
|
<itemizedlist>
|
|
|
|
<listitem><para>
|
|
|
|
BitBake executes tasks according to provided
|
|
|
|
metadata that builds up the tasks.
|
|
|
|
Metadata is stored in recipe (<filename>.bb</filename>),
|
|
|
|
configuration (<filename>.conf</filename>), and class
|
|
|
|
(<filename>.bbclass</filename>) files and provides
|
|
|
|
BitBake with instructions on what tasks to run and
|
|
|
|
the dependencies between those tasks.
|
|
|
|
</para></listitem>
|
|
|
|
<listitem><para>
|
|
|
|
BitBake includes a fetcher library for obtaining source
|
|
|
|
code from various places such as source control
|
|
|
|
systems or websites.
|
|
|
|
</para></listitem>
|
|
|
|
<listitem><para>
|
|
|
|
The instructions for each unit to be built (e.g. a piece
|
|
|
|
of software) are known as recipe files and
|
|
|
|
contain all the information about the unit
|
|
|
|
(dependencies, source file locations, checksums, description
|
|
|
|
and so on).
|
|
|
|
</para></listitem>
|
|
|
|
<listitem><para>
|
|
|
|
BitBake includes a client/server abstraction and can
|
|
|
|
be used from a command line or used as a service over XMLRPC and
|
|
|
|
has several different user interfaces.
|
|
|
|
</para></listitem>
|
|
|
|
</itemizedlist>
|
2014-01-17 14:58:20 +00:00
|
|
|
</para>
|
|
|
|
</section>
|
2014-01-17 15:00:48 +00:00
|
|
|
|
|
|
|
<section id="history-and-goals">
|
|
|
|
<title>History and Goals</title>
|
|
|
|
|
2014-01-17 14:58:20 +00:00
|
|
|
<para>
|
|
|
|
BitBake was originally a part of the OpenEmbedded project.
|
|
|
|
It was inspired by the Portage package management system
|
|
|
|
used by the Gentoo Linux distribution.
|
|
|
|
On December 7, 2004, OpenEmbedded project team member,
|
|
|
|
Chris Larson split the project into two distinct pieces:
|
|
|
|
<itemizedlist>
|
|
|
|
<listitem><para>BitBake, a generic task executor</para></listitem>
|
|
|
|
<listitem><para>OpenEmbedded, a metadata set utilized by
|
2014-02-03 13:29:54 +00:00
|
|
|
BitBake</para></listitem>
|
2014-01-17 14:58:20 +00:00
|
|
|
</itemizedlist>
|
|
|
|
Today, BitBake is the primary basis of the
|
|
|
|
<ulink url="http://www.openembedded.org/">OpenEmbedded</ulink>
|
2014-02-18 13:49:12 +00:00
|
|
|
project, which is being used to build and maintain Linux
|
|
|
|
distributions such as the Angstrom Distribution and which is used
|
|
|
|
as the build tool for Linux projects such as the Yocto Project.
|
2014-01-17 14:58:20 +00:00
|
|
|
</para>
|
2014-01-17 15:00:48 +00:00
|
|
|
|
|
|
|
<para>
|
|
|
|
Prior to BitBake, no other build tool adequately met the needs of
|
|
|
|
an aspiring embedded Linux distribution.
|
|
|
|
All of the build systems used by traditional desktop Linux
|
|
|
|
distributions lacked important functionality, and none of the
|
2014-01-17 15:15:24 +00:00
|
|
|
ad-hoc buildroot systems, prevalent in the
|
2014-01-17 15:00:48 +00:00
|
|
|
embedded space, were scalable or maintainable.
|
|
|
|
</para>
|
|
|
|
|
|
|
|
<para>
|
|
|
|
Some important original goals for BitBake were:
|
2013-01-11 17:45:15 +00:00
|
|
|
<itemizedlist>
|
2014-01-17 15:00:48 +00:00
|
|
|
<listitem><para>
|
|
|
|
Handle cross-compilation.
|
|
|
|
</para></listitem>
|
|
|
|
<listitem><para>
|
|
|
|
Handle inter-package dependencies (build time on
|
|
|
|
target architecture, build time on native
|
|
|
|
architecture, and runtime).
|
|
|
|
</para></listitem>
|
|
|
|
<listitem><para>
|
|
|
|
Support running any number of tasks within a given
|
|
|
|
package, including, but not limited to, fetching
|
|
|
|
upstream sources, unpacking them, patching them,
|
2014-02-03 13:29:54 +00:00
|
|
|
configuring them, and so forth.
|
2014-01-17 15:00:48 +00:00
|
|
|
</para></listitem>
|
|
|
|
<listitem><para>
|
2014-02-03 13:29:54 +00:00
|
|
|
Be Linux distribution agnostic for both build and
|
|
|
|
target systems.
|
2014-01-17 15:00:48 +00:00
|
|
|
</para></listitem>
|
|
|
|
<listitem><para>
|
2014-02-03 13:29:54 +00:00
|
|
|
Be architecture agnostic.
|
2014-01-17 15:00:48 +00:00
|
|
|
</para></listitem>
|
|
|
|
<listitem><para>
|
2014-02-03 13:29:54 +00:00
|
|
|
Support multiple build and target operating systems
|
|
|
|
(e.g. Cygwin, the BSDs, and so forth).
|
2014-01-17 15:00:48 +00:00
|
|
|
</para></listitem>
|
|
|
|
<listitem><para>
|
2014-02-03 13:29:54 +00:00
|
|
|
Be self contained, rather than tightly
|
2014-01-17 15:00:48 +00:00
|
|
|
integrated into the build machine's root
|
|
|
|
filesystem.
|
|
|
|
</para></listitem>
|
|
|
|
<listitem><para>
|
2014-02-03 13:29:54 +00:00
|
|
|
Handle conditional metadata on the target architecture,
|
|
|
|
operating system, distribution, and machine.
|
2014-01-17 15:00:48 +00:00
|
|
|
</para></listitem>
|
|
|
|
<listitem><para>
|
2014-02-03 13:29:54 +00:00
|
|
|
Be easy to use the tools to supply local metadata and packages
|
|
|
|
against which to operate.
|
2014-01-17 15:00:48 +00:00
|
|
|
</para></listitem>
|
|
|
|
<listitem><para>
|
2014-02-03 13:29:54 +00:00
|
|
|
Be easy to use BitBake to collaborate between multiple
|
|
|
|
projects for their builds.
|
2014-01-17 15:00:48 +00:00
|
|
|
</para></listitem>
|
|
|
|
<listitem><para>
|
2014-02-03 13:29:54 +00:00
|
|
|
Provide an inheritance mechanism that share
|
2014-01-17 15:00:48 +00:00
|
|
|
common metadata between many packages.
|
|
|
|
</para></listitem>
|
2013-01-11 17:45:15 +00:00
|
|
|
</itemizedlist>
|
2014-02-03 13:29:54 +00:00
|
|
|
Over time it became apparent that some further requirements
|
2014-01-17 15:00:48 +00:00
|
|
|
were necessary:
|
2013-01-11 17:45:15 +00:00
|
|
|
<itemizedlist>
|
2014-01-17 15:00:48 +00:00
|
|
|
<listitem><para>
|
2014-02-03 13:29:54 +00:00
|
|
|
Handle variants of a base recipe (e.g. native, sdk,
|
|
|
|
and multilib).
|
2014-01-17 15:00:48 +00:00
|
|
|
</para></listitem>
|
|
|
|
<listitem><para>
|
2014-02-03 13:29:54 +00:00
|
|
|
Split metadata into layers and allow layers
|
2014-01-17 15:00:48 +00:00
|
|
|
to override each other.
|
|
|
|
</para></listitem>
|
|
|
|
<listitem><para>
|
|
|
|
Allow representation of a given set of input variables
|
|
|
|
to a task as a checksum.
|
|
|
|
Based on that checksum, allow acceleration of builds
|
|
|
|
with prebuilt components.
|
|
|
|
</para></listitem>
|
2013-01-11 17:45:15 +00:00
|
|
|
</itemizedlist>
|
2014-01-17 15:00:48 +00:00
|
|
|
BitBake satisfies all the original requirements and many more
|
|
|
|
with extensions being made to the basic functionality to
|
|
|
|
reflect the additional requirements.
|
|
|
|
Flexibility and power have always been the priorities.
|
2014-02-03 13:29:54 +00:00
|
|
|
BitBake is highly extensible and supports embedded Python code and
|
2014-01-17 15:00:48 +00:00
|
|
|
execution of any arbitrary tasks.
|
|
|
|
</para>
|
2014-01-13 23:35:00 +00:00
|
|
|
</section>
|
|
|
|
|
|
|
|
<section id="Concepts">
|
|
|
|
<title>Concepts</title>
|
|
|
|
|
|
|
|
<para>
|
|
|
|
BitBake is a program written in the Python language.
|
|
|
|
At the highest level, BitBake interprets metadata, decides
|
|
|
|
what tasks are required to run, and executes those tasks.
|
|
|
|
Similar to GNU Make, BitBake controls how software is
|
|
|
|
built.
|
2014-02-03 13:29:54 +00:00
|
|
|
GNU Make achieves its control through "makefiles".
|
|
|
|
BitBake uses "recipes".
|
|
|
|
</para>
|
|
|
|
|
|
|
|
<para>
|
2014-01-13 23:35:00 +00:00
|
|
|
BitBake extends the capabilities of a simple
|
2014-02-03 13:29:54 +00:00
|
|
|
tool like GNU Make by allowing for much more complex tasks
|
2014-01-13 23:35:00 +00:00
|
|
|
to be completed, such as assembling entire embedded Linux
|
2014-02-03 13:29:54 +00:00
|
|
|
distributions.
|
|
|
|
</para>
|
|
|
|
|
|
|
|
<para>
|
|
|
|
The remainder of this section introduces several concepts
|
|
|
|
that should be understood in order to better leverage
|
|
|
|
the power of BitBake.
|
2014-01-13 23:35:00 +00:00
|
|
|
</para>
|
|
|
|
|
|
|
|
<section id='recipes'>
|
|
|
|
<title>Recipes</title>
|
|
|
|
|
|
|
|
<para>
|
2014-02-03 13:29:54 +00:00
|
|
|
BitBake Recipes, which are denoted by the file extension
|
|
|
|
<filename>.bb</filename>, are the most basic metadata files.
|
2014-02-18 13:49:12 +00:00
|
|
|
These recipe files provide BitBake with the following:
|
2014-01-13 23:35:00 +00:00
|
|
|
<itemizedlist>
|
2014-01-17 15:15:24 +00:00
|
|
|
<listitem><para>Descriptive information about the package</para></listitem>
|
|
|
|
<listitem><para>The version of the recipe</para></listitem>
|
2014-02-18 13:49:12 +00:00
|
|
|
<listitem><para>Existing Dependencies</para></listitem>
|
2014-01-17 15:15:24 +00:00
|
|
|
<listitem><para>Where the source code resides</para></listitem>
|
|
|
|
<listitem><para>Whether the source code requires any patches</para></listitem>
|
|
|
|
<listitem><para>How to compile the source code</para></listitem>
|
2014-02-03 13:29:54 +00:00
|
|
|
<listitem><para>Where on the target machine to install the
|
|
|
|
package being compiled</para></listitem>
|
2014-01-13 23:35:00 +00:00
|
|
|
</itemizedlist>
|
|
|
|
</para>
|
|
|
|
|
|
|
|
<para>
|
|
|
|
Within the context of BitBake, or any project utilizing BitBake
|
2014-01-17 15:15:24 +00:00
|
|
|
as it's build system, files with the <filename>.bb</filename>
|
2014-02-03 13:29:54 +00:00
|
|
|
extension are referred to as recipes.
|
|
|
|
<note>
|
|
|
|
The term "package" is also commonly used to describe recipes.
|
|
|
|
However, since the same word is used to describe packaged
|
|
|
|
output from a project, it is best to maintain a single
|
|
|
|
descriptive term, "recipes".
|
|
|
|
</note>
|
2014-01-13 23:35:00 +00:00
|
|
|
</para>
|
|
|
|
</section>
|
|
|
|
|
|
|
|
<section id='configuration-files'>
|
|
|
|
<title>Configuration Files</title>
|
|
|
|
|
|
|
|
<para>
|
2014-02-03 13:29:54 +00:00
|
|
|
Configuration files, which are denoted by the
|
|
|
|
<filename>.conf</filename> extension, define
|
|
|
|
various configuration variables that govern the project's build
|
2014-01-13 23:35:00 +00:00
|
|
|
process.
|
|
|
|
These files fall into several areas that define
|
|
|
|
machine configuration options, distribution configuration
|
|
|
|
options, compiler tuning options, general common
|
2014-02-03 13:29:54 +00:00
|
|
|
configuration options, and user configuration options.
|
2014-01-17 15:15:24 +00:00
|
|
|
The main configuration file is the sample
|
2014-02-03 13:29:54 +00:00
|
|
|
<filename>bitbake.conf</filename> file, which is
|
|
|
|
located within the BitBake source tree
|
|
|
|
<filename>conf</filename> directory.
|
2014-01-13 23:35:00 +00:00
|
|
|
</para>
|
|
|
|
</section>
|
|
|
|
|
|
|
|
<section id='classes'>
|
|
|
|
<title>Classes</title>
|
|
|
|
|
|
|
|
<para>
|
2014-02-03 13:29:54 +00:00
|
|
|
Class files, which are denoted by the
|
|
|
|
<filename>.bbclass</filename> extension, contain
|
2014-01-13 23:35:00 +00:00
|
|
|
information that is useful to share between metadata files.
|
2014-02-03 13:29:54 +00:00
|
|
|
The BitBake source tree currently comes with one class metadata file
|
|
|
|
called <filename>base.bbclass</filename>.
|
|
|
|
You can find this file in the
|
|
|
|
<filename>classes</filename> directory.
|
2014-02-18 13:49:12 +00:00
|
|
|
The <filename>base.bbclass</filename> is special since it
|
|
|
|
is always included automatically for all recipes
|
|
|
|
and classes.
|
2014-01-13 23:35:00 +00:00
|
|
|
This class contains definitions for standard basic tasks such
|
|
|
|
as fetching, unpacking, configuring (empty by default),
|
|
|
|
compiling (runs any Makefile present), installing (empty by
|
|
|
|
default) and packaging (empty by default).
|
2014-02-03 13:29:54 +00:00
|
|
|
These tasks are often overridden or extended by other classes
|
2014-01-13 23:35:00 +00:00
|
|
|
added during the project development process.
|
|
|
|
</para>
|
|
|
|
</section>
|
2014-02-14 19:55:03 +00:00
|
|
|
|
|
|
|
<section id='layers'>
|
|
|
|
<title>Layers</title>
|
|
|
|
|
|
|
|
<para>
|
|
|
|
Layers allow you to isolate different types of
|
|
|
|
customizations from each other.
|
|
|
|
While you might find it tempting to keep everything in one layer
|
|
|
|
when working on a single project, the more modular you organize
|
|
|
|
your metadata, the easier it is to cope with future changes.
|
|
|
|
</para>
|
|
|
|
|
|
|
|
<para>
|
|
|
|
To illustrate how you can use layers to keep things modular,
|
|
|
|
consider machine customizations.
|
|
|
|
These types of customizations typically reside in a special layer,
|
|
|
|
rather than a general layer, called a Board Specific Package (BSP) Layer.
|
|
|
|
Furthermore, the machine customizations should be isolated from
|
|
|
|
recipes and metadata that support a new GUI environment, for
|
|
|
|
example.
|
|
|
|
This situation gives you a couple of layers: one for the machine
|
|
|
|
configurations and one for the GUI environment.
|
|
|
|
It is important to understand, however, that the BSP layer can still
|
|
|
|
make machine-specific additions to recipes within
|
|
|
|
the GUI environment layer without polluting the GUI layer itself
|
|
|
|
with those machine-specific changes.
|
|
|
|
You can accomplish this through a recipe that is a BitBake append
|
|
|
|
(<filename>.bbappend</filename>) file.
|
|
|
|
</para>
|
|
|
|
</section>
|
2014-02-18 13:49:12 +00:00
|
|
|
|
|
|
|
<section id='append-bbappend-files'>
|
|
|
|
<title>Append Files</title>
|
|
|
|
|
|
|
|
<para>
|
|
|
|
Append files, which are files that have the
|
|
|
|
<filename>.bbappend</filename> file extension, add or
|
|
|
|
extend build information to an existing
|
|
|
|
recipe file.
|
|
|
|
</para>
|
|
|
|
|
|
|
|
<para>
|
|
|
|
BitBake expects every append file to have a corresponding recipe file.
|
|
|
|
Furthermore, the append file and corresponding recipe file
|
|
|
|
must use the same root filename.
|
|
|
|
The filenames can differ only in the file type suffix used
|
|
|
|
(e.g. <filename>formfactor_0.0.bb</filename> and
|
|
|
|
<filename>formfactor_0.0.bbappend</filename>).
|
|
|
|
</para>
|
|
|
|
|
|
|
|
<para>
|
|
|
|
Information in append files overrides the information in the
|
|
|
|
similarly-named recipe file.
|
|
|
|
</para>
|
|
|
|
</section>
|
2014-01-13 23:35:00 +00:00
|
|
|
</section>
|
2014-01-17 15:07:11 +00:00
|
|
|
|
|
|
|
<section id='obtaining-bitbake'>
|
|
|
|
<title>Obtaining BitBake</title>
|
|
|
|
|
|
|
|
<para>
|
2014-02-03 13:29:54 +00:00
|
|
|
You can obtain BitBake several different ways:
|
|
|
|
<itemizedlist>
|
|
|
|
<listitem><para><emphasis>Cloning BitBake:</emphasis>
|
|
|
|
Using Git to clone the BitBake source code repository
|
2014-02-18 13:49:12 +00:00
|
|
|
is the recommended method for obtaining BitBake.
|
|
|
|
Cloning the repository makes it easy to get bug fixes
|
|
|
|
and have access to stable branches and the master
|
|
|
|
branch.
|
|
|
|
Once you have cloned BitBake, you should use
|
|
|
|
the latest stable
|
|
|
|
branch for development since the master branch is for
|
|
|
|
BitBake development and might contain less stable changes.
|
|
|
|
</para>
|
|
|
|
<para>You usually need a version of BitBake
|
|
|
|
that matches the metadata you are using.
|
|
|
|
The metadata is generally backwards compatible but
|
|
|
|
not forward compatible.</para>
|
|
|
|
<para>Here is an example that clones the BitBake repository:
|
2014-02-03 13:29:54 +00:00
|
|
|
<literallayout class='monospaced'>
|
2014-01-17 15:07:11 +00:00
|
|
|
$ git clone git://git.openembedded.org/bitbake
|
2014-02-03 13:29:54 +00:00
|
|
|
</literallayout>
|
|
|
|
This command clones the BitBake Git repository into a
|
|
|
|
directory called <filename>bitbake</filename>.
|
|
|
|
Alternatively, you can
|
|
|
|
designate a directory after the
|
|
|
|
<filename>git clone</filename> command
|
|
|
|
if you want to call the new directory something
|
|
|
|
other than <filename>bitbake</filename>.
|
|
|
|
Here is an example that names the directory
|
|
|
|
<filename>bbdev</filename>:
|
|
|
|
<literallayout class='monospaced'>
|
2014-01-17 15:07:11 +00:00
|
|
|
$ git clone git://git.openembedded.org/bitbake bbdev
|
2014-02-03 13:29:54 +00:00
|
|
|
</literallayout></para></listitem>
|
2014-02-18 13:49:12 +00:00
|
|
|
<listitem><para><emphasis>Installation using your Distribution
|
|
|
|
Package Management System:</emphasis>
|
|
|
|
This method is not
|
|
|
|
recommended because the BitBake version that is
|
|
|
|
provided by your distribution, in most cases,
|
|
|
|
is several
|
|
|
|
releases behind a snapshot of the BitBake repository.
|
|
|
|
</para></listitem>
|
|
|
|
<listitem><para><emphasis>Taking a snapshot of BitBake:</emphasis>
|
|
|
|
Downloading a snapshot of BitBake from the
|
|
|
|
source code repository gives you access to a known
|
|
|
|
branch or release of BitBake.</para>
|
|
|
|
<para>The following example downloads a snapshot of
|
|
|
|
BitBake version 1.17.0:
|
|
|
|
<literallayout class='monospaced'>
|
|
|
|
$ wget http://git.openembedded.org/bitbake/snapshot/bitbake-1.17.0.tar.gz
|
|
|
|
$ tar zxpvf bitbake-1.17.0.tar.gz
|
|
|
|
</literallayout>
|
|
|
|
After extraction of the tarball using the tar utility,
|
|
|
|
you have a directory entitled
|
|
|
|
<filename>bitbake-1.17.0</filename>.
|
|
|
|
</para></listitem>
|
2014-02-03 13:29:54 +00:00
|
|
|
</itemizedlist>
|
2014-01-17 15:07:11 +00:00
|
|
|
</para>
|
|
|
|
</section>
|
2014-01-17 15:00:48 +00:00
|
|
|
</chapter>
|