sysmo-bts
/
generic-poky
9
0
Fork 0
Code Pull Requests Releases Activity
generic-poky/documentation/kernel-manual/kernel-how-to.xml

2179 lines
94 KiB
XML
Raw Blame History

<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd">
<chapter id='kernel-how-to'>
<title>Working with the Yocto Project Kernel</title>
<section id='actions-org'>
<title>Introduction</title>
<para>
This chapter describes how to accomplish tasks involving the kernel's tree structure.
The information covers the following:
<itemizedlist>
<listitem><para>Tree construction</para></listitem>
<listitem><para>Build strategies</para></listitem>
<!-- <listitem><para>Series &amp; Configuration Compiler</para></listitem>
<listitem><para>kgit</para></listitem> -->
<listitem><para>Workflow examples</para></listitem>
<!-- <listitem><para>Source Code Manager (SCM)</para></listitem>
<listitem><para>Board Support Package (BSP) template migration</para></listitem>
<listitem><para>BSP creation</para></listitem>
<listitem><para>Patching</para></listitem>
<listitem><para>Updating BSP patches and configuration</para></listitem>
<listitem><para>guilt</para></listitem>
<listitem><para>scc file example</para></listitem>
<listitem><para>"dirty" string</para></listitem>
<listitem><para>Transition kernel layer</para></listitem> -->
</itemizedlist>
</para>
</section>
<section id='tree-construction'>
<title>Tree Construction</title>
<para>
The Yocto Project kernel repository, as shipped with the product, is created by
compiling and executing the set of feature descriptions for every BSP/feature
in the product.
Those feature descriptions list all necessary patches,
configuration, branching, tagging and feature divisions found in the kernel.
</para>
<para>
You can find the files used to describe all the valid features and BSPs in the Yocto Project
kernel in any clone of the kernel git tree.
The directory <filename>meta/cfg/kernel-cache/</filename> is a snapshot of all the kernel
configuration and feature descriptions (.scc) used to build the kernel repository.
You should realize, however, that browsing the snapshot of feature
descriptions and patches is not an effective way to determine what is in a
particular kernel branch.
Instead, you should use git directly to discover the changes
in a branch.
Using git is a efficient and flexible way to inspect changes to the kernel.
For examples showing how to use git to inspect kernel commits, see the following sections
in this chapter.
</para>
<note><para>
Ground up reconstruction of the complete kernel tree is an action only taken by the
Yocto Project team during an active development cycle.
Creating a project simply clones this tree to make it efficiently available for building
and development.
</para></note>
<para>
The general flow for constructing a project-specific kernel tree is as follows:
<orderedlist>
<listitem><para>A top-level kernel feature is passed to the kernel build subsystem.
Normally, this is a BSP for a particular kernel type.</para></listitem>
<listitem><para>The file that describes the top-level feature is located by searching
these system directories:</para>
<itemizedlist>
<listitem><para>The in-tree kernel-cache directories</para></listitem>
<!-- <listitem><para>kernel-*-cache directories in layers</para></listitem> -->
<listitem><para>Recipe SRC_URIs</para></listitem>
<!-- <listitem><para>configured and default templates</para></listitem> -->
</itemizedlist>
<para>For a typical build a feature description of the format:
&lt;bsp name&gt;-&lt;kernel type&gt;.scc is the target of the search.
</para></listitem>
<listitem><para>Once located, the feature description is either compiled into a simple script
of actions, or an existing equivalent script that was part of the
shipped kernel is located.</para></listitem>
<listitem><para>Extra features are appended to the top-level feature description.
These features can come from the KERNEL_FEATURES variable in recipes.</para></listitem>
<listitem><para>Each extra feature is located, compiled and appended to the script from
step #3</para></listitem>
<listitem><para>The script is executed, and a meta-series is produced.
The meta-series is a description of all the branches, tags, patches and configuration that
needs to be applied to the base git repository to completely create the
BSP source (build) branch.</para></listitem>
<listitem><para>The base repository is cloned, and the actions
listed in the meta-series are applied to the tree.</para></listitem>
<listitem><para>The git repository is left with the desired branch checked out and any
required branching, patching and tagging has been performed.</para></listitem>
</orderedlist>
</para>
<para>
The tree is now ready for configuration and compilation.
</para>
<note><para>The end-user generated meta-series adds to the kernel as shipped with
the Yocto Project release.
Any add-ons and configuration data are applied to the end of an existing branch.
The full repository generation that is found in the
official Yocto Project kernel repositories is the combination of all
supported boards and configurations.</para>
<para>This technique is flexible and allows the seamless blending of an immutable
history with additional deployment specific patches.
Any additions to the kernel become an integrated part of the branches.
</para></note>
<!-- <note><para>It is key that feature descriptions indicate if any branches are
required, since the build system cannot automatically decide where a
BSP should branch or if that branch point needs a name with
significance. There is a single restriction enforced by the compilation
phase:
</para>
<para>A BSP must create a branch of the format &lt;bsp name&gt;-&lt;kernel type&gt;.</para>
<para>This means that all merged/support BSPs must indicate where to start
its branch from, with the right name, in its .scc files. The scc
section describes the available branching commands in more detail.
</para>
</note> -->
<!-- <para>
A summary of end user tree construction activities follow:
<itemizedlist>
<listitem><para>compile and link a full top-down kernel description from feature descriptions</para></listitem>
<listitem><para>execute the complete description to generate a meta-series</para></listitem>
<listitem><para>interpret the meta-series to create a customized git repository for the
board</para></listitem>
<listitem><para>migrate configuration fragments and configure the kernel</para></listitem>
<listitem><para>checkout the BSP branch and build</para></listitem>
</itemizedlist>
</para> -->
</section>
<section id='build-strategy'>
<title>Build Strategy</title>
<para>
There are some prerequisites that must be met before starting the compilation
phase of the kernel build system:
</para>
<itemizedlist>
<listitem><para>There must be a kernel git repository indicated in the SRC_URI.</para></listitem>
<listitem><para>There must be a BSP build branch - &lt;bsp name&gt;-&lt;kernel type&gt; in 0.9 or
&lt;kernel type&gt;/&lt;bsp name&gt; in 1.0.</para></listitem>
</itemizedlist>
<para>
You can typically meet these prerequisites by running the tree construction/patching phase
of the build system.
However, other means do exist.
For examples of alternate workflows such as bootstrapping a BSP, see
the<link linkend='workflow-examples'> Workflow Examples</link> section in this manual.
</para>
<para>
Before building a kernel it is configured by processing all of the
configuration "fragments" specified by the scc feature descriptions.
As the features are compiled, associated kernel configuration fragments are noted
and recorded in the meta-series in their compilation order.
The fragments are migrated, pre-processed and passed to the Linux Kernel
Configuration subsystem (lkc) as raw input in the form of a <filename>.config</filename> file.
The lkc uses its own internal dependency constraints to do the final
processing of that information and generates the final <filename>.config</filename> file
that is used during compilation.
</para>
<para>
Using the board's architecture and other relevant values from the board's template
the Kernel compilation is started and a kernel image is produced.
</para>
<para>The other thing that you will first see once you configure a kernel is that
it will generate a build tree that is separate from your git source tree.
This build tree has the name using the following form:
<literallayout class='monospaced'>
linux-&lt;BSPname&gt;-&lt;kerntype&gt;-build
</literallayout>
"kerntype" is one of the standard kernel types.
</para>
<para>
The existing support in the kernel.org tree achieves this default functionality.
</para>
<para>
What this means, is that all the generated files for a particular BSP are now in this directory.
The files include the final <filename>.config</filename>, all the <filename>.o</filename>
files, the <filename>.a</filename> files, and so forth.
Since each BSP has its own separate build directory in its own separate branch
of the git tree you can easily switch between different BSP builds.
</para>
</section>
<!-- <section id='scc'>
<title>Series &amp; Configuration Compiler (SCC)</title>
<para>
In early versions of the product, kernel patches were simply listed in a flat
file called "patches.list", and then quilt was added as a tool to help
traverse this list, which in quilt terms was called a "series" file.
</para>
<para>
Before the 2.0 release, it was already apparent that a static series file was
too inflexible, and that the series file had to become more dynamic and rely
on certain state (like kernel type) in order to determine whether a patch was
to be used or not. The 2.0 release already made use of some stateful
construction of series files, but since the delivery mechanism was unchanged
(tar + patches + series files), most people were not aware of anything really
different. The 3.0 release continues with this stateful construction of
series files, but since the delivery mechanism is changed (git + branches) it
now is more apparent to people.
</para>
<para>
As was previously mentioned, scc is a "series and configuration
compiler". Its role is to combine feature descriptions into a format that can
be used to generate a meta-series. A meta series contains all the required
information to construct a complete set of branches that are required to
build a desired board and feature set. The meta series is interpreted by the
kgit tools to create a git repository that could be built.
</para>
<para>
To illustrate how scc works, a feature description must first be understood.
A feature description is simply a small bash shell script that is executed by
scc in a controlled environment. Each feature description describes a set of
operations that add patches, modify existing patches or configure the
kernel. It is key that feature descriptions can include other features, and
hence allow the division of patches and configuration into named, reusable
containers.
</para>
<para>
Each feature description can use any of the following valid scc commands:
<itemizedlist>
<listitem><para>shell constructs: bash conditionals and other utilities can be used in a feature
description. During compilation, the working directory is the feature
description itself, so any command that is "raw shell" and not from the
list of supported commands, can not directly modify a git repository.</para></listitem>
<listitem><para>patch &lt;relative path&gt;/&lt;patch name&gt;: outputs a patch to be included in a feature's patch set. Only the name of
the patch is supplied, the path is calculated from the currently set
patch directory, which is normally the feature directory itself.</para></listitem>
<listitem><para>patch_trigger &gt;condition&lt; &gt;action&lt; &lt;tgt&gt;: indicate that a trigger should be set to perform an action on a
patch.</para>
<para>The conditions can be:
<itemizedlist>
<listitem><para>arch:&lt;comma separated arch list or "all"&gt;</para></listitem>
<listitem><para>plat:&lt;comma separated platform list or "all"&gt;</para></listitem>
</itemizedlist></para>
<para>The action can be:
<itemizedlist>
<listitem><para>exclude: This is used in exceptional situations where a patch
cannot be applied for certain reasons (arch or platform).
When the trigger is satisfied the patch will be removed from
the patch list.</para></listitem>
<listitem><para>include: This is used to include a patch only for a specific trigger.
Like exclude, this should only be used when necessary.
It takes 1 argument, the patch to include.</para></listitem>
</itemizedlist></para></listitem>
<listitem><para>include &lt;feature name&gt; [after &lt;feature&gt;]: includes a feature for processing. The feature is "expanded" at the
position of the include directive. This means that any patches,
configuration or sub-includes of the feature will appear in the final
series before the commands that follow the include.</para>
<para>
include searches the include directories for a matching feature name,
include directories are passed to scc by the caller using -I &lt;path&gt; and
is transparent to the feature script. This means that &lt;feature name&gt; must
be relative to one of the search paths. For example, if
/opt/kernel-cache/feat/sched.scc is to be included and scc is invoked
with -I /opt/kernel-cache, then a feature would issue "include
feat/sched.scc" to include the feature.
</para>
<para>
The optional "after" directive allows a feature to modify the existing
order of includes and insert a feature after the named feature is
processed. Note: the "include foo after bar" must be issued before "bar"
is processed, so is normally only used by a new top level feature to
modify the order of features in something it is including.</para></listitem>
<listitem><para>exclude &lt;feature name&gt;: Indicates that a particular feature should *not* be included even if an
'include' directive is found. The exclude must be issued before the
include is processed, so is normally only used by a new top level feature
to modify the order of features in something it is including.</para></listitem>
<listitem><para>git &lt;command&gt;: Issues any git command during tree construction. Note: this command is
not validated/sanitized so care must be taken to not damage the
tree. This can be used to script branching, tagging, pulls or other git
operations.</para></listitem>
<listitem><para>dir &lt;directory&gt;: changes the working directory for "patch" directives. This can be used to
shorten a long sequence of patches by not requiring a common relative
directory to be issued each time.</para></listitem>
<listitem><para>kconf &lt;type&gt; &lt;fragment name&gt;: associates a kernel config frag with the feature.
&lt;type&gt; can be
"hardware" or "non-hardware" and is used by the kernel configuration
subsystem to audit configuration. &lt;fragment name&gt; is the name of a file
in the current feature directory that contains a series of kernel
configuration options. There is no restriction on the chosen fragment
name, although a suffix of ".cfg" is recommended. Multiple fragment
specifications are supported.</para></listitem>
<listitem><para>branch &lt;branch name&gt;: creates a branch in the tree. All subsequent patch commands will be
applied to the new branch and changes isolated from the rest of the
repository.</para></listitem>
<listitem><para>scc_leaf &lt;base feature&gt; &lt;branch name&gt;: Performs a combination feature include and branch. This is mainly a
convenience directive, but has significance to some build system bindings
as a sentinel to indicate that this intends to create a branch that is
valid for kernel compilation.</para></listitem>
<listitem><para>tag &lt;tag name&gt;: Tags the tree. The tag will be applied in processing order, so will
be after already applied patches and precede patches yet to be applied.</para></listitem>
<listitem><para>define &lt;var&gt; &lt;value&gt;: Creates a variable with a particular value that can be used in subsequent
feature descriptions.</para></listitem>
</itemizedlist>
</para>
</section> -->
<!-- <section id='kgit-tools'>
<title>kgit Tools</title>
<para>
The kgit tools are responsible for constructing and maintaining the Wind
River kernel repository. These activities include importing, exporting, and
applying patches as well as sanity checking and branch management. From the
developers perspective, the kgit tools are hidden and rarely require
interactive use. But one tool in particular that warrants further description
is "kgit-meta".
</para>
<para>
kgit-meta is the actual application of feature description(s) to a kernel repo.
In other words, it is responsible for interpreting the meta series generated
from a scc compiled script. As a result, kgit-meta is coupled to the set of
commands permitted in a .scc feature description (listed in the scc section).
kgit-meta understands both the meta series format and how to use git and
guilt to modify a base git repository. It processes a meta-series line by
line, branching, tagging, patching and tracking changes that are made to the
base git repository.
</para>
<para>
Once kgit-meta has processed a meta-series, it leaves the repository with the
last branch checked out, and creates the necessary guilt infrastructure to
inspect the tree, or add to it via using guilt. As was previously mentioned,
guilt is not required, but is provided as a convenience. Other utilities such
as quilt, stgit, git or others can also be used to manipulate the git
repository.
</para>
</section> -->
<section id='workflow-examples'>
<title>Workflow Examples</title>
<para>
As previously noted, the Yocto Project kernel has built in git integration.
However, these utilities are not the only way to work with the kernel repository.
Yocto Project has not made changes to git or to other tools that
would invalidate alternate workflows.
Additionally, the way the kernel repository is constructed results in using
only core git functionality thus allowing any number of tools or front ends to use the
resulting tree.
</para>
<para>
This section contains several workflow examples.
</para>
<section id='change-inspection-kernel-changes-commits'>
<title>Change Inspection: Kernel Changes/Commits</title>
<para>
A common question when working with a BSP or kernel is:
"What changes have been applied to this tree?"
</para>
<para>
In projects that have a collection of directories that
contain patches to the kernel it is possible to inspect or "grep" the contents
of the directories to get a general feel for the changes.
This sort of patch inspection is not an efficient way to determine what has been done to the
kernel.
The reason it is inefficient is because there are many optional patches that are
selected based on the kernel type and the feature description.
Additionally, patches could exist in directories that are not included in the search.
</para>
<para>
A more efficient way to determine what has changed in the kernel is to use
git and inspect or search the kernel tree.
This method gives you a full view of not only the source code modifications,
but also provides the reasons for the changes.
</para>
<section id='what-changed-in-a-bsp'>
<title>What Changed in a BSP?</title>
<para>
Following are a few examples that show how to use git to examine changes.
Note that because the Yocto Project git repository does not break existing git
functionality and because there exists many permutations of these types of
commands there are many more methods to discover changes.
</para>
<note><para>
Unless you provide a commit range
(&lt;kernel-type&gt;..&lt;bsp&gt;-&lt;kernel-type&gt;), kernel.org history
is blended with Yocto Project changes.
</para></note>
<literallayout class='monospaced'>
# full description of the changes
&gt; git whatchanged &lt;kernel type&gt;..&lt;kernel type&gt;/&lt;bsp&gt;
&gt; eg: git whatchanged yocto/standard/base..yocto/standard/common-pc/base
# summary of the changes
&gt; git log --pretty=oneline --abbrev-commit &lt;kernel type&gt;..&lt;kernel type&gt;/&lt;bsp&gt;
# source code changes (one combined diff)
&gt; git diff &lt;kernel type&gt;..&lt;kernel type&gt;/&lt;bsp&gt;
&gt; git show &lt;kernel type&gt;..&lt;kernel type&gt;/&lt;bsp&gt;
# dump individual patches per commit
&gt; git format-patch -o &lt;dir&gt; &lt;kernel type&gt;..&lt;kernel type&gt;/&lt;bsp&gt;
# determine the change history of a particular file
&gt; git whatchanged &lt;path to file&gt;
# determine the commits which touch each line in a file
&gt; git blame &lt;path to file&gt;
</literallayout>
</section>
<section id='show-a-particular-feature-or-branch-change'>
<title>Show a Particular Feature or Branch Change</title>
<para>
Significant features or branches are tagged in the Yocto Project tree to divide
changes.
Remember to first determine (or add) the tag of interest.
</para>
<note><para>
Because BSP branch, kernel.org, and feature tags are all present, there are many tags.
</para></note>
<literallayout class='monospaced'>
# show the changes tagged by a feature
&gt; git show &lt;tag&gt;
&gt; eg: git show yaffs2
# determine which branches contain a feature
&gt; git branch --contains &lt;tag&gt;
# show the changes in a kernel type
&gt; git whatchanged yocto/base..&lt;kernel type&gt;
&gt; eg: git whatchanged yocto/base..yocto/standard/base
</literallayout>
<para>
You can use many other comparisons to isolate BSP changes.
For example, you can compare against kernel.org tags (e.g. v2.6.27.18, etc), or
you can compare against subsystems (e.g. git whatchanged mm).
</para>
</section>
</section>
<section id='development-saving-kernel-modifications'>
<title>Development: Saving Kernel Modifications</title>
<para>
Another common operation is to build a BSP supplied by Yocto Project, make some
changes, rebuild and then test.
Those local changes often need to be exported, shared or otherwise maintained.
</para>
<para>
Since the Yocto Project kernel source tree is backed by git, this activity is
much easier as compared to with previous releases.
Because git tracks file modifications, additions and deletions, it is easy
to modify the code and later realize that the changes should be saved.
It is also easy to determine what has changed.
This method also provides many tools to commit, undo and export those modifications.
</para>
<para>
There are many ways to save kernel modifications.
The technique employed
depends on the destination for the patches:
<itemizedlist>
<listitem><para>Bulk storage</para></listitem>
<listitem><para>Internal sharing either through patches or by using git</para></listitem>
<listitem><para>External submissions</para></listitem>
<listitem><para>Exporting for integration into another SCM</para></listitem>
</itemizedlist>
</para>
<para>
Because of the following list of issues, the destination of the patches also influences
the method for gathering them:
<itemizedlist>
<listitem><para>Bisectability</para></listitem>
<listitem><para>Commit headers</para></listitem>
<listitem><para>Division of subsystems for separate submission or review</para></listitem>
</itemizedlist>
</para>
<section id='bulk-export'>
<title>Bulk Export</title>
<para>
This section describes how you can export in "bulk" changes that have not
been separated or divided.
This situation works well when you are simply storing patches outside of the kernel
source repository, either permanently or temporarily, and you are not committing
incremental changes during development.
</para>
<note><para>
This technique is not appropriate for full integration of upstream submission
because changes are not properly divided and do not provide an avenue for per-change
commit messages.
Therefore, this example assumes that changes have not been committed incrementally
during development and that you simply must gather and export them.
</para></note>
<literallayout class='monospaced'>
# bulk export of ALL modifications without separation or division
# of the changes
&gt; git add .
&gt; git commit -s -a -m &gt;commit message&lt;
or
&gt; git commit -s -a # and interact with $EDITOR
</literallayout>
<para>
The previous operations capture all the local changes in the project source
tree in a single git commit.
And, that commit is also stored in the project's source tree.
</para>
<para>
Once the changes are exported, you can restore them manually using a template
or through integration with the <filename>default_kernel</filename>.
</para>
</section>
<section id='incremental-planned-sharing'>
<title>Incremental/Planned Sharing</title>
<para>
This section describes how to save modifications when you are making incremental
commits or practicing planned sharing.
The examples in this section assume that changes have been incrementally committed
to the tree during development and now need to be exported. The sections that follow
describe how you can export your changes internally through either patches or by
using git commands.
</para>
<para>
During development the following commands are of interest.
For full git documentation, refer to the git man pages or to an online resource such
as <ulink url='http://github.com'></ulink>.
<literallayout class='monospaced'>
# edit a file
&gt; vi &gt;path&lt;/file
# stage the change
&gt; git add &gt;path&lt;/file
# commit the change
&gt; git commit -s
# remove a file
&gt; git rm &gt;path&lt;/file
# commit the change
&gt; git commit -s
... etc.
</literallayout>
</para>
<para>
Distributed development with git is possible when you use a universally
agreed-upon unique commit identifier (set by the creator of the commit) that maps to a
specific change set with a specific parent.
This identifier is created for you when
you create a commit, and is re-created when you amend, alter or re-apply
a commit.
As an individual in isolation, this is of no interest.
However, if you
intend to share your tree with normal git push and pull operations for
distributed development, you should consider the ramifications of changing a
commit that you have already shared with others.
</para>
<para>
Assuming that the changes have not been pushed upstream, or pulled into
another repository, you can update both the commit content and commit messages
associated with development by using the following commands:
<literallayout class='monospaced'>
&gt; git add &gt;path&lt;/file
&gt; git commit --amend
&gt; git rebase or git rebase -i
</literallayout>
</para>
<para>
Again, assuming that the changes have not been pushed upstream, and that
no pending works-in-progress exist (use "git status" to check) then
you can revert (undo) commits by using the following commands:
<literallayout class='monospaced'>
# remove the commit, update working tree and remove all
# traces of the change
&gt; git reset --hard HEAD^
# remove the commit, but leave the files changed and staged for re-commit
&gt; git reset --soft HEAD^
# remove the commit, leave file change, but not staged for commit
&gt; git reset --mixed HEAD^
</literallayout>
</para>
<para>
You can create branches, "cherry-pick" changes or perform any number of git
operations until the commits are in good order for pushing upstream
or for pull requests.
After a push or pull, commits are normally considered
"permanent" and you should not modify them.
If they need to be changed you can incrementally do so with new commits.
These practices follow the standard "git" workflow and the kernel.org best
practices, which Yocto Project recommends.
</para>
<note><para>
It is recommended to tag or branch before adding changes to a Yocto Project
BSP or before creating a new one.
The reason for this recommendation is because the branch or tag provides a
reference point to facilitate locating and exporting local changes.
</para></note>
<section id='export-internally-via-patches'>
<title>Exporting Changes Internally by Using Patches</title>
<para>
This section describes how you can extract committed changes from a working directory
by exporting them as patches.
Once extracted, you can use the patches for upstream submission,
place them in a Yocto Project template for automatic kernel patching,
or apply them in many other common uses.
</para>
<para>
This example shows how to create a directory with sequentially numbered patches.
Once the directory is created, you can apply it to a repository using the
<filename>git am</filename> command to reproduce the original commit and all
the related information such as author, date, commit log, and so forth.
</para>
<note><para>
The new commit identifiers (ID) will be generated upon re-application.
This action reflects that the commit is now applied to an underlying commit
with a different ID.
</para></note>
<para>
<literallayout class='monospaced'>
# &lt;first-commit&gt; can be a tag if one was created before development
# began. It can also be the parent branch if a branch was created
# before development began.
&gt; git format-patch -o &lt;dir&gt; &lt;first commit&gt;..&lt;last commit&gt;
</literallayout>
</para>
<para>
In other words:
<literallayout class='monospaced'>
# Identify commits of interest.
# If the tree was tagged before development
&gt; git format-patch -o &lt;save dir&gt; &lt;tag&gt;
# If no tags are available
&gt; git format-patch -o &lt;save dir&gt; HEAD^ # last commit
&gt; git format-patch -o &lt;save dir&gt; HEAD^^ # last 2 commits
&gt; git whatchanged # identify last commit
&gt; git format-patch -o &lt;save dir&gt; &lt;commit id&gt;
&gt; git format-patch -o &lt;save dir&gt; &lt;rev-list&gt;
</literallayout>
</para>
<!--<para>
See the "template patching" example for how to use the patches to
automatically apply to a new kernel build.
</para>-->
</section>
<section id='export-internally-via-git'>
<title>Exporting Changes Internally by Using git</title>
<para>
This section describes how you can export changes from a working directory
by pushing the changes into a master repository or by making a pull request.
Once you have pushed the changes in the master repository you can then
pull those same changes into a new kernel build at a later time.
</para>
<para>
Use this command form to push the changes:
<literallayout class='monospaced'>
git push ssh://&lt;master server&gt;/&lt;path to repo&gt; &lt;local branch&gt;:&lt;remote branch&gt;
</literallayout>
</para>
<para>
For example, the following command pushes the changes from your local branch
<filename>yocto/standard/common-pc/base</filename> to the remote branch with the same name
in the master repository <filename>//git.mycompany.com/pub/git/kernel-2.6.37</filename>.
<literallayout class='monospaced'>
&gt; push ssh://git.mycompany.com/pub/git/kernel-2.6.37 yocto/standard/common-pc/base:yocto/standard/common-pc/base
</literallayout>
</para>
<para>
A pull request entails using "git request-pull" to compose an email to the
maintainer requesting that a branch be pulled into the master repository, see
<ulink url='http://github.com/guides/pull-requests'></ulink> for an example.
</para>
<note><para>
Other commands such as 'git stash' or branching can also be used to save
changes, but are not covered in this document.
</para></note>
<!--<para>
See the section "importing from another SCM" for how a git push to the
default_kernel, can be used to automatically update the builds of all users
of a central git repository.
</para>-->
</section>
</section>
<section id='export-for-external-upstream-submission'>
<title>Exporting Changes for External (Upstream) Submission</title>
<para>
This section describes how to export changes for external upstream submission.
If the patch series is large or the maintainer prefers to pull
changes, you can submit these changes by using a pull request.
However, it is common to sent patches as an email series.
This method allows easy review and integration of the changes.
</para>
<note><para>
Before sending patches for review be sure you understand the
community standards for submitting and documenting changes and follow their best practices.
For example, kernel patches should follow standards such as:
<itemizedlist>
<listitem><para><ulink url='http://userweb.kernel.org/~akpm/stuff/tpp.txt'></ulink></para></listitem>
<listitem><para><ulink url='http://linux.yyz.us/patch-format.html'></ulink></para></listitem>
<listitem><para>Documentation/SubmittingPatches (in any linux kernel source tree)</para></listitem>
</itemizedlist>
</para></note>
<para>
The messages used to commit changes are a large part of these standards.
Consequently, be sure that the headers for each commit have the required information.
If the initial commits were not properly documented or do not meet those standards,
you can re-base by using the "git rebase -i" command to manipulate the commits and
get them into the required format.
Other techniques such as branching and cherry-picking commits are also viable options.
</para>
<para>
Once you complete the commits, you can generate the email that sends the patches
to the maintainer(s) or lists that review and integrate changes.
The command "git send-email" is commonly used to ensure that patches are properly
formatted for easy application and avoid mailer-induced patch damage.
</para>
<para>
The following is an example of dumping patches for external submission:
<literallayout class='monospaced'>
# dump the last 4 commits
&gt; git format-patch --thread -n -o ~/rr/ HEAD^^^^
&gt; git send-email --compose --subject '[RFC 0/N] &lt;patch series summary&gt;' \
--to foo@yoctoproject.org --to bar@yoctoproject.org \
--cc list@yoctoproject.org ~/rr
# the editor is invoked for the 0/N patch, and when complete the entire
# series is sent via email for review
</literallayout>
</para>
</section>
<section id='export-for-import-into-other-scm'>
<title>Exporting Changes for Import into Another SCM</title>
<para>
When you want to export changes for import into another
Source Code Manager (SCM) you can use any of the previously discussed
techniques.
However, if the patches are manually applied to a secondary tree and then
that tree is checked into the SCM you can lose change information such as
commit logs.
Yocto Project does not recommend this process.
</para>
<para>
Many SCMs can directly import git commits, or can translate git patches so that
information is not lost.
Those facilities are SCM-dependent and you should use them whenever possible.
</para>
</section>
</section>
<section id='scm-working-with-the-yocto-project-kernel-in-another-scm'>
<title>Working with the Yocto Project Kernel in Another SCM</title>
<para>
This section describes kernel development in another SCM, which is not the same
as exporting changes to another SCM.
For this scenario you use the Yocto Project build system to
develop the kernel in a different SCM.
The following must be true for you to accomplish this:
<itemizedlist>
<listitem><para>The delivered Yocto Project kernel must be exported into the second
SCM.</para></listitem>
<listitem><para>Development must be exported from that secondary SCM into a
format that can be used by the Yocto Project build system.</para></listitem>
</itemizedlist>
</para>
<section id='exporting-delivered-kernel-to-scm'>
<title>Exporting the Delivered Kernel to the SCM</title>
<para>
Depending on the SCM it might be possible to export the entire Yocto Project
kernel git repository, branches and all, into a new environment.
This method is preferred because it has the most flexibility and potential to maintain
the meta data associated with each commit.
</para>
<para>
When a direct import mechanism is not available, it is still possible to
export a branch (or series of branches) and check them into a new repository.
</para>
<para>
The following commands illustrate some of the steps you could use to
import the yocto/standard/common-pc/base kernel into a secondary SCM:
<literallayout class='monospaced'>
&gt; git checkout yocto/standard/common-pc/base
&gt; cd .. ; echo linux/.git &gt; .cvsignore
&gt; cvs import -m "initial import" linux MY_COMPANY start
</literallayout>
</para>
<para>
You could now relocate the CVS repository and use it in a centralized manner.
</para>
<para>
The following commands illustrate how you can condense and merge two BSPs into a second SCM:
<literallayout class='monospaced'>
&gt; git checkout yocto/standard/common-pc/base
&gt; git merge yocto/standard/common-pc-64/base
# resolve any conflicts and commit them
&gt; cd .. ; echo linux/.git &gt; .cvsignore
&gt; cvs import -m "initial import" linux MY_COMPANY start
</literallayout>
</para>
</section>
<section id='importing-changes-for-build'>
<title>Importing Changes for the Build</title>
<para>
Once development has reached a suitable point in the second development
environment, you need to export the changes as patches.
To export them place the changes in a recipe and
automatically apply them to the kernel during patching.
</para>
<!--<para>
If changes are imported directly into git, they must be propagated to the
wrll-linux-2.6.27/git/default_kernel bare clone of each individual build
to be present when the kernel is checked out.
</para>
<para>
The following example illustrates one variant of this workflow:
<literallayout class='monospaced'>
# on master git repository
&gt; cd linux-2.6.27
&gt; git tag -d common_pc-standard-mark
&gt; git pull ssh://&lt;foo&gt;@&lt;bar&gt;/pub/git/kernel-2.6.27 common_pc-standard:common_pc-standard
&gt; git tag common_pc-standard-mark
# on each build machine (or NFS share, etc)
&gt; cd wrll-linux-2.6.27/git/default_kernel
&gt; git fetch ssh://&lt;foo&gt;@&lt;master server&gt;/pub/git/kernel-2.6.27
# in the build, perform a from-scratch build of Linux and the new changes
# will be checked out and built.
&gt; make linux
</literallayout>
</para> -->
</section>
</section>
<!-- <section id='bsp-template-migration-from-2'>
<title>BSP: Template Migration from 2.0</title>
<para>
The move to a git-backed kernel build system in 3.0 introduced a small new
requirement for any BSP that is not integrated into the GA release of the
product: branching information.
</para>
<para>
As was previously mentioned in the background sections, branching information
is always required, since the kernel build system cannot make intelligent
branching decisions and must rely on the developer. This branching
information is provided via a .scc file.
</para>
<para>
A BSP template in 2.0 contained build system information (config.sh, etc) and
kernel patching information in the 'linux' subdirectory. The same holds true
in 3.0, with only minor changes in the kernel patching directory.
The ".smudge" files are now ".scc" files and now contain a full description
of the kernel branching, patching and configuration for the BSP. Where in
2.0, they only contained kernel patching information.
</para>
<para>
The following illustrates the migration of a simple 2.0 BSP template to the
new 3.0 kernel build system.
</para>
<note><para>
Note: all operations are from the root of a customer layer.
</para></note>
<literallayout class='monospaced'>
templates/
`&dash;&dash; board
`&dash;&dash; my_board
|&dash;&dash; config.sh
|&dash;&dash; include
`&dash;&dash; linux
`&dash;&dash; 2.6.x
|&dash;&dash; knl-base.cfg
|&dash;&dash; bsp.patch
`&dash;&dash; my_bsp.smudge
&gt; mv templates/board/my_board/linux/2.6.x/* templates/board/my_board/linux
&gt; rm -rf templates/board/my_board/linux/2.6.x/
&gt; mv templates/board/my_board/linux/my_bsp.smudge \
templates/board/my_board/linux/my_bsp-standard.scc
&gt; echo "kconf hardware knl-base.cfg" &gt;&gt; \
templates/board/my_board/linux/my_bsp-standard.scc
&gt; vi templates/board/my_board/linux/my_bsp-standard.scc
# add the following at the top of the file
scc_leaf ktypes/standard my_bsp-standard
templates/
`&dash;&dash; board
`&dash;&dash; my_board
|&dash;&dash; config.sh
|&dash;&dash; include
`&dash;&dash; linux
|&dash;&dash; knl-base.cfg
|&dash;&dash; bsp.patch
`&dash;&dash; my_bsp-standard.scc
</literallayout>
<para>
That's it. Configure and build.
</para>
<note><para>There is a naming convention for the .scc file, which allows the build
system to locate suitable feature descriptions for a board:
</para></note>
<literallayout class='monospaced'>
&lt;bsp name&gt;-&lt;kernel type&gt;.scc
</literallayout>
<para>
if this naming convention isn't followed your feature description will
not be located and a build error thrown.
</para>
</section> -->
<section id='bsp-creating'>
<title>Creating a BSP Based on an Existing Similar BSP</title>
<para>
This section provides an example for creating a BSP
that is based on an existing, and hopefully, similar
one. It assumes you will be using a local kernel
repository and will be pointing the kernel recipe at
that. Follow these steps and keep in mind your
particular situation and differences:
<orderedlist>
<listitem><para>
Identify a machine configuration file that matches your machine.
</para>
<para>
You can start with something in <filename>meta/conf/machine</filename> - <filename>
meta/conf/machine/atom-pc.conf</filename> for example. Or, you can start with a machine
configuration from any of the BSP layers in the meta-intel repository at
<ulink url='http://git.yoctoproject.org/cgit/cgit.cgi/meta-intel/'></ulink>, such as
<filename>meta-intel/meta-emenlow/conf/machine/emenlow.conf</filename>.
</para>
<para>
The main difference between the two is that "emenlow" is in its own layer.
It is in its own layer because it needs extra machine-specific packages such as its
own video driver and other supporting packages.
The "atom-pc" is simpler and does not need any special packages - everything it needs can
be specified in the configuration file.
The "atom-pc" machine also supports all of Asus eee901, Acer Aspire One, Toshiba NB305,
and the Intel&reg; Embedded Development Board 1-N450 with no changes.
</para>
<para>
If you want to make minor changes to support a slightly different machine, you can
create a new configuration file for it and add it alongside the others.
You might consider keeping the common information separate and including it.
</para>
<para>
Similarly, you can also use multiple configuration files for different machines even
if you do it as a separate layer like meta-emenlow.
</para>
<para>
As an example consider this:
<itemizedlist>
<listitem><para>Copy meta-emenlow to meta-mymachine</para></listitem>
<listitem><para>Fix or remove anything you do not need.
For this example the only thing left was the kernel directory with a
<filename>linux-yocto_git.bbappend</filename>
file
and <filename>meta-mymachine/conf/machine/mymachine.conf</filename>
(linux-yocto is the kernel listed in
<filename>meta-emenlow/conf/machine/emenlow.conf</filename>)</para></listitem>.
<listitem><para>Add a new entry in the <filename>build/conf/bblayers.conf</filename>
so the new layer can be found by BitBake.</para></listitem>
</itemizedlist>
</para></listitem>
<listitem><para>
Create a machine branch for your machine.
</para>
<para>
For the kernel to compile successfully, you need to create a branch in the git repository
specifically named for your machine.
To create this branch first create a bare clone of the Yocto Project git repository.
Next, create a local clone of that:
<literallayout class='monospaced'>
$ git clone --bare git://git.yoctoproject.org/linux-yocto-2.6.37.git
linux-yocto-2.6.37.git
$ git clone linux-yocto-2.6.37.git linux-yocto-2.6.37
</literallayout>
</para>
<para>
Now create a branch in the local clone and push it to the bare clone:
<literallayout class='monospaced'>
$ git checkout -b yocto/standard/mymachine origin/yocto/standard/base
$ git push origin yocto/standard/mymachine:yocto/standard/mymachine
</literallayout>
</para></listitem>
<listitem><para>
In a layer, create a <filename>linux-yocto_git.bbappend</filename>
file with the following:
</para>
<para>
<literallayout class='monospaced'>
FILESEXTRAPATHS := "${THISDIR}/${PN}"
COMPATIBLE_MACHINE_mymachine = "mymachine"
# It is often nice to have a local clone of the kernel repository, to
# allow patches to be staged, branches created, and so forth. Modify
# KSRC to point to your local clone as appropriate.
KSRC ?= /path/to/your/bare/clone/for/example/linux-yocto-2.6.37.git
# KMACHINE is the branch to be built, or alternatively
# KBRANCH can be directly set.
# KBRANCH is set to KMACHINE in the main linux-yocto_git.bb
# KBRANCH ?= "${LINUX_KERNEL_TYPE}/${KMACHINE}"
KMACHINE_mymachine = "yocto/standard/mymachine"
SRC_URI = "git://${KSRC};nocheckout=1;branch=${KBRANCH},meta;name=machine,meta"
</literallayout>
</para>
<para>
After doing that, select the machine in <filename>build/conf/local.conf</filename>:
<literallayout class='monospaced'>
#
MACHINE ?= "mymachine"
#
</literallayout>
</para>
<para>
You should now be able to build and boot an image with the new kernel:
<literallayout class='monospaced'>
$ bitbake poky-image-sato-live
</literallayout>
</para></listitem>
<listitem><para>
Modify the kernel configuration for your machine.
</para>
<para>
Of course, that will give you a kernel with the default configuration file, which is probably
not what you want.
If you just want to set some kernel configuration options, you can do that by
putting them in a file.
For example, inserting the following into some <filename>.cfg</filename> file:
<literallayout class='monospaced'>
CONFIG_NETDEV_1000=y
CONFIG_E1000E=y
</literallayout>
</para>
<para>
And, another <filename>.cfg</filename> file would contain:
<literallayout class='monospaced'>
CONFIG_LOG_BUF_SHIFT=18
</literallayout>
<para>
These config fragments could then be picked up and
applied to the kernel .config by appending them to the kernel SRC_URI:
</para>
<literallayout class='monospaced'>
SRC_URI_append_mymachine = " file://some.cfg \
file://other.cfg \
"
</literallayout>
</para>
<para>
You could also add these directly to the git repository <filename>meta</filename>
branch as well.
However, the former method is a simple starting point.
</para></listitem>
<listitem><para>
If you're also adding patches to the kernel, you can do the same thing.
Put your patches in the SRC_URI as well (plus <filename>.cfg</filename> for their kernel
configuration options if needed).
</para>
<para>
Practically speaking, to generate the patches, you'd go to the source in the build tree:
<literallayout class='monospaced'>
build/tmp/work/mymachine-poky-linux/linux-yocto-2.6.37+git0+d1cd5c80ee97e81e130be8c3de3965b770f320d6_0+
0431115c9d720fee5bb105f6a7411efb4f851d26-r13/linux
</literallayout>
</para>
<para>
Then, modify the code there, using quilt to save the changes, and recompile until
it works:
<literallayout class='monospaced'>
$ bitbake -c compile -f linux-yocto
</literallayout>
</para></listitem>
<listitem><para>
Once you have the final patch from quilt, copy it to the
SRC_URI location.
The patch is applied the next time you do a clean build.
Of course, since you have a branch for the BSP in git, it would be better to put it there instead.
For example, in this case, commit the patch to the "yocto/standard/mymachine" branch, and during the
next build it is applied from there.
</para></listitem>
</orderedlist>
</para>
</section>
<section id='bsp-creating-bsp-without-a-local-kernel-repo'>
<title>Creating a BSP Based on an Existing Similar BSP Without a Local Kernel Repository</title>
<para>
If you are creating a BSP based on an existing similar BSP but you do not have
a local kernel repository, the process is very similar to the process in
the previous section (<xref linkend='bsp-creating'/>).
</para>
<para>
Follow the exact same process as described in the previous section with
these slight modifications:
</para>
<orderedlist>
<listitem><para>Perform Step 1 as is from the previous section.</para></listitem>
<listitem><para>Perform Step 2 as is from the previous section.</para></listitem>
<listitem><para>Perform Step 3 but do not modify the
KSRC line in the bbappend file.</para></listitem>
<listitem><para>Edit the <filename>local.conf</filename> so
that it contains the following:
<literallayout class='monospaced'>
YOCTO_KERNEL_EXTERNAL_BRANCH="&lt;your-machine&gt;-standard
</literallayout></para>
<para>Adding this statement to the file triggers BSP bootstrapping
to occur and the correct branches and base configuration to be used.
</para></listitem>
<listitem><para>Perform Step 4 as is from the previous section.</para></listitem>
<listitem><para>Perform Step 5 as is from the previous section.</para></listitem>
</orderedlist>
</section>
<!-- <section id='bsp-creating-a-new-bsp'>
<title>BSP: Creating a New BSP</title>
<para>
Although it is obvious that the structure of a new BSP uses the migrated
directory structure from the previous example,the first question is whether
or not the BSP is started from scratch.
</para>
<para>
If Yocto Project has a similar BSP, it is often easier to clone and update,
rather than start from scratch. If the mainline kernel has support, it is
easier to branch from the -standard kernel and begin development (and not be
concerned with undoing existing changes). This section covers both options.
</para>
<para>
In almost every scenario, the LDAT build system bindings must be completed
before either cloning or starting a new BSP from scratch. This is simply
because the board template files are required to configure a project/build
and create the necessary environment to begin working directly with the
kernel. If it is desired to start immediately with kernel development and
then add LDAT bindings, see the "bootstrapping a BSP" section.
</para>
<section id='creating-from-scratch'>
<title>Creating the BSP from Scratch</title>
<para>
To create the BSP from scratch you need to do the following:
<orderedlist>
<listitem><para>Create a board template for the new BSP in a layer.</para></listitem>
<listitem><para>Configure a build with the board.</para></listitem>
<listitem><para>Configure a kernel.</para></listitem>
</orderedlist>
</para>
<para>
Following is an example showing all three steps. You start by creating a board template for the new BSP in a layer.
<literallayout class='monospaced'>
templates/
`&dash;&dash; board
`&dash;&dash; my_bsp
|&dash;&dash; include
|&dash;&dash; config.sh
`&dash;&dash; linux
|&dash;&dash; my_bsp.cfg
`&dash;&dash; my_bsp-standard.scc
&gt; cat config.sh
TARGET_BOARD="my_bsp"
TARGET_LINUX_LINKS="bzImage"
TARGET_SUPPORTED_KERNEL="standard"
TARGET_SUPPORTED_ROOTFS="glibc_std"
BANNER="This BSP is *NOT* supported"
TARGET_PROCFAM="pentium4"
TARGET_PLATFORMS="GPP"
&gt; cat include
cpu/x86_32_i686
karch/i386
&gt; cat linux/my_bsp-standard.scc
scc_leaf ktypes/standard/standard.scc my_bsp-standard
&gt; cat linux/my_bsp.cfg
CONFIG_X86=y
CONFIG_SMP=y
CONFIG_VT=y
# etc, etc, etc
</literallayout>
</para>
<para>
Something like the following can now be added to a board build, and
a project can be started:
<literallayout class='monospaced'>
&dash;&dash;enable-board=my_bsp \
&dash;&dash;with-layer=custom_bsp
</literallayout>
</para>
<para>
Now you can configure a kernel:
<literallayout class='monospaced'>
&gt; make -C build linux.config
</literallayout>
</para>
<para>
You now have a kernel tree, which is branched and has no patches, ready for
development.
</para>
</section> -->
<!-- <section id='cloning-an-existing-bsp'>
<title>Cloning an Existing BSP</title>
<para>
Cloning an existing BSP from the shipped product is similar to the "from
scratch" option and there are two distinct ways to achieve this goal:
<itemizedlist>
<listitem><para>Create a board template for the new BSP in a layer.</para></listitem>
<listitem><para>Clone the .scc and board config.</para></listitem>
</itemizedlist>
</para>
<para>
The first method is similar to the from scratch BSP where you create a board template for the new
BSP. Although in this case, copying an existing board template from
wrll-wrlinux/templates/board would be appropriate, since we are cloning an
existing BSP. Edit the config.sh, include and other board options for the new
BSP.
</para>
<para>
The second method is to clone the .scc and board config.
To do this, in the newly created board template, create a linux subdirectory and export
the .scc and configuration from the source BSP in the published Yocto Project
kernel. During construction, all of the configuration and patches were
captured, so it is simply a matter of extracting them.
</para>
<para>
Extraction can be accomplished using four different techniques:
<itemizedlist>
<listitem><para>Config and patches from the bare default_kernel.</para></listitem>
<listitem><para>Clone default_kernel and checkout wrs_base.</para></listitem>
<listitem><para>Clone default_kernel and checkout BSP branch.</para></listitem>
<listitem><para>Branch from the Yocto Project BSP.</para></listitem>
</itemizedlist>
</para>
<para>
Technique 1: config and patches from the bare default_kernel
<literallayout class='monospaced'>
&gt; cd layers/wrll-linux-2.6.27/git/default_kernel
&gt; git show checkpoint_end | filterdiff -i '*common_pc*' | patch -s -p2 -d /tmp
# This will create two directories: cfg and patches.
&gt; cd /tmp/cfg/kernel-cache/bsp/common_pc/
# This directory contains all the patches and .scc files used to construct
# the BSP in the shipped tree. Copy the patches to the new BSP template,
# and add them to the .scc file created above. See "template patching" if
# more details are required.
</literallayout>
</para>
<para>
Technique 2: clone default_kernel and checkout wrs_base
<literallayout class='monospaced'>
&gt; git clone layers/wrll-linux-2.6.27/git/default_kernel windriver-2.6.27
&gt; cd windriver-2.6.27
&gt; git checkout wrs_base
&gt; cd wrs/cfg/kernel-cache/bsp/common_pc
# again, this directory has all the patches and .scc files used to construct
# the BSP
</literallayout>
</para>
<para>
Technique 3: clone default_kernel and checkout BSP branch
<literallayout class='monospaced'>
&gt; git clone layers/wrll-linux-2.6.27/git/default_kernel windriver-2.6.27
&gt; cd windriver-2.6.27
&gt; git checkout common_pc-standard
&gt; git whatchanged
# browse patches and determine which ones are of interest, say there are
# 3 patches of interest
&gt; git format-patch -o &lt;path to BSP template&gt;/linux HEAD^^^
# update the .scc file to add the patches, see "template patches" if
# more details are required
</literallayout>
</para>
<para>
Technique #4: branch from the Yocto Project BSP
<note><para>This is potentially the most "different" technique, but is actually
the easiest to support and leverages the infrastructure. rtcore BSPs
are created in a similar manner to this.
</para></note>
</para>
<para>
In this technique the .scc file in the board template is slightly different
and indicates that the BSP should branch after the base Yocto Project BSP
of the correct kernel type, so to start a new BSP that inherits the
kernel patches of the common_pc-standard, the following would be done:
<literallayout class='monospaced'>
&gt; cat linux/my_bsp-standard.scc
scc_leaf bsp/common_pc/common_pc-standard.scc my_bsp-standard
</literallayout>
</para>
<para>
And only kernel configuration (not patches) need be contained in the
board template.
</para>
<para>
This has the advantage of automatically picking up updates to the BSP
and not duplicating any patches for a similar board.
</para>
</section> -->
<!-- <section id='bsp-bootstrapping'>
<title>BSP: Bootstrapping</title>
<para>
The previous examples created the board templates and configured a build
before beginning work on a new BSP. It is also possible for advanced users to
simply treat the Yocto Project git repository as an upstream source and begin
BSP development directly on the repository. This is the closest match to how
the kernel community at large would operate.
</para>
<para>
Two techniques exist to accomplish this:
</para>
<para>
Technique 1: upstream workflow
<literallayout class='monospaced'>
&gt; git clone layers/wrll-linux-2.6.27/git/default_kernel windriver-2.6.27
&gt; cd windriver-2.6.27
&gt; git checkout -b my_bsp-standard common_pc-standard
# edit files, import patches, generally do BSP development
# at this point we can create the BSP template, and export the kernel
# changes using one of the techniques discussed in that section. For
# example, It is possible to push these changes, directly into the
# default_kernel and never directly manipulate or export patch files
</literallayout>
</para>
<para>
Technique 2: Yocto Project kernel build workflow
</para>
<para>
Create the BSP branch from the appropriate kernel type
<literallayout class='monospaced'>
&gt; cd linux
# the naming convention for auto-build is &lt;bsp&gt;-&lt;kernel type&gt;
&gt; git checkout -b my_bsp-standard standard
</literallayout>
</para>
<para>
Make changes, import patches, etc.
<literallayout class='monospaced'>
&gt; ../../host-cross/bin/guilt init
# 'wrs/patches/my_bsp-standard' has now been created to
# manage the branches patches
# option 1: edit files, guilt import
&gt; ../../host-cross/bin/guilt new extra-version.patch
&gt; vi Makefile
&gt; ../../host-cross/bin/guilt refresh
# add a header
&gt; ../../host-cross/bin/guilt header -e
# describe the patch using best practices, like the example below:
&dash;&dash;&dash;&gt;&dash;&dash;&dash;&gt;&dash;&dash;&dash;&gt; cut here
From: Bruce Ashfield &lt;bruce.ashfield@windriver.com&gt;
Adds an extra version to the kernel
Modify the main EXTRAVERSION to show our bsp name
Signed-off-by: Bruce Ashfield &lt;bruce.ashfield@windriver.com&gt;
&dash;&dash;&dash;&gt;&dash;&dash;&dash;&gt;&dash;&dash;&dash;&gt; cut here
# option 2: import patches
&gt; git am &lt;patch&gt;
or
&gt; git apply &lt;patch&gt;
&gt; git add &lt;files&gt;
&gt; git commit -s
# configure the board, save relevant options
&gt; make ARCH=&lt;arch&gt; menuconfig
# save the cfg changes for reconfiguration
&gt; mkdir wrs/cfg/&lt;cache&gt;/my_bsp
&gt; vi wrs/cfg/&lt;cache&gt;/my_bsp/my_bsp.cfg
# classify the patches
&gt; ../../host-cross/bin/kgit classify create &lt;kernel-foo-cache&gt;/my_bsp/my_bsp
# test build
&gt; cd ..
&gt; make linux TARGET_BOARD=my_bsp kprofile=my_bsp use_current_branch=1
</literallayout>
</para>
<para>
Assuming the patches have been exported to the correct location, Future
builds will now find the board, apply the patches to the base tree and make
the relevant branches and structures and the special build options are no
longer required.
</para>
</section>
</section> -->
<!-- <section id='patching'>
<title>Patching</title>
<para>
The most common way to apply patches to the kernel is via a template.
However, for more advanced applications (such as the sharing of patches between
multiple sub-features) it is possible to patch the kernel-cache.
This section covers both scenarios.
</para>
<section id='patching-template'>
<title>Patching: Template</title>
<para>
kernel
templates follow the same rules as any LDAT template. A directory should be
created in a recognized template location, with a 'linux' subdirectory. The
'linux' directory triggers LDAT to pass the dir as a potential patch location
to the kernel build system. Any .scc files found in that directory, will be
automatically appended to the end of the BSP branch (for the configured
board).
</para>
<para>
This behavior is essentially the same since previous product
releases. The only exception is the use of ".scc", which allows kernel
configuration AND patches to be applied in a template.
</para>
<note><para>
If creating a full template is not required, a .scc file can be placed at
the top of the build, along with configuration and patches. The build
system will pickup the .scc and add it onto the patch list automatically
</para></note>
<para>
As an example, consider a simple template to update a BP:
<literallayout class='monospaced'>
&gt; cat templates/feature/extra_version/linux/extra_version.scc
patch 0001-extraversion-add-Wind-River-identifier.patch
</literallayout>
</para>
<para>
To illustrate how the previous template patch was created, the following
steps were performed:
<literallayout class='monospaced'>
&gt; cd &lt;board build&gt;/build/linux
&gt; vi Makefile
# modify EXTRAVERSION to have a unique string
&gt; git commit -s -m "extraversion: add Yocto Project identifier" Makefile
&gt; git format-patch -o &lt;path to layer&gt;/templates/feature/extra_version/linux/
&gt; echo "patch 0001-extraversion-add-Wind-River-identifier.patch" &gt; \
&lt;path to layer&gt;/templates/feature/extra_version/linux/extra_version.scc
</literallayout>
</para>
<para>
This next example creates a template with a linux subdirectory, just as we
always have for previous releases.
<literallayout class='monospaced'>
&gt; mkdir templates/features/my_feature/linux
</literallayout>
</para>
<para>
In that directory place your feature description, your
patch and configuration (if required).
<literallayout class='monospaced'>
&gt; ls templates/features/my_feature/linux
version.patch
my_feature.scc
my_feature.cfg
</literallayout>
</para>
<para>
The .scc file describes the patches, configuration and
where in the patch order the feature should be inserted.
<literallayout class='monospaced'>
patch version.patch
kconf non-hardware my_feature.cfg
</literallayout>
</para>
<para>
Configure your build with the new template
<literallayout class='monospaced'>
&dash;&dash;with-template=features/my_feature
</literallayout>
</para>
<para>
Build the kernel
<literallayout class='monospaced'>
&gt; make linux
</literallayout>
</para>
</section>
<section id='patching-kernel-cache'>
<title>Patching: Kernel Cache</title>
<para>
As previously mentioned, this example is included for completeness, and is for more advanced
applications (such as the sharing of patches between multiple sub-features).
Most patching should be done via templates, since that interface is
guaranteed not to change and the kernel-cache interface carries no such
guarantee.
</para>
<para>
At the top of a layer, create a kernel cache. The build system will recognize
any directory of the name 'kernel-*-cache' as a kernel cache.
<literallayout class='monospaced'>
&gt; cd &lt;my layer&gt;
&gt;mkdir kernel-temp-cache
</literallayout>
</para>
<para>
Make a directory with the BSP
<literallayout class='monospaced'>
&gt; mkdir kernel-temp-cache
&gt; mkdir kernel-temp-cache/my_feat
</literallayout>
</para>
<para>
Create the feature files as they were in technique #1
<literallayout class='monospaced'>
&gt; echo "patch my_patch.path" &gt; kernel-temp-cache/my_feat/my_feature.scc
</literallayout>
</para>
<para>
Configure the build with the feature added to the kernel type
<literallayout class='monospaced'>
&dash;&dash;with-kernel=standard+my_feat/my_feature.scc
</literallayout>
</para>
<para>
Build the kernel
<literallayout class='monospaced'>
&gt; make linux
</literallayout>
</para>
</section>
</section>
<section id='bsp-updating-patches-and-configuration'>
<title>BSP: Updating Patches and Configuration</title>
<para>
As was described in the "template patching" example, it is simple
to add patches to a BSP via a template, but often, it is desirable
to experiment and test patches before committing them to a template.
You can do this by modifying the BSP source.
</para>
<para>
Start as follows:
<literallayout class='monospaced'>
&gt; cd linux
&gt; git checkout &lt;bspname&gt;-&lt;kernel name&gt;
&gt; git am &lt;patch&gt;
</literallayout>
</para>
<para>
Or you can do this:
<literallayout class='monospaced'>
&gt; kgit-import -t patch &lt;patch&gt;
&gt; cd ..
&gt; make linux
</literallayout>
</para>
<para>
For details on conflict resolution and patch application, see the
git manual, or other suitable online references.
<literallayout class='monospaced'>
&gt; git am &lt;mbox&gt;
# conflict
&gt; git apply &dash;&dash;reject .git/rebase-apply/0001
# resolve conflict
&gt; git am &dash;&dash;resolved (or git am &dash;&dash;skip, git am &dash;&dash;abort)
# continue until complete
</literallayout>
</para>
<para>
Here is another example:
<literallayout class='monospaced'>
# merge the patches
# 1) single patch
&gt; git am &lt;mbox&gt;
&gt; git apply &lt;patch&lt;
&gt; kgit import -t patch &lt;patch&gt;
# 2) multiple patches
&gt; git am &lt;mbox&gt;
&gt; kgit import -t dir &lt;dir&gt;
# if kgit -t dir is used, a patch resolution cycle such
# as this can be used:
&gt; kgit import -t dir &lt;dir&gt;
# locate rejects and resolve
# options:
&gt; wiggle &dash;&dash;replace &lt;path to file&gt; &lt;path to reject&gt;
&gt; guilt refresh
or
&gt; # manual resolution
&gt; git add &lt;files&gt;
&gt; git commit -s
or
&gt; git apply &dash;&dash;reject .git/rebase-apply/0001
&gt; git add &lt;files&gt;
&gt; git am &dash;&dash;resolved
or
&gt; # merge tool of choice
# continue series:
&gt; kgit import -t dir &lt;dir&gt;
or
&gt; git am &dash;&dash;continue
</literallayout>
</para>
<para>
Once all the patches have been tested and are satisfactory, they
should be exported via the techniques described in "saving kernel
modifications."
</para>
<para>
Once the kernel has been patched and configured for a BSP, it's
configuration commonly needs to be modified. This can be done by
running [menu|x]config on the kernel tree, or working with
configuration fragments.
</para>
<para>
Using menuconfig, the operation is as follows:
<literallayout class='monospaced'>
&gt; make linux.menuconfig
&gt; make linux.rebuild
</literallayout>
</para>
<para>
Once complete, the changes are in linux-&lt;bsp&gt;-&lt;kernel type&gt;-build/.config.
To permanently save these changes, compare the .config before and after the
menuconfig, and place those changes in a configuration fragment in the
template of your choice.
</para>
<para>
Using configuration fragments, the operation is as follows (using the
si_is8620 as an example BSP):
<literallayout class='monospaced'>
&gt; vi linux/wrs/cfg/kernel-cache/bsp/si_is8620/si_is8620.cfg
&gt; make linux.reconfig
&gt; make linux.rebuild
</literallayout>
</para>
<para>
The modified configuration fragment can simply be copied out of the
linux/wrs/.. directory and placed in the appropriate template for future
application.
</para>
</section>
<section id='tools-guilt'>
<title>Tools: guilt</title>
<para>
Yocto Project has guilt integrated as a kernel tool; therefore users that are
familiar with quilt may wish to use this tool to pop, push and refresh
their patches. Note: guilt should only be used for local operations, once
a set of changes has been pushed or pulled, they should no longer be popped
or refresh by guilt, since popping, refreshing and re-pushing patches
changes their commit IDs and creating non-fast forward branches.
</para>
<para>
The following example illustrates how to add patches a Yocto Project
BSP branch via guilt:
<literallayout class='monospaced'>
&gt; cd build/linux
&gt; git checkout common_pc-standard
&gt; guilt new extra.patch
# edit files, make changes, etc
&gt; guilt refresh
&gt; guilt top
extra.patch
# export that patch to an external location
&gt; kgit export -p top /tmp
</literallayout>
</para>
<para>
Other guilt operations of interest are:
<literallayout class='monospaced'>
> guilt push, guilt push -a
> guilt pop
> guilt applied, guilt unapplied
> guilt top
> guilt refresh
> guilt header -e
> guilt next
</literallayout>
</para>
<note><para>
Guilt only uses git commands and git plumbing to perform its operations,
anything that guilt does can also be done using git directly. It is provided
as a convenience utility, but is not required and the developer can use whatever
tools or workflow they wish.
</para></note>
<para>
The following builds from the above instructions to show how guilt can be
used to assist in getting your BSP kernel patches ready. You should follow
the above instructions up to and including 'make linux.config'. In this
example I will create a new commit (patch) from scratch and import another
fictitious patch from some external public git tree (ie, a commit with full
message, signoff etc.). Please ensure you have host-cross/bin in your path.
<literallayout class='monospaced'>
%> cd linux
%> guilt-init
%> guilt-new -m fill_me_in_please first_one.patch
%> touch somefile.txt
%> guilt-add somefile.txt
%> guilt-header -e
%> guilt-refresh
%> guilt-import path_to_some_patch/patch_filename
%> guilt-push
</literallayout>
</para>
<para>
Here are a few notes about the above:
<itemizedlist>
<listitem><para>guilt-header -e &dash;&dash; this will open editing of the patch header in
EDITOR. As with a git commit the first line is the short log and
should be just that short and concise message about the commit. Follow
the short log with lines of text that will be the long description but
note Do not put a blank line after the short log. As usual you will
want to follow this with a blank line and then a signoff line.</para></listitem>
<listitem><para>The last line in the example above has 2 dots on the end. If you
don't add the 2 periods on the end guilt will think you are sending
just one patch. The wrong one!</para></listitem>
<listitem><para>The advantage to using guilt over not using guilt is that if you have a
review comment in the first patch (first_one.patch in the case of this
example) it is very easy to use guilt to pop the other patches off
allowing you to make the necessary changes without having to use more
inventive git type strategies.</para></listitem>
</itemizedlist>
</para>
</section>
<section id='tools-scc-file-example'>
<title>Tools: scc File Example</title>
<para>
This section provides some scc file examples: leaf node, 'normal' mode, and transforms.
</para>
<section id='leaf-node'>
<title>Leaf Node</title>
<para>
The following example is a BSP branch with no child branches - a leaf on the tree.
<literallayout class='monospaced'>
# these are optional, but allow standalone tree construction
define WRS_BOARD &lt;name&gt;
define WRS_KERNEL &lt;kern type&gt;
define WRS_ARCH &lt;arch&gt;
scc_leaf ktypes/standard common_pc-standard
# ^ ^
# +&dash;&dash; parent + branch name
include common_pc.scc
# ^
# +&dash;&dash;&dash; include another feature
</literallayout>
</para>
</section>
<section id='normal-mode'>
<title>'Normal' Mode</title>
<para>
Here is an example of 'normal' mode:
<literallayout class='monospaced'>
# +&dash;&dash;&dash;&dash; name of file to read
# v
kconf hardware common_pc.cfg
# ^ ^
# | +&dash;&dash; 'type: hardware or non-hardware
# |
# +&dash;&dash;&dash; kernel config
# patches
patch 0002-atl2-add-atl2-driver.patch
patch 0003-net-remove-LLTX-in-atl2-driver.patch
patch 0004-net-add-net-poll-support-for-atl2-driver.patch
</literallayout>
</para>
</section>
<section id='transforms'>
<title>Transforms</title>
<para>
This section shows an example of transforms:
<literallayout class='monospaced'>
# either of the next two options will trigger an 'auto'
# branch from existing ones, since they change the commit
# order and hence must construct their own branch
# this changes the order of future includes, if the
# passed feature is detected, the first feature is
# included AFTER it
include features/rt/rt.scc after features/kgdb/kgdb
# this also changes the order of existing branches
# this prevents the named feature from ever being
# included
exclude features/dynamic_ftrace/dynamic_ftrace.scc
# inherit the standard kernel
include ktypes/standard/standard
# LTT supplies this, so we don't want the sub-chunk from RT.
patch_trigger arch:all exclude ftrace-upstream-tracepoints.patch
# ...but we still want the one unique tracepoint it added.
patch tracepoint-add-for-sched_resched_task.patch
# these will change the named patches in the series into
# &lt;patch name&gt;.patch.&lt;feature name&gt;
# where the substituted patch is in this directory
patch_trigger arch:all ctx_mod dynamic_printk.patch
patch_trigger arch:all ctx_mod 0001-Implement-futex-macros-for-ARM.patch
# unconditionally exclude a patch
patch_trigger arch:all exclude ftrace-fix-ARM-crash.patch
</literallayout>
</para>
</section>
</section> -->
<section id='tip-dirty-string'>
<title>"-dirty" String</title>
<para>
If kernel images are being built with "-dirty" on the end of the version
string, this simply means that modifications in the source
directory have not been committed.
<literallayout class='monospaced'>
&gt; git status
</literallayout>
</para>
<para>
You can use the git command above to report modified, removed, or added files.
You should commit those changes to the tree regardless of whether they will be saved,
exported, or used.
Once you commit the changes you need to rebuild the kernel.
</para>
<para>
To brute force pickup and commit all such pending changes enter the following:
<literallayout class='monospaced'>
&gt; git add .
&gt; git commit -s -a -m "getting rid of -dirty"
</literallayout>
</para>
<para>
Next, rebuild the kernel.
</para>
</section>
<!-- <section id='kernel-transition-kernel-layer'>
<title>Creating a Transition Kernel Layer</title>
<para>
You can temporarily use a different base kernel in Yocto Project by doing the following:
<orderedlist>
<listitem><para>Create a custom kernel layer.</para></listitem>
<listitem><para>Create a git repository of the transition kernel.</para></listitem>
</orderedlist>
</para>
<para>
Once you meet those requirements you can build multiple boards and kernels.
You pay the setup cost only once.
You can then add additional BSPs and options.
</para>
<para>
Once you have the transition kernel layer in place you can evaluate
another kernel's functionality with the goal of easing transition to an integrated and validated
Yocto Project kernel.
</para> -->
<!--<para>
The next few sections describe the process:
</para> -->
<!-- <section id='creating-a-custom-kernel-layer'>
<title>Creating a Custom Kernel Layer</title>
<para>
The custom kernel layer must have the following minimum
elements:
<itemizedlist>
<listitem><para>An include of the shipped Yocto Project kernel layer.</para></listitem>
<listitem><para>A kernel-cache with an override of the standard kernel type.</para></listitem>
</itemizedlist>
</para>
<para>
This allows the inheritance of the kernel build infrastructure,
while overriding the list of patches that should be applied to
the base kernel.
</para>
<para>
The kernel layer can optionally include an override to the base
Yocto Project Linux BSP to inhibit the application of BSP specific
patches. If a custom BSP is being used, this is not required.
</para>
</section> -->
<!-- <section id='git-repo-of-the-transition-kernel'>
<title>git Repo of the Transition Kernel</title>
<para>
The kernel build system requires a base kernel repository to
seed the build process. This repository must be found in the
same layer as the build infrastructure (i.e wrll-linux-2.6.27)
in the 'git' subdir, with the name 'default_kernel'
</para>
<para>Since Yocto Project Linux ships with a default_kernel
(the validated Yocto Project kernel) in the wrll-linux-2.6.27
kernel layer, that must be removed and replaced with the
transition kernel.
</para>
<para>If the Yocto Project install cannot be directly modified
with the new default kernel, then the path to the transition
kernel layer's 'git' subdir must be passed to the build
process via:
<programlisting>
linux_GIT_BASE=&lt;absolute path to layer&gt;/git
</programlisting>
</para>
<para>
If the transition kernel has not been delivered via git,
then a git repo should be created, and bare cloned into
place. Creating this repository is as simple as:
<literallayout class='monospaced'>
&gt; tar zxvf temp_kernel.tgz
&gt; cd temp_kernel
&gt; git init
&gt; git add .
&gt; git commit -a -m "Transition kernel baseline"
'temp_kernel' can now be cloned into place via:
&gt; cd &lt;path to git base&gt;/git
&gt; git clone &dash;&dash;bare &lt;path to temp_kernel/temp_kernel default_kernel
</literallayout>
</para>
</section> -->
<!-- <section id='building-the-kernel'>
<title>Building the Kernel</title>
<para>
Once these prerequisites have been met, the kernel can be
built with:
<literallayout class='monospaced'>
&gt; make linux
</literallayout>
</para>
<para>
The new base kernel will be cloned into place and have any patches
indicated in the transition kernel's cache (or templates) applied.
The kernel build will detect the non-Yocto Project base repo and
use the HEAD of the tree for the build.
</para>
</section> -->
<!-- <section id='example'>
<title>Example</title>
<para>
This example creates a kernel layer to build the latest
kernel.org tree as the 'common_pc' BSP.
<literallayout class='monospaced'>
&gt; cd &lt;path to layers&gt;
&gt; mkdir wrll-linux-my_version
&gt; cd wrll-linux-my_version
&gt; echo "wrll-linux-2.6.27" &gt; include
&gt; mkdir -p kernel-cache/ktypes/standard
&gt; mkdir -p kernel-cache/bsp/common_pc
&gt; echo "v2.6.29" &gt; kernel-cache/kver
&gt; echo "branch common_pc-standard" &gt; kernel-cache/bsp/common_pc/common_pc.scc
&gt; echo "kconf hardware common_pc.cfg" &gt;&gt; kernel-cache/bsp/common_pc/common_pc.scc
&gt; echo "CONFIG_FOO=y" &gt; kernel-cache/bsp/common_pc/common_pc.cfg
&gt; mkdir git
&gt; cd git
&gt; git clone &dash;&dash;bare git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6.git default_kernel
</literallayout>
</para>
<para>
Configure a build to use the new layer. This means that:
<literallayout class='monospaced'>
&dash;&dash;enable-kernel-version=my_version
</literallayout>
</para>
<para>
Should be used to override the shipped default.
</para>
<para>
To build the kernel:
<literallayout class='monospaced'>
&gt; cd build
&gt; make linux_GIT_BASE=&lt;layer path&gt;/wrll-linux-my_version/git linux
</literallayout>
</para>
<para>
If this is to build without some user intervention (passing of the
GIT_BASE), you must do the clone into the wrll-linux-2.6.27/git directory.
</para>
<note><para>Unless you define valid "hardware.kcf" and "non-hardware.kcf" some
non fatal warnings will be seen. They can be fixed by populating these
files in the kernel-cache with valid hardware and non hardware config
options.
</para></note>
</section> -->
<!-- </section> -->
</section>
<!-- <itemizedlist>
<listitem><para>Introduction to this section.</para></listitem>
<listitem><para>Constructing a project-specific kernel tree.</para></listitem>
<listitem><para>Building the kernel.</para></listitem>
<listitem><para>Seeing what has changed.</para></listitem>
<listitem><para>Seeing what has changed in a particular branch.</para></listitem>
<listitem><para>Modifying the kernel.</para></listitem>
<listitem><para>Saving modifications.</para></listitem>
<listitem><para>Storing patches outside of the kernel source repository (bulk export).</para></listitem>
<listitem><para>Working with incremental changes.</para></listitem>
<listitem><para>Extracting commited changes from a working directory (exporting internally through
patches.</para></listitem>
<listitem><para>Pushing commited changes.</para></listitem>
<listitem><para>Exporting for external (upstream) submission.</para></listitem>
<listitem><para>Exporting for import into another Source Control Manager (SCM).</para></listitem>
<listitem><para>Working with the Yocto Project kernel in another SCM.</para>
<itemizedlist>
<listitem><para>Exporting the delivered kernel to an SCM.</para></listitem>
<listitem><para>Importing changed for the build.</para></listitem>
</itemizedlist></listitem>
<listitem><para>Migrating templates from version 2.0.</para></listitem>
<listitem><para>Creating a new Board Support Package (BSP).</para>
<itemizedlist>
<listitem><para>Creating from scratch.</para></listitem>
<listitem><para>Cloning.</para></listitem>
</itemizedlist></listitem>
<listitem><para>BSP bootstrapping.</para></listitem>
<listitem><para>Applying patches to the kernel through a template.</para></listitem>
<listitem><para>Applying patches to the kernel without using a template.</para></listitem>
<listitem><para>Updating patches and configurations for a BSP.</para></listitem>
<listitem><para>Using guilt to add and export patches.</para></listitem>
<listitem><para>Using scc.</para></listitem>
<listitem><para>Building a 'dirty' image.</para></listitem>
<listitem><para>Temporarily using a different base kernel.</para></listitem>
<listitem><para>Creating a custom kernel layer.</para></listitem>
<listitem><para>Creating the git repository of the transition kernel.</para></listitem>
</itemizedlist> -->
</chapter>
<!--
vim: expandtab tw=80 ts=4
-->
View Git Blame Copy Permalink