Classes (.bbclass files) can be overlayed in a layer although they are
currently located by BitBake in a different way (via BBPATH instead of
using layer priority) and thus it is useful to be able to see when this
is in effect and which layer's class is actually being used.
(Bitbake rev: f6493e4bad005a82580380d800ebf4c438292f5b)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a show-recipes subcommand which lists all available recipes, with
the layer they are provided by. You can optionally filter the output by
recipe name (PN).
(This is a generalised version of the show-overlayed subcommand.)
(Bitbake rev: 05e86ba966f5a26721891c82b21afa48768a67cc)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Make the following improvements to the show-overlayed subcommand:
* Show recipes that are overlayed when the version is higher or lower,
not just when it is the same. This gives a much better picture of the
influence each layer is having over the metadata used for building.
This can be disabled with the -s option if you just want to see
recipes with the same version as before.
* Default to showing name (PN), layer and version rather than the full
path and filename. The old style formatting can be used by specifying
the -f option.
* Mark skipped recipes as such in the output, and print them in the
correct sorted place in the list rather than at the end
* Prefix/suffix title line with === so it can be filtered out easily in
shell scripts if desired
(Bitbake rev: 43b473275d3cb2e60a14e4a52cdc4654b3f4e5e7)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
It turns out that using the collection name as specified within
layer.conf (i.e. what gets added to BBFILE_COLLECTIONS) as a name to
refer to the layer is not particularly useful, since layer creators
aren't necessarily setting these to a meaningful value - e.g. OE-Core
uses "normal", meta-oe uses "openembedded-layer", etc. In any case,
BitBake uses the directory name in its list of configured layers in the
system information presented upon starting a build, so let's just do the
same here and avoid confusion.
Also rename the get_append_layer function to get_file_layer since it is
in no way specific to bbappends.
(Bitbake rev: 35d2c1c618826e961dbf4b9889b829f469346d74)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Rather than the slightly awkward underscores, use dashes in subcommands
e.g. show-layers instead of show_layers. (The old underscored forms
continue to be accepted however.)
(Bitbake rev: 6d311ddc1be04ae5bd0a1ebee94b44968e8a3f27)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
[YOCTO #1126]
A local PR service will be started and stopped automatically along
with the bitbake invocation/ternimation.
This local PR service will be started only and if only when the
PRSERV_HOST is set to 'localhost' and PRSERV_PORT is set to '0'.
When started, the sqlite3 database is stored at
"${PERSISTEN_DIR}/prserv.sqlite3" or "${CACHE}/prserv.sqlite3".
(Bitbake rev: 9d8f45407c67ed0d3c4f820cf646de3c385067c7)
Signed-off-by: Lianhao Lu <lianhao.lu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
[YOCTO #1556]
1. Added the package_arch into the index to the DB table. Because the
change in PACKAGE_ARCH will results in different checksum, and it is
better to have seperate PR value domains for differnt PACKAGE_ARCH of
the same pakcage.
2. Changed the PR service to operate in no history mode. In this mode,
the for a given query tuple (version, pkgarch, checksum), the returned
value will be the largest among all the values of the same (version,
pkgarch). This means the PR value returned can NOT be decremented.
3. Added export function. For each (version, pkgarch) tuple, only the
record with the maximum value will be exported.
4. Added import function. The record will only be imported if the
imported value is larger than the value stored in the DB with the same
(version, pkgarch, checksum) tuple.
(Bitbake rev: 379567ee879dcdc09a51f7f1212bde1076147a6f)
Signed-off-by: Lianhao Lu <lianhao.lu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
It's recommended practice to close files when finished with them and the
code in this function was not doing this.
(Bitbake rev: 470a160813ce6cf04f83258d46ded5c2ab8bc520)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If you flatten layers that have different directory structures you may
not end up with a usable layer in the output directory - some files
won't be picked up by BitBake.
To try to avoid this problem, once flattening has completed, get the
BBFILES entries that correspond to the layer from which the output
layer's conf/layer.conf came from, and check through all of the
.bb/.bbappend files in the output directory to see if any will not be
referred to by BBFILES in the output layer. If any are found, show a
warning to the user.
(Bitbake rev: 8e4dc97614f2022855143b49d18795ca0352b237)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
You can now optionally specify two or more layers to flatten into the
output, rather than flattening all of the layers in the current
configuration (but this is still the default behaviour if no layers are
specified). Note that this means the output layer may still contain
bbappends where the corresponding recipes are not present in the list of
layers to flatten. There is also a caveat when a layer not being
flattened would be "inbetween" the flattened layers (see the command
help for details.)
Implements feature request in [YOCTO #1564].
(Bitbake rev: 379b12107ec921b4458eda320078374a509164c1)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Create a new option "--server-only" for bitbake command, which allows
bitbake runs as a server, and let frontend connect the server itself.
"--server-only" should work with "-t xmlrpc", or bitbake will exit.
bitbake --server-only -t xmlrpc will print out the server address and
port information.
(Bitbake rev: 2677254cf9792cee083605267570d93b425cc2db)
Signed-off-by: Dongxiao Xu <dongxiao.xu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If "bitbake-layers show_layers" was run when the cache was dirty forcing
a parse, it failed with the following error:
ERROR: Failure expanding variable SRCPV, expression was
${@bb.fetch2.get_srcrev(d)} which triggered exception AttributeError:
'module' object has no attribute 'fetch2'
A simple import of bb.fetch2 in bitbake-layers fixes this.
Fixes [YOCTO #1855].
(Bitbake rev: a228f0a32c2c14d62effbbba5f4fada4cd3817f6)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Since bitbake switched back to the fork instead of the exec model,
it no longer used bitbake-runtask and the code has suffered some bitrot.
bitbake-runtask is a useful tool for excuting the task without
the scheduler of bitbake, so that the external tool can invoke it
easily. It also provides a useful example of how to invoke exec_task()
with low overhead without a lot of the bitbake threading/UI overhead.
Significant changes:
* This patch changes the argument order so that the commonly used
and mandatory arguments come first.
* The taskhash file and dryrun options are now optional
* It now uses the bitbake logging mechanisms to provide processed
logging output to the console.
* The process handling to do with stdout/stderr redirection
are removed since they're no longer required.
[YOCTO #1229]
RP: Logging updates to the patch based on Roberts original patch
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The recent change which modified inheritFromOS to use the intial
environment, rather than the current environment, introduced a bug such
that variables which had been cleaned from the environment where still set
in the data store.
This patch changes things such that a list of approved environment
variables is saved after the environment is cleaned and only the variables
in this list are inherited in inheritFromOS.
CC: James Limbouris <james.limbouris@gmail.com>
CC: Chris Larson <clarson@kergoth.com>
(Bitbake rev: cb6c07054e8baf94614713ec257c643b22266d75)
Signed-off-by: Joshua Lock <josh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Make show_layers print layer name, path, and priority in a simple table
rather than just the value of BBLAYERS.
(Bitbake rev: 29afbf10c00b5ccdd8d2c063dda3f3e2b6aa1624)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The INFO: prefix makes sense in bitbake itself but in bitbake-layers is
just noise. Changing this exposed the fact that the logger was not being
set up correctly (which should be done anyway to correctly support
logging from BitBake), so this has also been fixed.
[RP: Sync with logging changes]
(Bitbake rev: 79751524cb1f0388975c6228f8c112e6af80f154)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Instead of running show_appends when no command is specified, show the
default help text. Running show_appends by default made sense previously
when that was the only real command provided, but now we have several
useful commands implemented.
(Bitbake rev: 62a2e0aa1ac8a459928d1f72783b6ca9c1756350)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Improve default help output and make unrecognised commands print it out
rather than just an error.
(Bitbake rev: 84f20eb874e7e0ae59bc76883ba9698b0cfcdb6b)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
At the moment it bugs me a lot that we only have one effective logging
level for bitbake, despite the logging module having provision to do
more advanced things. This patch:
* Changes the core log level to the lowest level we have messages of
(DEBUG-2) so messages always flow through the core logger
* Allows build.py's task logging code to log all the output regardless
of what output is on the console and sets this so log files now
always contain debug level messages even if these don't appear
on the console
* Moves the verbose/debug/debug-domains code to be a UI side setting
* Adds a filter to the UI to only print the user requested output.
The result is more complete logfiles on disk but the usual output to the
console.
There are some behaviour changes intentionally made by this patch:
a) the -v option now controls whether output is tee'd to the console.
Ultimately, we likely want to output a message to the user about where the
log file is and avoid placing output directly onto the console for every
executing task.
b) The functions get_debug_levels, the debug_levels variable, the
set_debug_levels, the set_verbosity and set_debug_domains functions are
removed from bb.msg.
c) The "logging" init function changes format.
d) All messages get fired to all handlers all the time leading to an
increase in inter-process traffic. This could likely be hacked around
short term with a function for a UI to only request events greater than
level X. Longer term, having masks for event handlers would be better.
e) logger.getEffectiveLevel() is no longer a reliable guide to what
will/won't get logged so for now we look at the default log levels instead.
[YOCTO #304]
(Bitbake rev: 45aad2f9647df14bcfa5e755b57e1ddab377939a)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The Cooker requires a copy of the environment mapping, modify
bitbake-layers to take one and pass it to the cooker.
(Bitbake rev: f5653e557b68a27e99a2a6a5c5a31d0ba0b56bcb)
Signed-off-by: Joshua Lock <josh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Create a data store in the cooker containing the values of the environment
from when BitBake is launched such that child processes can replicate
(and/or use values from) the host environment, rather than the cleaned up
environment that the main BitBake process uses.
(Bitbake rev: 54c7206165c0e7cfe5f7b243c80461baf5e7dfb1)
Signed-off-by: Joshua Lock <josh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
do_EOF is only needed when using the Python cmd class in line-oriented
mode - we are just sending single commands to it.
(Bitbake rev: 0cbf5dcaf9f67522bd58d868aa01f28e846dfc19)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If you run "bitbake-layers help commandname" it now provides some useful
help text.
(Bitbake rev: 1548edcd18a78bab74cde39167169f11ba3c4d58)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Don't always parse on initialisation - instead check for errors and then
parse when we know we need to. Avoids keeping the user waiting.
(Bitbake rev: 86adaca6ce959ad5e908c394625bc9880f3c0216)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Default server is process, adjust comments and messages accordingly.
(Bitbake rev: 8ba4d0e98401cdb808f727703913ad8ba87f8e71)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The -b option doesn't handle dependencies so note this in the help.
(Bitbake rev: b133b2e2dd5bcde705397eb38fa20a5c4da6e3b3)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fix bitbake-layers "TypeError: 'NoneType' object is not iterable" error on
startup. Commit f3be8e9a7d changed to expect
prefile and postfile to be populated and no longer expects file.
(Bitbake rev: 0f45fcab5c724aabb5b2933dfcdf88ebe256cba9)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Structure of cooker was changed since the patch for bitbake commit
edacf98cceb2fe1275042595d3fce6822fa411ca was created - cooker.pkg_pn now
has string keys, so sort it accordingly.
(Bitbake rev: 9c2a064ab7fd9b8bcca58dfeb1abfae2555f0088)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This patch moves the logic of show_appends_with_no_recipes from bitbake-layers
into bitbake. By default, a fatal message is printed; we can also define a variable
BB_DANGLINGAPPENDS_WARNONLY to make the message only a warning(the variables
could be defined in conf/local.conf with a value "yes", "true" or "1").
(Bitbake rev: f5ba7c795df7cbd58124e35970ddc5bd84cbfb8e)
Signed-off-by: Dexuan Cui <dexuan.cui@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Sort packages alphabetically but ensure appends are left in their
original order (layer priority).
(Bitbake rev: edacf98cceb2fe1275042595d3fce6822fa411ca)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Takes the current layer configuration and builds a "flattened" directory
containing the contents of all layers, with any overlayed recipes removed
and bbappends appended to the corresponding recipes. Note that some manual
cleanup may still be necessary afterwards, in particular:
* where non-recipe files (such as patches) are overwritten (the flatten
command will show a warning for these)
* where anything beyond the normal layer setup has been added to
layer.conf (only the lowest priority layer's layer.conf is used)
* Overridden/appended items from bbappends will need to be tidied up
(Bitbake rev: 296c83cc22ce281223fe91ef84bc89034cd141e7)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a show_overlayed action to list overlayed recipes.
(Bitbake rev: f0c2175dc943160e45ebd72fc932dd16ee361bfb)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Useful if you want to load a configuration file that sets values which may
also be set in bitbake.conf or one of the files it includes.
(Bitbake rev: a8246ae5400c23df0d3ee29c36f4d9f257d1e6d1)
Signed-off-by: Joshua Lock <josh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add -t options in bitbake for configuring server type.
(Bitbake rev: 5591329948648927154024bcb882c45766defac2)
Signed-off-by: Liping Ke <liping.ke@intel.com>
Signed-off-by: Joshua Lock <josh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Update exception handling syntax to use the modern style:
except ExcType as localvar
(Bitbake rev: dbf5f42b06bef81749b13aa99945cc1292a6676d)
Signed-off-by: Scott Garman <scott.a.garman@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Report bbappends correctly for skipped recipes instead of reporting the
bbappends as not having any matching recipe.
(From Poky rev: 39d72ecf6bc1da85aefb11e6481719185f0cb953)
(Bitbake rev: a71183762e5d7d9f8153832efb9595e3a21d62f1)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
At best it gave 1-2% improvement now, its 32 bit x86 only and isn't
supported after python 2.6.
PyPy is probably a better option now.
(Bitbake rev: 3c3bd0c2fa80d747f25401c17b785c7c2f3787ca)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Paul Eggleton
Richard Purdie
Lianhao Lu
Dongxiao Xu
Robert Yang
Joshua Lock
Dexuan Cui
Liping Ke
Scott Garman