We saw builds where runtime providers were sometimes changing order and the
build result was therefore non-deterministic. For example it could show:
DEBUG: providers for lib32-initd-functions are: ['lib32-lsbinitscripts', 'lib32-initscripts']
or
DEBUG: providers for lib32-initd-functions are: ['lib32-initscripts', 'lib32-lsbinitscripts']
which could cause a test to pass or fail.
This change ensures we don't rely on the random order of dictonaries in
memory and act deterministically.
(Bitbake rev: eddab52a4e7c49bfcdc2c26004d0bc777a1e5c25)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Now that the datastore works dynamically we don't need the update_data calls
so we can just remove them. They're not actually done anything at all for
a while.
(Bitbake rev: 2300beb50333bb620013b058a7309e7f2042101d)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
getVar() now defaults to expanding by default, thus remove the True
option from getVar() calls with a regex search and replace.
Search made with the following regex: getVar ?\(( ?[^,()]*), True\)
(Bitbake rev: 3b45c479de8640f92dd1d9f147b02e1eecfaadc8)
Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
A "bitbake world -c unpack" currently breaks as not all tasks have an
unpack task. This change allows addition of world targets only if the
specified task exists which makes certain commands possible when otherwise
you just get errors which can't easily be avoided.
(Bitbake rev: ca4f5e6d01b5c8cf315f59bc86194d63c0d3d042)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Various misc changes to convert bitbake to python3 which don't warrant
separation into separate commits.
(Bitbake rev: d0f904d407f57998419bd9c305ce53e5eaa36b24)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Sometimes you can end up in a situation where you need to specify that
a specific runtime entity should be provided by a specific entry.
An example of this is bluez where you could end up in a situation where
for example:
NOTE: multiple providers are available for runtime libasound-module-bluez (bluez4, bluez5)
NOTE: consider defining a PREFERRED_PROVIDER entry to match libasound-module-bluez
NOTE: multiple providers are available for runtime bluez-hcidump (bluez-hcidump, bluez5)
NOTE: consider defining a PREFERRED_PROVIDER entry to match bluez-hcidump
The only option here is to set something like PREFERRED_PROVIDER_bluez4 = "bluez4"
which is clearly not very informative.
I've actually held off adding RPROVIDER support for a long while as this
does have sigificant potential for misuse. It doesn't for example allow
multiple runtime providers of the same name to coexist, that simply isn't
supported. It therefore doesn't replace some of the name mappings such
as busybox verses coreutils that OE-Core faces as that is a different
problem with different constraints. This mechanism is simply to provide
bitbake with a hint to decide what the dependency tree should look like.
Also, this allows us to stop printing a confusing message telling the user
to set PREFERRED_PROVIDER when the setting needed would be rather ambiguous.
[YOCTO #5044]
(Bitbake rev: 62eb39d1474d024b204634689071700605c6095c)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Back in history the code did depend on previous build results. This was
bad for determinism and we no longer do that. Update comments to match
the current behaviour.
(Bitbake rev: c3fa7e561c22786d3ac57d04c367aa50f1b3b820)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
PN can contain '_', e.g. gcc-cross-x86_64 and an override cannot
hence we do this manually rather than use OVERRIDES.
(Bitbake rev: 7a6baf02617d1edced4eaff235e73d746e2a3b68)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In certain circumstances it can be useful to get access to the world
targets list from a recipe in order to add dependencies on some or all
of the items in it. If a special function, 'calculate_extra_depends' is
defined in the recipe, and the recipe is to be built, then call it at
the right point before we calculate which tasks should be run. The
function can append items to the "deps" list in order to add
dependencies. This is not as tidy a solution as I would have liked, but
it does at least do the job.
As part of this change, the buildWorldTargets function was moved to
bb.providers to make it possible to call from taskdata.
Part of the implementation of [YOCTO #8600].
(Bitbake rev: aba0dce57c889495ec5c13919991a060aeff65d2)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
A run of "bitbake bash -c unpack" when the task has already been
completed resulted in about 9000 calls to logger.debug(). With this
patch which comments out some noisy/less usefull logging and moves
other logging calls outside loops, this number is reduced to 1000
calls. This results in cleaner logs and gives a small but
measurable 0.15s speedup. The log size dropped from 900kb to 160kb.
(Bitbake rev: d2677f084fe1d8846db77d89ef5e6ffb18dc171a)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The runtime provider debug message is the same as the build time debug
message, make them different would be better.
[YOCTO #5067]
(Bitbake rev: 92b624cbc2711d3d859994099fb63918dfd0031a)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We can't know the dynamic pacakge's name exactly, there might be a
problem, for example, when we use:
IMAGE_INSTALL_append += "ncurses-lib12344"
The ncurses-lib12344 matches ncurses' dynamic packages pattern:
PACKAGES_DYNAMIC = "^${PN}-lib.*"
so there is no errors before the rootfs creation though there is no
ncurses-lib12344.
We can warn this, but I think that we'd better not since there are many
dynamic packages, or there would be too many warnings, for example, the
perl and kernel modules, maybe we can print a debug message for it.
[YOCTO #4798]
(Bitbake rev: df372ca057f0c8c2152223b3e26ad9a30958bab6)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This lambda function is equivalent to the default sort used by sorted,
so we can simply remove this. The syntax isn't compatible with python 3.
(Bitbake rev: da8550fc884596222daa3f8794dce1abd01e5612)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
For some reason the code calls int() on the epoch component of any
PREFERRED_VERSION. Since this is compared against strings, the comparison
would always fail. This removes the stray cast and allows epochs
in preferred_version to work correctly.
[YOCTO #3187]
(Bitbake rev: 117b47553970fc5307374cbf500744b7c302efb4)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently if there are multiple preferred providers available for
a runtime dependency, bitbake will print an Error message and let
the build go on. Anyways the build should abort while any Errors
occured.
[YOCTO #2734]
(Bitbake rev: 5f81a714f4fca785780bef555b419f0250e5ec1c)
Signed-off-by: Wenzong Fan <wenzong.fan@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Make the following improvements to the show-overlayed subcommand:
* Show recipes that are overlayed when the version is higher or lower,
not just when it is the same. This gives a much better picture of the
influence each layer is having over the metadata used for building.
This can be disabled with the -s option if you just want to see
recipes with the same version as before.
* Default to showing name (PN), layer and version rather than the full
path and filename. The old style formatting can be used by specifying
the -f option.
* Mark skipped recipes as such in the output, and print them in the
correct sorted place in the list rather than at the end
* Prefix/suffix title line with === so it can be filtered out easily in
shell scripts if desired
(Bitbake rev: 43b473275d3cb2e60a14e4a52cdc4654b3f4e5e7)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We have a problem knowing when to show the user debug information and
when not to since the code has already shown the user suitable information
about why a failure is occurring.
This patch adds a bb.BBHandledException exception class which can be used
to identify those exceptions which don't need further explanation to
the user.
This patch uses this class for the bb.providers exceptions and ensures the
command handling code correctly filters the exceptions meaning that
"bitbake invalid"
now shows an simple error message and not a python traceback.
[YOCTO #1141 partial]
(Bitbake rev: eac9249b40ae1e3aa21e016010c862664e59a8d4)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In verbose mode there are hundreds of these "checking PREFERRED_PROVIDER_%s"
messages, cluttering the output and obscuring the more important resulting
"selecting %s to satisfy runtime %s due to %s" messages. Individual "checking"
lines are more suited for debug mode, similar to "sorted providers for %s
are: %s", hence convert logger.verbose() to logger.debug().
(Bitbake rev: 85dfbec26abb5b944758f5c4749b7df16c0fb2e6)
Signed-off-by: Denys Dmytriyenko <denys@ti.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If the specified PREFERRED_VERSION is not available then list the
available versions in the output. (PR is omitted.)
(Bitbake rev: eea5ff9f34bb9b2e29f5fa43deb80d4aa6ef7ddc)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Overrides on the right are the highest priority and in this case, pn-PN
and PN should take priority over any other override so fix the code to
do this.
Also, since overrides will have been processed by bitbake, we shouldn't
then be specifically looking up PREFERRED_VERSION_${PN} but just using
PREFERRED_VERSION.
This patch corrects the behaviours to match what the code is expected
to do.
(Bitbake rev: 606f1acc6fb8ccec45d6a52ed6ae6dc128011402)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Take a real world testcase where you have two recipes, each of which
contains PACKAGES_DYNAMIC = "gdk-pixbuf-loaders-*" and recipes which
RDEPEND on some gdk-pixbuf-loaders-xxx package. To select between these
you need to set a PREFERRED_PROVIDER.
These are specified in the PN namespace so the locgical conclusion is
that setting PREFERRED_PROVIDER_gdk-pixbuf = "gtk+" should work. It
doesn't and instead checks crazy things.
The code was correctly finding the two possible providers, gtk+ and
gdk-pixbuf. It was however only accepting PREFERRED_PROVIDER_gtk+
= "gdk-pixbuf" to resolve this problem which reads as the exact
opposite to what was wanted.
This patch changes the code to do something that makes sense. I suspect
that before these changes it was pretty much a null operation rubber
stamping the single provider case. For Poky at least it exposes a few
cases where -nativesdk recipes were providing the same things as their
normal counterparts but these are genuine bugs in the metadata.
I've also attempted to make the multiple provider error message human
readable as I counldn't understand it and I doubt anyone else could
either.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We use a custom Logger subclass for our loggers
This logger provides:
- 'debug' method which accepts a debug level
- 'plain' method which bypasses log formatting
- 'verbose' method which is more detail than info, but less than debug
(Bitbake rev: 3b2c1fe5ca56daebb24073a9dd45723d3efd2a8d)
Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
dict objects provide an __iter__ method for the iteration which gives you the
keys, so calling keys directly is unnecessary, and isn't really a best
practice. The only time you really need to call the keys is if there's a
danger of the dict changing out from underneith you, either due to external
forces or due to modification of the iterable in the loop. Iterations over
os.environ are apparently subject to such changes, so they must continue to
use keys().
As an aside, also switches a couple spots to using sorted() rather than
creating a temporary list with keys() and sorting that.
(Bitbake rev: 5b6ccb16c6e71e23dac6920cd2df994d67c2587b)
Signed-off-by: Chris Larson <clarson@mvista.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
This breaks preferred providers functionality
This reverts commit ee9afccf33b220a21b74fab279925eeb4771249b.
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* File licence headers were sanitised causing most of the diff.
* cooker.py was created from bin/bitbake.
* cvs fetcher port option was added
* The -f force option was fixed to work correctly
* Multiple entries in rrecrdeps are now handled correctly
(allows adding do_deploy to image depends)
git-svn-id: https://svn.o-hand.com/repos/poky/trunk@1129 311d38ba-8fff-0310-9ca6-ca027cbcb966
Richard Purdie
Joshua Lock
Paul Eggleton
Robert Yang
Wenzong Fan
Denys Dmytriyenko
Chris Larson
Richard Purdie
Chris Larson
Bernhard Reutner-Fischer
Richard Purdie