If you have a layer with a blank BBFILE_PATTERN the layer was ignored
when processing the list of layers with priorities. This list is not
only used for processing recipes, but also by additional programs such
as bitbake-layers show-layers.
Without this change, a layer that provides configuration or classes
only does now show up in show-layers, which is used by the
yocto-compat-layer.py script. This causes a failures in the compatibility
check.
(Bitbake rev: 1db8336cee8cd22aaaa67e3dde5b9218840ba4a1)
Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The default layers are missing the recipe link definitions in
the fixture files, and because they are predefined they do not
get the updated information from the Layer Index.
[YOCTO #12006]
(Bitbake rev: b408dfae3685494ae34417e72c586b9eb0ddace9)
Signed-off-by: David Reyna <David.Reyna@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The list of columns in the many 'Edit Columns' pop-ups became unsorted
with the 'Toaster Table' implementation. These entries need to be
gathered and sorted in the column processing.
[YOCTO #12004]
(Bitbake rev: d06a8ff74be11e4315feac6768064355ec15611f)
Signed-off-by: David Reyna <David.Reyna@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If no external layers are defined, pokydirname is not set. Rectify
this by taking the 'be.sourcedir' as the pokydirname.
[YOCTO #12015]
(Bitbake rev: 4bcf15abbd6c399b7094762394393d41e0f347e7)
Signed-off-by: Olaf Mandel <o.mandel@menlosystems.com>
Signed-off-by: David Reyna <David.Reyna@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
One of the debug messages during build contains a list of all layers
but without spaces or other separators between them. Use pformat
instead.
[YOCTO #12014]
(Bitbake rev: 00fb88c9c969ae9b38004adf236c057628b3227d)
Signed-off-by: Olaf Mandel <o.mandel@menlosystems.com>
Signed-off-by: David Reyna <David.Reyna@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The build page provides tables related to performance that
cover build time, CPU time and disk IO. The "Edit columns"
drop down does not allow selection of the Order column and
makes it hidden as well which is not accurate from user
interaction point of view.
This patch enables the hideable property for the Order column
so it is hidden by default but the user can enable it through
the drop down if need be.
[YOCTO #11040]
(Bitbake rev: a3063e3c7e3ae21bcebe23fcf0a4fda559252fb5)
Signed-off-by: Awais Belal <awais_belal@mentor.com>
Signed-off-by: David Reyna <David.Reyna@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There must be at least one FSTYPE selected in the Toaster bitbake
variable editor page. When the user deselects all the "Save"
button gets disabled, but the error message is missing.
[YOCTO #8126]
(Bitbake rev: 407bb26eebe4ec8a68547fd98b0a15cb18895d9a)
Signed-off-by: David Reyna <David.Reyna@windriver.com>
Signed-off-by: David Reyna <David.Reyna@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In collect_bbfiles() we're monkey-patching os.listdir in order to find
which directories to watch, and then undoing that when we're finished -
however if an exception occurred for any reason there was nothing to
ensure the latter occurred. This may not have caused any issues, but as
this kind of thing really ought to be secured using try...finally just
in case, so do that.
(Bitbake rev: 021e0eda63bd629af56b5e1c380c5f98868f7332)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The code that was supposed to watch directories along BBFILES for
creation of new files wasn't working in the case where the directory did
not initially contain any matching files - since in updateCache() we are
passing the directory path to add_filewatch() and the latter function
calls os.path.dirname() on the path on the assumption that it is a file
path, and thus the parent of the directory got watched but not the
directory itself. (If the directory wasn't empty everything worked fine
since add_filewatch() was called elsewhere with the path to one of the
files in that directory, and thus the directory got watched). Add a
parameter to add_filewatch() to tell it we are passing it directory
path(s) rather than file path(s).
(Bitbake rev: b0b36ff889715227e394c59a36326c31ea21b96b)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Some of the directories searched by BBFILES are not currently being added
to the inotify watch list. This can mean that added append files are not
noticed leading to misleading metadata results when using
BB_SERVER_TIMEOUT != 0.
We use glob to expand the BBFILES references and without writing our own
implentation, figuring out the directories it searches is hard. We use
some horrible hacks here to intecept the listdir calls, I'm open
to better ways to handle this but this does ensure we have the right
watches set.
(Bitbake rev: a161ea021a1e309c25d1adc09e6e3fc58442c893)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When the 'noweb' option for Toaster is used, perform the database
check/create if the Toaster database does not yet exist.
This will allow Toaster to not fail if the first use is with 'noweb'.
This avoids potentially clashing database updates if there are
multiple overlaping 'noweb' sessions (for example with a CI system).
If the user wished to update the database, they can either use the
explicit "lsupdate" command or (re)start a web hosted Toaster session
(which is gated by the webserver's PID).
[YOCTO #11378]
(Bitbake rev: 4966a56bea7e404c574c197bdaaeb12ead895021)
Signed-off-by: David Reyna <David.Reyna@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The method "get_last_build_id" missing the "()" in several calls. This
has been failing silently with Django 1.8 but intermittently crash with
Django 1.9.
[YOCTO #11570]
(Bitbake rev: 98e424c2cd3ce8bc7df247263e44a6d6d9ede3e6)
Signed-off-by: David Reyna <David.Reyna@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The MockEvent needs to not only stand in for Toaster and Bitbake quick events,
it also needs to stand in for LogRecord, and for that it needs to provide
the new getMessage method.
[YOCTO #11440]
(Bitbake rev: e95ed33726b2a6d03382e07c6efe2ab574f84ee8)
Signed-off-by: David Reyna <David.Reyna@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
For layers directories with sub-layers (for example meta-intel),
Toaster breaks when trying to exact match a dependency event for
that sub-layer against the top level layer directory paths.
Given that top the level layer directory paths are unique, adding
a test to see if the dependency path is a subset of a top level
path is also unique.
Also, since a warning was issued the processing should not fail
on the assert but instead should continue and gather the recipes
and tasks.
[YOCTO #11149]
(Bitbake rev: e1c769eed2258bb2697e9558f484dd62a3da7399)
Signed-off-by: David Reyna <David.Reyna@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add IDs to the build page menu lines, for example "Tasks", "Recipes",
"Time", "CPU Time", "Disk I/O", and so forth. This will support test
automation.
[YOCTO #11337]
(Bitbake rev: 0bd1fcdbfa6b9bf4642f1a2b1d46767c7c15a5d1)
Signed-off-by: David Reyna <David.Reyna@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add IDs to the navigation bar links "All builds", "All projects",
and "Documentation. This will support test automation.
[YOCTO #11335]
(Bitbake rev: 4734001672cb35789be5a8d5c8b91ca57821bdba)
Signed-off-by: David Reyna <David.Reyna@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The code in these two functions is meant to be equivlanet in behaviour
but isn't. Add in code to ensure files that don't exist are handled
consistently by both functions. Users did report being able to generate
tracebacks otherwise.
(Bitbake rev: 1b66c57d0f8a9bd9f9feb2a85759e18d9a1d674b)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Ensure that when an item fetched from a premirror has an invalid checksum the
fetcher falls back to the usual logic of trying the upstream and any configured
mirrors.
(Bitbake rev: cc52b9b12c60810142252b9cb5d4268e42371b8e)
Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Move the release name lookup into the layer index
from 'morty' to the 'pyro'.
Move the bitbake branch from 1.32 to 1.34.
[YOCTO #11377]
(Bitbake rev: 21d963149b5d97452420230a252101115b708d85)
Signed-off-by: David Reyna <David.Reyna@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If you have, for example, no value set for a variable VARIABLE and a
then VARIABLE_pn-something = "value" and then you parse something.bb,
you expect getVar('VARIABLE') on the resulting datastore to return
"value", but the code here assumed that if the variable wasn't set
without overrides then we didn't need to return the overridedata and
thus we didn't get the overridden value.
In OE this affected the ability to get RECIPE_MAINTAINER for a recipe
in a script using tinfoil (since this is only set from an inc file with
_pn- overrides for each recipe, and no default is set).
(Bitbake rev: b3d2c9917c5fd8278878328794daa107ddf79b64)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In a multiconfig environment, a tinfoil call such as
tinfoil.parse_recipe("multiconfig:arduino-101-sss:gcc")
can fail with an error such as:
File "/data/master/poky/bitbake/lib/bb/tinfoil.py", line 373, in get_recipe_file
raise bb.providers.NoProvider('Unable to find any recipe file matching "%s"' % pn)
bb.providers.NoProvider: Unable to find any recipe file matching "multiconfig:arduino-101-sss:gcc"
The culprit is findBestProvider, which does not handle multiconfig.
This patch fixes the error and in the case mentioned above the tinfoil call returns:
"multiconfig:arduino-101-sss:/data/master/poky/meta/recipes-devtools/gcc/gcc_6.3.bb"
[YOCTO#11210]
(Bitbake rev: e9c03fbfd7b057b28645affa263cb4aebfa24b04)
Signed-off-by: Juro Bystricky <juro.bystricky@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fixed:
bitbake-layers show-recipes | less
press "q" to exit
There will be a Broken pipe error output as follows:
"BrokenPipeError: [Errno 32] Broken pipe"
(Bitbake rev: 4fca9a07f2d6b0544977112672b786982d7bb8f2)
Signed-off-by: Zhixiong Chi <zhixiong.chi@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Recipes/variables that raise a SkipRecipe exception are intentionally
skipped, and should not generate warnings.
[YOCTO #11319]
(Bitbake rev: edf6e6094a9f7ad4b2ba06eef8fd34756edbedce)
Signed-off-by: Nathan Rossi <nathan@nathanrossi.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We clear append/prepend on newly set variables, we should also clear
remove operations. If we don't do this, there is no way we can actually
delete a remove operation. Bitbake internally uses parsing=True to avoid
these side effects when making its own internal calls.
Also add a testcase to bitbake-selftest to ensure we remain consistent going
forward from here.
(Bitbake rev: 3a319f079d699c870d8531e051ab65e6278d1fa5)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Use the TaskArtifacts event to scan the SDK and ESDK manifests
to cleanly collect the respective artifact files.
The previous method was broken when the SDK file deployment moved
from the do_populate_sdk[_ext] tasks to their sstate tasks. That
method is disabled (but not yet removed) in preparation for the
rest of refactor work for the parent #10283 work.
[YOCTO #10850]
(Bitbake rev: 1360d7b847cc01031edb2f4b289fac9560d72fa7)
Signed-off-by: David Reyna <David.Reyna@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In the move over to argparse we've made the two signature file arguments
optional and thus if -t is not in use we need to explicitly check if at
least one signature file has been specified - and if not, show an error
and the command-line help.
(Bitbake rev: 9011366689b26305281fcb2d412dcacece917e18)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fix a missing close quote in the layer add button, and
use the layer index for the xhr_layer call.
[YOCTO #11213]
(Bitbake rev: 54d61e7c64c97799de85b3563119f0f5d051675e)
Signed-off-by: David Reyna <David.Reyna@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
A syntax fix inadvertantly broke the Toaster filter feature.
[YOCTO #11317]
(Bitbake rev: 29c9fa31d0f03553e24391568d2a5cb588595420)
Signed-off-by: David Reyna <David.Reyna@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If the output is a TTY, add colour to the output in order to make it
easier to read. At the moment this is fairly basic, just add colour to
the "titles" of each change and to the diff output.
I tried to introduce this without changing the code too much - rather
than moving everything over to the new python formatting style, I've
introduced a color_format() function which takes care of the colour
formatting, either accepting additional format arguments or
alternatively leaving the caller to use the old-style formatting (%) to
insert values.
(Bitbake rev: 04a023c8fdea1e1812fcdcaf00345aab59f9abe1)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If a variable value has changed and either the new or old value contains
spaces, a word diff should be appropriate and may be a bit more readable.
Import the "simplediff" module and use it to show a word diff (in the
style of GNU wdiff and git diff --word-diff).
Also use a similar style diff to show changes in the runtaskhashes list.
I didn't use an actual word-diff here since it's a little different - we
can be sure that the list is a list and not simply a free-format string.
(Bitbake rev: 20db6b6553c80e18afc4f43dc2495435f7477822)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If we just want to drill down to the actual differences then we don't
need to see certain things in the output, e.g. basehash changing or the
signature of dependent tasks. This will be used for comparing signatures
within buildhistory-diff in OE-Core; the default mode as used by
bitbake-diffsigs and bitbake -S printdiff remains unchanged for the
moment.
(Bitbake rev: 6543a59b1ebd3194a7c6421cffc66ebe31a67c62)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
With the -t option which recurses to find the ultimate cause of a
signature change, it was hardcoded to take the last two executions of
the specified task. On the other hand, if you have two specific task
hashes (say from bitbake output, or some other tool) then you'll want to
pick those, so provide an option to specify those as well. (Note, the
new -s option needs to be specified alongside -t rather than instead of
it.)
(Bitbake rev: d9813b1a4223cf8dc80cab90e467ddf4bf8d1078)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Argparse is a bit easier to deal with than optparse, and since we're
about to add some options, migrate this script over.
(Bitbake rev: 7f130e0b5ce6cfc6b35176465f260092cd3b3d64)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If the runtaskdeps list hasn't actually changed (but the signatures of
some of the tasks did) then it doesn't make sense to print out the old
and new lists as they are both the same and may be very long, e.g. for
do_rootfs in OE.
(Bitbake rev: cb170543605288b3e8badfac3a54c588f4c95413)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When dumping changes to signatures e.g. output of bitbake -s printdiff,
if for example a function has changed, it's much more readable to see a
unified diff of the changes rather than just printing the old function
followed by the new function, so use difflib to do that.
Note: I elected to keep to one item in the returned list per change,
rather than one line per line of output, so that the caller can still
look at changes individually if needed. Thus I've added some handling to
bitbake-diffsigs to split the change into lines so that each line is
displayed indented.
(Bitbake rev: 4d254ae63a35231c98e3f73f669b040ed1144042)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This logic doesn't work in practice, certainly not with current versions
where sigdata files are preserved in the stamps directory and therefore
there will often be multiple sigdata files - you can now easily get
files for the same signature from sstate and the stamps directory with the
result that bitbake-diffsigs reports nothing has changed. Instead, let's
change the find_siginfo function in OE-Core to simply not return
duplicates so we don't have to filter them out here.
(Bitbake rev: f0d7ab259d8ef95643e7229474b7850608aa4426)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If just one of the two signatures we want to compare aren't available,
report that one rather than misleadingly claiming both are missing.
(Bitbake rev: c87764b9147792a10efad3ed5378f36f0a055bc6)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Printing "pbzip2pbzip2_1.1.13.bb" is ugly, we need to add a separating
slash so that we get "pbzip2/pbzip2_1.1.13.bb" instead.
(Bitbake rev: 55cd4045a37afc954f4d5091f524756b266064fc)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We weren't picking the right files to compare here - according to the
order in which the list is sorted (by mtime), we need to be taking the
last two items and not the first two.
(Bitbake rev: 99f49b56115b1f8d1a0a0b911da62ffd1f997b5f)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Valid machine names cannot include spaces anywhere
in the name and doing so will result in a build failure.
This implements a mechanism to alert the user against
such a misconfiguration and does not allow input of
such machine names.
[YOCTO #8721]
(Bitbake rev: 6fb642935a3787659aa316ca906025d2d87964cf)
Signed-off-by: Awais Belal <awais_belal@mentor.com>
Signed-off-by: David Reyna <David.Reyna@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently if you specify a recrdeptask, it must exist for all recipes or
you get a python traceback. This is a bug and it should be possible to have
recipes which don't have the specified task.
As well as preventing such a traceback (which shouldn't happen, it should be
a user readable error), this allows us to fix issues in OE-Core which would
otherwise trigger the traceback.
(Bitbake rev: f93a77f2f188e18de9e3d812e86d77c2f3c71889)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This has long since been deprecated and is no longer used anywhere, FILESPATH
is the commonly used varaible which offers much more flexibility. Remove
the FILESDIR code and references from bitbake.
(Bitbake rev: 751c9dc51fd01fa64a1ff37ba2638110335f71af)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a few comments at the top of the file explaining what it's for, and
a comment pointing out that you need to increment the cache version when
changing any code that changes the output.
(Bitbake rev: c1780bc1872bb35bc28c76eff9110717f02d9db2)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add some tests to verify that we are extracting "contains" information
from python expressions in the code in the bb.data and bb.codeparser
modules.
(Bitbake rev: 88fda492df875dd79b7aecf1f34b38517fc1eb33)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Ensure we handle bb.utils.contains_any() as separate items, rather than
how we handle contains() where every item must be in the list.
Additionally, enable handling bb.utils.filter() which for the purposes
of looking at dependencies is the same as contains_any().
Additionally bump the codeparser cache and recipe cache versions to
invalidate the user's existing caches (ensuring that the changes take
effect and avoiding "taskhash mismatch" errors respectively).
(Bitbake rev: 496e3c84820a2a889d99d3604659e47a550941d5)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The code that determines variable dependencies uses the codeparser to
find references to "contains" type operations e.g. bb.utils.contains().
That function can take multiple items to check, and all specified items
have to be present. However this code didn't handle that - it assumed
that only one item would be specified and thus it was treating the
multiple items as a single item with spaces in between. Split the value
and check if all words are present in order to determine whether the
check is "set" or "unset".
(Bitbake rev: 272f1245acdd4fb85cb78612aa03627a9c246d8f)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Mark Hatle
David Reyna
Olaf Mandel
Awais Belal
Paul Eggleton
Richard Purdie
Joshua Lock
Juro Bystricky
Zhixiong Chi
Nathan Rossi
Markus Lehtonen