Sometimes you can end up in a situation where you need to specify that
a specific runtime entity should be provided by a specific entry.
An example of this is bluez where you could end up in a situation where
for example:
NOTE: multiple providers are available for runtime libasound-module-bluez (bluez4, bluez5)
NOTE: consider defining a PREFERRED_PROVIDER entry to match libasound-module-bluez
NOTE: multiple providers are available for runtime bluez-hcidump (bluez-hcidump, bluez5)
NOTE: consider defining a PREFERRED_PROVIDER entry to match bluez-hcidump
The only option here is to set something like PREFERRED_PROVIDER_bluez4 = "bluez4"
which is clearly not very informative.
I've actually held off adding RPROVIDER support for a long while as this
does have sigificant potential for misuse. It doesn't for example allow
multiple runtime providers of the same name to coexist, that simply isn't
supported. It therefore doesn't replace some of the name mappings such
as busybox verses coreutils that OE-Core faces as that is a different
problem with different constraints. This mechanism is simply to provide
bitbake with a hint to decide what the dependency tree should look like.
Also, this allows us to stop printing a confusing message telling the user
to set PREFERRED_PROVIDER when the setting needed would be rather ambiguous.
[YOCTO #5044]
(Bitbake rev: 62eb39d1474d024b204634689071700605c6095c)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Back in history the code did depend on previous build results. This was
bad for determinism and we no longer do that. Update comments to match
the current behaviour.
(Bitbake rev: c3fa7e561c22786d3ac57d04c367aa50f1b3b820)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We now prefix log messages coming from worker task context with the
PF and task info, however parsing messages all have to be manually
prefixed which is ugly and error prone. This change modifies the log
handler filter so this happens automatically, meaning we don't have
to change every message to include that information. This makes error
messages longer but more usable.
(Bitbake rev: 1af0ccaac81e182c4ca520037dda362d180e5605)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
An invalid task causes bitbake to exit incorrectly, firing a
CommandCompleted event rather than a CommandFailed one. This
means that clients listening for CommandFailed events are
unable to detect the build failure even though one occurred.
Passing an exception string to finishAsyncCommand when a task
fails causes the CommandFailed event to be fired correctly.
[YOCTO #9087]
(Bitbake rev: 98a2c37e077b16e3bc8bb102bd18b293130d15a4)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When multiple recipes which both provide something are being built, bitbake
informs us that most likely one of them provides something the other doesn't,
which is usually correct, but unfortunately it's rather painful to figure out
exactly what that is.
This patch dumps two sets of information, one is the provides information for
each recipe, filtered so only common components are removed. The other is a list
of dependees on the recipe, since sometimes this can easily identify why something
is being built.
Its not straightforward for bitbake to obtain the information but since the
warning/error code path isn't the normal one, we can afford to go through some
less than optimal processing to aid debugging.
Also provide the same information even if we're showing a warning since its still
useful.
[YOCTO #8032]
(Bitbake rev: 96fc889b8e62ba4463c71158c4b7286c48d68cd8)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Bitbake variables don't include ":" characters so exclude these from the variable
expansion regexp.
This assists when parsing shell code which does A=${B:-C} as we don't want a
dependency on a variable called "B:-C".
(Bitbake rev: 14ed41a292123374d94f5c786a619881f2ddea42)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There were no tests that verified the value of origvalue in the callback
routines used by edit_metadata(). This patch adds one for a simple
multiline variable.
(Bitbake rev: ece3a4d02d8162dee78c2062c10291b5fd625c36)
Signed-off-by: Randy Witt <randy.e.witt@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
edit_metadata() would corrupt a variable that was multiline, but
had the ending quotes on the same line as the last value. For example:
TEST_VAR = " foo \
bar"
would become " foo ba" because the code would always delete the last
character on the line and then do it again if the line ended in the
quote. This however doesn't show up if you have:
TEST_VAR = " foo \
bar \
"
which is how all the test cases were written.
This patch fixes that bug and adds and fixes a test that matched the bugs
behavior rather than the expected behavior.
(Bitbake rev: 14f05cbdc2ad8d59a94af1c8816567d93c39c88c)
Signed-off-by: Randy Witt <randy.e.witt@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We have been seeing UnicodeDecodeErrors when handling the
ImagePkgList MetadataEvent in ORMWrapper's
save_target_file_information() if the event includes filenames
that include non-ASCII characters.
In the short term work around this by converting paths to the
unicode type when passing them to Django's ORM. This is a bit of
a hack but it's too late in the cycle to do anything more invasive.
[YOCTO #9142]
(Bitbake rev: f50fff03b3de02e73a3cc2eb9935f7c345dbddc4)
Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
buildinfohelper stores current Build object in its internal
state. Any changes to Build object will be lost if internal
state is not updated as current buildinfohelper code
saves Build object from internal state when build is
completed.
This bug causes incorrect build state when build is cancelled.
Updating internal state should fix it.
Note, that this commit updates internal state after status of
the build is changed to Build.CANCELLED. There are several other
places in the code where Build object is updated without updating
internal state. They should be carefully analyzed and fixed.
(Bitbake rev: d056cf40fc55530cb1736aedfb9a3c355884991e)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When bitbake doesn't need to build anything it still sends
ImagePkgList event with empty 'pkgdata', 'imgdata' and 'filedata'
fields. This causes crash in buildinfohelper code as it's assumed
that above mentioned fields always have data keyed by build target:
ERROR: u'core-image-minimal'
Traceback (most recent call last):
File "toasterui.py", line 423, in main
buildinfohelper.store_target_package_data(event)
File "buildinfohelper.py", line 1218, in store_target_package_data
imgdata = BuildInfoHelper._get_data_from_event(event)['imgdata'][target.target]
KeyError: u'core-image-minimal'
Fixed this by using dict.get method with empty dictionary as default
return value instead of trying to get value without checking if target
key is in the data.
(Bitbake rev: c39cc463e6d9594bf2c5ac8bb74e834f6f2cf7c8)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When a build is cancelled the build (action) is complete if it has been
caused the request being cancelled then update the build outcome
accordingly.
(Bitbake rev: d94d12914d351bf560b06d6f4e45c294b04ecaa3)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
toasterui exits event loop on one of the following events:
CommandCompleted, CommandFailed or CommandExit.
Unfortunately none of them come from bitbake when build fails.
This is normai if toasterui runs in observer mode. However, if it's
in build mode this causes toasterui to stuck in the infinite loop
waiting for new events.
The only event we can rely on is BuildCompleted as it always
comes from bitbake unlike 3 above mentioned events.
Modified the code to always shutdown toasterui in build mode
on BuildCompleted event.
(Bitbake rev: 9cd60f98b13cf7b1c518851a51e1cbaa596d8f81)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The keys 'started', 'ended', 'cpu_time_user', 'disk_io_read' and
'disk_io_write' were added to the event recently, so they don't
exist in the events generated by bitbake server from older releases.
Checking if task_to_update structure has these keys before using
them should fix build of older releases.
(Bitbake rev: 79611d0ea742263074fbb0bf5f1e39df75fd9f55)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
buildinfohelper.brbe is lost when buildinfohelper is closed.
This causes incorrect report of brbe when build is done.
Saved brbe attribute before closing buildinfohelper and used
it to report correct brbe.
Got rid of useless and confusing 'ToasterUI build done 1'
log message.
(Bitbake rev: 5d7cce0d0ed70f6b3ebd6cbad300d86964a13398)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
DepTreeGenerated event doesn't contain 'providermap' data in jethro.
Modified buildinfohelper to handle events without this data. This
should make it possible to handle jethro events coming from jethro
bitbake server by the latest buildinfohelper.
(Bitbake rev: f6dcb1c9967f042beae024146781cb8235a9e1f2)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Return value of self.BBServer.registerEventHandler differs between
jethro and master. To be able to build jethro toaster should be
able to communicate with jethro bitbake server i.e. it must work
with both old and new registerEventHandler call.
(Bitbake rev: f356c154016c428a3b53af61a075de6f14d9d1d9)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In current toaster code BRBE(build request:build environment) value
is passed from toaster to buildinfohelper through the 'SetBRBE' event.
Passing it through environment variable is easier as it doesn't
involve rpc communication between toaster and bitbake server.
It also eliminates the need in running bitbake observer process.
Added parameter 'brbe' to BuildInfoHelper.__init__
Used environment variable TOASTER_BRBE to set brbe for
buildinfohelper object.
(Bitbake rev: a0c8e2b309055e5927a8ff729d292ccaa69d0575)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
It was used for workaround git 1.7.9.2 which was released in 2012 which
should not be existed on nowadays host, so remove it to avoid
confusions.
(Bitbake rev: 6140d0cc9aecf1029ca16fed47071dfcc92f4269)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Python 2's sqlite3 module defaults to returning Unicode strings for SQL
text queries, which could trickle down to other parts of bitbake code and
cause unexpected Unicode conversions. Using byte strings avoids this issue.
For example, the git fetcher's AUTOREV support caches HEAD SHA1's using
bb.persist_data, so sometimes the git command strings passed to fetch2's
runfetchcmd() were unicode, potentially causing UnicodeDecodeErrors when
it appended the values of environment variables containing non-ASCII chars.
[YOCTO #9382]
(Bitbake rev: 09623a0811c613a47a01ae465b822d8156faca30)
Signed-off-by: Daniel Klauer <daniel.klauer@gin.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
sstate.bbclass for example writes siginfo files to a separate location
but we need to read taint data from the standard path.
(Bitbake rev: da444c9761ee15a59ea8880e3f812a5d3f1a1aaa)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The taint values need to be passed from the server to the workers to
ensure they see the same stamp values. Also ensure that the "nostamp:"
prefix isn't included in the checksum value to match the server
calculation. This ensures the checksums are all consistent.
(Bitbake rev: f80ba20e90f3746f7faee3e0ff7f249025fec8ee)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In theory all the information to recalcuate the task signatures was written
into the siginfo/sigdata files. In reality, some of the information was
written into the filename.
Firstly this patch duplicates that info into the file itself just for easy
of use since its small.
Secondly, we abstract out the existing "calculate the checksum" code for
the taskhash, and add a function to calculate the bashhash based on the
informaiton within the file.
Finally, we call these functions when we're writing out the data to check
that the data we're writing is consistent. I've found a couple of places
it wasn't and its good to know about these in advance, rather than having
a siginfo/sigdata file which a given hash in its filename but a contents
which give a different result.
This should all combine to avoid a certain class of checksum bugs making
it into world, and identifying problems in advance.
(Bitbake rev: 0f50a18d7a0ea0d68edd8e5217e29111f4b1ea0b)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When I enabled debugging of the checksum code, I found the value calculated
from siginfo/sigdata files for do_fetch tasks never matched. This was due
to an error in the way the data was being stored for these, it wasn't ordered
correctly. This patch fixes things so the checksums calculated from
siginfo/sigdata files is correct when file checksums are present.
(Bitbake rev: 046c1be7594fae2eec3d1f242ba3e9a85f1a1880)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The real method is a few lines later, this one is incorrect and
just causing confusion. Remove it.
(Bitbake rev: a896f263300f069400eae533be0daf5dedf41c95)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The tests were skipped when running without network even though they
didn't require network. This commit also adds a test case for URLs with
ports in them (the ports should not be considered when doing trusted
network checks).
(Bitbake rev: 77747de6b20538063eef3b188489a35bef225359)
Signed-off-by: Olof Johansson <olof.johansson@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Bitbake would fail to classify the following URL as belonging to a
allowed network, because of the port number in the url.
BB_ALLOWED_NETWORKS = "*.example.com"
SRC_URI = "http://git.example.com:8080/foo.tar.gz"
Since protocols aren't specified in the BB_ALLOWED_NETWORKS variable,
it's reasonable to believe that this should work regardless of protocol
being used.
(Bitbake rev: ff603df23037e10fb2cfdf150429cba3f65072cd)
Signed-off-by: Olof Johansson <olof.johansson@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add additional metadata to the layer created for build history to be
able to identify the layer and recipe later on. Specifically this is the
branch and release to which the recipe and layer are associated with
enabling differentiation of two recipes which are local release and
master and 'master' release.
[YOCTO #8528]
[YOCTO #8545]
(Bitbake rev: 3deebd887bddbbd02fd9829a180aab494b1af7c4)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
It's very possible that we added layer as:
BBLAYERS += "/path/to/meta/"
then there would be warning:
WARNING: No bb files matched BBFILE_PATTERN_core '^/path/to/meta//'
This patch can fix the problem.
(Bitbake rev: 2b1cb21d18fb18399e682021b866babeced9a4aa)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
With the code as it stands today it not possible to execute a python function
and get "normal" python exception handling behaviour. If a python function
raises an exception, it forces a traceback to be printed and the exception
becomes a FuncFailed exception.
This adds in a parameter 'pythonexception' which allows standard python
exceptions to be passed unchanged with no traceback. Ultimately we may want
to change to this convention in various places but at least it means we can
start to add sane functions now.
(Bitbake rev: 85cf22fd0ed26bb7dc7738ef2a10441891f38ae2)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There are some codepaths where the file checksum is verified and can
be found to mismatch but the 'rename' logic doesn't kick in. If code
relies on the presence of a file for the checksum having been checked
(e.g. uninative.bbclass) then it can be used when the checksum hasn't
matched.
Therefore rename the file whenever an invalid checksum is encountered.
(Bitbake rev: 69ef6c8a9db02bfa0e3fac72481ec26586a29a01)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Prevent a hang when shutdown() is called during parsing (e.g. after
SIGINT). We must not append 'None' to the jobs queue. Otherwise the
worker loop inside Parser.realrun() may break out at the wrong point,
causing the results queue thread blocking bitbake indefinitely.
[YOCTO #9319]
(Bitbake rev: 7ebea3e9a60232222efa8a546a0ff28a53029949)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently bbappend files in a layer are applied in the order they're
found on disk (as reported by glob) which means things are not
deterministic.
By sorting the glob results, the order becomes deterministic, the parsing
order for .bb files also should be deterministic as a result of this change.
[YOCTO #9138]
(Bitbake rev: 3f8febc4212fbd3485ac9bdd4ac71b8fb0a05693)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Before this patch, directory symlinks mathcing filename pattern (either
a file name or a glob pattern) were followed. However, directory
symlinks deeper in the search chain were omitted by os.walk(). Now
directory traversal behaves consistently, ignoring syminks on all
levels.
One reason for choosing not to "walk into" directory symlinks is that
dir symlinks in externalsrc.bbclass in oe-core are causing problems in
source tree checksumming.
[YOCTO #8853]
(Bitbake rev: 66dff37ebcd1dd14ebd6933d727df9cf0a641866)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If local.conf contains an invalid line, e.g.:
APPEND += " igor"
(note the leading space) then nasty tracebacks are shown which confuse the
user. Change so the parse error is simply shown without a traceback, improving
the user experience.
[YOCTO #9332]
(Bitbake rev: 148aa1fb45dcb37a756a08301a7daf270e753180)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When prefix is part of the version directory it need to ensure that
only version directory is used so remove previous directories if exists.
Example: pfx = '/dir1/dir2/v' and version = '2.5' the expected result
is 'v2.5' instead of '/dir1/dir2/v2.5'.
[YOCTO #8778]
(Bitbake rev: c760531c6dbf88135ab9f8e6f0784ccbf2cce1e4)
Signed-off-by: Aníbal Limón <limon.anibal@gmail.com>
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Little improvement for reference tokens by names instead of index.
(Bitbake rev: e8ea15eeb1857ed4bb6337836bd2fb1f5dbb1bdf)
Signed-off-by: Aníbal Limón <limon.anibal@gmail.com>
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We recently dropped lockfiles for file:// urls which in itself makes
sense.
If a file url redirects to something like an http:// mirror, we'd have
no lock taken for the original file and could race against others
trying to download the file. We therefore need to ensure there is a
lock taken in the mirror handling code.
This adds code to take such a lock, assuming it isn't the same lock
as the parent url.
(Bitbake rev: 913b6ce22cd50eac96e8937c5ffc704bfce2c023)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This fixes a problem when the repository contains multiple levels of submodules via a resursive submodule init.
(Bitbake rev: dbafbe229360ffe5908b106a9c10e274712b9b17)
Signed-off-by: Derek Straka <derek@asterius.io>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently xmlrpc server implicitly sets itself into single use mode
when bitbake server is started with anonymous port (0) or no port is
provided in command line. In this mode bitbake shuts down xmlrpc server
after build is done. This assumption is incorrect in some cases.
For example Toaster uses bitbake in this mode and expects xmlrpc server
to stay in memory.
Till recent changes single use mode was always unset due to the bug.
When the bug was fixed it broke toaster builds as Toaster couldn't
communicate with bitbake server in single use mode.
Reimplemented logic of setting single use mode. The mode is explicity
set when --server-only command line parameter is not provided to bitbake.
It doesn't depend on the port number anymore.
[YOCTO #9275]
[YOCTO #9240]
[YOCTO #9252]
(Bitbake rev: afc0dd5c532684f6201b1e12bbf4c226ea19062d)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
bb.event.ParseStarted event is not processed by toasterui, but
present in event list. This causes the following error:
WARNING: Unknown event: <bb.event.ParseStarted object at ...
and non-zero return code:
WARNING: Return value is 1
(Bitbake rev: 1cc102f3d83d9467a3a3c422254333796ba95605)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Remove the very verbose log dump from toasterui. This generates several
megabytes of not that useful debug information and actually hinders
finding the original exception.
(Bitbake rev: a21dc134bdce2c9eb5e47c770094660f0c45c398)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This makes it possible to prevent a recipe to be cached, and thus,
parsed every time.
Use with care.
[YOCTO #8853]
(Bitbake rev: 78335c1fbe5266116700c2413aac28b00423a75b)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Use a constant to define the name for the toaster custom images layer;
this constant is then used to identify this layer in various places.
(Bitbake rev: 2540969ec71612af7f9041cadcc401513e9b357b)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Update the upstream url used for testing cups versions after upstream website
changes.
(Bitbake rev: 5f06041d4936fc22297945bbbad7020bfa9083c6)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fixes [YOCTO #9231]
npm when given an invalid registry URL with --registry actually goes and
fetches from the default registry, but this commit makes sure it goes to the
specified one.
(Bitbake rev: 7c849be7c70a5db4f66fe3041486abb923b5e4ee)
Signed-off-by: Brendan Le Foll <brendan.le.foll@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Instead of trying one time with a timeout of 20 seconds try 4 times with
a timeout of 5 seconds, to account for a slow server start.
(Bitbake rev: 4a7fe63126dd8177baa5ad21e59e0bebeea8c596)
Signed-off-by: Lucas Dutra Nunes <ldnunes@ossystems.com.br>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The data available from buildstats is now more fine grained than
previously, so take advantage of that to enrich the data we save
against tasks:
* Store the CPU usage for user and system separately, and display
them separately.
* Disk IO is now measured in bytes, not ms. Also store the
read/write bytes separately.
* Store started and ended times, as well as elapsed_time. This
will enable future features such as showing which tasks were
running at a particular point in the build.
There was also a problem with how we were looking up the Task
object, which meant that the buildstats were being added to
new tasks which weren't correctly associated with the build. Fix
how we look up the Task (only looking for tasks which match the
build, and the task and recipe names in the build stats data) so
the build stats are associated with the correct task.
[YOCTO #8842]
(Bitbake rev: efa6f915566b979bdbad233ae195b413cef1b8da)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
For some reason, the values for SRC_URI[md5sum] and SRC_URI[sha256sum]
were not being expanded. That lead to the following code not working
as expected:
SRC_URI = "http://.../${PN}-${PV}.tar.gz"
MD5SUM = "123abc..."
SHA256SUM = "abcd1234..."
SRC_URI[md5sum] = "${MD5SUM}"
SRC_URI[sha256sum] = "${SHA256SUM}"
(Bitbake rev: ba011470df0ea8bd89f01c0b02ec4b3969e60ce7)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
XMLRPCServer.single_use attribute was always set to False.
This caused xmlrpc server to keep running after build is done as
BitBakeServerCommands.removeClient only shuts down server if its
single_use attribute is set to True.
(Bitbake rev: 0a60b0928a0a746a60d2c2f294ff1903963c7086)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Without this you get a rather odd traceback instead of the proper
exception message.
(Bitbake rev: 2fe1826d3077eeda6cde433d3a1e6620f74e08dd)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The output of "npm view dependencies" isn't entirely JSON if there are
multiple results, but the code here was just discarding the output if
the entire thing didn't parse as JSON. Split the output into lines and
iterate over it, parsing JSON fragments as we find them; this way we end
up with the last package's dependencies since it'll be last in the
output.
Digging further, it seems that the dependencies field reported by "npm
view" also includes optional dependencies. That wouldn't be a problem
except some of these optional dependencies may be OS-specific; for
example the "chokidar" module has "fsevents" in its optional
dependencies, but fsevents only works on MacOS X (and is only needed
there). If we erroneously pull in fsevents, not only is it unnecessary
but it causes "npm shrinkwrap" to throw a tantrum. In the absence of a
better approach, look at the os field and discard the module (along with
any of its dependencies) if it isn't for Linux.
As part of this, we can reduce the calls to npm view to one per package
since we get the entire json output rather than querying twice for two
separate fields. Overall the time taken has probably increased since we
are being more thorough about dependencies, but it's not quite as bad as
it could have been.
(Bitbake rev: 436d67fe7af89ecfbd11749a6ae1bc20e81f2cc8)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
"2 || 3" is a valid version specification for a dependency in an npm
package.json file, but of course that looks like something else when
sent to a shell. Quote the version value to avoid this.
(Bitbake rev: bea0246831a46d943d2e27d6b38f6e498bd3413c)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Continue after processing BuildStarted event to fix
WARNING: Unknown event: <bb.event.BuildStarted object at 0x2554150>
(Bitbake rev: 12f1fb8c9b70fea0c9145f881bcceb8af32df6af)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Toasterui exits only if bitbake observer shuts down.
In build mode it should exit when build is done.
Made toasterui exit on bb.command.CommandCompleted,
bb.command.CommandFailed and bb.command.CommandExit events
when it's running in build mode.
(Bitbake rev: b11f9d6d3c2eb615335901e1dcea699daf3afb4c)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently toasterui works only in observer mode. This is
artificial limitation which was made to support current toaster
design. As we decided to stop using bitbake server we'll
need to run toasterui also in build mode.
[YOCTO #7880]
(Bitbake rev: d4b5796899c3ca5c7becd7322291afd8afb35a31)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently toasterui ignores return value of setEventMask
command, which created confusing difference between set of
events set by this command and the real set used in the code.
Checked if setEventMask succeeded. Print error message and
exit if it's not.
(Bitbake rev: 6e3f13ffb47102b5df2da91fbc3f5da3179245b2)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Executing setEventMask command when bitbake server is in readonly
mode causes runCommand to fail with the following error:
'Not able to execute not readonly commands in readonly mode'
Set readonly attribute for setEventMask command to make it working
for Toaster UI. This should not do any harm as this command doesn't
influence cooker state.
(Bitbake rev: 8a47d30b2555255fbf6049c5ed69b29664c32b17)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Removed events not used in the code from the list.
Added events that are used in the code.
(Bitbake rev: 16b14ec16049cc2040a60ad5fc95f6e19dda91a6)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Reformatted and reordered list of events to make changes
easily and see them clearly in the diffs.
(Bitbake rev: 42a2d1115f2b23dc063a3172285ca3be73cf70bb)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Tarballs that are fetched down via npm repositories seem to often have
unknown headers. This doesn't affect our ability to extract the contents
though so we don't really care to see those warnings.
(Bitbake rev: b38975103e52a0c25e9ad9032c8cca1c47cbdcc2)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
npm allows you to specify a dependency as a Github username+path, or
omit the version entirely. You can hit these if you don't use a
shrinkwrap file, with the result that the code later fails due to the
output of "npm view" being empty; so handle this lazily by just ignoring
this part of the dependency if it's not really a version.
(Bitbake rev: 7b7a65c44dbdd5ba9366d4e2093f76df8758d546)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
No code changes, just fix to use four spaces.
(Bitbake rev: 66a9ee7d54ca9c25209f72da079f260ccdcc872a)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This ui does not work in master, nor has it been updated for several years.
[YOCTO #9178]
(Bitbake rev: 9fad1d13eed1f725971e6d12d3977cd31e07019a)
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We've been gearing up the Toaster web UI to replace the Hob (GTK+ based) UI
for some time now; Hob has basically been on life support for the past few
releases. As of late last month in master, Toaster has the capability to
select the packages in an image, removing the last thing that Hob could do
that Toaster couldn't.
To recap, the reasons why Hob is being removed include:
- The code is tightly woven into BitBake, making it fragile. This means it
needs significant QA and maintenance on an ongoing basis.
- Some of the implementation is not ideal; we'll be able to remove some cruft
from BitBake and OE-Core at the same time.
- It's GTK+ 2 based, not the current GTK+ 3.
- Toaster is now a much more capable UI and is being actively maintained
The discussion about removing hob can be found at:
http://lists.openembedded.org/pipermail/openembedded-architecture/2016-February/000082.html
(Bitbake rev: be2cceea159c6ca9111eff3df87b98513eab6d72)
Signed-off-by: bavery <brian.avery@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
npm-shrinkwrap will sometimes resolve a git URL which instead of a http url, in
this case go and grab the dist.tarball via npm instead of using the resolved
URL.
(Bitbake rev: eb53b927ff59aa19cf28bc46beb9f9a185a59990)
Signed-off-by: Brendan Le Foll <brendan.le.foll@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The previous commit breaks absolute pathnames in file:// urls, this
fixes it.
(Bitbake rev: b8113a1800687a37a26ac28deafdbafd74cc138e)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When in SRC_URI appears file://dir;subdir=foo unpacker copies 'dir' to ${WORKDIR}, not
${WORKDIR}/foo as it should be.
These changes are fixing following bugs as well:
Bug 6128 - Incorrect wildcard unpack behaviour in fetcher
Bug 6129 - Local directories unpack to a different location than local files
(Bitbake rev: e659a3b0c2771679057ee3e13cd42e6c62383ff2)
Signed-off-by: Alexander Shashkevich <alex@stunpix.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
For some reason the enablement piece of the patch went missing, add it.
(Bitbake rev: 0270b5a3873ed0aeca3a66198c87a6164fb644b8)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
PN can contain '_', e.g. gcc-cross-x86_64 and an override cannot
hence we do this manually rather than use OVERRIDES.
(Bitbake rev: 7a6baf02617d1edced4eaff235e73d746e2a3b68)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
For local files, there are no races with downloads, we don't need ".done"
stamps and we don't need lockfiles.
This considerably cleans up DL_DIR and all the pointless ".done" files
as well as removes stalls over local files with the same name.
(Bitbake rev: 48e903745db578d9b9b425a8d411c1369df0eb94)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Rather than create ".lock" and ".done" files with no name, error,
forcing us to fix the cases where this is a problem.
(Bitbake rev: 81158071508cc68c39db7d501370872f44d335cc)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If we specify urls such as npm://somehost;someparams the fetcher currently
does a poor job of handling mirrors of these urls due to deficiencies in the
way decodeurl works. This is because "somehost" is returned as a path, not
a host.
This tweaks the code so that unless its a file url, the host is returned
correctly.
This patch also adds test cases for these urls to the exist set of test
urls.
We need to tweak the URI() class since this thinks this is a relative url
which is clearly isn't. We also need to handle the fact that encodeurl will
error if passed a url of this form (it would want the path to be '/'.
(Bitbake rev: 83203cd2e677706e0111892a7843b83263cb8bd9)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If we wget something which looks directory like we end up with lock files
and done stamps without names, they also all use the same lockfile.
This change ensures that we use separate lock files based on the url
and avoid creating the mysterious ${DL_DIR}/.done files.
(Bitbake rev: 20bc82086018832e047345a672d74b6c1c113650)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
npm fetcher with support for shrinkwrap files and lockdown files to easily
download and install an npm package with strict dependency resolution.
The SRC_URI should be in the format of:
SRC_URI = "npm://registry.npmjs.org/;name=${PN};version=${PV}"
To add a shrinkwrap and lockdown file use:
NPM_SHRINKWRAP := "${THISDIR}/${PN}/npm-shrinkwrap.json"
NPM_LOCKDOWN := "${THISDIR}/${PN}/lockdown.json"
(Bitbake rev: dec75bbc5d075acb322dad8b1c40d6bd518dc9fd)
Signed-off-by: Brendan Le Foll <brendan.le.foll@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This is useful as npm-lockdown uses sha1 because npm releases the sha1 of
packages and whilst this is undocumented it seems no other algorithm is
supported
(Bitbake rev: fd5d9011f6dd7029895b64d8a02d33185b9aa8ae)
Signed-off-by: Brendan Le Foll <brendan.le.foll@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
At some point in the future, getVarFlag should expand by default. To
get there from the current position, we need a period of time where the
expand parameter is mandatory.
This patch starts that process. Clear errors will result from any code
which doesn't provide this. Layers can be fixed with an expression
like:
sed -e 's:\(\.getVarFlag([^,()]*, [^,()]*\)):\1, False):g' -i `grep -ril getVar *`
(Bitbake rev: aa3faebdf6af66ab34f74d328b2113de0b08c7ee)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
At some point in the future, getVar should expand by default. To get
there from the current position, we need a period of time where the
expand parameter is mandatory.
This patch starts that process. Clear errors will result from any code
which doesn't provide this. Layers can be fixed with an expression
like:
sed -e 's:\(\.getVar([^,()]*\)):\1, False):g' -i `grep -ril getVar *`
(Bitbake rev: fab717d303df0bcef737661f6917f275f35215a4)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Define a new bitbake configuration variable BB_HASH_CHECKSUM_CACHE_FILE
that can be used to define the cache file to use for file checksum
cache.
(Bitbake rev: a965b390d6240e279c190b92b17c0573e9bd604c)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If no cache file name is given a default from class variable is used,
like before.
(Bitbake rev: 2602a312818f564961de7dfa63c429d45ff9e5ac)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Move the local file checksum functionality from bb.fetch2 into
bb.checksum module.
(Bitbake rev: 4f60933283f377d68f191db849dac6c1dc7a0aed)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Before this patch the usage of cache was quite useless as the file
checksums were not actually cached on disk but re-calculated every time.
This patch utilises the new writeout_file_checksum_cache() method of the
SignatureGenerator class to do the job.
(Bitbake rev: 5ac9cbf405841ed3f65e6f99a3cee032567fb182)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Extend the API in order to be able to write out the file checksum cache
onto disk. SignatureGeneratorBasic class now implements a method that
update the fetcher local files checksum cache with the task file
dependency checksums.
(Bitbake rev: ecdabd321d48fa367b89ebffc00aa525b6eaa95c)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Drop unused 'd' argument from the cache save methods, simplifying the
API.
(Bitbake rev: 81bc1f20662c39ee8db1da45b1e8c7eb64abacf3)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The current behavior of Hob is that there is a "Run Image" button which becomes visible only for qemu images.
My suggested change is:
- if an image is selected and it is qemu-compatible, let the "Run image" button be named "Run qemu image"
- if an image is selected and it is not qemu-compatible, let the same button show up with the name "Run custom image", and besides that, an option shows-up to allow the selection of the custom script (by default it points out to runqemu script) to be used for launching this custom image
Note: in case there is more than one toggled image (qemu runnable or deployable), when the user clicks the "Run custom image" button, a dialog will be presented, allowing to choose between any of the existing images.
[YOCTO #8940]
(Bitbake rev: cc4cfc2370297b8feb2dc39d4262e73adf06c09a)
Signed-off-by: Mirela Rabulea <mirela.rabulea@nxp.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
An example prefix: `perl-5.22.1-r0 do_compile:`
(Bitbake rev: 792b759e59e31d2e43d525a6e50d866b4f51f072)
Signed-off-by: Christopher Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This lets us filter and use -l to show messages from that source specifically.
(Bitbake rev: 7946927156dec33364418988eb921ddb273660eb)
Signed-off-by: Christopher Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If the worker segfaults, we may never see a TaskFailed event from it, only
a runQueueTaskFailed event. In this case, return_value isn't getting set
leading to an incorrect exit code from bitbake. Fix by setting return_value
in both places.
(Bitbake rev: e5dd50e0d95d532fe31dde61f8c6b1a7a72321e9)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If there is a missing provider and we're using "-k" mode alongside "-w",
we could get a traceback since there was no provider. Add tests to avoid this.
(Bitbake rev: 90a4805e4e770a433b4394ea99792731e9a4b546)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We're seeing issues where the self test, which uses tinfoil doesn't
notice the changed contents of include files. The issue is
cached_statements in the parser being reused when the files have changed.
Whilst looking at this, I realised there were some other issues:
* We need to also invalidate the mtime cache when cooker restarts
* We should pass full filenames to the file invalidation code
* We should process cached_statements as part of inotify invalidation
With these fixes, the caching is more reliable for memory resident
bitbake too. It does raise some questions about cache validation and
lifecycles and indicates bitbake does need more work in the area,
preferably with the removal of the globals. This at least highlights
and works around some of the current issues.
(Bitbake rev: 3f507ff8bc467fba936cf3f31bb8ea8e02f168e8)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
CookerParser.shutdown code doesn't do all required work to shutdown
parser processes. As a result bitbake hangs if interrupted during
parsing. Putting None into the parser_quit queue should fix this issue
as it makes parsers to quit main loop.
(Bitbake rev: f67307977e8f089ce6d208d3e9de2a6a1768757e)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The brbe variable is set on the bitbake server when the Toaster
UI starts. This enables Toaster to associate events with the
correct build and build environment.
However, the brbe variable is also used when a build starts to
identify whether a new build needs to be created, or an existing
one looked up. This causes a bug for command-line builds which
happen after a Toaster-triggered build: because the brbe variable
is never unset on the server or the buildinfohelper, the new
command-line build events are treated as originating from the
previous build.
Ensure the brbe variable is reset when the buildinfohelper "closes"
a build, so that each build then either sets the brbe variable
(Toaster-triggered builds) or leaves it blank (command-line builds).
Also modify the localhostbecontroller so that the brbe variable
is not set on the server and not looked up from the server. This
ensures that it is only set when the triggerBuild() method is
called, and that it remains as None for command-line builds.
[YOCTO #9021]
(Bitbake rev: 4a6a8d0074f62208d843b06344be31ae73d9b745)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If something fails in a exec_func_python() the current stack trace shows
incorrect filenames and linenumbers. For example:
The stack trace of python calls that resulted in this exception/failure was:
File: '/media/build1/poky/meta/recipes-sato/images/core-image-sato.bb', lineno: 200, function: <module>
0196: chksum = bb.utils.sha256_file(fn)
0197: f.write('%s\t%s\n' % (chksum, os.path.relpath(fn, baseoutpath)))
0198:
0199:
*** 0200:copy_buildsystem(d)
0201:
File: '/media/build1/poky/meta/recipes-sato/images/core-image-sato.bb', lineno: 9, function: copy_buildsystem
0005:IMAGE_FEATURES += "splash package-management x11-base x11-sato ssh-server-dropbear hwcodecs"
0006:
0007:LICENSE = "MIT"
0008:
*** 0009:inherit core-image
0010:
0011:IMAGE_INSTALL += "packagegroup-core-x11-sato-games"
File: '/usr/lib/python2.7/subprocess.py', lineno: 535, function: check_call
0531: The arguments are the same as for the Popen constructor. Example:
0532:
0533: check_call(["ls", "-l"])
0534: """
*** 0535: retcode = call(*popenargs, **kwargs)
0536: if retcode:
0537: cmd = kwargs.get("args")
0538: if cmd is None:
0539: cmd = popenargs[0]
The problem is the use of "FILE" to obtain the current filename. Instead,
we therefore inject the function being executed into the methodpool which
allows us to correct its linenumber and filename information. We can then
clearly mark the initial piece as autogenerated and the rest of the linenumber
and filename information should be correct. Afterwards the trace starts:
The stack trace of python calls that resulted in this exception/failure was:
File: 'exec_python_func() autogenerated', lineno: 2, function: <module>
0001:
*** 0002:copy_buildsystem(d)
0003:
File: '/media/build1/poky/meta/classes/populate_sdk_ext.bbclass', lineno: 66, function: copy_buildsystem
0062: import glob
0063: import oe.copy_buildsystem
0064: import subprocess
0065:
*** 0066: subprocess.check_call("foo")
0067:
0068: oe_init_env_script = d.getVar('OE_INIT_ENV_SCRIPT', True)
0069:
0070: conf_bbpath = ''
File: '/usr/lib/python2.7/subprocess.py', lineno: 535, function: check_call
0531: The arguments are the same as for the Popen constructor. Example:
0532:
0533: check_call(["ls", "-l"])
0534: """
*** 0535: retcode = call(*popenargs, **kwargs)
0536: if retcode:
0537: cmd = kwargs.get("args")
0538: if cmd is None:
0539: cmd = popenargs[0]
We can't inject into methodpool at parsing time, since there may be
_append or other override operations against the function before its
execution.
(Bitbake rev: fae153095d23157dd7e72c29f683f86149ee33a8)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Right now, if you have some python code like:
X = "a"
def somefunction(d):
d.setVar("X", "b")
d.setVar("Y", "${X}")
then any sane person would expect that Y = "b" at the end of the
function. This is not the case, Y = "a".
This is due to the python function being expanded before execution, the
executed code would read d.setVar("Y", "a"). This understandably
confuses people, it also makes it near impossible to write ${} in a
python function without unintended things happening.
I think there is general agreement we should fix this and standardise
on non-expansion of python functions. We already don't expand anonymous
python (mostly).
I've checked OE-Core with buildhistory before and after this change and
there were a small number of issues this exposed which I've sent
patches for.
I propose we default to not expanding python code and then deal with
any consequences from that if/as/where identified. This will improve
new user understanding and usability of the system, it also allows
several long standing weird expansion issues to be fixed.
(Bitbake rev: 8bf33a8e92c0e188fa392030025756196c96fcbb)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We don't want to expand python functions since they aren't expanded
at execution time (e.g. anonymous python). They can also have side
effects.
This function is primarily used by toaster for variable dumps for later
display. The lack of expansion of python functions won't matter in this case
and actively helps some variable handling (e.g. SRCPV).
(Bitbake rev: 3f5520b4844a4bdd615046479ba08ed192bdc8cd)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Expanding python functions for variable dependencies doesn't really make sense,
not least since this causes execution of any inline python, it also makes it
impossible to write expressions like d.expand("${X}") of d.setVar("X", "${Y}")
which may have the wrong values if expanded now.
This starts to standardise the approach across bitbake for handling python code.
(Bitbake rev: 765a2480dbe288f64562a9611dd93b6b6dd0a64e)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We don't expand anonymous python before execution, so nor should
we do this when calculating checksums for them.
(Bitbake rev: 5f10987edda35b08970a6dd6ccf9febad271ce3e)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The functionality overlap between these two functions is significant and
its clearer to handle both things together since they are intimately
linked. There should be no behaviour change, just clearer code.
(Bitbake rev: 391aa4afc91be90d8d3ee47e1bf797d6ebe61a71)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When we use functions from the data store, they now have correct line number
and filename information. This function would attempt to correct line numbers
which doesn't need correcting, leading to misleading messages to the user.
Therefore remove this code as being obsoleted.
(Bitbake rev: 918bec86bc8ee94feb82380ff410d9fdcbe9e720)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Extended the dot styling of dependencies created by bitbake -g in dot syntax to differentiate between the various kinds.
depends: solid
rdepends: dashed
rrecommends: dotted
The change observed is that depends get an explicit style which is the same as dot default behavior and the runtime recommends get
dotted while before they were dashed. This helps to distinguish them graphically as well as eases post processing by script.
(Bitbake rev: 86e78e0ca7aa5452411f35239942ecee3d8824ec)
Signed-off-by: Henning Schroeder <henning.schroeder@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Instead of keeping the original dependency information for the pool of
CustomImagePackage reset it with each new build.
(Bitbake rev: a0b97ffc7a468bad081ce3276c74728bf6830250)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This adds the concept of CustomImagePackage this is similar to the way
layers and recipes work in that we have a set of data which is part of
the build history and a set of data which is part of the configuration
data that toaster uses to guide people in configuring their project. We
create a set of built_packages for every build but only create a package
for configuration purposes if we don't already have one, so that the
CustomImagePackage only ever contains a unique list of packages that are
available to be added and removed from a CustomImageRecipe.
(Bitbake rev: f81bb65883baa6c0f8a4d48a4de3291a10543992)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In order to use in other modules since is a common function
when needs to get proxies working.
(Bitbake rev: 85c529044381895556d603a3974de22392646a22)
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently any not catched exception in cooker causes bitbake
to hang because of not terminated children of CookerParser.
Long term solution would be to reimplement Cooker as a context
manager and terminate parser children in its __exit__ method.
Partial fix is to call CookerParser.shutdown in Cooker.shutdown in
hope that all Cooker exceptions are caught and shutdown method is
called.
[YOCTO #8900]
(Bitbake rev: 3f67600dc3292bc8208644ce89e8bf7ab95cf2e7)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Since people do copy and paste these things, clean up old syntax styles.
(Bitbake rev: 4fb028b0bd14d3e4b3fd7a89c643528728566476)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This shouldn't be in here, use a variable instead.
(Bitbake rev: 2e25d09a1ab62ccc3573d13114d59838cf4b07f2)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Anonymous functions are python functions, set the variable
flags as such so we can detect them and avoid expansion where
needed.
(Bitbake rev: 1b303785c578bbae3a89be8d751d80fba860f62e)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently bitbake doesn't parse into data.expand() expressions,
relying on high level expansion of python code to handle this.
One of the tests does however test this works.
We don't really want to be doing string expansion on python code,
so specifically parse into expand() function calls so that when
the high level behaviour is tweaked, the self tests continue to
pass and that we do continue to handle expand() function calls as
best we can.
(Bitbake rev: b12c17be5e4a74c9680876605c87f46501f78d28)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This sets the scene for removing the default False for expansion from
getVarFlag. This would later allow True to become the expand default.
On the most part this is an automatic translation with:
sed -e 's:\(\.getVarFlag([^,()]*, [^,()]*\)):\1, False):g' -i `grep -ril getVar *`
There should be no functional change from this patch.
(Bitbake rev: 7c3b99c6a716095af3ffce0b15110e91fb49c913)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This fix a problem when checking out a commit that changes the submodules
previously checkout.
Example:
Recipe uses branch A and then it updates to use branch B, but branch B has
different submodules dependencies then what branch A previously had.
(Bitbake rev: 54a3864246f2be0b62761f639a1d5c9407aded4f)
Signed-off-by: Felipe F. Tonello <eu@felipetonello.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
I before E, except after C...
(Bitbake rev: 14c9593265f7469cb8a205a46f845ac7491246df)
Signed-off-by: Phil Blundell <pb@pbcl.net>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If we're only going to parse one recipe, no point in starting
a large number of threads.
(Bitbake rev: b977faf59dc08050a44a16032fe52d1bbb80f2a1)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When running universe builds, we don't expect an error exit code for
provider warnings. Change the error messages to warnings in this case.
This deals with errors causing problems on our autobuilders amongst
other issues.
(Bitbake rev: d4989fb0355476de172169f0698757f7360e9a1f)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
findServerDetails function can be removed safely
from the source tree. Couldn't find any files
calling this function.
(Bitbake rev: 46871f769db13ccd36deedc5b6f3dbc0a3d31c4b)
Signed-off-by: Sujith H <sujith.h@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The recent change to verify_checksum() to only show checksum warnings
if no checksums are supplied made it possible to simplify the logic a
bit more.
(Bitbake rev: 1dc00b874acae44bbba9d8028d94f7bc97ddcd76)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This means that when you view the process tree, the processes
have meaningful names, aiding debugging:
$ pstree -p 30021
bash(30021)───KnottyUI(115579)───Cooker(115590)─┬─PRServ(115592)───{PRServ Handler}(115593)
├─Worker(115630)───bash:sleep(115631)───run.do_sleep.11(115633)───sleep(115634)
└─{ProcessEQueue}(115591)
$ pstree -p 30021
bash(30021)───KnottyUI(117319)───Cooker(117330)─┬─Cooker(117335)
├─PRServ(117332)───{PRServ Handler}(117333)
├─Parser-1:2(117336)
└─{ProcessEQueue}(117331)
Applies to parse threads, PR Server, cooker, the workers and execution
threads, working within the 16 character limit as best we can.
Needed to tweak the bitbake-worker magic values to tell the
workers apart.
(Bitbake rev: 539726a3b2202249a3f148d99e08909cb61902a5)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Being able to tell the bitbake processes apart is useful for debugging.
Add a helper function which allows this without making it a hard
dependency. Errors are ignored, this is just nice to have.
(Bitbake rev: fd7f1a94d196b8a3c445e313d9e699b352b1da97)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
A newline is always appended to the function body when it's written
out, so strip any trailing newlines which may be there already.
(Bitbake rev: 8a3f50936113e15d2f2822f6aee494204fa1c24f)
Signed-off-by: Andre McCurdy <armccurdy@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fix quoting of $BASH_COMMAND and avoid wrapping at 80 columns (the
script which follows is likely to contain some very long lines, so
line wrapping in bb_exit_handler() looks somewhat out of place).
(Bitbake rev: 8e12c8f8441a7c6a03e603c5789d6037945704c1)
Signed-off-by: Andre McCurdy <armccurdy@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Before, BBMASK was only permitted to contain one regular expression.
This made it hard to add to the BBMASK in multiple places as one was
supposed to separate the different regular expressions with a "|"
rather than with whitespace as is customary in BitBake variables.
Now one can specify any number of regular expressions in BBMASK. This
makes it possible to, e.g., mask out recipes in another layer from the
layer.conf file.
This also properly ignores any regular expressions that do not compile
(before an invalid regular expression would cause a ParseError in the
first bbappend file found stating that it was not a BitBake file...)
(Bitbake rev: 2c778ad50aceaffb855baf5f4aa0fed98c880870)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The core change here is to fall back to GET requests if HEAD is rejected in the
checkstatus() method, as you can't do a HEAD on Amazon S3 (used by Github
archives). This meant removing the monkey patch that the default method was GET
and adding a fixed redirect handler that doesn't reset to GET.
Also, change the way the opener is constructed from an if/elif cluster to a
conditionally constructed list.
(Bitbake rev: 6ec70d5d2e330b41b932b0a655b838a5f37df01e)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If one checksum is supplied to a SRC_URI, we really don't want to show
warnings about the other type which isn't present as one checksum
is really good enough for most cases.
(Bitbake rev: 43358a9b595b2928458a5f463cf1949394160c3a)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In certain circumstances it can be useful to get access to the world
targets list from a recipe in order to add dependencies on some or all
of the items in it. If a special function, 'calculate_extra_depends' is
defined in the recipe, and the recipe is to be built, then call it at
the right point before we calculate which tasks should be run. The
function can append items to the "deps" list in order to add
dependencies. This is not as tidy a solution as I would have liked, but
it does at least do the job.
As part of this change, the buildWorldTargets function was moved to
bb.providers to make it possible to call from taskdata.
Part of the implementation of [YOCTO #8600].
(Bitbake rev: aba0dce57c889495ec5c13919991a060aeff65d2)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The rprovides maybe contain duplicated lines when parse again, we need
check it before add to cachedata.rproviders, similar to what we had done
to cachedata.providers.
(Bitbake rev: 6c488afb0fe30a9655ec62a1d22f9f388365f012)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This is useful for newbie, for example:
$ bitbake rpm-build
ERROR: Nothing PROVIDES 'rpm-build'. Close matches:
pm-utils
rpm RPROVIDES rpm-build
[YOCTO #8881]
(Bitbake rev: 4b59eb8cc2321fe72f2988b6c9c0fecd4883255b)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If variables are unset, the code simply doesn't expand them, there
aren't errors. If the code is a python expression, this can get a bit
messy, see the attached test case. The python expansion code sees the }
of the unexpanded value rather than the close of the python expression
and then raises a SyntaxError exception.
Ideally, we'd update the code to match pairs of brackets. I don't know
how to do that with the current regex and this is unfortunately a
performance sensitive piece of code. We also run the risk of breaking
existing code in OE-Core where there are "{" characters but not "}"
to close them (PKGE and PE).
Rather than raising the exception, matching the existing "just return
the expression" behaviour seems more consistent with the standard
variable behaviour.
This addresses an issue found in the recent image.bbclass code where
there are some variables we choose not to expand (TMPDIR/DATETIME).
This patch also adds a test case for this behaviour. It wouldn't preclude
improved bracket matching code in the future either.
(Bitbake rev: d80d39e73223a50fda0090784303d2c57167bb4c)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
A recent change in bitbake added filename/lineno information to the
parameters of bb.data.build_dependencies(). The codeparser tests
required a little adaption to the changes, adding the flags to the FOO
variable used in the tests.
The error seen when running the tests is a TypeError exception raised
in bb.codeparser:
TypeError: int() argument must be a string or a number, not 'NoneType'
(Bitbake rev: f1fe674397ac5cd355696d5b4cc90b7cfa6c867f)
Signed-off-by: Olof Johansson <olof.johansson@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This reverts commit b22592af81.
That commit isn't entirely clear about why this change is needed but
I do have a usecase where this breaks things. If for example you run
"bitbake X -c packagedata" and that packagedata is in sstate, you'd
expect this to work.
If sstate doesn't contain a do_populate_sysroot for a dependency, you
would still expect the command above to succeed and you would not
expect
it to rebuild that dependency. With the current code, this isn't what
happens. The code finds the sstate for do_populate_sysroot missing,
this makes the task "uncovered" and this in turn makes it unskippable.
The example I found with this was avahi-ui, where it would trigger
a build of libdaemon to obtain its populate_sysroot.
Since this behaviour seems completely incorrect, revert the older patch
and we'll address any issues that crop up as a result.
(Bitbake rev: 36a9840a5da17cc14561881fdd6a4f2cb0a75e49)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
OSErrors occurring in toaster.bbclass are converted to
OSErrorException metadata events. They were then being swallowed
as unprocessed events by toasterui, which made them difficult
to spot.
Explicitly catch OSErrorException events and log them so they
are easier to spot and debug.
(Bitbake rev: 69f2b2bc373ce114609600b59a6b6ccef20771c9)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The event mask for toasterui doesn't include MetadataEvents.
This means that we're missing the ArtifactFileSize event
(among others), which is the one we use to populate the SDK
artifact table.
Add that event type to the toasterui event mask so we can
record SDK artifacts as they are created.
[YOCTO #7603]
(Bitbake rev: d0276a831bb8cffd42c8367895633eaa1fa1ed30)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This reverts commit 4193e99adce8e88f12ac88d7578ad39575f7e346.
It seems the underlying issue was caused by ":" in the url which isn't
supported. The patch was therefore incorrect.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This reverts commit e130dca85bac82bd4d88f94a6bf9fe36e8ad4d7c.
This is in fact a valid use case, for example the sstate.bbclass code
sets up SSTATE_MIRRORS as PREMIRRORS. Its quite common to map those
file:// urls to remote http:// urls and with the above change, this
no longer works.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This goes undetected most of the time, but when updating a repository,
if the ud.fullmirror file is not present, you end up getting an
exception instead of carrying on because the errno module is not
loaded (specifically "if exc.errno != errno.ENOENT").
(Bitbake rev: e6fca8480731ce817df9bee61438347a5e3d3017)
Signed-off-by: Kristian Amlie <kristian.amlie@mender.io>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The 'stamp-base' and 'stamp-base-clean' related codes are no longer useful,
clean them up.
[YOCTO #8468]
(Bitbake rev: 7b4c42b315d4a26dd8f2ceb874a94737bf9f183e)
Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Debugging suggests that setscene tasks are being a little greedy about their
dependencies, for example, lsof is insisting that gcc-runtime's do_package
is installed. If it isn't, its requiring gcc to rebuild.
If gcc-runtime do_package_write_xxx and do_packagedata is available, there
is no reason do_package should be needed.
The reason this is happening appears to be from the batching up of task
dependencies code, rather than setscene tasks stopping when passing over
a setscene task, they were being carried forward. This patch fixes it
so the data is 'zeroed' when passing over a setscene task boundary,
which gives the dependency graph that is expected.
After this patch, lsof will rebuild quite happily without
gcc-runtime:do_package being present, as expected. This should lead to
less dependencies being installed for builds from sstate and generally
better performance in general.
(Bitbake rev: f8bcb0a1e3b008b71c9a7cd21f76d0906f2d8068)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Added new entries to Provides model and link them to
Recipe_Dependency using 'via' field.
This data will be used by Toaster UI to show 'Provides:'
information for the recipes.
[YOCTO #6169]
(Bitbake rev: 336ddc8df611d4c8f1c3d3a06d0a85bb544c38bc)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Used providermap in store_dependency_information function
to find virtual dependencies. This should fix annoying
warnings "stpd: KeyError saving recipe dependency"
[YOCTO #6169]
(Bitbake rev: 85c416ca338c886db6e79651e44727482df9fb07)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Added providermap information to the result of buildDependTree API.
This will be used by Toaster to map virtual dependencies to recipes.
(Bitbake rev: d3e07368549f30265f59846a260efa8230a225ca)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Added optional parameter 'prefix' to filter out names that
don't start with specified prefix. Changed existing call
of get_providermap according to changed API.
Optimized the code: got rid of extra loop and temporary
list variable virts.
(Bitbake rev: df5a1392d6f91ccb44a99721c7d847da242121bb)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Its turning out that we really need a way to have bitbake just run
the setscene tasks but not any real tasks, particularly for SDK
operations.
Add an option for this since its pretty straight forward. This allows
various nasty workarounds in OE-Core to be removed.
(Bitbake rev: e4a2aafa1650a227a04d92a8a0b31efaed2c310e)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Some servers, e.g. bitbucket.org can't cope with ssh:// as part of
the git url syntax. git itself is happy enough with this but you
get server side errors when using it.
This changes the git fetcher to use the more common ssh url format
which also means we need a : before the path.
Seems a shame to have to do this due to broken servers however
it should be safe enough since this other form is the one most people
use on the commandline so it should be safe enough.
[YOCTO #8864]
(Bitbake rev: 4193e99adce8e88f12ac88d7578ad39575f7e346)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
TERM=dumb bitbake X
shows no output for task status which is suboptimal. Use the non-interactive
mode if the terminal doesn't support what we need for interactive mode giving
a better user experience. Also print a note to the console to say this has
happened.
[YOCTO #8768]
(Bitbake rev: 6f84cf4bd77f35fcd07e0b2f5149f1d6866a414d)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The documentation for findFilesMatchingInDir() was inconsistant with the
implementation: the regex was escaped before searching so effectively it's a
pure textual substring, and the machine example was broken.
(Bitbake rev: 6bef981488ec94b46dbe3797acfecf9c4b6ecbbc)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
No point counting all instances when we just want to know if there's any or not.
(Bitbake rev: 3369072efb653339da8dbd1ca864ff8e1ff899ca)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Where we add in mappings for EXPORT_FUNCTIONS, add dummy filename
and lineno data so ensure the assumption that all python functions
have this is correct.
(Bitbake rev: 547128731e62b36d2271c4390b3fee2b16c535dc)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Setup of event queue includes registering of UI handler.
This operation can fail when cooker is busy. However, there is
no need in registering UI handler for terminating the server.
Moved the call of connection.terminateServer before setting up
of the event queue. This should make terminating server to work
more reliably as it doesn't depend on setting up the event queue
and registering UI handler anymore.
This should also help Toaster backend to restart bitbake server
and observer without getting "Could not register UI event handler"
errors.
[YOCTO #8776]
(Bitbake rev: 0c5a9349f797d05c282c2ada1893e187e05f0576)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Current code in connect method sets up event queue, which requires
registering UI handler. This functionality may not be needed for
some operations, e.g. for server termination.
Moved functionality of setting up event queue in from 'connect'
method to 'setupEventQueue' in BitBakeXMLRPCServerConnection class.
(Bitbake rev: 4429871da76d6bd29e023ff42740fe7daa6b40fa)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Replaced 'while' loop with 'for' loop.
Made the code more compact and hopefully more understandable.
(Bitbake rev: 4e1e497c8432536b3522295e5b1284844ccea056)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This attribute was introduced by mistake. EventHandle is used in the
code for the same purpose.
(Bitbake rev: 8d505ec8913a7d51de48b4f52bb64c5d6a0bb08e)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Current code throws Exception("Could not register UI event handler")
if event handler can't be registered. The real reason of this is that
cooker is in busy state. Error message lacks information about this.
Added error message to the return value of registerEventHandler.
Included returned error message into the log message and exception
text.
(Bitbake rev: 07de1ca7d57dcd0cc37406feae2949da12a3fa7a)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Added method to convert state code into the human readable name.
It will be used in logging and error reporting.
(Bitbake rev: 9ec6379b27d210214d0b3f2e55962f721b7f5f51)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
7-Zip is a file archiver claiming the highest compression ratio.
This patch allows using 7-Zip commpressed files in bitbake recipes.
Two common formats are supported:
SRC_URI = "file://abc.tar.7z"
SRC_URI = "file://abc.7z"
(Bitbake rev: 7120f5bfaae54e91bc95da5667831424724ce613)
Signed-off-by: Juro Bystricky <juro.bystricky@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Poking around the ast to correct linenumbers works well for runtime failures
but not for parsing ones. We can use blank linefeeds to correct the line
numbers instead, with the advantage that we don't need to double compile.
(Bitbake rev: 10256ac3e7be7e691176ecc5d55856d88f1fe940)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The PREMIRROR isn't useful for "file://", so avoid using it, this is
good for searching speed and can reduce useless lines in log.do_fetch.
(Bitbake rev: e130dca85bac82bd4d88f94a6bf9fe36e8ad4d7c)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This makes no sense as just a note, its at least a warning and useful
to get an idea of which codepath is failing.
(Bitbake rev: 0194cf0da24dc72dab0612cd54aa5190e6cd92f2)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This codepath can be triggered by a python indentation error for example.
Showing it as an ExpansionError is misleading.
Change the code to add a warning about where the failure came from (in
particular giving the variable key name that triggered it) but raise the
proper exception.
(Bitbake rev: d49d46533704e8b4404e29abfb5a7383d704c91a)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The SyntaxError exception simply shows the recipe that failed to parse
which is pretty useless without the actual exception. We could make it
print more info, however we can just use one of the more generic handlers
instead and remove this one.
For a python indentation error, this leads to a much more readable error
message.
(Bitbake rev: 9241eb10847634e34c5ff8767ed8c114f66ff6cf)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If parse_python() fails, the output is confusing. Passing in the extra
file/line data isn't expensive and improves readability significantly.
(Bitbake rev: a4bb753488d322e0e31c31d6377ba780f2f824c4)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently, if there is something like a python indentation error in a
python function, the linenumbers and even file aren't reported correctly.
This allows lineno and filename parameters to be passed in to correct this.
The lack of a lineno parameter to python's compile() function is worked
around by using empty linefeeds. Ugly, but effective and with minimal
performance overhead.
(Bitbake rev: 5796ed550d127853808f38257f8dcc8c1cf59342)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
With the addition of function line number handling, the overhead of
the compile functions is no longer negligible. We tend to compile
the same pieces of code over and over again so wrapping a cache around
this is beneficial and removes the overhead of line numbered functions.
Life cycle of a cache using a global like this is in theory problematic
although in reality unlikely to be an issue. It can be dealt with
if/as/when we deal with the other global caches.
(Bitbake rev: 98d7002d1dca4b62042e1589fd5b9b3805d57f7a)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Now we're actively using the line numbers for other thins, having
magic values like IN_PYTHON_EOF causes problems, in particular, 32
bit overflow on 32 bit machines.
There is a neater way to signal eof to feeder(), just using an extra
parameter so use this instead and drop the IN_PYTHON_EOF magic values.
This has the added bonus that line numbers are then correct for
python functions at the end of files.
(Bitbake rev: e0f05871c2a6f1e86ae19ad343c7c6f822ddb67e)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Instead of:
"""
can only concatenate tuple (not "int") to tuple
"""
we now see:
"""
Traceback (most recent call last):
File "/media/build1/poky/bitbake/lib/bb/ui/knotty.py", line 324, in main
termfilter.updateFooter()
File "/media/build1/poky/bitbake/lib/bb/ui/knotty.py", line 210, in updateFooter
lines = 1 + int(len(content) / (self.columns + 1))
TypeError: can only concatenate tuple (not "int") to tuple
"""
which makes tacking down and fixing the problem much easier.
Also ensure we set an error exit code.
(Bitbake rev: d965bcae6cfd268406a3bd1ef77c5bb6c6e1c6d7)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When the row handling was introduced, one of the callbacks was
missed resulting in:
TypeError: can only concatenate tuple (not "int") to tuple
Fix it.
(Bitbake rev: 0b77cea2bf5b5f5704e2650fb0332f5d78037781)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This updates buildinfo helper for the recent buildstats layout change
(Bitbake rev: 30311bbe667e9f22de17fae00ff58da06a7c3e23)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When a mirror tarball is fetched, the original fetch method is called, which
unpacks the mirror tarball. After the original method is called, it checks the
localpath of the mirror tarball rather than the clone path, which isn't ideal,
particularly if the mirror tarball was removed due to being out of date. We
know the original fetch method will do what it needs to do to get its content
in the form it needs from the mirror tarball, so we can use its localpath
instead.
(Bitbake rev: 1732ad65d6c7d67b7d07cb30c074f5016adadbea)
Signed-off-by: Christopher Larson <chris_larson@mentor.com>
Signed-off-by: Awais Belal <awais_belal@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If you break the internals of better_exec(), you get a very weird
error about tb_next not being a method of None. Fix this by checking
we can step back a trace level.
(Bitbake rev: 1d710ed484f68fca0789022dde7ba877b9a894f5)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently bitbake tracebacks can have places where the line numbers are
inaccurate and filenames may be missing. These changes start to try and
correct this.
The only way I could find to correct line numbers was to compile as a
python ast, tweak the line numbers then compile to bytecode. I'm open
to better ways of doing this if anyone knows of any.
This does mean passing a few more parameters into functions, and putting
more data into the data store about functions (i.e. their filenames
and line numbers) but the improvement in debugging is more than worthwhile).
Before:
----------------
ERROR: Execution of event handler 'run_buildstats' failed
Traceback (most recent call last):
File "run_buildstats(e)", line 43, in run_buildstats(e=<bb.build.TaskStarted object at 0x7f7b7c57a590>)
NameError: global name 'notexist' is not defined
ERROR: Build of do_patch failed
ERROR: Traceback (most recent call last):
File "/media/build1/poky/bitbake/lib/bb/build.py", line 560, in exec_task
return _exec_task(fn, task, d, quieterr)
File "/media/build1/poky/bitbake/lib/bb/build.py", line 497, in _exec_task
event.fire(TaskStarted(task, logfn, flags, localdata), localdata)
File "/media/build1/poky/bitbake/lib/bb/event.py", line 170, in fire
fire_class_handlers(event, d)
File "/media/build1/poky/bitbake/lib/bb/event.py", line 109, in fire_class_handlers
execute_handler(name, handler, event, d)
File "/media/build1/poky/bitbake/lib/bb/event.py", line 81, in execute_handler
ret = handler(event)
File "run_buildstats(e)", line 43, in run_buildstats
NameError: global name 'notexist' is not defined
----------------
After:
----------------
ERROR: Execution of event handler 'run_buildstats' failed
Traceback (most recent call last):
File "/media/build1/poky/meta/classes/buildstats.bbclass", line 143, in run_buildstats(e=<bb.build.TaskStarted object at 0x7efe89284e10>):
if isinstance(e, bb.build.TaskStarted):
> trigger = notexist
pn = d.getVar("PN", True)
NameError: global name 'notexist' is not defined
ERROR: Build of do_package failed
ERROR: Traceback (most recent call last):
File "/media/build1/poky/bitbake/lib/bb/build.py", line 560, in exec_task
return _exec_task(fn, task, d, quieterr)
File "/media/build1/poky/bitbake/lib/bb/build.py", line 497, in _exec_task
event.fire(TaskStarted(task, logfn, flags, localdata), localdata)
File "/media/build1/poky/bitbake/lib/bb/event.py", line 170, in fire
fire_class_handlers(event, d)
File "/media/build1/poky/bitbake/lib/bb/event.py", line 109, in fire_class_handlers
execute_handler(name, handler, event, d)
File "/media/build1/poky/bitbake/lib/bb/event.py", line 81, in execute_handler
ret = handler(event)
File "/media/build1/poky/meta/classes/buildstats.bbclass", line 143, in run_buildstats
trigger = notexist
NameError: global name 'notexist' is not defined
----------------
(Bitbake rev: 1ff860960919ff6f8097138bc68de85bcb5f88b0)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
It can be useful to run all tasks up to but not including a specific task. The
main reason this was never added was the lack of a good syntax. This patch
uses the syntax <taskname>- to denote this behaviour which is simple, not
invasive and fits what we need from good syntax IMO, hence we can add this.
(Bitbake rev: 99ccfd411ab3f7baa111f9f3d50fae68816a9a83)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
buildinfohelper, with the new import paths for our Django models
and Django 1.8, was not getting an active connection to the database.
In buildinfohelper, call django.setup() explicitly to make sure
that the database connection is ready and models can be queried
and saved.
[YOCTO #8364]
(Bitbake rev: 671aaab8cb7c494cd5c7621b45a6f41a203d8bb5)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Rather than maintain data as part of the migrations (as was
done for the default project previously), create the default
(cli builds) project on demand as a by-product of getting
it from the database.
[YOCTO #8364]
(Bitbake rev: 5fd8e90ab9b81d1bd0d301bc1c91228ecbbea74b)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The buildinfohelper runs outside of Django, but still needs
access to our Django app classes.
Previously, the imports referenced the toaster.* app, which worked
fine. But in Django 1.8, this causes an error about the same
module being loaded multiple times from different paths.
Change the paths to our Django modules so they don't cause
this error to be thrown. We can do this as we've added our
application libraries to sys.path in the buildinfohelper anyway.
[YOCTO #8364]
(Bitbake rev: 070da64cf32c32b5ffc34d611b463c3a3960b419)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently BRBE is known to toasterui only when build is
started. It's passed to it with BuildStarted event. This is
too late as if build fails earilier than build starts toasterui
can not inform Toaster about the failure.
Set BRBE as soon as it's provided by Toaster.
This should make toasterui to be able to inform Toaster
about early build failures, e.g. failures during recipe parsing.
(Bitbake rev: d7819508dac488a64be3caec88db285cda9599ab)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If subprocess raises a CalledProcessError() error, e.g. from a call
like subprocess.check_call("false"), bitbake would try and pass the
object over IPC and fail, leading to an unusual error:
('__init__() takes at least 3 arguments (1 given)', <class 'subprocess.CalledProcessError'>, ())%
To avoid this, we turn the value into a string which prevents the
issues the IPC has trying to deal with the object (for the same reason
we deal with tracebacks here too).
[YOCTO #8752]
(Bitbake rev: 05695424b918fc81b16cbac70d79d8271a0b6045)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Toaster is not able to see ParseStarted and ParseProgress events
for command-line builds. This means it's not possible for Toaster
to detect failed builds, if the failure occurs at a point before
the BuildStarted event, as the build won't show up at all.
Add these events to the event mask, so that Toaster's toasterui
can detect and respond to them.
(Bitbake rev: 16bfd3e3d145705a2b3a05648ddbcacc7a338dfa)
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If there are more tasks running than there are lines on the terminal, the data
scrolls in ways the UI wasn't designed for. This patch adjusts the UI just to
show the processes which fit onto the number of rows in the terminal window.
You can see the total number running from the counter in the top left as usual
and this makes warning and errors messages scrolling from the top of the window
work as designed.
Ultimately, scrolling would be nice but is for another time, this fixes the
biggest UI issue on highly parallel machines.
(Bitbake rev: 67b77658e2bfa849f6f55c9c262cb11d6bfdb399)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Rename REGEX to UPSTREAM_CHECK_REGEX, REGEX_URI to UPSTREAM_CHECK_URI, and
GITTAGREGEX to UPSTREAM_CHECK_GITTAGREGEX to better reflect their purpose
and to reflect a common namespace.
(Bitbake rev: f0a9e783f9969573fd74edfa241ef14f18ac684e)
Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Removed buildinfohelper code which was trying to guess layer version of
the recipe using build request information. The code caused creation of
duplicated recipes as it resulted in layer version from layer index
instead of returning build layer version. As a result of this Toaster
UI was not showing any information about recipes.
Default approach used to find layer version seems to work much better as
it finds proper layer version. Now toaster will use it as the only
way to find layer version.
(Bitbake rev: 101690bda7ad55dc0657483233c90c374713755b)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When build information is the 'original' source of the information we
need to return the recipe that was created rather than the copy of the
recipe that is taken for keeping build history. We do this already for
command line triggered builds, but we also have this case for custom
images. We can simply check if the built_recipe exists instead of
special casing this.
(Bitbake rev: 9a8653bf602b2111dee7ee6a459682a68a695b22)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
buildinfohelper code expects only one HelpText object per
build/variable/description.
Current code creates more than one such an object, which causes
toastergui to crash with this exception:
MultipleObjectsReturned: get() returned more than one HelpText -- it returned 2!
Used git_or_create API to ensure that only one HelpText object is
created.
(Bitbake rev: e9b46803eb6f1f4044919abf90c8aeb3536e73ed)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Toaster needs bb server to be running all the time due
to merged analysis and managed modes. Server gets restarted
before every build triggered by UI, but it shouldn't be
terminated as it will influence command line builds.
[YOCTO #8279]
(Bitbake rev: e69c87c7842e796ffcd7193ecde22c8f688498f5)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Updated attributes of buildinfohelper object as they can
be changed for every build. For example brbe is set by
runbuilds for every build triggered by Toaster UI.
(Bitbake rev: ea3bc8d01704dc64f6cb7b4f5fe66c312a575174)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Make the following improvements to edit_bblayers_conf():
* Support ~ in BBLAYERS entries
* Handle where BBLAYERS items are added over multiple lines with +=
instead of one single long item
Also add some comments documenting the function arguments and return
values as well as a set of bitbake-selftest tests.
(This function is used by the bitbake-layers add, remove and
layerindex-fetch subcommands, as well as devtool when adding the
workspace layer).
(Bitbake rev: e9a0858023c7671e30cc8ebb08496304b7f26b31)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If you tried to delete the variable on the first line passed to
edit_metadata() this failed because the logic for trimming extra blank
lines didn't expect the list to be empty at that point - fix that bad
assumption.
(Bitbake rev: 8bce6fefdc5c046b916588962a2b429c0f648133)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
For two reasons:
1) The important one: we hit the following bug when doing upstream version checks
on some webpages:
https://bugs.launchpad.net/beautifulsoup/+bug/1471755
2) Also, documentation for beautifulsoup states that memory usage and
speed is improved that way.
(Bitbake rev: 7546d4aeb3ba8fda9832081b84d93138dc5e58d6)
Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Our current OVERRIDES handling means we end up caching and checking for
a lot of possible override combinations which turn out to very unlikely.
A typical example is the SRC_URI variable where we have to check if
"URI" is an override. Having spent many hours working in this code, I've
realised all the actual overrides we use are lower case and our standard
variables are mostly uppercase.
This means we could gain quite some speed advantage if we write this
into the code, that overrides only consist of lowercase characters. This
patch shows how simple this is and the resulting speed gains are
significant. This is a significant change but tests show we don't appear
to have any users of capitals in overrides in any OE-Core metadata.
Before "time bitbake -p":
real 2m4.224s
user 7m32.312s
sys 0m7.116s
After "time bitbake -p":
real 1m26.009s
user 5m10.484s
sys 0m4.640s
This check could also be made conditional however I'm not seeing a need
to do that at present.
(Bitbake rev: c9b9443faa76ee7366b1400a56f826f3f9dec1be)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This looks reasonable until you realise self.localpath is a function. Data
expansion of something which isn't a string is the original value so this
code just wastes CPU cycles and makes no sense. Remove it.
(Bitbake rev: 37214ea9bf484998b75dbc1200d53f1afc5257ed)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Trying to expand a variable which isn't a string doesn't make sense.
(Bitbake rev: 62367cca1f1793eb9827406bcdd5980fdeb80a60)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Trying to expand a variable which isn't a string doesn't make sense.
(Bitbake rev: 54d0ddd166a6707b4f8c8535639e3055b783363b)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The server state gets reset multiple times during startup and currently
we reload the codeparser cache each time. This is pointless and causes
unnecessary interaction time with bitbake.
(Bitbake rev: 4f700316933e8e7b2d27366e5ce6176895b913e7)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Rather than relying on bug 8411, which is conveniently creating
separate log files for each of our builds, create our own
log file for each build.
The log files are created in the same tmp directories that
bitbake users, but are timestamped to the millisecond to avoid
name collisions.
Each log file is opened on a ParseStarted event (for builds
triggered by Toaster) or BuildStarted event (for builds on the
command line: Toaster doesn't get the ParseStarted event
for command-line builds).
The log file is closed on the BuildCompleted event, or if the
build fails.
Because we start logging on ParseStarted for Toaster builds,
we're able to capture the "Build Configuration" section which
bitbake writes to output.
[YOCTO #8373]
(Bitbake rev: 7974203cd8bc66dff1fcc55f8723dedefaf72840)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Similarly to BB_TASK_NICE_LEVEL, add BB_TASK_IONICE_LEVEL which allows the ioprio
of tasks to be adjusted. This is in response to various qemu runtime timeouts
which have been witnessed on the autobuilder, seemingly due to IO starvation (we
already use NICE_LEVEL to adjust tasks). This has a fairly urgent need to deal
with certain 'random' failures we're seeing on the autobuilders in testing.
The format of the data in the variable is BB_TASK_IONICE_LEVEL = "<class>.<prio>".
For <class>, 2 is best effort (the default), 1 is real time and 3 is idle. You'd
need superuser privileges to use realtime. The <prio> value is a default of 4,
and can be set between 0 and 7 with 7 being lowest priority and 0 the highest.
The user can set this freely with normal privileges
Note that in order for this to take effect, you need the cfq scheduler selected
for the backing block device.
We could use nice wrapper functions for ioprio from modules like psutil however
that would complicate bitbake dependencies. This version has some magic numbers
but works on the main 32 and 64 bit x86 build architectures and can easily be
extended if ever needed. When we move to python 3.x, we can likely replace this
with standard calls.
(Bitbake rev: b9471ad147b102c45d65f5ffd9521864df7ff9c1)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The console log data is written to is created at console initialisation
time and does not change over reset events. This ensures the
BB_CONSOLELOG value is correct over such resets by preserving it.
(Bitbake rev: 335eb2db228f7543a49de71f063ac72b865c947a)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Metadata can define BB_CONSOLELOG as containing ${DATETIME} and
this can get expanded to a different value each time the variable
is read. In the case of BB_CONSOLELOG, this behaviour is not
desireable.
The values of DATE/TIME are locked down at build time but this is too
late for the purposes of ensuring the system can figure out the real
value of BB_CONSOLELOG.
The best way to do this is to set the variable into the datastore, thereby
preserving its value.
[YOCTO #8411]
(Bitbake rev: 021f2eb55ab5863b57ed1b3f19f1b329bc1ad477)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Ross Burton
Richard Purdie
Randy Witt
Joshua Lock
Ed Bartosh
Michael Wood
Robert Yang
Klauer, Daniel
Olof Johansson
Markus Lehtonen
Aníbal Limón
Derek Straka
Brendan Le Foll
Lucas Dutra Nunes
Elliot Smith
Peter Kjellerstedt
Paul Eggleton
brian avery
bavery
Alexander Shashkevich
Mirela Rabulea
Christopher Larson
Schroeder, Henning
Aníbal Limón
Felipe F. Tonello
Phil Blundell
Sujith H
Andre McCurdy
Kristian Amlie
Chen Qi
Juro Bystricky
Alexander Kanavin