Commit Graph

322 Commits

Author SHA1 Message Date
Alexandru DAMIAN b611de9429 bitbake: cooker: make sure state is properly used to accept clients
This patch fixes a bug where if the build is force stopped,
subsequent clients cannot connect to the server due to
unnecessary limits on setFeature.

Additionally, we make sure that the state is properly reset
even if the BuildCompleted event firing excepts for some reason.

(Bitbake rev: 0b66b05169688aa4ddc4c54d175bb961b2f27fec)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-03-09 14:33:46 +00:00
Alexandru DAMIAN a9cd657a01 bitbake: cooker: read file watches on server idle
The inotify facility monitoring changes to the config files
could be overwhelmed by massive changes to the watched files
while server is running.

This patch adds verification the notification watches to the
server idle functions, in addition to the cooker updateCache
command which executes only infrequently, thus preventing
overflowing the notification buffer.

[YOCTO #7316]

(Bitbake rev: 996e663fd5c254292f44eca46f5fdc95af897f98)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-03-09 14:33:43 +00:00
Paul Eggleton c4da9b949a bitbake: cooker: rework LAYERDEPENDS versioning so that it is actually useful
We've had versioned dependency support in LAYERDEPENDS for quite a long
time, but I can say with pretty good certainty that almost nobody has
used it up to now because it was too strict - the specified version had
to exactly match the version in your configuration or you would get an
error; there was no "greater than or equal" option, which is usually
what you will want given that LAYERVERSION does get bumped from time to
time.

However, users mismatching layer branches and then having their builds
fail later on with some incomprehensible error is still a pretty common
problem. We can't simply use the git branch because not everyone is
always on a branch and the branch names don't always match up (and
that's not an issue). To provide a practical means to address branch
mismatching, I have reworked LAYERDEPENDS version specifications to use
the more familiar "dependency (>= version)" syntax as used with package
dependencies, support non-integer versions, and clarified the error
message a little. If we then take care to bump the version on every
breaking change, it is at least possible to have layers depend on these
changes when they update to match; we can now even support a major.minor
scheme to allow retrospectively adding a version limiter to old branches
when a new branch is created and yet still allow the old branch minor
version to be bumped if needed.

Fixes [YOCTO #5991].

(Bitbake rev: 408be9cdf2b1e32e64ea488d8051a546fb54c144)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-02-16 09:10:42 +00:00
Richard Purdie 9538ee9191 bitbake: cooker: Use expanded_data for ASSUME_PROVIDED
Users expect operators like _remove to work on this variable. We need
to use expanded_data to ensure this happens correctly.

[YOCTO #7135]

(Bitbake rev: cc4c8478fc547ea0ebf827a8d319496b39f25684)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-01-23 11:33:55 +00:00
Richard Purdie 4eb3ffe571 bitbake: cooker: Rename event_data -> expanded_data
event_data would be better named expanded_data, then we can widen its scope
to other places in cooker where we need to access an expanded data store.
We certainly don't want multiple expanded data stores.

(Bitbake rev: 1a3c1c9203e1a1452314954f1cfd771e5c1ce89b)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-01-23 11:33:54 +00:00
Robert Yang c6f933e5cf bitbake: cooker.py: trap the ENOSPC error and translate it
Trap the ENOSPC error and translate it into a human readable error
message, which is good for debugging.

(Bitbake rev: 2b084dff6ff0d274fbbf7ab07022507f7249e427)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-01-21 14:37:39 +00:00
Richard Purdie e63723f59a bitbake: cooker: Further optimise pyinotify
We currently add crazy numbers of watches on files. The per user limit is 8192
by default and on a system handling multiple builds, this can be an issue.

We don't need to watch all files individually, we can watch the directory containing
the file instead. This gives better resource utilisation and better performance
further reverting some of the performance regression seen with the introduction
of pyinotify.

(Bitbake rev: a2d441237916a99405b800c1a3dc39f860100a8c)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-01-21 14:37:39 +00:00
Richard Purdie db45ddeeaf bitbake: command/cooker/knotty: Fix memres handling of command environment changes
If the environment changes, we need memory resident bitbake to adapt to those
changes. This adds in functionality to handle this alongside the configuration
option handling code. This means that the common usage:

MACHINE=X bitbake Y

now works with the memory resident server.

(Bitbake rev: 4d1343010da757a0c126bc22475354da44aaf8e3)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-01-21 14:37:39 +00:00
Richard Purdie 967f13fda0 bitbake: cooker: Improve pyinotify performance
Benchmarks show that the introduction of pyinotify regressed
performance. This patch ensures we only call the add_watch() function
for new entries, not ones we've already processed which does improve
performance as measured by "time bitbake -p".

This doesn't completely remove the overhead but it does substantially
reduce it.

(Bitbake rev: 493361f35f6cc332d4ea359a2695622c2c91a9c2)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-01-16 08:25:30 +00:00
Richard Purdie eb7480fc4d bitbake: cooker: Fix pyinotify handling of ENOENT issues
We try and add watches for files that don't exist but if they did, would influence
the parser. The parent directory of these files may not exist, in which case we need
to watch any parent that does exist for changes. This change implements that fallback
handling.

(Bitbake rev: 979ddbe4b7340d7cf2f432f6b1eba1c58d55ff42)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-01-14 13:34:06 +00:00
Richard Purdie 6c3c3e11f6 bitbake: cooker/cache/parse: Implement pyinofity based reconfigure
Memory resident bitbake has one current flaw, changes in the base configuration
are not noticed by bitbake. The parsing cache is also refreshed on each invocation
of bitbake (although the mtime cache is not cleared so its pointless).

This change adds in pyinotify support and adds two different watchers, one
for the base configuration and one for the parsed recipes.

Changes in the latter will trigger a reparse (and an update of the mtime cache).
The former will trigger a complete reload of the configuration.

Note that this code will also correctly handle creation of new configuration files
since the __depends and __base_depends variables already track these for cache
correctness purposes.

We could be a little more clever about parsing cache invalidation, right now we just
invalidate the whole thing and recheck. For now, its better than what we have and doesn't
seem to perform that badly though.

For education and QA purposes I can document a workflow that illustrates this:

$ source oe-init-build-env-memres
$ time bitbake bash
[base configuration is loaded, recipes are parsed, bash builds]
$ time bitbake bash
[command returns quickly since all caches are valid]
$ touch ../meta/classes/gettext.bbclass
$ time bitbake bash
[reparse is triggered, time is longer than above]
$ echo 'FOO = "1"' >> conf/local.conf
$ time bitbake bash
[reparse is triggered, but with a base configuration reload too]

As far as changes go, I like this one a lot, it makes memory resident bitbake
truly usable and may be the tweak we need to make it the default.

The new pyinotify dependency is covered in the previous commit.

(Bitbake rev: 0557d03c170fba8d7efe82be1b9641d0eb229213)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-01-14 11:32:02 +00:00
Richard Purdie 1393e224b5 bitbake: cooker: Shut down the parser in error state
If the cooker is in an error state, we shouldn't continue to try parsing.
This fixes an issue where an invalid PR server is detected when bitbake
is started and ensures bitbake exits cleanly rather than hanging.

[YOCTO #6934]

(Bitbake rev: 294bb9cad294423d4f8998405ceff58655f12660)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-01-08 09:42:24 +00:00
Paul Eggleton d31b7bdf11 bitbake: cooker: add ability to ignore unmatched regexes in BBFILE_PATTERN
Add a BBFILE_PATTERN_IGNORE_EMPTY variable to allow ignoring the fact
that a regex specified in BBFILE_PATTERN for a particular collection
doesn't match any recipes. This will be used in OpenEmbedded in the
workspace layers created by "devtool" which may not always contain any
recipes (which is not cause for warning the user).

(Bitbake rev: 19c74aaa2836a88d666f0032452fac521689ab6b)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-12-23 10:18:21 +00:00
Alexandru DAMIAN 85a17f86ea bitbake: add option to write offline event log file
This patch adds a "-w/--write-log" option to bitbake
that writes an event log file for the current build.

The name of the file is passed as a parameter to the "-w"
argument. If the parameter is the empty string '', the file
name is generated in the form bitbake_eventlog_DATE.json,
where DATE is the current date and time, with second precision.

The "-w" option can also be supplied as the BBEVENTLOG
environment variable.

We add a script, toater-eventreplay, that reads an event
log file and loads the data into a Toaster database, creating
a build entry.

We modify the toasterui to fix minor issues with reading
events from an event log file.

Performance impact is undetectable under no-task executed builds.

(Bitbake rev: 1befb4a783bb7b7b387d4b5ee08830d9516f1ac2)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-12-18 10:24:06 +00:00
Richard Purdie 2fc06a12a1 bitbake: cooker: Allow featureset in error state
Currently, if an invalid PR service is selected the server will error
with a traceback. This is because its set into the error state and the
setFeature code will then fail since its not in the initial state.

Modifying the featureset in the error state is acceptable, we just need
to ensure we don't trigger a reset, that would happen from whichever
code handles the error.

[YOCTO #6934]

(Bitbake rev: c52841445d8db8f84c4da34203b195fea5874247)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-11-28 14:03:02 +00:00
Robert Yang c09e56168e bitbake: bitbake-worker: exit normally when SIGHUP
Fixed:
1) Run "bitbake recipe" in the terminal
2) Close the terminal while building
3) $ ps aux | grep bitbake-worker
There will be many processes, and they will keep the resources (e.g.,
memory), and won't exit unless kill or kill -9.

(Bitbake rev: 40d2ae0723de2bf5fee343faafb4afda40546839)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-11-20 14:15:32 +00:00
Brendan Le Foll edaeb89408 bitbake: cooker.py: fix loginfo op being set to an invalid value
loginfo "op" was being set to invalid values in saveConfigurationVar it was
working because set is a python global but append is not. This replaces both
the "op" with strings and removes the crash when calling appendConfigurationVar

(Bitbake rev: 2a8e847de85546d43600a561f9c63aa36bd69222)

Signed-off-by: Brendan Le Foll <brendan.le.foll@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-11-12 17:04:50 +00:00
Richard Purdie ff5fba8462 bitbake: knotty: Ensure commandline parameters are updated in memres server
When using options like -k, -f, -v and so on with the memory resident
server, they'd currently only be set on the initial values passed to
the original command. This ensures they now match those specified
on the commandline for the options where this makes sense.

To make this work, a command to update the options on the server side
is required so this is added.

[YOCTO #5292]

(Bitbake rev: 1c75cc4d0c8b606c1fe76e6bf60bf6a32298b105)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-09-23 20:06:06 +01:00
Bernhard Reutner-Fischer 21f41f6c43 bitbake: cooker: tweak CookerCollectFiles::find_bbfiles
since python-2.5 string.endswith() takes a tuple

(Bitbake rev: 86a67a1fd4244da9343dbf14deed1ad0d3003f32)

Signed-off-by: Bernhard Reutner-Fischer <rep.dot.nop@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-08-23 23:02:00 +01:00
Namhyung Kim f401e8f88c bitbake: cooker: Get rid of duplicate 'import re'
The re module was imported at the top, no need to do it again.

(Bitbake rev: c29c2370635075949c75937c237602600f4f10dc)

Signed-off-by: Namhyung Kim <namhyung@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-08-02 09:57:11 +01:00
Robert P. J. Day 653b557770 bitbake: "per-package" should say "per-recipe"
(Bitbake rev: 1cd369883469747a8158826bb8d67dcca2a8577f)

Signed-off-by: Robert P. J. Day <rpjday@crashcourse.ca>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-07-23 10:02:13 +01:00
Marius Avram 56c776de3b bitbake: cooker: Overwrite IMAGE_BASENAME to default in custom image
This solves a problem of custom images which inherit a base
image with IMAGE_BASENAME overwritten in their recipe by a
different value than its default one: ${PN}.

The value of IMAGE_BASE causes a crash when hob will try to
create symbolic links to the resulting images from the deploy
directory, because it will look for names similar to
<original_recipe_name>-edited-timestamp-machine.rootfs.*
which might be different from the actual resulting image.

The solution is to simply overwrite IMAGE_BASENAME in the
custom recipe to the default value in the case IMAGE_BASENAME
is found in the base recipe.

Some recipes which were affected by this problem are those
from meta-fsl-demos (e.g.: fsl-image-test).

[YOCTO #6017]

(Bitbake rev: e42ee93519000f827be49659b6b5fb7717b3d592)

Signed-off-by: Marius Avram <marius.avram@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-04-04 15:08:26 +01:00
Alexandru DAMIAN f688f6b566 bitbake: bitbake: cooker: mark setFeatures command as read-only
This patch makes sure that the setFeatures command is marked
as read-only and that it can only run if the cooker is in
the initial state.

Additionally, remove logging from the XMLRPC module in favor
of sending the exception to the client for easy processing.

	[YOCTO #6089]

(Bitbake rev: f0a1a3e24757f7658d272035620465f92a3e4c3c)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-04-01 12:53:26 +01:00
Richard Purdie ef28fcd7fe bitbake: bin/bitbake/cooker: Ensure initial featureset is optimal
If the featureset didn't match the defaults, we'd pay the price of two
base configuration parses which showed up adversely in the performance
benchmarks. This also passes the feature set into the cooker creation so
in the common case we don't have to reset the server. This speeds up
both knotty and hob startup. If the featureset doesn't match, the system
will reset as before, this just streamlines the common case.

(Bitbake rev: 1249543c4dbf3edeac033d888497864cfc807a4e)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-03-30 10:10:36 +01:00
Richard Purdie f80334d8aa bitbake: cooker: Only change self.data if it exists
With the change to more optimal default featureset behaviour, a race was
exposed by hob where the code may try and change self.data before it
exists. This change avoids that.

When the datastore is created, the cooker configuration is used so
data tracking is correctly handled regardless.

(Bitbake rev: 9d8f7efbc39d64124936ccaeb3c47a112e595d78)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-03-30 10:10:36 +01:00
Richard Purdie 8c1e43ca58 bitbake: cooker/event: Overhaul sanity test mechanism
Sanity tests are currently a pain as its hard to control when they run. This results
in issues where for example the bitbake -e output is not useful as the sanity tests
prevent it from executing. The sanity tests should run later than the base configuration.

This patch changes the sanity tests to always be event triggered with the option of
returning either events on the status, or raising errors. A new cooker feature is used
to change the behaviour depending on the controlling UI.

This does need a change to sanity.bbclass in the OE metadata but its worth the pain
for the increased flexibility and control this offers UIs and the improvement to the
user experience.

(Bitbake rev: 32e171bcc92c6e27fefee971e8714ddf8e1a8ac1)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-03-27 09:42:08 +00:00
Marius Avram 22af8031cd bitbake: cooker: delVar in removeConfigurationVar
When a variable was removed from a configuration file it was not
removed from memory. This also had the effect of not allowing
to set a new value for the same variable with saveConfigurationVar.

(Bitbake rev: 30cd1fab6633aaf50ef53eefccc6d69d598eb293)

Signed-off-by: Marius Avram <marius.avram@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-03-25 13:26:54 +00:00
Marius Avram e665de5500 bitbake: bitbake: cooker: some IMAGE_FEATURES not recognized
Fixes an issue in hob which happened when the local.conf file was
modified externally by appending "eclipse-debug" to the IMAGE_FEATURES
variable. The reason of the problem is that some IMAGE_FEATURES are
not available in the image.bbclass file and they are declared in the
core-image.bbclass. Now a default hob image will inherit core-image.

[YOCTO #5711]

(Bitbake rev: 81413d94f40f58d790d7a7dc4259108f9c5d4fc0)

Signed-off-by: Marius Avram <marius.avram@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-03-14 07:25:40 -07:00
Richard Purdie 0b4ae1c774 bitbake: cooker: Handle SIGTERM more gracefully
If the cooker receives a SIGTERM it currently hangs using 100% CPU,

This patch adds in an intercept for the event and puts the cooker into
shutdown mode allowing it to exit cleanly/safely and avoiding the hang.

(Bitbake rev: 00c22434123739b0819b31d7b1d353901a3e12da)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-03-10 11:10:00 -07:00
Richard Purdie d51bf8d9f6 bitbake: cooker/command: Add setFeatures command
Add a command to allow backends to set particular 'features' on the
cooker (server).

(Bitbake rev: f547d6ec6cfd677d71fa96dd3c69823c00dc6c69)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-03-10 11:09:59 -07:00
Irina Patru 14e548dbb7 bitbake: bitbake: hob: check if parser has attribute 'shutdown'
It must be checked first if parser has the attribute 'shutdown' when
user hits Stop button and the forceshutdown state is given.

[HOB #5579]

(Bitbake rev: 46943b442ea4fa778f70590b6dcce483595efaf8)

Signed-off-by: Irina Patru <irina.patru@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-01-28 00:52:57 +00:00
Richard Purdie 3c5b68356d bitbake: cooker: Add option of specifying indvidual tasks for targets
Currently its near enough impossible to tell bitbake to run target X,
task Y and target A, task B. We could hack various parts of the API
around but it would mean incompatibilities.

An alternative is to accept the syntax "<target>:do_<task>" as a target
name. The default task would be used where the task is unspecified.

This has the advantage that its neat/clean code and works from all
current APIs including the commandline.

(Bitbake rev: 55f6bee3114e582333a1784caeddb197b9163d02)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-01-28 00:52:56 +00:00
Robert Yang f4e85cad1b bitbake: bitbake/lib/bb/cooker.py: remove a duplicated self.data
(Bitbake rev: 34afbdd0fc809b8fb20696aeef3e6a61d6812e16)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-01-03 15:16:42 +00:00
Richard Purdie d1e66643ae bitbake: cooker/command/hob: Cleanup configuration init/reset functions and commands
initConfigurationData and loadConfigurationData are similar functions, the only
reason for them appears to be to be able to reset the pre/post configuration
files. The current code is confusing and unmaintainable.

Instead this patch creates a new Sync command which allows these to be explicitly
set. The init and load functions can then be merged into one. There is then no
need for a parseConfiguration command, we can simply reset the server to have the
settings take effect.

The reset fuction is not an instant value return and triggers an event so it should
be an Async command, not a sync one.

The number of calls for the set pre/post command is probably higher than it
need be but someone with more familiarity with the hob code base can probably
figure out the right places its needed (maybe just init_cooker?).

(Bitbake rev: bae5210d7e048022f083361964ebec7daf1608f7)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-12-03 17:45:54 +00:00
Jason Wessel a67fde0ab1 bitbake: bitbake.lock: Add host:port to bitbake.lock for memres server
The idea is to build on the --status-only option for bitbake and
expose a mechanism where the oe init scripts can easily switch between
memres server and the non-memres server.

In the case of the standard oe init script the following
can shut down the server:

if [ -z "$BBSERVER" ] && [ -f bitbake.lock ] ; then
    grep ":" bitbake.lock > /dev/null && BBSERVER=`cat bitbake.lock` bitbake --status-only
    if [ $? = 0 ] ; then
       echo "Shutting down bitbake memory resident server with bitbake -m"
       BBSERVER=`cat bitbake.lock` bitbake -m
    fi
fi

A similar function can be used to automatically detect if the server
is already running for the oe memres init script.  This new
functionality allows for the memres init script to be started in a new
shell and connect up to an alaready running server without seeing the
error of trying to start the server multiple times.

(Bitbake rev: b1803958de8d7c3c3279841e38604a08dc2316cc)

Signed-off-by: Jason Wessel <jason.wessel@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-12-02 17:33:14 +00:00
Nathan Rossi f91a3f46a1 bitbake: cooker: Fix support for wildcards in bbappend filenames
The 'bbappend in f' incorrectly compares the current recipe with the avaliable
bbappends recipes. This comparsion causes unrequested bbappend files to be
appended, e.g. in the case of 'libgcc_4.8.bb', the bbappends for 'libgcc_4.8.bb'
and 'gcc_4.8.bb' are added to the filelist (because 'gcc_4.8.bb' is contained in
the 'libgcc_4.8.bb' string) which in turn causes the gcc_4.8.bbappend files to
be appended to the libgcc_4.8 recipe.

This should be a 'bbappend == f' to match the previous implementation of this
function, such that if no wildcard is present the recipe names must match
exactly.

This issue was introduced by commit 31bc9af9cd56e7b318924869970e850993fafc5f,
which it related to [YOCTO #5411].

(Bitbake rev: 991cbeedbde8bd25ce08c669b1bfac8b99e33149)

Signed-off-by: Nathan Rossi <nathan.rossi@xilinx.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-11-29 09:53:00 +00:00
Saul Wold 381d592018 bitbake: cooker: add support for using % as a wildcard in bbappend filename
There has been a continuing call for supporting wildcard in bbappend
filenames. The wildcard is actually allow matching of the name and
version up to the point of encountering the %.  This approach will
allow for matching of the major or major.minor.

Exampes:

busybox_1.21.1.bb
busybox_1.21.%.bbappend will match
busybox_1.2%.bbappend will also match

if we update to busybox_1.3.0.bb the above won't match, but a busybox_1.%.bb
will.

[YOCTO #5411]

(Bitbake rev: 31bc9af9cd56e7b318924869970e850993fafc5f)

Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-11-24 12:25:36 +00:00
Alexandru DAMIAN e19ee57c96 bitbake: bitbake: cooker, toaster: mark interrupted builds as failed
This patch changes bitbake to log an error to the
command line when the build is interrupted via Ctrl-C.

This is needed to inform the user that not all tasks
required for the build have been executed, and
the build is not complete.

Internally, the Bitbake server will return a CommandFailed
event that will be logged by Toaster as build failure.

(Bitbake rev: 9a658e8b1511f1b9f91663f546f748fdfbc8965f)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-11-20 14:05:52 +00:00
Alexandru DAMIAN 9fdd8adc17 bitbake: cooker, toaster: variable definition tracking
In order to track the file where a configuration
variable was defined, this patch bring these changes:

* a new feature is defined in CookerFeatures, named
BASEDATASTORE_TRACKING. When a UI requests BASEDATASTORE_TRACKING,
the base variable definition are tracked when configuration
is parsed.

* getAllKeysWithFlags now includes variable history in the
data dump

* toaster_ui.py will record the operation, file path
and line number where the variable was changes

* toaster Simple UI will display the file path
and line number for Configuration page

There is a change in the models to accomodate the recording
of variable change history.

[YOCTO #5227]

(Bitbake rev: 78e58fed82f2a71f052485de0052d7b9cca53ffd)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-11-15 11:48:53 +00:00
Paul Eggleton ad65846ba8 bitbake: cooker: warn if user specifies a target listed in ASSUME_PROVIDED
If the user explicitly asks to build a target that is listed in the
value of ASSUME_PROVIDED, show a warning mentioning that it will be
ignored.

(Bitbake rev: 65be09f5e2a4a41e65c9232e208d8154b822fc4e)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-11-15 11:48:52 +00:00
Cristiana Voicu ea92671d98 bitbake: cooker: replace "w" file opening mode with "a" mode
Also removed some redundant file manipulation code
Based on patch sent by Stefan Stanacar <stefanx.stanacar@intel.com>

(Bitbake rev: e054c1e7c8581f66082fcdfb89769401ca6e78a3)

Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-11-12 17:01:37 +00:00
Alexandru DAMIAN 0087db6257 bitbake: cooker: do not recreate recipecache in buildfile mode
When building a single file, the cooker will recreate
the recipecache from scratch.

I suspect this is a remnant of past code, since:
* the current recipecache works fine
* the new recipecache will not have all the fields as
requested by HOB_EXTRA_CACHES setting

This patch disables recreating the recipecache, leading
to shorter times when building single build files
(-b option) and better compatibility with Toaster.

(Bitbake rev: 618d69b00075981b8553513130d7deb1aed61578)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-11-04 09:36:22 +00:00
Alexandru DAMIAN a670859103 bitbake: cooker: add data to the dependency tree dump
Toaster needes to record extra data that needs to
be moved at the time of the dependency tree dump.

This data includes:
* layer priorities for recording in the layer section
* the inherit list for each PN which allows to determine
the type of the PN (regular package, image, etc).

This patch adds this data to the dependency tree dump.

(Bitbake rev: 7636aba37320aaf9b044d3832ddc21af51ccd69c)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-11-04 09:36:21 +00:00
Richard Purdie 3ca820de91 bitbake: cooker/command: Add error state for the server and use for pre_serve errors
Currently if errors occur when starting the PR service, there is a race that
occurs since the UI runs various commands including starting builds before
processing the CookerExit(). By adding the error state and refusing to run
async commands in this mode, builds are prevented from starting and the
UI reaches the exit code with the system shutting down cleanly.

(Bitbake rev: 42fa34142ea685f91115a551e74416ca28ef1c91)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-10-14 16:59:12 +01:00
Cristiana Voicu 81e678bc0a bitbake: bitbake: replace comment added in conf files
The patch that implements removeConfigurationVar method was made before
merging the patch that replaces "added by bitbake" with "added by hob".
This patch corrects this issue.

[YOCTO #5284]
(Bitbake rev: 42601a5edef0316767b952b162123534aa8fee18)

Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-10-07 12:52:05 +01:00
Cristiana Voicu ee7e64f116 bitbake: bitbake/hob: removing extra parameters from conf files using hob
In Hob settings, there is a tab to add/remove extra settings. This
patch implements a way to "remove" variables from conf files, through
bitbake. But, to keep the history assigment of the variables synchronized,
instead of removing, it replaces the lines with blank lines.

[YOCTO #5284]
(Bitbake rev: bd720fb63cef6b399619b8fbcaeb8d7710f2d6df)

Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-10-07 09:37:34 +01:00
Valentin Popa 45392cc67a bitbake: HOB: MACHINE should be saved in conf files using ?=
MACHINE var is saved using early assignment operator.
Calling  MACHINE=x bitbake core-image-... works properly.
Comment "#added by bitbake" is replaced with "#added by hob".

[YOCTO #5070]
(Bitbake rev: 2d0ec8ff083b636a6cf98de3278900eb95c3def6)

Signed-off-by: Valentin Popa <valentin.popa@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-10-07 09:37:33 +01:00
Alexandru DAMIAN a05d474e5a bitbake: bitbake: cooker, command: add a command to return global data
Adding the 'getAllKeysWithFlags' read-only command that will
return a dump of the global data state, together with specified
flags for each key. The flag list is passed in as the first
parameter to the command.

This will be used by UI clients to get the build configuration.

(Bitbake rev: 3e094da513e1220319288806cb76ddf804772afd)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-09-22 12:19:44 +01:00
Alexandru DAMIAN 0c51d610e1 bitbake: bitbake: cooker: get extra information from recipe cache
The loaded cache modules may add extra attributes to
the recipecache, that will be populated by the cache
classes required by the UI. These attributes
will be used by the UI to display relevant information.

Adds cachefields cache class field to specify
for each cache class which attributes will be set
in the recipecache.

Adds code to automatically expand depends tree with the
fields exported by the extra cache class.

Fixes a cache field name in the HOB UI.

(Bitbake rev: 47c171005fb3803d936e65fcd4436c643883ae16)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-09-22 12:19:44 +01:00
Alexandru DAMIAN f7621f47d8 bitbake: bitbake: cooker,runqueue: send the task dependency tree
Adding a CookerFeature that allows UIs to enable
receving a dependency tree once the task data has been
computed and the runQueue is ready to start.

This will allow the clients to display dependency
data in an efficient manner, and not recompute the runqueue
specifically to get the dependency data.

(Bitbake rev: 75466a53b6eece5173a9bfe483414148e4c06517)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-09-22 12:19:43 +01:00