When using the dry run option (-n), bitbake would still try and fire
a specific fakeroot worker. This is doomed to failure since it might
well not have been built.
Add in some checks to prevent the failures.
[YOCTO #5367]
(Bitbake rev: 78ae96e667d3fbb8649fe25eb073e15a99d61cc8)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This utility doesn't take any special arguments, but it's nice if it at
least knows how to deal with no arguments, --help and errors properly.
(Bitbake rev: 0cabdf1d0cde6687bc1372675a0d6242587c87a0)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* Set up a logger independent of BitBake so we can log errors ourselves
* Handle common errors without printing a traceback
(Bitbake rev: 77b5f5b8dca4deebb06eeb06a8e7f2ccdbfff46f)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* Use OptionParser to parse the two options to -t rather than trying to
pick them out ourselves.
* Add a description shown with --help output
(Bitbake rev: daab42d19463b4108968fc88b207936e5ac84154)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Adjust the task name automatically if the -t option is specified with
a task name that doesn't start with do_ (e.g. "configure" instead of
"do_configure").
(Bitbake rev: d182cbc63745303ef2dc9fa2cbbf5d87a68e0b52)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Getting an error message about --remote-server being set when really
BBSERVER was is confusing, clarify the message.
(Bitbake rev: d7b5938a30a9b0ed83f899a06a88786e8392f8bd)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When starting a server we don't want to show the debug event queue which
is reserved for when when errors have occurred. This patch copies the UI
code to ensure the user doesn't see confusing output.
(Bitbake rev: a886cda58415085981646fb9a024fa7641f55865)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
For BBHandledExceptions, we've already displaced a sensible error to
the user so we don't need to do it again. Just exit with an error
value.
(Bitbake rev: 1ff5ec26eba70ab1c85674a60b7dac77317bf349)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If for example you try "bitbake -m" with an invalid BBSERVER, error
messages are not displayed. This change ensures logging is in place
to catch and display such errors.
(Bitbake rev: 719808f95adc7820fcc09743c592513414d03ce1)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The --help text was rather inconsistent in style and plain incorrect in places,
using confusing terminology in others. I guess most people know what the options
do and don't read this but its confusing to new users.
This updates it to use the terms recipe and task consistently, remove
the references to stage, bbread and generally try and make the output
more useful.
[YOCTO #4856]
(Bitbake rev: 516311946c7bd14c84947dc44c3bb0563e5a9667)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Implementing feature set selection that allows a client
to enable specific features in the server at connection time.
Only enabling of features is supported, as there is
no way to safely remove data loaded into the cooker.
Once enabled, a feature will remain enabled for the
life of the cooker.
Client-server connection now supports specifying the feature
set required by the client. This is implemented in the Process
server using a managed proxy list, so the server cooker
will now load dynamically needed features based on what client
connects to it.
In the XMLRPC server the feature set is requested by
using a parameter for registerUIHandler function.
This allows observer-only clients to also specify features
for the server.
The server code configuration now is completly separated
from the client code. All hardcoding of client knowledge is
removed from the server.
The extra_caches is removed as the client can now specify
the caches it needs using the feature. The UI modules
now need to specify the desired featureSet. HOB is modified
to conform to the featureSet specification.
The only feature available is CookerFeatures.HOB_EXTRA_CACHES
which forces loading the bb.cache_extra:HobRecipeInfo class.
(Bitbake rev: 98e594837aab89ea042cfa9f3740d20a661b14e2)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Due to the worker split the ${DATE} and ${TIME} variables could end up
with different values for different workers.
E.g., a task like do_rootfs that is run within a fakeroot environment
had a slightly different view of the time than another task that was not
fakerooted which made it impossible to correctly refer to the image
generated by do_rootfs from the other task.
(Bitbake rev: 756cc69ebf8bfe8455d0c90f288dd51be2499773)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
BUILDNAME is set from cooker by default, so since the worker split it
will not be set when executing functions. In OpenEmbedded this results
in /etc/version (which is populated from BUILDNAME) not having any
content. Pass this variable value through to the worker explicitly to
fix the issue.
Fixes [YOCTO #4818].
(Bitbake rev: 92940b0427d9b2b3f95e27c230ec1e36638a34bc)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
bitbake-worker makes use of the signal module
but it doesn't import it. This patch fixes the issue.
[YOCTO #4750]
(Bitbake rev: c2ed639690f135994199eb24d964e37f57259e3a)
Signed-off-by: Valentin Popa <valentin.popa@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This change allows bitbake to connect to a remote server
by reading the BBSERVER environment variable and effecting
the config. Basically a shortcut to the command line parameters.
(Bitbake rev: 81929f86c57ed0a4ad0cda7aaa820fceabaa61e9)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
I add an option to terminate a remote server gracefully
as not to need a kill command.
(Bitbake rev: 7495f835666a9561c2c7d84da7aaa74e4df55b9a)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
I add an observer only mode for the knotty UI and
the XMLRPC server that will allow the UI to register
a callback with a server in order to receive events.
The observer-UI is able to send read-only commands to the
server, and also is able to register as an event handler.
Read-only commands are the commands that do not change
the state of the server and have been marked as such in
the command module.
The observer can switch to a full client if it calls addClient
at any time, and the server has no other client running.
(Bitbake rev: 4de9ee21f1fa4d04937cc7430fb1fc8b7a8f61e2)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
With the change to bitbake-worker we need to ensure the workers know
how to contact the PR service, the magic 0 port and singleton is
no longer enough.
(Bitbake rev: c761751e259bb8e940552a28794b45887b5a72d9)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This is a pretty fundamental change to the way bitbake operates. It
splits out the task execution part of runqueue into a completely
separately exec'd process called bitbake-worker.
This means that the separate process has to build its own datastore and
that configuration needs to be passed from the cooker over to the
bitbake worker process.
Known issues:
* Hob is broken with this patch since it writes to the configuration
and that configuration isn't preserved in bitbake-worker.
* We create a worker for setscene, then a new worker for the main task
execution. This is wasteful but shouldn't be hard to fix.
* We probably send too much data over to bitbake-worker, need to
see if we can streamline it.
These are issues which will be followed up in subsequent patches.
This patch sets the groundwork for the removal of the double bitbake
execution for psuedo which will be in a follow on patch.
(Bitbake rev: b2e26f1db28d74f2dd9df8ab4ed3b472503b9a5c)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Two fixes in bitbake related to running remote servers -
* can now specify correctly the bind port
* the information in print conforms to common server infrastructure
(Bitbake rev: b657208ee15ae065e5fcc2dd6e0051e03d246727)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Added code in XMLRPC server that creates a stub local server
for a client-only connection and is able to connect to
a remote server, and receive events from the remote server.
Added the option to start a client with a remote server in
bitbake.
Original code by Bogdan Marinescu <bogdan.a.marinescu@intel.com>
(Bitbake rev: 25b2af76104d5aaf6435de8c158e0407512f97ce)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This is a code sanitization targeted at making further
server-related changes easier (launch a server
separately or creating a mockup-server) to do.
(Bitbake rev: eac00258d213137ef73aed255c92b7981e2f1c75)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The collection of the extra caching data should not
be performed by the cooker, but supplied to it.
This patch will also streamline the code for launching servers
without a UI attached.
Based on a patch by Bogdan Marinescu <bogdan.a.marinescu@intel.com>
(Bitbake rev: f0b54280a6bce522508e4741e5f507bc284113a8)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Slight change in bitbake as to retrieve the entire UI module
for further processing instead of just the main function.
Based on a patch by Bogdan Marinescu <bogdan.a.marinescu@intel.com>
(Bitbake rev: f49341a9599d971829ef65b2b02732543740a3c9)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
With the removal of the none server type, we can remove the launchUI
method and simplify the code slightly.
(Bitbake rev: 9bef2f2dd0bcaa59528ebcb3c1ce053b7dff1ec6)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The process server backend has been serving well as the default for a long
time now and the UI model is much better thought out that it used to be. With
the move to make bitbake a memory resident process, the none server is now
looking rather pointless and complicates the code needlessly. Lets therefore
now remove it.
(Bitbake rev: 9af03a89605e3db9bce3cea1e0f2d0b6cfaa6fe1)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fixes the show-appends and flatten subcommands for recent refactoring in
cooker.
[YOCTO #4536]
[YOCTO #4535]
(Bitbake rev: 54817d0de667941ee68fe3490684159e2d90e9c5)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
calc_bbfile_priority is now in CookerCollectFiles which can be accessed
on the collection attribute of a cooker instance.
Fixes [YOCTO #4513].
(Bitbake rev: 5d941631ad7198737d9a5c5a920a9062fa0431f8)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
As the code stands today its hard to know which configuration variables
are used by which parts of the system. Some are used by the UIs, some
by bin/bitbake itself, some by cooker.
This patch changes the configuration to just contain the variables cooker
uses, and changes bin/bitbake to access the variables it needs directly
which hopefully lets us start to untangle this mess.
(Bitbake rev: e57497a24b6157c92519a34accd66035a39ad1f8)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Building up a set of actions for the server is tricky since we depend upon the
commandline but fall back to values from the datastore. We should be able to build
a datastore without a commandline and vice versa. Ultimately the UI should send
the commands to the server.
This patch amounts to code rearranging, moving the heavy lifting to the UI, though
a helper in the configuration option. This will need further cleanup/tweaking but
this should be the only update needed to the UIs. The code now queries the server
for any missing data should it need to.
This code allows various knowledge of configuration variables to move to the UI side
only, partcularly pkgs_to_build but also all the command specifiers. It should also
be possible to move cmd eventually, I'm just unsure if any callers call the commands
expecting this to default to something sane right now.
(Bitbake rev: 2dbbb1d51dafd4451fef8fe16f095bcd4b8f1177)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently the UI and server configuration is one big incestuous mess. To
start to untangle this we creater cookerdata, a new module which contains
various confiuration modules and the code for building the base datastore.
To start with we add a ConfigParameters() class which contains information
about both the commandline configuration and the original environment.
The CookerConfiguration class is created to contain the cooker.configuration
options. This means we can transfer new paramters to the server over something
like XMLRPC and then build a new configuration from these on the server.
Based on a patch from Alexandru Damian <alexandru.damian@intel.com>
(Bitbake rev: 35bd5997e8d8e74bc36019030cc10c560a8134f9)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Due to the internal event processing, this excepting handler usually raises an
Empty error, masking the underlying failure. Ensure the original exception is
raised.
(Bitbake rev: 7d548568a55adfe84a976f2a549995e42da1afef)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Python multiprocessing needs /dev/shm, and if it doesn't exist (e.g. in
a chroot environment) or is not writable, you'll get
"OSError: [Errno 38] Function not implemented" or
"OSError: [Errno 13] Permission denied" with a traceback, which doesn't
really help to explain what is wrong.
Implements [YOCTO #4407].
(Bitbake rev: e7460d01f281f913a94192e1f4bd20688164bdd4)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently if errors occur early in the init process, the errors may
not be shown to the user. This change ensures that if a failure does
occur, the messages are flushed from the queue and shown to the user.
(Bitbake rev: fda84c4285e0bc31c249b6dd5464aeb6ad622a9a)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In case of an error in bitbake, the stack trace is limited to 5
items. This is an endless source of confusion and it makes bugs
reports impractical, since a full stack trace can't be included
in the bug report. This patch simply removes the depth limit.
(Bitbake rev: 02629c42fb09413d9da16cfe43e03338ce7db3ff)
Signed-off-by: Bogdan Marinescu <bogdan.a.marinescu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There are two "help" lines when run bitbake help under python 2.6.6
(perhaps all python 2.6):
$ bitbake-layers help
[snip]
help
display general help or help on a specified command
[snip]
help
display general help or help on a specified command
This is because the cmd.py in python 2.6.6 also has a do_help() function
as our bitbake-layers, and the cmd.Cmd.get_names() doesn't consider the
overload, this has been fixed in python 2.7, use set() to remove the
duplicated would fix the problem.
[YOCTO #3428]
(Bitbake rev: 324ed96e28ec31cff8cef1824d20d40f9a5d46ad)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The "bitbake-layers show-cross-depends" print the absolute path in the
past, now it will print the relative path (relative to the layer dir) by
default, and add the "-f" option to make it print the absolute path.
For example:
$ bitbake-layers show-cross-depends
[snip]
meta-intel/meta-jasperforest/conf/machine/jasperforest.conf requires meta/conf/machine/include/ia32-base.inc
[snip]
$ bitbake-layers show-cross-depends -f
[snip]
/path/to/poky/meta-intel/meta-jasperforest/conf/machine/jasperforest.conf requires /path/to/poky/meta/conf/machine/include/ia32-base.inc
[snip]
[YOCTO #3824]
(Bitbake rev: df217701318d60559be0b10214b883b8ce4f5d2a)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We need to empty out the environment whilst we build the cooker but
we need the environment for the UIs since hob uses DISPLAY and other
session variables.
This patch adapts the utils functions to return removed environment
components so we can reinject them for use by the UI, allowing hob
to work again.
(Bitbake rev: fc330d810099c57fefd4e706159a73ad8401d97c)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Figure out the dependency between recipes that crosses a layer boundary
* Introduction:
- For the .bb file, we need check the following depends:
> Check the DEPENDS, RDEPENDS and inherits according to the
cooker_data.
> The cooker_data doesn't have the info about "require/include xxx",
so we need check them manually.
- For the .bbclass, .inc and .conf file, we can't get their file
depends from the cooker_data, we need check them manually.
* Usage:
$ bitbake-layer show-cross-depends
I don't like the name "show-cross-depends", figure-out-cross-depends
might be better, but most of the commands are "show-xxx" for bitbake-layers.
* Output: (snipped)
/work/poky/meta/recipes-support/libusb/libusb-compat_0.1.4.bb inherits /work/poky/meta-yocto/classes/poky-sanity.bbclass
/work/poky/meta-intel/meta-cedartrail/recipes-cdv-media/images/core-image-cdv-media.bb requires /work/poky/meta/recipes-sato/images/core-image-sato.bb
/work/poky/meta-yocto/recipes-core/tiny-init/tiny-init.bb RDEPENDS /work/poky/meta/recipes-core/busybox/busybox_1.20.2.bb
/work/poky/meta-intel/common/recipes-bsp/amt/lms_7.1.20.bb DEPENDS /work/poky/meta/recipes-devtools/autoconf/autoconf_2.69.bb
/work/poky/meta/classes/distrodata.bbclass includes /work/poky/meta-yocto/conf/distro/include/package_regex.inc
/work/poky/meta-intel/meta-sys940x/conf/machine/sys940x.conf requires /work/poky/meta/conf/machine/include/tune-atom.inc
[YOCTO #3387]
(Bitbake rev: 100502ef722bf19b6c125a5e017f31686ad18421)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This means we can run in parallel with other bitbake processed we're not in
task mode.
(Bitbake rev: 20222f2f16130e91eed5e68286188ee0e8f8f3bf)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie
Paul Eggleton
Alexandru DAMIAN
Peter Kjellerstedt
Valentin Popa
Bogdan Marinescu
Robert Yang