This patch properly shuts down toaster in interactive mode
under Mac OSx.
(Bitbake rev: a00cd5135a4bbd61d311fa17569894f974ab4420)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This patch modifies the toasterui to properly return the exit
code based on the errors found in the toaster itself.
The upload event file API call will not delete event logs for which
toasterui showed an error. This will facilitate debugging.
Minor enhancement in the buildinfohelper to reduce the number
of lookups on unknown layer objects (prevented testing of the patch).
(Bitbake rev: 1ddd6a9e4280a4adf971132ff1fe7ec9b3252905)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Used sys.exit instead of assigning exit code to
variable. This way it's more clear when bitbake
exists and which exit code is used.
(Bitbake rev: 5ecb8817bd49223652ede4fe513f1a42f2196798)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Bitbake program and core versions must match.
Moved __version__ from main.py back to bin/bitbake.
Implemented check for version match in bin/bitbake.
(Bitbake rev: 2fe7d8c574ddf6a30278cff1a5a5c4089dc56d6d)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
tbs
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Some error messages were lost because BBMainException was
inherited from bb.BBHandledException. When bb.BBHandledException
is processed error messages are not printed as they suppose to
be printed before raising this exception.
Stopped to inherit BBMainException from bb.BBHandledException.
Handled BBMainException in bin/bitbake and printed error message
to the stderr.
(Bitbake rev: c8e2a40c4e9865ebef9936d23644f2602a5c90f5)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
For now, `bitbake-diffsigs -t <recipe> <task>' doesn't work. It always outputs
nothing.
The problem is that bitbake-diffsigs are comparing sigdata and siginfo files
that have the same hash value. This is not what we want. These two files are
actually duplicates considering the purpose of bitbake-diffsigs. So we need
to remove one of them so that bitbake-diffsigs could actually compare the
correct signature files.
(Bitbake rev: c34613eb69fd19770cbfc78ab8384221f10d5587)
Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In show-recipes and show-overlayed, we only list one variant of each
recipe where multiple exist, therefore we should show the main one (e.g.
in OpenEmbedded, we now show openssl and not nativesdk-openssl which
would otherwise sort first.)
Fixes [YOCTO #7514].
(Bitbake rev: c4172fe33f0c63eda7f4af2bd36a5e9738e13595)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This is particularly problematic since qemu images switched to ext4 by
default and now cannot work properly with UIs like hob.
This patch adds in ext4 to the appropriate variable fixing this.
[YOCTO #7426]
(Bitbake rev: a1431b1b85bf90ed5f771552d0ce91de288047d4)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Moved most of functionality of bin/bitbake to lib/bb/main.py
to be able to call bitbake from python code.
(Bitbake rev: d377f7f88d73f4e5d2dffef03d6acee809827ac6)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We use logger.plain() to produce actual output which needs to go to
stdout. We could use more advanced filtering (and probably should in
future) but for the moment let's just fix the regression.
Fixes [YOCTO #7458].
(Bitbake rev: e96fc0ccfc9f5be2c41c9733c92dc81df3df5065)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The recent refactoring caused use of wildcards to trigger a traceback
because the old variable name was being used.
Fixes [YOCTO #7367].
(Bitbake rev: 7182cdd3c4a534a87147bb0aad7b360ffef6426b)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In order to bring in transaction.set_commit support,
we need Django 1.6, discarding support for Django 1.5
This patch removes 1.5 from the list of acceptable Django
versions.
(Bitbake rev: 8640769296c3fffa2cf56cfb5327ca24533f5a41)
Signed-off-by: Alexandru Damian <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a command to query layer dependencies from a layer index such as the
OpenEmbedded Layer Index at http://layers.openembedded.org. Fetches the
layer and its dependencies and adds them into conf/bblayers.conf.
[YOCTO #5348]
(Bitbake rev: 4b8fcf9a5bc802793bf332334217faace55f14f6)
Signed-off-by: Chong Lu <Chong.Lu@windriver.com>
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Logging output was simply not getting printed.
(Bitbake rev: 62b825b19b13a914cbff5303d541bd5dbbec90a7)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This makes help formatting and option handling a lot more standardised
and allows us to drop a bunch of code. We also gain slightly more
straightforward error handling.
One side-effect however is that the old subcommand syntax using
underscores is no longer supported. The dashed form has been supported
(and displayed in the help text) for quite a while now so I wouldn't
imagine that will be much of an issue.
(Bitbake rev: 6e2f09b58882d3949026b9dd545f789ad3fe6fab)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This patch improves the logging facilities for toaster in order
to help diagnose bugs that happen on user machines.
The logs are stored now under "/tmp/toaster_$$" where $$ is a
PID-based unique identifier. On shutdown, toaster will automatically
erase all logs unless errors are listed in the log file.
On error, Toaster provides suggestions on what to do.
This patch includes a minor fix found as a result of logging
improvements.
(Bitbake rev: 8a8248f7b7e30469f592e2f8adbf6ce21e8685c5)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This patch brings fixes and enhancements in the startup sequence.
- the dependency version checking is now correctly enforced when
starting in both modes
- fixed errors in looking up environment variables
- added message with 'daemon' program start location
- presenting menu for selecting config file
[YOCTO #7283]
[YOCTO #7291]
[YOCTO #7273]
(Bitbake rev: c5ddd9d88910857a1b745b1c237df0390dd56835)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
A small typo slipped in the message included in
the fix for [YOCTO #6023].
(Bitbake rev: f28fb3d660d8726b75e6c31fa36bc22b54e1bca7)
Signed-off-by: Belen Barros Pena <belen.barros.pena@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This patch makes sure that all the toaster conf files are actually
written from the build enviroment controllers.
Additionally, toaster checks that the 'daemon' program, which is used
to start the build system, is available (currently for localhost).
[YOCTO #7171]
(Bitbake rev: 0a1db7d1531f8254955e1152bcd8e6db4ec1d277)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This patch updates the prompts to the user to better explain
what Toaster is doing and the information it needs from the user.
Additionally, fixes a check in loadconf command.
[YOCTO #6785]
[YOCTO #7251]
(Bitbake rev: 90ef8975f35e2da824bc1c80e41ca26d9af0b208)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This patch makes Toaster managed mode obey the "noweb" parameter
by not starting the web server and launching the web browser
command if the "noweb" parameter is specified.
The web browser will be pointed at 127.0.0.1 instead of 0.0.0.0
[YOCTO #7039]
(Bitbake rev: 4037f8b08bc9fb5c4c9f260efb847105be718a32)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We change the setting of variables from directly injection
into the set-up cooker to writing a conf file that is pre-read
on bitbake server startup. This is needed because the injection
can only happen after the variable set is parsed, and the variables
already inferred, so setting up variables happens too late.
[YOCTO #7045]
(Bitbake rev: 854f680b5b9d2d0fa796af84cb1218545fbfc55a)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The bug has a long discussion of this. Basically, in some environments,
the exact details of which aren't understood, a Ctrl+C signal to the
UI is being transmitted to all the process children. Looking at the output
of "ps ax -O tpgid", its clear the main process is still the terminal
owner of these processes.
stty -a on a problematic system shows: "-ignbrk brkint"
and on a working system shows: "-ignbrk -brkint"
The description of brkint would suggest this is the problem, setting up
that terminal environment wasn't able to reproduce the problem though.
It was confirmed that using setsid() caused the problem to be resolved
and is probably the right thing to be doing anyway, so lets do it.
[YOCTO #6949]
(Bitbake rev: 461aa73fff0ab616032d28c4fd0322eb88838be6)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* Fix add-layer error message when a layer is already in BBLAYERS
* Ensure we show an error message if we can't find BBLAYERS at all
(Bitbake rev: 1c743fd2103730e27699dd55efc6914d3b0c3702)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add add-layer and remove-layer commands for easily adding and removing
layers to/from bblayers.conf.
(Bitbake rev: 0e94aed033917b91ea6ba0522b14a040c3bdc987)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This patch adds a "-w/--write-log" option to bitbake
that writes an event log file for the current build.
The name of the file is passed as a parameter to the "-w"
argument. If the parameter is the empty string '', the file
name is generated in the form bitbake_eventlog_DATE.json,
where DATE is the current date and time, with second precision.
The "-w" option can also be supplied as the BBEVENTLOG
environment variable.
We add a script, toater-eventreplay, that reads an event
log file and loads the data into a Toaster database, creating
a build entry.
We modify the toasterui to fix minor issues with reading
events from an event log file.
Performance impact is undetectable under no-task executed builds.
(Bitbake rev: 1befb4a783bb7b7b387d4b5ee08830d9516f1ac2)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This patch modifies how layers are identified and matched.
Layers were primarely organized by the source of layer information,
and Releases were separated by both layer git branches and originating
source of layer information. This setup prevented mixing layers from
different sources for a certain release, which didn't match the way
people use Yocto Project / bitbake.
This patch brings name-based indentification, where layers with the
same name are assumed to be equivalent, in the sense of being able
to substitute one another. To facilitate this identification to
humans, layers are differentiated by GIT URI instead of layer sources,
which was a rather arbitrary abstraction.
Additional changes include modification to models in order accomodate
for the new data structure, and to config file loading to match
the new toasterconf.json layout.
(Bitbake rev: 4357200aed522ad56cfd84917f877645b83b6a70)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When toaster is used by multiple users on the same machine,
it is possible that files in /tmp have a name race.
This patch makes sure that the files have unique names.
(Bitbake rev: d4a47bc84f762666a847f1152cc2e75c9ef36092)
Signed-off-by: Belen Barros Pena <belen.barros.pena@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fixed:
1) Run "bitbake recipe" in the terminal
2) Close the terminal while building
3) $ ps aux | grep bitbake-worker
There will be many processes, and they will keep the resources (e.g.,
memory), and won't exit unless kill or kill -9.
(Bitbake rev: 40d2ae0723de2bf5fee343faafb4afda40546839)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We add the "webport=" command line option as to allow starting
the web server on a custom port.
The bitbake server port is now auto-allocated. This is needed
to be able to run multiple toaster environments on a single machine.
We tackle bug 6023 (toaster refusing to start when lock file is present)
by using more specific checks, and automatically recover from
bitbake server down / webserver up error mode.
Command line parameters are now read on both interactive and managed
modes.
The localhost and ssh controllers are updated to work with the modified
toaster launcher script.
[YOCTO #6023]
(Bitbake rev: cd3eb5b051743463cfe51dba97cae4da75420048)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fix the startup message by making clear which
server runs on which port.
(Bitbake rev: 26d601ab2a677a422bab425696a2cf6357ff3576)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We avoid a race between the setting the TOASTER_BRBE variable
and reading the variable in toaster ui by supplying the variable
at server startup time through the toaster.conf post-read file.
Additional small changes are included, including marking the
build request with the environment id of where the build took place.
(Bitbake rev: 7c333350418c4140e6c988c5272940f8057d327d)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Without this you see:
File "bitbake/bin/bitbake-worker", line 201, in fork_off_task
os._exit(child())
TypeError: an integer is required
(Bitbake rev: cd477b5e77ab0373248b8a8fa30e1c7b8ea984fd)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We need to transfer some of the siggen data from the core/cooker into
the worker instances. There was a partial API created for this but
its ugly and its not possible to extend it from the siggen class.
This patch completes the interface/abstraction for the data and
means the class can extend/customise it in any siggen class.
(Bitbake rev: cf2d642052979d236185c5b8ca2c5478c06e62ae)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We change the toaster starting script to make the
bitbake server listen on all interfaces on the local machine.
This is needed to be able to receive a controlling client
running on a remote machine.
(Bitbake rev: 137179eafca8d1a5a69b6302f8cc8961be3b45c4)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We enhance the checksettings command to try to automatically
detect settings for running on localhost.
The checksettings will look for a nearby poky layer source,
for a nearby build directory, and will try to import settings
from "toasterconf.json" files found in the local layer.
On new configuration, it will also perform updates from the
layer source.
(Bitbake rev: 2aab77dfccb538e2b09829841ea6c464d40cafb1)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently we get no profiling oversight into either the main bitbake worker
process, or the overall parsing before task execution. This adds in extra
profiling hooks so we can truly capture all parts of bitbake's execution
into the profile data.
To do this we modify the 'magic' value passed to bitbake-worker to trigger
the profiling, before the configuration data is sent over to the worker.
(Bitbake rev: 446e490bf485b712e5cee733dab5805254cdcad0)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When processes terminate, we really want all of the child processes to
terminate too. This was not happening for worker processes which spawned their
own multiprocessing pools, leading to build hangs. This change ensures any
sigterm gets passed to the whole process group. In local tests, this resolved
some hanging process workloads I could generate. It does rely on signals
being delivered in a timely fashion and there is a multiprocessing bug we have
to work around there.
(Bitbake rev: 96f8ea07ace1379380fab2d78eb592fa40c867d4)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The SIGNATURE_HANDLER can be defined by the user, so we can't use choices.
(Bitbake rev: 0036b36e333a68c6d30a6bf5c21bfd4fd3e8441b)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Since Toaster adds a number of specific requirements managed
by pip, we document these requirements in order to be able
to run bitbake and toaster under virtualenv. The target here
is to break the dependency on distro packages for specific
python libraries.
In order to start bitbake/Toaster in a distro-independent manner,
we use virtualenv and pip. We add venv to the .gitignore list
to make room for the virtualenv deployment in this directory.
Use this command sequence to setup the virtualenv:
$ virtualenv venv
$ . venv/bin/activate
$ cat toaster-requirements.txt | xargs pip install
bitbake and Toaster will then run normally under virtualenv.
We also fix bugs related to toaster starting in managed mode
through this script:
* toaster will not use bldcontrol application in interactive mode
* a proper message will be displayed if toaster detects a managed
start request in a build environment
(Bitbake rev: b5789c7ea7e6d1a4b67ec60fc0e4233ec543ba3a)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We add explicit absolute paths for a directory where
the layer sources will be checked out (sourcedir) and
where the build activities will take place.
Adding minimal checking when starting the application in
order to make sure that BuildEnvironment (BE) settings are
usable. This check is ran by the toaster script at startup.
Modify the localhost bbcontroller to use the BE settings
instead of trying to self-configure on checked out sources.
(Bitbake rev: d17500d3f73fdeeef5f11fb3773a65e927be3f02)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(Bitbake rev: 1cd369883469747a8158826bb8d67dcca2a8577f)
Signed-off-by: Robert P. J. Day <rpjday@crashcourse.ca>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The bitbake -S had been updated which always reuires an argument, so
update the help info:
- Add the two args in the help message: none and printdiff
- Use type="choice" so that we can get more friendly error messages,
for example:
bitbake: error: option -S: invalid choice: 'printdiffX' (choose from 'none', 'printdiff')
- dump-signatures=DUMP_SIGNATURES -> dump-signatures=SIGNATURE_HANDLER
(Bitbake rev: 021b778fa4685bdde39e1a0f6c7c57632dcf792a)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We add the new project and project page skeletons.
In the process, we add an identifier in the settings.py
to detect whenever Toaster is running in managed mode,
and a context processor to make this value available
to the template processor.
(Bitbake rev: 927a27c68e24cfe13f62ca5f0e60878b04fa4e24)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We add the build control functionality to toaster.
* The bldcontrol application gains bbcontroller classes
that know how to manage a localhost build environment.
* The toaster UI now detects it is running under build
environment controller, and update the build controller
database and will shut down the bitbake server once
the build is complete.
* The toaster script can now run in standalone mode,
launching the build controller and the web interface instead
of just monitoring the build, as in the interactive mode.
* A fixture with the default build controller entry for
localhost is provided.
[YOCTO #5490]
[YOCTO #5491]
[YOCTO #5492]
[YOCTO #5493]
[YOCTO #5494]
[YOCTO #5537]
(Bitbake rev: 10988bd77c8c7cefad3b88744bc5d8a7e3c1f4cf)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We add support to specify a connection token in the command line
and in the environment variable BBTOKEN.
When a client registers to a bitbake server, that client
will have exclusive access to the server. The client is identified
by a server-supplied token. If a client terminates, we cannot
reconnect to the server as the token is lost.
This patch adds the capability to specify the connection token
in the command line for xmlrpc clients. This allows us
to have bitbake work as an already-authenticated
client with the server and resume sending commands to a server.
(Bitbake rev: db5390940c0afbcdc9fbcf1225761968ae51d4a7)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We create the model classes that store information
about triggering builds, and the available build
environments.
We add a fixture with a default build environment
for build control, using a "build/" directory under
the poky checkout directory.
We enable the bldcontrol in toaster starting script
and in the toaster settings as to allow the actual database
to be kept in sync with the source code.
(Bitbake rev: d4bfe9059f765f11244b97e324c0131f32f8e400)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This fixes toaster startup failing after the preparatory patches
for bldcontrol application have been merged.
I slipped a bldcontrol-specific line into the toaster startup
script and this prevents toaster from starting. We comment
out this line until the bldcontrol application have been merged.
(Bitbake rev: d095eec6fc958d0aeb514cdc206734617fd7c930)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The 'bitbake-layers show-appends' command can use the built-in
get_file_appends() method which correctly identifies bbappends that
include a '%' wildcard in their filename.
(Bitbake rev: 2732dbae67c1945b668c38cc4cc5678c4aafe3d6)
Signed-off-by: Ash Charles <ashcharles@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We disable bitbake self start to prevent race condition.
ToasterUI will shutdown the server when the build is done
if running in managed mode.
We fix usage of kill server flag in the bitbake binary.
(Bitbake rev: 30159dbda3a40fa596302f91c705cb5f148c97a9)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The toaster starter script was designed to be sourced in
a build environment and set up the build recording environment
to be used in an interactive mode.
For the standalone web server mode, we modify the toaster
script to be run directly from the checked-out sources,
without a build environment set up, and run the web server
alone. In the standalone mode, the build environemnts and
all build activities are controled through the web interface.
(Bitbake rev: c1db4ccf27bedcbab2f03e7539fdb11b042c4fb9)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We add an option to the startup script, named "noweb" that
will start toaster without the embedded web server.
This is useful to start the system for build-only environments,
where the web server code is running on a different machine.
(Bitbake rev: c39838201301b9732581288a93783400bebe6591)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When we use the XMLRPC client API to connect to a bitbake server,
we want to receive errors from the API instead of having the
API exiting without warning.
Thus the "bb.fatal" calls have been replaced with "bb.warn" calls,
and we re-raise the original exception for handling by the
original caller.
The bitbake starting script has been modified to properly test
for failures in calling the client API and handle them.
Additional error handling added in the client, as to prevent
fatal crashes.
(Bitbake rev: eb63f08c33644f64752aaae2146a000956ce894a)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The configuration reading code should live in the
main bitbake entry point, and the server modules should
be supplied with correct configuration instead of attempting
to parse from configuration files.
This patch moves the endpoint address reading from XMLRPC
to the bitbake main script.
(Bitbake rev: ac5753274ff932e1d6f073ab4dab7bd6fe5355a1)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
RRECOMMENDS must be satisfied at build time, and these could cross layer
boundaries, so report these if they exist.
(Bitbake rev: 5569b3dca61e6d962494ca65c7aad09b2eb2ae63)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Overlayed recipes caused this to show false positives because the
overlaying version appeared to be satisfying the overlayed version's
RDEPENDS; but you'd never be building both at the same time.
(Bitbake rev: b94318174fe7f92b9a20eabb0bc4055066cb3d51)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
It's not particularly useful to show globally inherited classes here
since they do not normally represent a dependency.
(Bitbake rev: d16948bb88fcf44d861985838030be7c08697963)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
By default, show-cross-depends shows dependencies on OE-Core (i.e.
"meta") which is not particularly useful. Add an option to allow you to
hide those. For example, to hide all dependencies on OE-Core:
bitbake-layers show-cross-depends -i meta
Multiple layers can be specified by using commas as separators (no
spaces).
(Bitbake rev: 0e9062e65acbb05c1d9b3a9145eb866c3d562309)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If bitbake-diffsigs is run without arguments it will error:
Traceback (most recent call last):
File "./bin/bitbake-diffsigs", line 121, in <module>
if output:
NameError: name 'output' is not defined
Fix this by moving the check for output into the inner else-clause.
(Bitbake rev: 7d3545a66863ad7183a7650b2af57eee091c45ae)
Signed-off-by: Jacob Kroon <jacob.kroon@mikrodidakt.se>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Iterating through and calling setVar on this number of variables has significant
overhead in the profiling data. By not setting this, we save 3,000 calls
to setVar which gives a noticeable improvement to the speed of task execution.
The BBHASH variables have since been replaced by accessing that data through
the siggen code and going forward, that is the preferred way work with it.
(Bitbake rev: 92526eadd09d19938762290e0492076174367583)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The connection may have failed before the event queue has been setup.
Handle this correctly in the exception handler.
(Bitbake rev: db4d80b5c2d32117cdf06333b9627202998b1512)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We remove trapping SIGCHLD due to a weird interaction
with the bash version used with Ubuntu 14.04 LTS.
(Bitbake rev: 59f2f33440449c586c23dd3a192698a37aaf0595)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If the featureset didn't match the defaults, we'd pay the price of two
base configuration parses which showed up adversely in the performance
benchmarks. This also passes the feature set into the cooker creation so
in the common case we don't have to reset the server. This speeds up
both knotty and hob startup. If the featureset doesn't match, the system
will reset as before, this just streamlines the common case.
(Bitbake rev: 1249543c4dbf3edeac033d888497864cfc807a4e)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There is no easy way to make this change. We really need parameters for the -S
(dump signatures) handling code. Such a parameter can then be used within the
codebase to handle the signatures in different ways.
For now, "none" is the recommended default and "printdiff" will execute the
new (and more expensive) comparison algorithms.
(Bitbake rev: b9873588696507dfb6aade6821f6f75cb9a19e0a)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This patch adds the capability to have the Toaster UI
detect when the Bitbake server exited and cleanly
trigger a clean shutdown of the system through the toaster
starting script.
(Bitbake rev: a9cfa3eacfc99550e1ad3f8bb61b2a0bc9b44332)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The children of the worker should have the default SIGTERM handler,
else they'll try and do cleanup which should only happen in the
parent leading to all kinds of bizarre build failures.
(Bitbake rev: a53c8d1f846d94082aa459996c4114f10970b8ef)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When passing -t with an incorrect server type, the error message was using a
variable that doesn't exist.
(Bitbake rev: 98b991287df06cd89955c1d0591fce3b5d4403d1)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We remove the checks for the stop command, since we assume
the user really means it wants to shutdown the system when
he issues the command.
[YOCTO #5376]
(Bitbake rev: 5d6594de8876be8ce4df56b846fee0fc687a6261)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently if bitbake-worker handles a SIGTERM, it leaves the child
processes to complete or hang. It shouldn't do this so hook the SIGTERM
event and gracefully shutdown any children.
(Bitbake rev: 551406f3f9ee94de09d2da6e16fea054c6dbfdb7)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If you are just trying to fix one test at a time, it can be useful to be
able to specify an individual test(s) rather than running them all:
bitbake-selftest bb.tests.codeparser bb.tests.cow
You can even specify the test class or function to run, e.g.:
bitbake-selftest bb.tests.fetch.URITest
bitbake-selftest bb.tests.fetch.FetcherNetworkTest.test_fetch
(Bitbake rev: 4df9c72663e972437131a848e6ddcf3769ae1d2b)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Toaster start script lunches multiple process components
of the toaster system.
This patch adds logic into the startup script to
safely fail startup and do proper cleanup on any error
that may happen during system start.
Bitbake needs to return 0 if it will successfully lunches
the server-mode.
(Bitbake rev: f43d284e7ae752049711d8215a6020bee6966d45)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
As Toaster advances, database schema alteration
will force users to delete old versions and lose
all data collected.
In order to prevent this, and to allow database
updates to happen without having to delete old data,
we use South to handle migrations for the ORM
application which stores the Toaster data.
[YOCTO #5559]
(Bitbake rev: 6936faed8d94f3a2ab4055049cd27d02d8229003)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In order to remain up to date with the relevant technologies,
Toaster is updated with this patch to Django 1.5. This also
makes headways to allow usage of emerging
Django-related technologies.
Changes include the startup script Django version check, usage
of TemplateView instead of deprecated simple function to do
redirects, and update to the new form of the _url_ template tag.
Support for Django 1.4.5 is now deprecated.
[YOCTO #5558]
(Bitbake rev: 2d37a1731a2b681bc976f3f391d65abb7745b6f9)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The idea is to build on the --status-only option for bitbake and
expose a mechanism where the oe init scripts can easily switch between
memres server and the non-memres server.
In the case of the standard oe init script the following
can shut down the server:
if [ -z "$BBSERVER" ] && [ -f bitbake.lock ] ; then
grep ":" bitbake.lock > /dev/null && BBSERVER=`cat bitbake.lock` bitbake --status-only
if [ $? = 0 ] ; then
echo "Shutting down bitbake memory resident server with bitbake -m"
BBSERVER=`cat bitbake.lock` bitbake -m
fi
fi
A similar function can be used to automatically detect if the server
is already running for the oe memres init script. This new
functionality allows for the memres init script to be started in a new
shell and connect up to an alaready running server without seeing the
error of trying to start the server multiple times.
(Bitbake rev: b1803958de8d7c3c3279841e38604a08dc2316cc)
Signed-off-by: Jason Wessel <jason.wessel@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This patch adds the ability to dynamically select a port for the
bitbake memory resident server when the BBSERVER port is set to -1.
This allows for running multiple instances of the bitbake memory
resident server on the same system in different build directories.
The client portion of the bitbake instance can also request that the
server automatically start when using the auto port feature. This is
to deal with a bitbake instance that eventually times out and exits or
that has died for some unknown reason.
The new functionality allows for lazy startup of the server after
sourcing the init script for the memory resident functionality.
(Bitbake rev: d6abc07ff385357d312d8435b89e0a9c1f965433)
Signed-off-by: Jason Wessel <jason.wessel@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The --status-only option is purely to check the health of the server.
The idea is to use it from the oe environment script so as to know if
the server needs to be started or not.
(Bitbake rev: 46b26ef5c27bfa9c91680f494f02750305b8098c)
Signed-off-by: Jason Wessel <jason.wessel@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently tasks have no knowledge of which other tasks they depend
upon. This makes it impossible to do at least two things which would be
desirable/interesting:
a) Have the ability to create per recipe sysroots
b) Allow the aclocal files to be present only for the entries in
DEPENDS (directly and indirectly)
By exporting task data through this new variable, tasks can inspect
their dependencies and then take actions based upon this.
(Bitbake rev: 84f1dde717dac22435005b79d03ee0b80a3e8e62)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In recent versions of bitbake, it is not possible to initialise a
BBCooker object without having it load the configuration first. Thus we
should avoid creating the Tinfoil object here in bitbake-layers which
does that internally until we actually need to, so you can run
"bitbake-layers help" and not have to wait several seconds for the
output.
(Bitbake rev: 8f1e280fbbb6432d7bcc1fb4241f402668c6c5ea)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When using the dry run option (-n), bitbake would still try and fire
a specific fakeroot worker. This is doomed to failure since it might
well not have been built.
Add in some checks to prevent the failures.
[YOCTO #5367]
(Bitbake rev: f34d0606f87ce9dacadeb78bac35879b74f10559)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In order to use toaster, now you have to set INHERIT+="toaster buildhistory"
To keep it simple, I've done some changes in order to automate it. When toaster
is started, this line is added to a new file called toaster.conf.
This file is passed to the bitbake server with the --postread parameter.
Based on a patch by Cristiana Voicu <cristiana.voicu@intel.com>
(Bitbake rev: 029e868044989eda370340f8bf4200cfd2670fca)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If the terminal where the server was started is closed,
the bitbake server should shutdown. Currently the system
is left in hanging state.
This patch uses "trap" command to make sure the servers
are closed on terminal exit.
[YOCTO #5376]
(Bitbake rev: 5f8b97010f7b465753b6ff6275d18426006ee14b)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This patch adds the Toaster component to Bitbake.
Toaster is a module designed to record the progress of a
Bitbake build, and data about the resultant artifacts.
It contains a web-based interface and a REST API allowing
post-facto inspection of the build process and artifacts.
Features present in this build:
* toaster start script
* relational data model
* Django boilerplate code
* the REST API
* the Simple UI web interface
This patch has all the development history squashed together.
Code portions contributed by Calin Dragomir <calindragomir@gmail.com>.
(Bitbake rev: d24334a5e83d09b3ab227af485971bb768bf5412)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This utility doesn't take any special arguments, but it's nice if it at
least knows how to deal with no arguments, --help and errors properly.
(Bitbake rev: 0cabdf1d0cde6687bc1372675a0d6242587c87a0)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* Set up a logger independent of BitBake so we can log errors ourselves
* Handle common errors without printing a traceback
(Bitbake rev: 77b5f5b8dca4deebb06eeb06a8e7f2ccdbfff46f)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* Use OptionParser to parse the two options to -t rather than trying to
pick them out ourselves.
* Add a description shown with --help output
(Bitbake rev: daab42d19463b4108968fc88b207936e5ac84154)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Adjust the task name automatically if the -t option is specified with
a task name that doesn't start with do_ (e.g. "configure" instead of
"do_configure").
(Bitbake rev: d182cbc63745303ef2dc9fa2cbbf5d87a68e0b52)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Getting an error message about --remote-server being set when really
BBSERVER was is confusing, clarify the message.
(Bitbake rev: d7b5938a30a9b0ed83f899a06a88786e8392f8bd)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When starting a server we don't want to show the debug event queue which
is reserved for when when errors have occurred. This patch copies the UI
code to ensure the user doesn't see confusing output.
(Bitbake rev: a886cda58415085981646fb9a024fa7641f55865)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
For BBHandledExceptions, we've already displaced a sensible error to
the user so we don't need to do it again. Just exit with an error
value.
(Bitbake rev: 1ff5ec26eba70ab1c85674a60b7dac77317bf349)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If for example you try "bitbake -m" with an invalid BBSERVER, error
messages are not displayed. This change ensures logging is in place
to catch and display such errors.
(Bitbake rev: 719808f95adc7820fcc09743c592513414d03ce1)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The --help text was rather inconsistent in style and plain incorrect in places,
using confusing terminology in others. I guess most people know what the options
do and don't read this but its confusing to new users.
This updates it to use the terms recipe and task consistently, remove
the references to stage, bbread and generally try and make the output
more useful.
[YOCTO #4856]
(Bitbake rev: 516311946c7bd14c84947dc44c3bb0563e5a9667)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Implementing feature set selection that allows a client
to enable specific features in the server at connection time.
Only enabling of features is supported, as there is
no way to safely remove data loaded into the cooker.
Once enabled, a feature will remain enabled for the
life of the cooker.
Client-server connection now supports specifying the feature
set required by the client. This is implemented in the Process
server using a managed proxy list, so the server cooker
will now load dynamically needed features based on what client
connects to it.
In the XMLRPC server the feature set is requested by
using a parameter for registerUIHandler function.
This allows observer-only clients to also specify features
for the server.
The server code configuration now is completly separated
from the client code. All hardcoding of client knowledge is
removed from the server.
The extra_caches is removed as the client can now specify
the caches it needs using the feature. The UI modules
now need to specify the desired featureSet. HOB is modified
to conform to the featureSet specification.
The only feature available is CookerFeatures.HOB_EXTRA_CACHES
which forces loading the bb.cache_extra:HobRecipeInfo class.
(Bitbake rev: 98e594837aab89ea042cfa9f3740d20a661b14e2)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Due to the worker split the ${DATE} and ${TIME} variables could end up
with different values for different workers.
E.g., a task like do_rootfs that is run within a fakeroot environment
had a slightly different view of the time than another task that was not
fakerooted which made it impossible to correctly refer to the image
generated by do_rootfs from the other task.
(Bitbake rev: 756cc69ebf8bfe8455d0c90f288dd51be2499773)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
BUILDNAME is set from cooker by default, so since the worker split it
will not be set when executing functions. In OpenEmbedded this results
in /etc/version (which is populated from BUILDNAME) not having any
content. Pass this variable value through to the worker explicitly to
fix the issue.
Fixes [YOCTO #4818].
(Bitbake rev: 92940b0427d9b2b3f95e27c230ec1e36638a34bc)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
bitbake-worker makes use of the signal module
but it doesn't import it. This patch fixes the issue.
[YOCTO #4750]
(Bitbake rev: c2ed639690f135994199eb24d964e37f57259e3a)
Signed-off-by: Valentin Popa <valentin.popa@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This change allows bitbake to connect to a remote server
by reading the BBSERVER environment variable and effecting
the config. Basically a shortcut to the command line parameters.
(Bitbake rev: 81929f86c57ed0a4ad0cda7aaa820fceabaa61e9)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
I add an option to terminate a remote server gracefully
as not to need a kill command.
(Bitbake rev: 7495f835666a9561c2c7d84da7aaa74e4df55b9a)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
I add an observer only mode for the knotty UI and
the XMLRPC server that will allow the UI to register
a callback with a server in order to receive events.
The observer-UI is able to send read-only commands to the
server, and also is able to register as an event handler.
Read-only commands are the commands that do not change
the state of the server and have been marked as such in
the command module.
The observer can switch to a full client if it calls addClient
at any time, and the server has no other client running.
(Bitbake rev: 4de9ee21f1fa4d04937cc7430fb1fc8b7a8f61e2)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
With the change to bitbake-worker we need to ensure the workers know
how to contact the PR service, the magic 0 port and singleton is
no longer enough.
(Bitbake rev: c761751e259bb8e940552a28794b45887b5a72d9)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This is a pretty fundamental change to the way bitbake operates. It
splits out the task execution part of runqueue into a completely
separately exec'd process called bitbake-worker.
This means that the separate process has to build its own datastore and
that configuration needs to be passed from the cooker over to the
bitbake worker process.
Known issues:
* Hob is broken with this patch since it writes to the configuration
and that configuration isn't preserved in bitbake-worker.
* We create a worker for setscene, then a new worker for the main task
execution. This is wasteful but shouldn't be hard to fix.
* We probably send too much data over to bitbake-worker, need to
see if we can streamline it.
These are issues which will be followed up in subsequent patches.
This patch sets the groundwork for the removal of the double bitbake
execution for psuedo which will be in a follow on patch.
(Bitbake rev: b2e26f1db28d74f2dd9df8ab4ed3b472503b9a5c)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Two fixes in bitbake related to running remote servers -
* can now specify correctly the bind port
* the information in print conforms to common server infrastructure
(Bitbake rev: b657208ee15ae065e5fcc2dd6e0051e03d246727)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Added code in XMLRPC server that creates a stub local server
for a client-only connection and is able to connect to
a remote server, and receive events from the remote server.
Added the option to start a client with a remote server in
bitbake.
Original code by Bogdan Marinescu <bogdan.a.marinescu@intel.com>
(Bitbake rev: 25b2af76104d5aaf6435de8c158e0407512f97ce)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This is a code sanitization targeted at making further
server-related changes easier (launch a server
separately or creating a mockup-server) to do.
(Bitbake rev: eac00258d213137ef73aed255c92b7981e2f1c75)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The collection of the extra caching data should not
be performed by the cooker, but supplied to it.
This patch will also streamline the code for launching servers
without a UI attached.
Based on a patch by Bogdan Marinescu <bogdan.a.marinescu@intel.com>
(Bitbake rev: f0b54280a6bce522508e4741e5f507bc284113a8)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Slight change in bitbake as to retrieve the entire UI module
for further processing instead of just the main function.
Based on a patch by Bogdan Marinescu <bogdan.a.marinescu@intel.com>
(Bitbake rev: f49341a9599d971829ef65b2b02732543740a3c9)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
With the removal of the none server type, we can remove the launchUI
method and simplify the code slightly.
(Bitbake rev: 9bef2f2dd0bcaa59528ebcb3c1ce053b7dff1ec6)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The process server backend has been serving well as the default for a long
time now and the UI model is much better thought out that it used to be. With
the move to make bitbake a memory resident process, the none server is now
looking rather pointless and complicates the code needlessly. Lets therefore
now remove it.
(Bitbake rev: 9af03a89605e3db9bce3cea1e0f2d0b6cfaa6fe1)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fixes the show-appends and flatten subcommands for recent refactoring in
cooker.
[YOCTO #4536]
[YOCTO #4535]
(Bitbake rev: 54817d0de667941ee68fe3490684159e2d90e9c5)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
calc_bbfile_priority is now in CookerCollectFiles which can be accessed
on the collection attribute of a cooker instance.
Fixes [YOCTO #4513].
(Bitbake rev: 5d941631ad7198737d9a5c5a920a9062fa0431f8)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
As the code stands today its hard to know which configuration variables
are used by which parts of the system. Some are used by the UIs, some
by bin/bitbake itself, some by cooker.
This patch changes the configuration to just contain the variables cooker
uses, and changes bin/bitbake to access the variables it needs directly
which hopefully lets us start to untangle this mess.
(Bitbake rev: e57497a24b6157c92519a34accd66035a39ad1f8)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Building up a set of actions for the server is tricky since we depend upon the
commandline but fall back to values from the datastore. We should be able to build
a datastore without a commandline and vice versa. Ultimately the UI should send
the commands to the server.
This patch amounts to code rearranging, moving the heavy lifting to the UI, though
a helper in the configuration option. This will need further cleanup/tweaking but
this should be the only update needed to the UIs. The code now queries the server
for any missing data should it need to.
This code allows various knowledge of configuration variables to move to the UI side
only, partcularly pkgs_to_build but also all the command specifiers. It should also
be possible to move cmd eventually, I'm just unsure if any callers call the commands
expecting this to default to something sane right now.
(Bitbake rev: 2dbbb1d51dafd4451fef8fe16f095bcd4b8f1177)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently the UI and server configuration is one big incestuous mess. To
start to untangle this we creater cookerdata, a new module which contains
various confiuration modules and the code for building the base datastore.
To start with we add a ConfigParameters() class which contains information
about both the commandline configuration and the original environment.
The CookerConfiguration class is created to contain the cooker.configuration
options. This means we can transfer new paramters to the server over something
like XMLRPC and then build a new configuration from these on the server.
Based on a patch from Alexandru Damian <alexandru.damian@intel.com>
(Bitbake rev: 35bd5997e8d8e74bc36019030cc10c560a8134f9)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Due to the internal event processing, this excepting handler usually raises an
Empty error, masking the underlying failure. Ensure the original exception is
raised.
(Bitbake rev: 7d548568a55adfe84a976f2a549995e42da1afef)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Python multiprocessing needs /dev/shm, and if it doesn't exist (e.g. in
a chroot environment) or is not writable, you'll get
"OSError: [Errno 38] Function not implemented" or
"OSError: [Errno 13] Permission denied" with a traceback, which doesn't
really help to explain what is wrong.
Implements [YOCTO #4407].
(Bitbake rev: e7460d01f281f913a94192e1f4bd20688164bdd4)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently if errors occur early in the init process, the errors may
not be shown to the user. This change ensures that if a failure does
occur, the messages are flushed from the queue and shown to the user.
(Bitbake rev: fda84c4285e0bc31c249b6dd5464aeb6ad622a9a)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In case of an error in bitbake, the stack trace is limited to 5
items. This is an endless source of confusion and it makes bugs
reports impractical, since a full stack trace can't be included
in the bug report. This patch simply removes the depth limit.
(Bitbake rev: 02629c42fb09413d9da16cfe43e03338ce7db3ff)
Signed-off-by: Bogdan Marinescu <bogdan.a.marinescu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There are two "help" lines when run bitbake help under python 2.6.6
(perhaps all python 2.6):
$ bitbake-layers help
[snip]
help
display general help or help on a specified command
[snip]
help
display general help or help on a specified command
This is because the cmd.py in python 2.6.6 also has a do_help() function
as our bitbake-layers, and the cmd.Cmd.get_names() doesn't consider the
overload, this has been fixed in python 2.7, use set() to remove the
duplicated would fix the problem.
[YOCTO #3428]
(Bitbake rev: 324ed96e28ec31cff8cef1824d20d40f9a5d46ad)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The "bitbake-layers show-cross-depends" print the absolute path in the
past, now it will print the relative path (relative to the layer dir) by
default, and add the "-f" option to make it print the absolute path.
For example:
$ bitbake-layers show-cross-depends
[snip]
meta-intel/meta-jasperforest/conf/machine/jasperforest.conf requires meta/conf/machine/include/ia32-base.inc
[snip]
$ bitbake-layers show-cross-depends -f
[snip]
/path/to/poky/meta-intel/meta-jasperforest/conf/machine/jasperforest.conf requires /path/to/poky/meta/conf/machine/include/ia32-base.inc
[snip]
[YOCTO #3824]
(Bitbake rev: df217701318d60559be0b10214b883b8ce4f5d2a)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We need to empty out the environment whilst we build the cooker but
we need the environment for the UIs since hob uses DISPLAY and other
session variables.
This patch adapts the utils functions to return removed environment
components so we can reinject them for use by the UI, allowing hob
to work again.
(Bitbake rev: fc330d810099c57fefd4e706159a73ad8401d97c)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Figure out the dependency between recipes that crosses a layer boundary
* Introduction:
- For the .bb file, we need check the following depends:
> Check the DEPENDS, RDEPENDS and inherits according to the
cooker_data.
> The cooker_data doesn't have the info about "require/include xxx",
so we need check them manually.
- For the .bbclass, .inc and .conf file, we can't get their file
depends from the cooker_data, we need check them manually.
* Usage:
$ bitbake-layer show-cross-depends
I don't like the name "show-cross-depends", figure-out-cross-depends
might be better, but most of the commands are "show-xxx" for bitbake-layers.
* Output: (snipped)
/work/poky/meta/recipes-support/libusb/libusb-compat_0.1.4.bb inherits /work/poky/meta-yocto/classes/poky-sanity.bbclass
/work/poky/meta-intel/meta-cedartrail/recipes-cdv-media/images/core-image-cdv-media.bb requires /work/poky/meta/recipes-sato/images/core-image-sato.bb
/work/poky/meta-yocto/recipes-core/tiny-init/tiny-init.bb RDEPENDS /work/poky/meta/recipes-core/busybox/busybox_1.20.2.bb
/work/poky/meta-intel/common/recipes-bsp/amt/lms_7.1.20.bb DEPENDS /work/poky/meta/recipes-devtools/autoconf/autoconf_2.69.bb
/work/poky/meta/classes/distrodata.bbclass includes /work/poky/meta-yocto/conf/distro/include/package_regex.inc
/work/poky/meta-intel/meta-sys940x/conf/machine/sys940x.conf requires /work/poky/meta/conf/machine/include/tune-atom.inc
[YOCTO #3387]
(Bitbake rev: 100502ef722bf19b6c125a5e017f31686ad18421)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This means we can run in parallel with other bitbake processed we're not in
task mode.
(Bitbake rev: 20222f2f16130e91eed5e68286188ee0e8f8f3bf)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The get_file_layer can't handle the nested layer correctly, e.g.:
meta-intel/conf/layer.conf
meta-intel/meta-cedartrail/conf/layer.conf
They are two layers, the file:
meta-intel/common/recipes-bsp/amt/lms_7.1.20.bb
belongs to meta-intel's layer, but the get_file_layer() may return
meta-cedartrail accordig to BBLAYERS' order, since it uses:
for layerdir in self.bblayers:
if regex.match(os.path.join(layerdir, 'test')):
return self.get_layer_name(layerdir)
which doesn't care the filename, re-match the filename would fix the
problem.
[YOCTO #3387]
(Bitbake rev: 7a31b9eac4d894c302f0e052a132c31b17d13d1f)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Since hig.py was becoming too large (which impacted maintenance and
operations like git merges) it was split into individual files for
the classes that were implemented in hig.py. hig is now a Python
package (lib/bb/ui/crumbs/hig/). The patch was tested by
building core-image-basic/qemux86 in Hob and accessing the various
UI elements implemented in hig. Note that the patch does not change
the functionality of Hob in any way, it's just a code refactoring
that should make life a bit easier for Hob developers.
[YOCTO #3200]
(Bitbake rev: a7a2b730f915cafe7aa30539622dd1ca64ae41f5)
Signed-off-by: Bogdan Marinescu <bogdan.a.marinescu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The method this older code was accessing doesn't exist. A library shouldn't
be changing behaviour based on the environment anyway when sepcific function
calls exist to correctly setup the logging. Fix the crash if BBDEBUG was used
by moving the code to bin/bitbake.
[YOCTO #3319]
(Bitbake rev: 3a1b75037fe1e5282ffd40b037e93353405faa45)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This code is dead and doesn't do anything so lets remove it.
(Bitbake rev: 8d45739f49618757a5d7d79782deda355e3981ec)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
It is possible to lose critical log data when python exits in an
unorderly fashion via segmentation fault or certain types of crashes.
This is because the buffer characteristics are inherited from the top
level stdout, which should be set to unbuffered, for the purpose of
all the forked children.
This pushes the buffering to the OS, instead of having python managing
the buffers in its stream handler class.
This change is also to provide the ability to tail logs written from
processes in "real time" because they would be written in an orderly
fashion depending upon the OS characteristics for the file I/O.
(Bitbake rev: c6a367bc3224adafca698a4ffc5414ad83842c16)
Signed-off-by: Jason Wessel <jason.wessel@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Mainly intended for the purpose of debugging or forcing builds
from source, the --no-setscene will prevent any setscene
tasks from running.
(Bitbake rev: 440e479f3e248482c38c149643403c6907ac7034)
Signed-off-by: Jason Wessel <jason.wessel@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add the ability to compare the two most recent runs of a specified task,
and follow dependent hash changes recursively. This enables you to trace
back and find exactly why a task was re-run after the fact.
Note that this relies on the metadata providing a function, hooked in
as bb.siggen.find_siginfo, which allows searching in the appropriate
places to find signature data files.
(Bitbake rev: cc70181659c07e04c205e17832846acf1ff31d28)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>