Used $TOASTER variable(path to the toaster script) instead of
current directory name to get path to the toasterconf.json
This way script should work in both modes and it doesn't depend
on the current directory.
(Bitbake rev: 5368d9af4e2c121938fc2421a536fb7f20a987fd)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The toaster startup script conditionally migrates the database
tables depending on whether you are in managed mode or not. This
means that if you are in analysis mode, some of the bldcontrol*
database tables used by managed mode are not available.
As a consequence, some of the code in toaster which refers to
those tables can break in analysis mode, as there's no clean
isolation of the two modes.
To prevent this from happening, always run the migrations for
managed mode and create the bldcontrol* tables, even if in
analysis mode.
Also clean up the function which starts up toaster so the
logic is easier to follow.
[YOCTO #8277]
(Bitbake rev: b1fc592131286ebbede2693be8c86636f0039011)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
removed layer dir questions and base off TOASTER_DIR
removed build dir questions and base off TOASTER_DIR
base configuration file off of TOASTER_CONF
fixed some pylint issues
[YOCTO #8217]
(Bitbake rev: d5811968b5f22093365f381fb7e75ab46e5269c2)
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
added TOASTER_CONF env var for toaasterconf.json
added TOASTER_DIR env var for working dir
added bugfix so WEB_PORT env variable is honored
(Bitbake rev: c4b351e40060a359d59ab51b5ed2f9713ca1a9a5)
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
added comment explaining how to create one later
(Bitbake rev: 782b7c74d5f10ed255538ba0c975ff342a34f5bf)
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In the forked child, we may use multiprocessing. There is only one event
pipe to the worker controlling process and if we're unlucky, multiple
processes can write to it at once corrupting the data by intermixing it.
We don't see this often but when we do, its quite puzzling. I suspect it
only happens in tasks which use multiprocessng (do_rootfs, do_package)
and is much more likely to happen when we have long messages, usually many
times PAGE_SIZE since PAGE_SIZE writes are atomic. This makes it much more
likely within do_roofs, when for example a subprocess lists the contents
of a rootfs.
To fix this, we give each child a Lock() object and use this to serialise
writes to the controlling worker.
(Bitbake rev: 3cb55bdf06148943960e438291f9a562857340a3)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a carriage return to build environment warning message to
avoid it being mixed up with shell prompt.
(Bitbake rev: d3675cca419946cc19b3f280446fe1f656f11902)
Signed-off-by: Cristian Iorga <cristian.iorga@intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If the pipe is closed, we want to ensure that we kill any child processes
by triggering the sigterm handler before we exit. This code does that,
hopefully avoiding the remaining process left behind issues on the autobuilder.
(Bitbake rev: 60f6c2818f38c4d9c2d9aaa42acf3071636f4a3b)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If we SIGKILL cooker (the parent process), ensure the worker notices
and shuts down gracefully. To do this:
* trigger the sigterm handler if the parent exits
* ensure broken pipe writes don't trigger backtraces which interfer with
other exit work
* notice if our command pipe is broken due to EOF and sigterm if so
(Bitbake rev: c43d6a8d711db8d3bd9a1976b9f8e3efdb4cb4ae)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
start/end are unused here and we can improve the code conditional blocks.
(Bitbake rev: 68f53dd77fe0bbfa044bd037a9484e0e1c9088b4)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a -i/--inherits option to filter the output to include only recipes
inheriting the specified class(es).
Implements [YOCTO #7475].
(Bitbake rev: e4ec622d9dc5f882049f5e8027331ec1e17681de)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
bitbake-layers needs to map recipe and class files to the layer they
came from within the show-recipes and show-overlayed commands. However,
it turns out that mapping a file to the layer it came from is not as
trivial as it might seem. To do it properly we need to match the path to
an entry in BBFILES then map that to the collection name using
BBFILE_PATTERN, then map that to the actual layer using variable
history. If it doesn't match any entry in BBFILES, then we can fall back
to BBFILE_PATTERN (to handle classes and conf files).
This fixes the layer name not showing up properly in the output of the
show-recipes and show-overlayed commands for recipes in layers such as
meta-intel that have subdirectories in BBFILE_PATTERN. It also fixes the
priority not showing up in show-layers for such layers.
As part of this I've added a function to VariableHistory which for a
space-separated list variable gives you a dict mapping the items added
to the files in which they were added. I've also fixed
bb.utils.get_file_layer() and reduced some of the duplication by using
this function in bitbake-layers. Also fixes the priority not showing up
for layers such as meta-intel
Fixes [YOCTO #8160].
(Bitbake rev: e852f6cabd7489585477ab567a1afeb2252377ac)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If the specified layer isn't a path, then just match on the directory.
Fixes [YOCTO #7839].
(Bitbake rev: b4c45bf6c42b4d319ba868f4ce77e86c8b585818)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We were printing an error here, but not exiting.
(Bitbake rev: ddcaf8950a0b1cc74806e1ad7b49a1de0ea0d2b1)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
It's best practice to use "with open..." rather than open & close where
practical.
(Bitbake rev: 1ef38549cae5639f2c8bcc2b270c5c82a5e29e3c)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
flatten support currently looks broken since it doesn't appear to
deal with handling "%" support in bbappend file names.
This patch converts it to use collections.get_file_appends() which
correctly handles "%" support.
(Bitbake rev: 0448714c52bc1e9584a5282cffdcaa404fb0618a)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Also document BB_SKIP_NETTESTS=yes parameter in --help output.
(Bitbake rev: 5196bfa9639eed2b1e6452f45775551203f8eeb4)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Made it working in dash. Note, that due to dash limitations
script will not work if sourced.
(Bitbake rev: febf8bedf8f9d37659831cdde208d14ece2fb322)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We enable a "nobrowser" parameter that inhibits
the launch of a browser when toaster starts.
This is useful for integration with automated startup scripts
and enables headless testing.
(Bitbake rev: ccf7f39d470fe6d743b58b1140c19cb8da31ecaf)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In an effort to fix client-side session storage using
cookies, we enable server-side session support for all pages.
(Bitbake rev: ba10b6f89767c0dad8a2b064f42a84956764e9da)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
I noticed that I was seeing loss of the log files when hitting
control-c while debugging a function in bitbake. In fact if you
take a recipe and replace its compile function as shown below let
it run for a few seconds and hit control-c, you will see first
hand that log data is not there.
do_compile () {
while [ 1 ] ; do
echo -n "Output date: "
date
sleep 1
done
}
It turns out there was a regression introduced by commit:
d0f0e5d9e6 which created the bitbake worker. Since the bitbake
worker is started in its own process space, it needs the exact
same code added from commit: 88429f018b where the problem was
fixed the first time around.
(Bitbake rev: 8d1748f75763b4a66516cc46d5457ee6404b1b68)
Signed-off-by: Jason Wessel <jason.wessel@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The bitbake observer is now started using python subprocess. This should
allow for toaster to run without installing the "daemon" application.
[Yocto #7271]
(Bitbake rev: 1dd599ddfcdb547bee49bd7d86acddf64c675b42)
Signed-off-by: Randy Witt <randy.e.witt@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This tests very basic usage of the parser and then adds a test
to ensure that incomplete functions raise an exception.
(Bitbake rev: b7bcef141b56fe8eb03724ea5251e3251fc63817)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Replaced tabs with 4 spaces.
Removed useless semicolons at the end of lines.
(Bitbake rev: 5bfe5eb0bf1e53ddb04c31e786e1ffc15542e6e8)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fixed shell syntax not understood by zsh. Made toaster
script to work for both shells in both modes: sourced and
directly called.
[YOCTO #6964]
(Bitbake rev: e7b27843e2f37619dbe555a8883ac67f418f32dc)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This patch properly shuts down toaster in interactive mode
under Mac OSx.
(Bitbake rev: a00cd5135a4bbd61d311fa17569894f974ab4420)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This patch modifies the toasterui to properly return the exit
code based on the errors found in the toaster itself.
The upload event file API call will not delete event logs for which
toasterui showed an error. This will facilitate debugging.
Minor enhancement in the buildinfohelper to reduce the number
of lookups on unknown layer objects (prevented testing of the patch).
(Bitbake rev: 1ddd6a9e4280a4adf971132ff1fe7ec9b3252905)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Used sys.exit instead of assigning exit code to
variable. This way it's more clear when bitbake
exists and which exit code is used.
(Bitbake rev: 5ecb8817bd49223652ede4fe513f1a42f2196798)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Bitbake program and core versions must match.
Moved __version__ from main.py back to bin/bitbake.
Implemented check for version match in bin/bitbake.
(Bitbake rev: 2fe7d8c574ddf6a30278cff1a5a5c4089dc56d6d)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
tbs
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Some error messages were lost because BBMainException was
inherited from bb.BBHandledException. When bb.BBHandledException
is processed error messages are not printed as they suppose to
be printed before raising this exception.
Stopped to inherit BBMainException from bb.BBHandledException.
Handled BBMainException in bin/bitbake and printed error message
to the stderr.
(Bitbake rev: c8e2a40c4e9865ebef9936d23644f2602a5c90f5)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
For now, `bitbake-diffsigs -t <recipe> <task>' doesn't work. It always outputs
nothing.
The problem is that bitbake-diffsigs are comparing sigdata and siginfo files
that have the same hash value. This is not what we want. These two files are
actually duplicates considering the purpose of bitbake-diffsigs. So we need
to remove one of them so that bitbake-diffsigs could actually compare the
correct signature files.
(Bitbake rev: c34613eb69fd19770cbfc78ab8384221f10d5587)
Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In show-recipes and show-overlayed, we only list one variant of each
recipe where multiple exist, therefore we should show the main one (e.g.
in OpenEmbedded, we now show openssl and not nativesdk-openssl which
would otherwise sort first.)
Fixes [YOCTO #7514].
(Bitbake rev: c4172fe33f0c63eda7f4af2bd36a5e9738e13595)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This is particularly problematic since qemu images switched to ext4 by
default and now cannot work properly with UIs like hob.
This patch adds in ext4 to the appropriate variable fixing this.
[YOCTO #7426]
(Bitbake rev: a1431b1b85bf90ed5f771552d0ce91de288047d4)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Moved most of functionality of bin/bitbake to lib/bb/main.py
to be able to call bitbake from python code.
(Bitbake rev: d377f7f88d73f4e5d2dffef03d6acee809827ac6)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We use logger.plain() to produce actual output which needs to go to
stdout. We could use more advanced filtering (and probably should in
future) but for the moment let's just fix the regression.
Fixes [YOCTO #7458].
(Bitbake rev: e96fc0ccfc9f5be2c41c9733c92dc81df3df5065)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The recent refactoring caused use of wildcards to trigger a traceback
because the old variable name was being used.
Fixes [YOCTO #7367].
(Bitbake rev: 7182cdd3c4a534a87147bb0aad7b360ffef6426b)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In order to bring in transaction.set_commit support,
we need Django 1.6, discarding support for Django 1.5
This patch removes 1.5 from the list of acceptable Django
versions.
(Bitbake rev: 8640769296c3fffa2cf56cfb5327ca24533f5a41)
Signed-off-by: Alexandru Damian <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a command to query layer dependencies from a layer index such as the
OpenEmbedded Layer Index at http://layers.openembedded.org. Fetches the
layer and its dependencies and adds them into conf/bblayers.conf.
[YOCTO #5348]
(Bitbake rev: 4b8fcf9a5bc802793bf332334217faace55f14f6)
Signed-off-by: Chong Lu <Chong.Lu@windriver.com>
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Logging output was simply not getting printed.
(Bitbake rev: 62b825b19b13a914cbff5303d541bd5dbbec90a7)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This makes help formatting and option handling a lot more standardised
and allows us to drop a bunch of code. We also gain slightly more
straightforward error handling.
One side-effect however is that the old subcommand syntax using
underscores is no longer supported. The dashed form has been supported
(and displayed in the help text) for quite a while now so I wouldn't
imagine that will be much of an issue.
(Bitbake rev: 6e2f09b58882d3949026b9dd545f789ad3fe6fab)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This patch improves the logging facilities for toaster in order
to help diagnose bugs that happen on user machines.
The logs are stored now under "/tmp/toaster_$$" where $$ is a
PID-based unique identifier. On shutdown, toaster will automatically
erase all logs unless errors are listed in the log file.
On error, Toaster provides suggestions on what to do.
This patch includes a minor fix found as a result of logging
improvements.
(Bitbake rev: 8a8248f7b7e30469f592e2f8adbf6ce21e8685c5)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This patch brings fixes and enhancements in the startup sequence.
- the dependency version checking is now correctly enforced when
starting in both modes
- fixed errors in looking up environment variables
- added message with 'daemon' program start location
- presenting menu for selecting config file
[YOCTO #7283]
[YOCTO #7291]
[YOCTO #7273]
(Bitbake rev: c5ddd9d88910857a1b745b1c237df0390dd56835)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
A small typo slipped in the message included in
the fix for [YOCTO #6023].
(Bitbake rev: f28fb3d660d8726b75e6c31fa36bc22b54e1bca7)
Signed-off-by: Belen Barros Pena <belen.barros.pena@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This patch makes sure that all the toaster conf files are actually
written from the build enviroment controllers.
Additionally, toaster checks that the 'daemon' program, which is used
to start the build system, is available (currently for localhost).
[YOCTO #7171]
(Bitbake rev: 0a1db7d1531f8254955e1152bcd8e6db4ec1d277)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This patch updates the prompts to the user to better explain
what Toaster is doing and the information it needs from the user.
Additionally, fixes a check in loadconf command.
[YOCTO #6785]
[YOCTO #7251]
(Bitbake rev: 90ef8975f35e2da824bc1c80e41ca26d9af0b208)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This patch makes Toaster managed mode obey the "noweb" parameter
by not starting the web server and launching the web browser
command if the "noweb" parameter is specified.
The web browser will be pointed at 127.0.0.1 instead of 0.0.0.0
[YOCTO #7039]
(Bitbake rev: 4037f8b08bc9fb5c4c9f260efb847105be718a32)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We change the setting of variables from directly injection
into the set-up cooker to writing a conf file that is pre-read
on bitbake server startup. This is needed because the injection
can only happen after the variable set is parsed, and the variables
already inferred, so setting up variables happens too late.
[YOCTO #7045]
(Bitbake rev: 854f680b5b9d2d0fa796af84cb1218545fbfc55a)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The bug has a long discussion of this. Basically, in some environments,
the exact details of which aren't understood, a Ctrl+C signal to the
UI is being transmitted to all the process children. Looking at the output
of "ps ax -O tpgid", its clear the main process is still the terminal
owner of these processes.
stty -a on a problematic system shows: "-ignbrk brkint"
and on a working system shows: "-ignbrk -brkint"
The description of brkint would suggest this is the problem, setting up
that terminal environment wasn't able to reproduce the problem though.
It was confirmed that using setsid() caused the problem to be resolved
and is probably the right thing to be doing anyway, so lets do it.
[YOCTO #6949]
(Bitbake rev: 461aa73fff0ab616032d28c4fd0322eb88838be6)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* Fix add-layer error message when a layer is already in BBLAYERS
* Ensure we show an error message if we can't find BBLAYERS at all
(Bitbake rev: 1c743fd2103730e27699dd55efc6914d3b0c3702)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add add-layer and remove-layer commands for easily adding and removing
layers to/from bblayers.conf.
(Bitbake rev: 0e94aed033917b91ea6ba0522b14a040c3bdc987)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This patch adds a "-w/--write-log" option to bitbake
that writes an event log file for the current build.
The name of the file is passed as a parameter to the "-w"
argument. If the parameter is the empty string '', the file
name is generated in the form bitbake_eventlog_DATE.json,
where DATE is the current date and time, with second precision.
The "-w" option can also be supplied as the BBEVENTLOG
environment variable.
We add a script, toater-eventreplay, that reads an event
log file and loads the data into a Toaster database, creating
a build entry.
We modify the toasterui to fix minor issues with reading
events from an event log file.
Performance impact is undetectable under no-task executed builds.
(Bitbake rev: 1befb4a783bb7b7b387d4b5ee08830d9516f1ac2)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This patch modifies how layers are identified and matched.
Layers were primarely organized by the source of layer information,
and Releases were separated by both layer git branches and originating
source of layer information. This setup prevented mixing layers from
different sources for a certain release, which didn't match the way
people use Yocto Project / bitbake.
This patch brings name-based indentification, where layers with the
same name are assumed to be equivalent, in the sense of being able
to substitute one another. To facilitate this identification to
humans, layers are differentiated by GIT URI instead of layer sources,
which was a rather arbitrary abstraction.
Additional changes include modification to models in order accomodate
for the new data structure, and to config file loading to match
the new toasterconf.json layout.
(Bitbake rev: 4357200aed522ad56cfd84917f877645b83b6a70)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When toaster is used by multiple users on the same machine,
it is possible that files in /tmp have a name race.
This patch makes sure that the files have unique names.
(Bitbake rev: d4a47bc84f762666a847f1152cc2e75c9ef36092)
Signed-off-by: Belen Barros Pena <belen.barros.pena@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fixed:
1) Run "bitbake recipe" in the terminal
2) Close the terminal while building
3) $ ps aux | grep bitbake-worker
There will be many processes, and they will keep the resources (e.g.,
memory), and won't exit unless kill or kill -9.
(Bitbake rev: 40d2ae0723de2bf5fee343faafb4afda40546839)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We add the "webport=" command line option as to allow starting
the web server on a custom port.
The bitbake server port is now auto-allocated. This is needed
to be able to run multiple toaster environments on a single machine.
We tackle bug 6023 (toaster refusing to start when lock file is present)
by using more specific checks, and automatically recover from
bitbake server down / webserver up error mode.
Command line parameters are now read on both interactive and managed
modes.
The localhost and ssh controllers are updated to work with the modified
toaster launcher script.
[YOCTO #6023]
(Bitbake rev: cd3eb5b051743463cfe51dba97cae4da75420048)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fix the startup message by making clear which
server runs on which port.
(Bitbake rev: 26d601ab2a677a422bab425696a2cf6357ff3576)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We avoid a race between the setting the TOASTER_BRBE variable
and reading the variable in toaster ui by supplying the variable
at server startup time through the toaster.conf post-read file.
Additional small changes are included, including marking the
build request with the environment id of where the build took place.
(Bitbake rev: 7c333350418c4140e6c988c5272940f8057d327d)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Without this you see:
File "bitbake/bin/bitbake-worker", line 201, in fork_off_task
os._exit(child())
TypeError: an integer is required
(Bitbake rev: cd477b5e77ab0373248b8a8fa30e1c7b8ea984fd)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We need to transfer some of the siggen data from the core/cooker into
the worker instances. There was a partial API created for this but
its ugly and its not possible to extend it from the siggen class.
This patch completes the interface/abstraction for the data and
means the class can extend/customise it in any siggen class.
(Bitbake rev: cf2d642052979d236185c5b8ca2c5478c06e62ae)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We change the toaster starting script to make the
bitbake server listen on all interfaces on the local machine.
This is needed to be able to receive a controlling client
running on a remote machine.
(Bitbake rev: 137179eafca8d1a5a69b6302f8cc8961be3b45c4)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We enhance the checksettings command to try to automatically
detect settings for running on localhost.
The checksettings will look for a nearby poky layer source,
for a nearby build directory, and will try to import settings
from "toasterconf.json" files found in the local layer.
On new configuration, it will also perform updates from the
layer source.
(Bitbake rev: 2aab77dfccb538e2b09829841ea6c464d40cafb1)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently we get no profiling oversight into either the main bitbake worker
process, or the overall parsing before task execution. This adds in extra
profiling hooks so we can truly capture all parts of bitbake's execution
into the profile data.
To do this we modify the 'magic' value passed to bitbake-worker to trigger
the profiling, before the configuration data is sent over to the worker.
(Bitbake rev: 446e490bf485b712e5cee733dab5805254cdcad0)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When processes terminate, we really want all of the child processes to
terminate too. This was not happening for worker processes which spawned their
own multiprocessing pools, leading to build hangs. This change ensures any
sigterm gets passed to the whole process group. In local tests, this resolved
some hanging process workloads I could generate. It does rely on signals
being delivered in a timely fashion and there is a multiprocessing bug we have
to work around there.
(Bitbake rev: 96f8ea07ace1379380fab2d78eb592fa40c867d4)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The SIGNATURE_HANDLER can be defined by the user, so we can't use choices.
(Bitbake rev: 0036b36e333a68c6d30a6bf5c21bfd4fd3e8441b)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Since Toaster adds a number of specific requirements managed
by pip, we document these requirements in order to be able
to run bitbake and toaster under virtualenv. The target here
is to break the dependency on distro packages for specific
python libraries.
In order to start bitbake/Toaster in a distro-independent manner,
we use virtualenv and pip. We add venv to the .gitignore list
to make room for the virtualenv deployment in this directory.
Use this command sequence to setup the virtualenv:
$ virtualenv venv
$ . venv/bin/activate
$ cat toaster-requirements.txt | xargs pip install
bitbake and Toaster will then run normally under virtualenv.
We also fix bugs related to toaster starting in managed mode
through this script:
* toaster will not use bldcontrol application in interactive mode
* a proper message will be displayed if toaster detects a managed
start request in a build environment
(Bitbake rev: b5789c7ea7e6d1a4b67ec60fc0e4233ec543ba3a)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We add explicit absolute paths for a directory where
the layer sources will be checked out (sourcedir) and
where the build activities will take place.
Adding minimal checking when starting the application in
order to make sure that BuildEnvironment (BE) settings are
usable. This check is ran by the toaster script at startup.
Modify the localhost bbcontroller to use the BE settings
instead of trying to self-configure on checked out sources.
(Bitbake rev: d17500d3f73fdeeef5f11fb3773a65e927be3f02)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(Bitbake rev: 1cd369883469747a8158826bb8d67dcca2a8577f)
Signed-off-by: Robert P. J. Day <rpjday@crashcourse.ca>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The bitbake -S had been updated which always reuires an argument, so
update the help info:
- Add the two args in the help message: none and printdiff
- Use type="choice" so that we can get more friendly error messages,
for example:
bitbake: error: option -S: invalid choice: 'printdiffX' (choose from 'none', 'printdiff')
- dump-signatures=DUMP_SIGNATURES -> dump-signatures=SIGNATURE_HANDLER
(Bitbake rev: 021b778fa4685bdde39e1a0f6c7c57632dcf792a)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We add the new project and project page skeletons.
In the process, we add an identifier in the settings.py
to detect whenever Toaster is running in managed mode,
and a context processor to make this value available
to the template processor.
(Bitbake rev: 927a27c68e24cfe13f62ca5f0e60878b04fa4e24)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We add the build control functionality to toaster.
* The bldcontrol application gains bbcontroller classes
that know how to manage a localhost build environment.
* The toaster UI now detects it is running under build
environment controller, and update the build controller
database and will shut down the bitbake server once
the build is complete.
* The toaster script can now run in standalone mode,
launching the build controller and the web interface instead
of just monitoring the build, as in the interactive mode.
* A fixture with the default build controller entry for
localhost is provided.
[YOCTO #5490]
[YOCTO #5491]
[YOCTO #5492]
[YOCTO #5493]
[YOCTO #5494]
[YOCTO #5537]
(Bitbake rev: 10988bd77c8c7cefad3b88744bc5d8a7e3c1f4cf)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We add support to specify a connection token in the command line
and in the environment variable BBTOKEN.
When a client registers to a bitbake server, that client
will have exclusive access to the server. The client is identified
by a server-supplied token. If a client terminates, we cannot
reconnect to the server as the token is lost.
This patch adds the capability to specify the connection token
in the command line for xmlrpc clients. This allows us
to have bitbake work as an already-authenticated
client with the server and resume sending commands to a server.
(Bitbake rev: db5390940c0afbcdc9fbcf1225761968ae51d4a7)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We create the model classes that store information
about triggering builds, and the available build
environments.
We add a fixture with a default build environment
for build control, using a "build/" directory under
the poky checkout directory.
We enable the bldcontrol in toaster starting script
and in the toaster settings as to allow the actual database
to be kept in sync with the source code.
(Bitbake rev: d4bfe9059f765f11244b97e324c0131f32f8e400)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This fixes toaster startup failing after the preparatory patches
for bldcontrol application have been merged.
I slipped a bldcontrol-specific line into the toaster startup
script and this prevents toaster from starting. We comment
out this line until the bldcontrol application have been merged.
(Bitbake rev: d095eec6fc958d0aeb514cdc206734617fd7c930)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The 'bitbake-layers show-appends' command can use the built-in
get_file_appends() method which correctly identifies bbappends that
include a '%' wildcard in their filename.
(Bitbake rev: 2732dbae67c1945b668c38cc4cc5678c4aafe3d6)
Signed-off-by: Ash Charles <ashcharles@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We disable bitbake self start to prevent race condition.
ToasterUI will shutdown the server when the build is done
if running in managed mode.
We fix usage of kill server flag in the bitbake binary.
(Bitbake rev: 30159dbda3a40fa596302f91c705cb5f148c97a9)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The toaster starter script was designed to be sourced in
a build environment and set up the build recording environment
to be used in an interactive mode.
For the standalone web server mode, we modify the toaster
script to be run directly from the checked-out sources,
without a build environment set up, and run the web server
alone. In the standalone mode, the build environemnts and
all build activities are controled through the web interface.
(Bitbake rev: c1db4ccf27bedcbab2f03e7539fdb11b042c4fb9)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We add an option to the startup script, named "noweb" that
will start toaster without the embedded web server.
This is useful to start the system for build-only environments,
where the web server code is running on a different machine.
(Bitbake rev: c39838201301b9732581288a93783400bebe6591)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When we use the XMLRPC client API to connect to a bitbake server,
we want to receive errors from the API instead of having the
API exiting without warning.
Thus the "bb.fatal" calls have been replaced with "bb.warn" calls,
and we re-raise the original exception for handling by the
original caller.
The bitbake starting script has been modified to properly test
for failures in calling the client API and handle them.
Additional error handling added in the client, as to prevent
fatal crashes.
(Bitbake rev: eb63f08c33644f64752aaae2146a000956ce894a)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The configuration reading code should live in the
main bitbake entry point, and the server modules should
be supplied with correct configuration instead of attempting
to parse from configuration files.
This patch moves the endpoint address reading from XMLRPC
to the bitbake main script.
(Bitbake rev: ac5753274ff932e1d6f073ab4dab7bd6fe5355a1)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
RRECOMMENDS must be satisfied at build time, and these could cross layer
boundaries, so report these if they exist.
(Bitbake rev: 5569b3dca61e6d962494ca65c7aad09b2eb2ae63)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Overlayed recipes caused this to show false positives because the
overlaying version appeared to be satisfying the overlayed version's
RDEPENDS; but you'd never be building both at the same time.
(Bitbake rev: b94318174fe7f92b9a20eabb0bc4055066cb3d51)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
It's not particularly useful to show globally inherited classes here
since they do not normally represent a dependency.
(Bitbake rev: d16948bb88fcf44d861985838030be7c08697963)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
By default, show-cross-depends shows dependencies on OE-Core (i.e.
"meta") which is not particularly useful. Add an option to allow you to
hide those. For example, to hide all dependencies on OE-Core:
bitbake-layers show-cross-depends -i meta
Multiple layers can be specified by using commas as separators (no
spaces).
(Bitbake rev: 0e9062e65acbb05c1d9b3a9145eb866c3d562309)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If bitbake-diffsigs is run without arguments it will error:
Traceback (most recent call last):
File "./bin/bitbake-diffsigs", line 121, in <module>
if output:
NameError: name 'output' is not defined
Fix this by moving the check for output into the inner else-clause.
(Bitbake rev: 7d3545a66863ad7183a7650b2af57eee091c45ae)
Signed-off-by: Jacob Kroon <jacob.kroon@mikrodidakt.se>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Iterating through and calling setVar on this number of variables has significant
overhead in the profiling data. By not setting this, we save 3,000 calls
to setVar which gives a noticeable improvement to the speed of task execution.
The BBHASH variables have since been replaced by accessing that data through
the siggen code and going forward, that is the preferred way work with it.
(Bitbake rev: 92526eadd09d19938762290e0492076174367583)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The connection may have failed before the event queue has been setup.
Handle this correctly in the exception handler.
(Bitbake rev: db4d80b5c2d32117cdf06333b9627202998b1512)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Ed Bartosh
Elliot Smith
brian avery
Richard Purdie
Cristian Iorga
Paul Eggleton
Alexandru DAMIAN
Jason Wessel
Randy Witt
Alexander Kanevskiy
Chen Qi
Chong Lu
Belen Barros Pena
Ross Burton
Belen Barros Pena
Robert Yang
Robert P. J. Day
Ash Charles
Jacob Kroon