This cleanup fixes a few issues:
1) Removes the superflous code to set toasterconf.json
- this isn't used and referenced meta-yocto
2) Changes exit to return so we don't surprise the user by exiting their shell
- this is necessary because it is being sourced
3) Removes the last references to the old TOASTER_MANAGED variable
- this is historical and no longer used.
4) Adds -t parameter to lsof
- This stops it from dying on odd filesystems and is much
faster since all we are using are processes anyway
5) Handles start and stop as params
- it was easy to confuse the script especially
if we were calling it with parameters.
- if start/stop isn't specified, it will still toggle
(Bitbake rev: 88fddbe80f56828026bf93560037af52b5dab628)
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This sets the scene for removing the default False for expansion from
getVarFlag. This would later allow True to become the expand default.
On the most part this is an automatic translation with:
sed -e 's:\(\.getVarFlag([^,()]*, [^,()]*\)):\1, False):g' -i `grep -ril getVar *`
There should be no functional change from this patch.
(Bitbake rev: 7c3b99c6a716095af3ffce0b15110e91fb49c913)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
I before E, except after C...
(Bitbake rev: 14c9593265f7469cb8a205a46f845ac7491246df)
Signed-off-by: Phil Blundell <pb@pbcl.net>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This means that when you view the process tree, the processes
have meaningful names, aiding debugging:
$ pstree -p 30021
bash(30021)───KnottyUI(115579)───Cooker(115590)─┬─PRServ(115592)───{PRServ Handler}(115593)
├─Worker(115630)───bash:sleep(115631)───run.do_sleep.11(115633)───sleep(115634)
└─{ProcessEQueue}(115591)
$ pstree -p 30021
bash(30021)───KnottyUI(117319)───Cooker(117330)─┬─Cooker(117335)
├─PRServ(117332)───{PRServ Handler}(117333)
├─Parser-1:2(117336)
└─{ProcessEQueue}(117331)
Applies to parse threads, PR Server, cooker, the workers and execution
threads, working within the 16 character limit as best we can.
Needed to tweak the bitbake-worker magic values to tell the
workers apart.
(Bitbake rev: 539726a3b2202249a3f148d99e08909cb61902a5)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Toaster script run bitbake with --postread conf/toaster.conf.
It turned out that variables set this way don't influence the
build. Changing --postread to --read makes bitbake to read
variables earlier and should fix this issue.
[YOCTO #8781]
(Bitbake rev: 76c9871740ef42ac35fdfdcb89a68478cca370cd)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Sometimes Toaster connection to the bitbake server fails with
this error in the log:
ERROR: Could not connect to server 0.0.0.0:46572
: Could not register UI event handler
It happens more often on slow machines.
Increasing timeout after restarting bitbake should fix this issue.
[YOCTO #8776]
(Bitbake rev: b64a0adc1efd72aa4d46df3e043dd14543abae3d)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Removed nobrowser and brbe script parameters as both
are confusing and nobrowser is not used anywhere.
brbe parameter usage can only be justified if toaster
doesn't work properly and user has to manually connect
toaster to running bitbake server. Even in this scenario
it's very unlikely to achieve as toaster script is not
designed for this kind of usage.
(Bitbake rev: 0fd04ede3fda6894d97a5ef830b79dbbc9c6cf51)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Set default values of script parameters just before
they are parsed to increase readability.
(Bitbake rev: 627f0d6adcfe281ef0487bf15a35151f1ceff194)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Moved addr:port, bitbake.lock and toastermain.pid checks
to the place where the rest of Toaster starting happens.
(Bitbake rev: 0e7812b5512e609ea815db30be1d7caca896ab60)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Removed check_pidbyfile and notify_chldexit functions from
toaster script as they're not called in the script.
(Bitbake rev: ead2823e5457c9c4d0321c2191fb99982b282d26)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This function is useless as it's called just once
and makes code less readable.
(Bitbake rev: 96e85159905e44533882ad8290b1c9dd252ff3ea)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Updated GPL information, years of development and
usage information. Removed outdated information about
2 ways of starting Toaster.
(Bitbake rev: b4789c9e3093e1ec8e70f67305c88709443578e6)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Used MANAGE variable to avoid repeating path to
manage.py in many places in toaster script.
(Bitbake rev: e8e19168ff43b72a2481771e7a40d9d7c155cfd2)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Removed unused variable RUNNING from the toaster script.
(Bitbake rev: 8b3f0b2c6efa2962d9c16e453f4d2c1af6419c6c)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Made toaster script more readable by splitting long lines and
removing and adding whitespace.
(Bitbake rev: 6d61d7903eb4fb12464bfea54d3a225f99ad394f)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Used new management command checksocket to check if
Toaster can listen on address:port.
[YOCTO #8775]
(Bitbake rev: 8a306ffe5a3642fe833c875362a183096a39f641)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Checked django version considering information from
toaster-requirements.txt, e.g. if requirements file contains
line "Django>1.8,<1.9" toaster should be able to check that
requirement correctly.
(Bitbake rev: 49976eca4a6e37e7653814c569badcd3e0fb719a)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
One manage.py migrate should be enough to apply migrations for
all applications. Extra calls just slow toaster script down.
Removed calls of migrate for orm and bldcontrol apps.
(Bitbake rev: 9299d7fa88e51294c4fd3f1354874d8253d25aff)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Instead of hard-coding the required Django version in the start
script, look it up from the toaster-requirements.txt file.
[YOCTO #8364]
(Bitbake rev: 2149ff805424692f9fde29618f2ae0eb99f710d2)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
syncdb is deprecated in favour of migrate in Django 1.7:
https://docs.djangoproject.com/en/1.8/releases/1.7/#schema-migrations
Update to the "migrate" command in Toaster's start script.
[YOCTO #8364]
(Bitbake rev: 7ff1630574180e1895b90ecef1ea0caf51304446)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Upgrade Django to long-term support version.
Django now provides its own migration framework, so remove
requirement for South.
[YOCTO #8364]
(Bitbake rev: 648b62654c52116451c6a68a46d7264db3a34d09)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Toaster script sets environment variables when it starts.
It makes sense to unset them when toaster stops as they can
cause other programs to behave incorrectly.
(Bitbake rev: 7a2e9745ff3ef6d6360f74b222248d4bd35e6a08)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This file is not needed anymore as toaster UI doesn't
write variables to it. It sets them directly on bitbake
server.
(Bitbake rev: 624240460ec32af918dcea8c04a913e3b94e012b)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This implementation allows to have functionality of restarting
bitbake in toaster script. It can be used by toaster script and
build controllers.
[YOCTO #8279]
(Bitbake rev: 24cd26b39014419d883b7cf45e4943c301d585cc)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Separated functionality of starting bitbake server and observer
processes.
This functionality will be used by build controllers to restart
bitbake processes.
(Bitbake rev: d8e9ce7ba3f5b89c51e769bd4a46c2026a110d40)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Separated functionality of stopping bitbake server and observer
processes.
This functionality will be used by build controllers to restart
bitbake processes.
(Bitbake rev: dd40d70b707a3ff59edfa69923d4d9b671928a66)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Running runbuilds machinery in background allows Toaster to
start builds initiated by toaster UI.
Added runbuilds pid file to the list in webserverKillAll to
kill runbuilds the same way as runserver process.
(Bitbake rev: 91bf323e873c31d883b01ea993c005f45382e83f)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Assigned TOASTER_DIR to the parent of the build directory.
This should fix local controller crash as it assumes that
TOASTER_DIR is a root of local poky and tries to clone it.
(Bitbake rev: 124eb2bb15de18e4d2d6483c3549e1d887234736)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Moved check for toasterconf.json after check of build environment.
We'll need some variables from build environment to find toasterconf.json
better way.
(Bitbake rev: a4f6809237dffb1abff78c6a8c82bcc4852aedfa)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Removed variable RUNNING as it's not used anymore
in the toaster script.
(Bitbake rev: 187c31a8789c37723b9288afd35f1785f1993895)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This changes the way we pick a toasterconf.json file so that it
matches the way oe-setup-builddir does it. We now base it on TEMPLATECONF
found in the .templateconf file.
(Bitbake rev: 8476db82ea80edf2b70bc2082bcecee09037556d)
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Run bitbake in toaster script the same way as it's run
by localbuildcontroller.
(Bitbake rev: c5d5da095d5eace03f8a50910f5a60ab35acbb1a)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Set environment variable DATABASE_URL in bitbake/bit/toaster
Variable value is obtained by running 'manage.py get-dburl'
(Bitbake rev: ac4800cc8b765b270f27ea9a5938f3f2a2e0ced4)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Removed support of starting toaster as a script.
Sourcing a toaster script becomes the only way to start it.
It's consistent with the way oe build system is started by sourcing
oe-init-build-env. It also returns user back to shell, so user can
continue running builds without having to open new terminal window.
[YOCTO #8279]
(Bitbake rev: 7e8632c0527245c1474e487dd9a972948e794471)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When toaster script is sourced 'return' should be used as
using 'exit' will cause script to exit the shell and close terminal
window. If script is called 'exit' should be used. Otherwise shell
will throw and error:
return: can only `return' from a function or sourced script
Used 'return' or 'exit' depending on the toaster mode using
TOASTER_MANAGED variable.
(Bitbake rev: 643c42069a2788e9f40fa1bc9845f32aed3d096c)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Set TOASTER_MANAGED to 1 if toaster is run as a script.
(Bitbake rev: 9f68f317e33d617558fce3427a94e14ef65c7d88)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Used TOASTER variable instead of SRCFILE as they're
essentially the same.
(Bitbake rev: 4b8cb1098ad73b4e9dc65b5c6de7d11d2e2468c7)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Used $TOASTER variable(path to the toaster script) instead of
current directory name to get path to the toasterconf.json
This way script should work in both modes and it doesn't depend
on the current directory.
(Bitbake rev: 5368d9af4e2c121938fc2421a536fb7f20a987fd)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The toaster startup script conditionally migrates the database
tables depending on whether you are in managed mode or not. This
means that if you are in analysis mode, some of the bldcontrol*
database tables used by managed mode are not available.
As a consequence, some of the code in toaster which refers to
those tables can break in analysis mode, as there's no clean
isolation of the two modes.
To prevent this from happening, always run the migrations for
managed mode and create the bldcontrol* tables, even if in
analysis mode.
Also clean up the function which starts up toaster so the
logic is easier to follow.
[YOCTO #8277]
(Bitbake rev: b1fc592131286ebbede2693be8c86636f0039011)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
removed layer dir questions and base off TOASTER_DIR
removed build dir questions and base off TOASTER_DIR
base configuration file off of TOASTER_CONF
fixed some pylint issues
[YOCTO #8217]
(Bitbake rev: d5811968b5f22093365f381fb7e75ab46e5269c2)
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
added TOASTER_CONF env var for toaasterconf.json
added TOASTER_DIR env var for working dir
added bugfix so WEB_PORT env variable is honored
(Bitbake rev: c4b351e40060a359d59ab51b5ed2f9713ca1a9a5)
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
added comment explaining how to create one later
(Bitbake rev: 782b7c74d5f10ed255538ba0c975ff342a34f5bf)
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In the forked child, we may use multiprocessing. There is only one event
pipe to the worker controlling process and if we're unlucky, multiple
processes can write to it at once corrupting the data by intermixing it.
We don't see this often but when we do, its quite puzzling. I suspect it
only happens in tasks which use multiprocessng (do_rootfs, do_package)
and is much more likely to happen when we have long messages, usually many
times PAGE_SIZE since PAGE_SIZE writes are atomic. This makes it much more
likely within do_roofs, when for example a subprocess lists the contents
of a rootfs.
To fix this, we give each child a Lock() object and use this to serialise
writes to the controlling worker.
(Bitbake rev: 3cb55bdf06148943960e438291f9a562857340a3)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a carriage return to build environment warning message to
avoid it being mixed up with shell prompt.
(Bitbake rev: d3675cca419946cc19b3f280446fe1f656f11902)
Signed-off-by: Cristian Iorga <cristian.iorga@intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If the pipe is closed, we want to ensure that we kill any child processes
by triggering the sigterm handler before we exit. This code does that,
hopefully avoiding the remaining process left behind issues on the autobuilder.
(Bitbake rev: 60f6c2818f38c4d9c2d9aaa42acf3071636f4a3b)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If we SIGKILL cooker (the parent process), ensure the worker notices
and shuts down gracefully. To do this:
* trigger the sigterm handler if the parent exits
* ensure broken pipe writes don't trigger backtraces which interfer with
other exit work
* notice if our command pipe is broken due to EOF and sigterm if so
(Bitbake rev: c43d6a8d711db8d3bd9a1976b9f8e3efdb4cb4ae)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
brian avery
Richard Purdie
Phil Blundell
Ed Bartosh
Elliot Smith
Cristian Iorga