This is a combined set of fixes for the project, layers and targets
pages in the project section of toaster.
The fixes correct behaviour and look in various parts of the page,
including submitting XHR commands and updating the DOM with the correct
info.
(Bitbake rev: 96d7738f964784871c928c376cb9fbc9a275cf00)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We add a static copy of the Angular's animate plugin,
version 1.2.23
Licensed under MIT.
(Bitbake rev: 12162fda32af4bc805d6ff5b0348d93befe2f623)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We update the release model to pinpoint to specific branch,
instead of holding a branch name that needs to be matched.
This is needed because we move away from mixing branches on different
layer sources.
Various minor changes to models file.
(Bitbake rev: d58aacc69b1832f99758b941803123329e06082d)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We add a logger, enabled through the settings.py, that
logs all SQL queries, with timestamp and duration, to console.
This is helpful in profiling the SQL queries.
The facility is disabled by default.
(Bitbake rev: 54fd0a76fa3154adfab5688ecc96963f42cded11)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Improve the performance of data logging in toasterui.
We modify the data queries used to:
* cache searching in memory
* insert in bulk (i.e. multiple values per insert, where possible)
On development test rig (networked mysql), on no-op build,
time for data recording is reduced from 4:10 to 1:30 (minutes).
We also improve the logging, so it is easier to detect
toasterui errors.
(Bitbake rev: d42784432f927f58730caf80546c66772e0fec89)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We add the "webport=" command line option as to allow starting
the web server on a custom port.
The bitbake server port is now auto-allocated. This is needed
to be able to run multiple toaster environments on a single machine.
We tackle bug 6023 (toaster refusing to start when lock file is present)
by using more specific checks, and automatically recover from
bitbake server down / webserver up error mode.
Command line parameters are now read on both interactive and managed
modes.
The localhost and ssh controllers are updated to work with the modified
toaster launcher script.
[YOCTO #6023]
(Bitbake rev: cd3eb5b051743463cfe51dba97cae4da75420048)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
To support yocto on systems with python3 as default version, scripts
should use /usr/bin/env python in the shebang, as this allows the use of
a fake env to mimic python2 as default version.
This patch simply replaces occurrences of #!/usr/bin/python with
#!/usr/bin/env python and was done with this oneliner:
git grep -lE '^#!/usr/bin/python' | xargs \
sed -i 's|/usr/bin/python|/usr/bin/env python|'
(Bitbake rev: 90d9bf1bda282fb5d053a42438a8cfd02475b9cd)
Signed-off-by: Martin Hundebøll <martin@hundeboll.net>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Being able to query whether updated versions of a url are available
is useful, not least for the package reporting system. Since such code
is closely linked to the url type and the url itself, the fetcher
makes a locical place to contain this code.
For wget based urls this means taking upstream directory listings
and searching those for later versions, returning those that are
found.
The patch also adds unittests for this function so that if
improvements are made, the original test urls can be used
to evaulate the those changes.
This is based on code from Irina Patru <irina.patru@intel.com>.
(Bitbake rev: a8272e22b7819e0e8afd8e291d276f5f28fc0007)
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Being able to generate a version string representing the most recent git commit
given git is useful, not least for the package reporting system.
This adds in a latest_versionstring method to the git fetcher
which allows users to query the latest version using ls-remote
and filtering the responses.
The patch also adds unittests for this function so that if
improvements are made, the original test urls can be used
to evaulate the those changes.
This is based on code from Irina Patru <irina.patru@intel.com>.
(Bitbake rev: f71c8c0354e87fed80bc845db6728e6e18ce9c4d)
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If we split variables only at whitespaces, a slipped in tab will render
a value unremovable.
(Bitbake rev: 9f171ea755644ecd9d2b3d7ed13bf8ec09ec917a)
Signed-off-by: Stefan Müller-Klieser <s.mueller-klieser@phytec.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Similar to the last shared work task signature bug, we've found another
one. Looking at the improved output of diffsigs in this case:
runtaskdeps changed from [
'autoconf_2.69.bb.do_populate_sysroot:virtual:native',
'gnu-config_20120814.bb.do_populate_sysroot:virtual:native',
'libgcc-initial_4.9.bb.do_patch:virtual:nativesdk'
] to [
'autoconf_2.69.bb.do_populate_sysroot:virtual:native',
'gcc-crosssdk-initial_4.9.bb.do_patch',
'gnu-config_20120814.bb.do_populate_sysroot:virtual:native'
]
so we can get a different task hash since libgcc sorts before gnu-config
and gcc sorts after it. We could do with a way of fixing this, the best
I can come up with is to include a single parent directory. Since
recipes are never at the top of any metadata trees I've seen, this
should suffice for now.
I'm planning to burn the concept of shared work within bitbake
and do something at the metadata level in the 1.8 timeframe as its just
too fragile as things stand and hard to fix well.
(Bitbake rev: d753644c67d163f338f2bdc3d600203e8b1a5734)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This is useful code to double check the computed checksum value if nothing
else. Might as well have it in tree.
(Bitbake rev: 54ecf96c6f031927ee2410f6efde4e16f19bbf66)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Order of runtaskdeps is important. If the hashes differ we should print output.
This is complicated by shared work where the filenames themselves can differ,
but the checksum should not.
This fixes a case where two different checksums could show no output with
bitbake-diffsigs.
(Bitbake rev: 40c95cb9def282dc88234cd72ff462d7a01e47c1)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We need to pass the URL for the build action as javascript
parameter, since the project ID is different for each build,
instead of using the project id of the most-recently-used project.
(Bitbake rev: 35c1de4b05c64ee1c436be2dfbd97dcf9ed9ed71)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We add improvements in data reading, following
issues discovered in testing.
- elapsed_time is now read from buildstats
- we add safeguards to not fail logging if the build was triggered
with a toaster_brbe configuration, but it's running in
1.6 mode
- added log markups for build finish to let other programs
known when the work is done.
[YOCTO #6833]
[YOCTO #6685]
[YOCTO #6887]
(Bitbake rev: 0b225035cefee3d3713a93f9a432e5e4d4e174f1)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Ideally, we want the PR service to have minimal influence from
queued disk IO. sqlite tends to be paranoid about data loss and
locks/fsync calls. There is a "WAL mode" which changes the journalling
mechanism and would appear much better suited to our use case.
This patch therefore switches the database to use WAL mode. With this
change, write overhead appears significantly reduced.
(Bitbake rev: 0cdd48261daeb17efc528b5de0ac81c8836e8565)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This has always bothered me:
NOTE: Preparing runqueue
NOTE: Executing RunQueue Tasks
This patch changes the messages to be consistent.
(Bitbake rev: 72ac9f9227fbfb4dc8b933b357d21aa0e4060959)
Signed-off-by: Gary Thomas <gary@mlbassoc.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Added Beautifulsoup module because fetch/wget latest_versionstring
method depends on it.
This provides support to fetch/wget.py module for search new package
versions in upstream sites.
(Bitbake rev: 4626c9b77e5eded97507b6f9ca0d891f9a54bb8a)
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This makes it possble to fetch Gerrit review references which are
normally stored under refs/changes.
Please disregard previous patch with the same topic.
(Bitbake rev: 268e9c0c6830e8e621c418f20c2ca12dc840e48b)
Signed-off-by: Fredrik Svensson <fredrik.svensson@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The sync/commit calls are happening in the submission thread which can
race against the handler. The handler may start new transactions which
then causes the submission thread to error with "cannot start a
transaction within a transaction".
The fix is to move the calls to the correct thread.
(Bitbake rev: 92e128a0e331e563cfe48827e95939041c16c88e)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fix the startup message by making clear which
server runs on which port.
(Bitbake rev: 26d601ab2a677a422bab425696a2cf6357ff3576)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We update the layer vcs web fields to record extra URLs
for directory and file viewing. Updating the layers
view to show this data.
(Bitbake rev: 14762e182c8af22fd0fa96f0ba0db1ecd2495fbc)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We add, for the localhost environments, the capability to
download build artifacts. This is a pontentially dangerous API,
because it gives unrestricted read access to the build environment
file system - do not expose the functionality directly to the
web layer, but use filtering/translation code, such as
exemplified in the build_artifact view.
The capability for remote build environments is dependent
on bug 6835, as to use the collect storage as intermediary
storage for serving files.
[YOCTO #6834]
(Bitbake rev: 5fce7f6e83c6143244faa9618b7ed20c1106e08f)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We avoid a race between the setting the TOASTER_BRBE variable
and reading the variable in toaster ui by supplying the variable
at server startup time through the toaster.conf post-read file.
Additional small changes are included, including marking the
build request with the environment id of where the build took place.
(Bitbake rev: 7c333350418c4140e6c988c5272940f8057d327d)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The default django URL validator marks git URLs as not valid,
so we change the default validators to a URLValidator-derived
that adds git and ssh protocol schemas and inhibits the
frontend URL validator.
We update the branch names based on the white-listed
branch names in the Releases name.
Adds proper class string to ease usage in admin section.
Other small improvements.
(Bitbake rev: 7d51f895bd56bd18f2258059b377851c2ce82114)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Conflicts:
bitbake/lib/toaster/orm/admin.py
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The default django URL validator marks git URLs as not valid,
so we change the default validators to a URLValidator-derived
that adds git and ssh protocol schemas and inhibits the
frontend URL validator.
(Bitbake rev: 7223619505c0939358287e5baf84da0a91cb8d34)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This patch brings in a new landing page to be shown
when there are no builds and no projects available.
The builds page now displays only only the builds part,
without the landing page bits.
There is a new projects page that displays the All Projects
table as specified in the design.
[YOCTO #6682]
(Bitbake rev: c6c7c05521daa9bf16c122d7d472330ca4c05e88)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We make sure we don't throw an exception when reading
non-existent variables from the database, and we return
empty data that can be displayed without ill effects.
This fixes the Configuration page on empty builds.
(Bitbake rev: 5b13eabdec1e8a052d343fd67654d7fe8974224f)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We refactor the objects.filter()[0] pattern to the proper
Django QuerySet usage of objects.get() to make sure we
get one and only one object for a query.
(Bitbake rev: 6abec6631fbadaecc474f2a4e7981fde3c58657c)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This patch definitivates the all layers page, providing interactivity
for adding and removing layers inside the project from this page.
[YOCTO #6590]
(Bitbake rev: e690080f83ad53c5e4a31e8c0fba2cc744eea1eb)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This patch rewrites the Project page and the additional
infrastructure in order to fix a bug that makes triggering
builds through UI impossible, and to introduce data feeds
for suggestions for the user.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We fix problems in the layer source update code related to
non-null field restrictions.
We add missing layer dependency update.
(Bitbake rev: e1169127405ce10a044ac9cdf4a892838a99d01f)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We add the Angular JS library, release 1.2.23
https://angularjs.org/
We add the Bootstrap directives for AngularUI, release 0.11.0
http://angular-ui.github.io/bootstrap/
The license ued for all libraries is MIT
(Bitbake rev: 8b4d33a6182486795cca53b2832967de583281e1)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If we are in managed mode, and we inspect a build that comes
from a project, we modify the breadcrumb as to display
the project from where the build originated.
[YOCTO #6776]
(Bitbake rev: fe568c9350a691c863f9f5bf8703d639c12a9961)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We change the saving of the build id to the build request as
soon is the build is created, as to allow for a consistent
display of build data while build is in progress.
(Bitbake rev: 9504ca6a69ba6da21f88b3cc77fa5910d886b6c3)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The build dashboard is supposed to show failed
tasks (if any) in the "Tasks" section.
(Bitbake rev: fc2a7be3982ea5441beef1b7e9e187caf5df2beb)
Signed-off-by: Belen Barros Pena <belen.barros.pena@linux.intel.com>
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Modify the help text we show for the "Task covered by"
information to indicate that a task can be covered by
more than one task.
(Bitbake rev: cad45dfebaabe1875f786ba7ceeb3ee377e926b3)
Signed-off-by: Belen Barros Pena <belen.barros.pena@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
For builds that generate no rootfs files, an extra <div>
was being added between the build status and the build
summary section. This commit removes the extra <div> and the
white space it created.
(Bitbake rev: e84db294f733464b4952a3c92120e5b19e8c2750)
Signed-off-by: Belen Barros Pena <belen.barros.pena@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The sorting in this column is tied to the sorting by
"Size". If we make both columns sortable, when you sort
by one of them it also sorts the other, and the table looks
as if you have sorted by 2 columns at the same time.
This was fixed in YOCTO #6061, but turned up again.
(Bitbake rev: acf372eb5a083451ab1507dd24e3a84380420c9a)
Signed-off-by: Belen Barros Pena <belen.barros.pena@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Change the background colour of table rows with the .error
class from white to transparent, to make sure the blue
highlight animation works when you click on the order
number in the task details page.
(Bitbake rev: 25fd347b6a972164af9a0b5b56f803a296290246)
Signed-off-by: Belen Barros Pena <belen.barros.pena@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Make sure the reverse dependencies tab for included packages
shows a notification when the number of reverse dependencies
is 0, instead of an empty table.
(Bitbake rev: 8dc3e582eb5f1d6e9a79de59a53014495a48e862)
Signed-off-by: Belen Barros Pena <belen.barros.pena@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The "Prebuilt task could be based on" table we provide in the task
details page for prebuilt tasks is missing the task descriptions.
This patch adds them.
[YOCTO #6328]
(Bitbake rev: 64894d640ee70a301f6f399db118ed17c0ccde08)
Signed-off-by: Belen Barros Pena <belen.barros.pena@linux.intel.com>
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In the templates basetable_bottom.html and basetable_top.html
remove the '2' and '5' options from the 'show rows' menu, and
add the '150' option as per design specs.
Alex edited conflicts.
(Bitbake rev: 03f7ba3d8a101e05c106e5fb65eb379a6e4cb476)
Signed-off-by: Belen Barros <belen.barros.pena@intel.com>
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
For those packages renamed at packaging time, the help text
in the 'Reverse runtime dependencies' tab was missing the
original package name.
[YOCTO #6327]
(Bitbake rev: ca8fc7bd4ad940c8eeefa57619a6c8423f116fc7)
Signed-off-by: Belen Barros Pena <belen.barros.pena@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We update the checksettings command to allow for upstream
git url detection at runtime by looking up a named remote
of a local repository.
Also various small fixes in check settings.
(Bitbake rev: c127ecb3b5b27b705b8714af063d910b944c8bbc)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We need to consistently use LogMessage.INFO/WARNING/ERROR to make sure toaster knows
how to categories these rather than passing in the "raw" loglevel value
which in best case comes from python logging but worst case any value.
[YOCTO 6885]
(Bitbake rev: 3aa13bc1d8218c97c76581a895fa5f03ff807fbe)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In the commit 'prserv: Ensure data is committed', the PR server moved to
only committing transactions to the database when the PR server is
stopped. This improves performance, but it means that if the machine
running the PR server loses power unexpectedly or if the PR server
process gets SIGKILL, the uncommitted package revision data is lost.
To fix this issue, sync the database periodically, once per 30 seconds
by default, if it has been marked as dirty. To be safe, continue to
sync the database at exit regardless of its status.
(Bitbake rev: 424df64f2e9679043f0ce2b4f7dfc59c3d404304)
Signed-off-by: Ben Shelton <ben.shelton@ni.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
See http://www.nongnu.org/lzip/lzip.html for details on the compression
format.
(Bitbake rev: 9027b1273b5405c7269b013604ab417771b5eafe)
Signed-off-by: Koen Kooi <koen@dominion.thruhere.net>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Forcing strict to be a string, to avoid problems when performing comparisons
[YOCTO #6762]
(Bitbake rev: b8ed2098bdea2afd93ab4e3e1b834f3a31cb60de)
Signed-off-by: Alejandro Hernandez <alejandro.hernandez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We fix the rendering of the size field in dirinfo-related pages
by directly calling filtered_filesizeformat and not rendering
it through the template engine.
Additionally, we enable error dumping into logs if an
Exception happens.
[YOCTO #6669]
(Bitbake rev: afa2431c21b8271b05dc4cca4265f98d9f338007)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If localhost resolves to a remote address (due to a misconfigured network),
starting the pr server will fail without useful information.
To reproduce, add '<bogus ip> localhost' to /etc/hosts and run
'bitbake -p'. The error message will be:
ERROR: Timeout while attempting to communicate with bitbake server
ERROR: Could not connect to server False:
Running 'bitbake-prserv --host=localhost --port=0 --start' will fail with:
error: [Errno 99] Cannot assign requested address
Since these errors does not show the IP address of the attempted socket
binding, this results in a lot of wasted time looking at firewall rules, etc.
This patch results in the following error message if the socket binding fails:
PR Server unable to bind to <bogus ip>:0
(Bitbake rev: fae5914030bcf4c061c22fc61034c40c87b7121a)
Signed-off-by: Konrad Scherer <Konrad.Scherer@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
hob was adding the redundant characters "\1" in SSTATE_MIRRORS variable. If
needed it is expected the user will add this instead so remove the code
that was doing this.
[YOCTO #6600]
(Bitbake rev: 73bf120062fc00c7e26dc4e77a7d140658d89daf)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This fixes urls of the form file://some/path/file;subdir=b. It also
adds in a couple of tests so we now tests these corner cases.
(Bitbake rev: 46306912a96444790efa9418d934dfdd36773ba1)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Check if the 'subdir' parameter exists and assign it to 'destdir' so that
files are copied in ${WORKDIR}/destdir. This fixes urls that are of the form
file://a;subdir=b.
(Bitbake rev: 836a986b365eb9798563ec08d90b346596de7791)
Signed-off-by: Roxana Ciobanu <roxana.ciobanu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Writing a log that the filesystem isn't being monitored for inode usage just
confuses users who are not aware about the nature of inodes in their filesystem,
so don't say anything, just silently disable the monitor. In general this only
happens on filesystems which don't have a limit on inodes.
(Bitbake rev: ca93bc84ee5fb94a50c11c47e4d212d7da649e24)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
I could apply the label "release_1.0" to a super project that contains
many sub projects. If my recipes have SRC_URI's that use that label but
grab different sub-folders, than there's a bug where the cached localfile
(tar.gz) will not be unique and reused at the wrong times.
SRC_URI = "p4://perforce::1234@//depot/SuperProject/MiniProjectAAA/...;label=release_1.0 \
p4://perforce::1234@//depot/SuperProject/MiniProjectBBB/...;label=release_1.0"
(Bitbake rev: 3b5b1703b77490116dda895b29737cea51a3d6a0)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When using options like -k, -f, -v and so on with the memory resident
server, they'd currently only be set on the initial values passed to
the original command. This ensures they now match those specified
on the commandline for the options where this makes sense.
To make this work, a command to update the options on the server side
is required so this is added.
[YOCTO #5292]
(Bitbake rev: 1c75cc4d0c8b606c1fe76e6bf60bf6a32298b105)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Without this you see:
File "bitbake/bin/bitbake-worker", line 201, in fork_off_task
os._exit(child())
TypeError: an integer is required
(Bitbake rev: cd477b5e77ab0373248b8a8fa30e1c7b8ea984fd)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If you have:
FOO = "${bindir}/X Y"
FOO_remove = "${bindir}/X"
the expected result is "Y". Currently this doesn't work since the removed
expressions are not expanded first. This patch adjusts things so the
expressions are expanded before being processed for removal.
Also add a test to ensure this case continues to work.
[YOCTO #6624]
(Bitbake rev: 72a1ca4a104ccab73d6abcbd44db9c2636a58572)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We need to transfer some of the siggen data from the core/cooker into
the worker instances. There was a partial API created for this but
its ugly and its not possible to extend it from the siggen class.
This patch completes the interface/abstraction for the data and
means the class can extend/customise it in any siggen class.
(Bitbake rev: cf2d642052979d236185c5b8ca2c5478c06e62ae)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Unfortunately it seems like the external use of the cooker
enableDataTracking() function broke at some point since the code that
reads it now runs within BBCooker's constructor. Since this now has to
be done early, add a parameter to Tinfoil's constructor to allow
enabling variable history tracking.
Fixes [YOCTO #6676].
(Bitbake rev: a9439b136f55f3f0e80ff053cd3b159da69ba362)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Code is related to the basetable templates files.
It had mixed tabs and spaces and was miss aligned in various places,
making it hard to read.
(Bitbake rev: cdaea8951df6b707afd1fefbf22295088256dd6f)
Signed-off-by: Marius Avram <marius.avram@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Until now cookies were used to save which columns were shown and which
were hidden in toaster tables. The tables from the templates also have
functionalities like sorting the entries on a certain column and
limiting the number of entries displayed on a page. The later however
were not saved using cookies. This patch brings this new feature.
The cookies are not saved only in the front-end. They are saved both
in the frontend in case the user uses the inputs/buttons to change
a parameter and also in the backend in case the user specifies manually
using GET variables the value of the parameters.
When no GET parameters are given the views will redirect the url to one
containg the parameters saved as cookies. When no cookies exist, default
values will be used.
[YOCTO #6126]
(Bitbake rev: 880b58c845e3a501fa90d24e1bd89c87ca84b709)
Signed-off-by: Marius Avram <marius.avram@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In order to avoid problems when using JSON serializer for
saving sessions, we move from storing the objects themselves
in the session to storing the object id and reloading the
object when retrieved.
This allows, for example, to use cookie-storage sessions if the
infrastructure owner so desires.
(Bitbake rev: 39d0f0c2e87d4b161f1eeaa2657e61b5a6bc9ee2)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We enable support for starting builds on remote machines
through SSH. The support is limited to poky-based distributions.
We refactor localhost build support and we update
bldcontrol application tests to uniformely test the APIs
of localhost and SSH build controllers.
[YOCTO #6240]
(Bitbake rev: c2ad9c9bb83f61c171434324df8c4d5ee655a556)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We change the toaster starting script to make the
bitbake server listen on all interfaces on the local machine.
This is needed to be able to receive a controlling client
running on a remote machine.
(Bitbake rev: 137179eafca8d1a5a69b6302f8cc8961be3b45c4)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We rename the projecttags in bldviewer.templatetags to
simple_projecttags in order to avoid conflict with the
similarly named tagset in toastergui.
The conflict leads to an intermittent bug where proper
tags are not read correctly since Django uses only the
module name as global tag library identificator.
(Bitbake rev: a37f2c194d7e59611177cb8755524b7ad702fe91)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
With the improved exception handling added in an earlier commit bitbake
now stops when recieving a SIGWINCH. This happens frequently when
disconnecting and reconnecting tmux sessions and bitbake didn't survive.
Restore old behaviour of ignoring interrupted system calls but keep
proper exception handling for other errors.
(Bitbake rev: 418358a595c75f45b8d15160ec42bbe569562d91)
Signed-off-by: Dan McGregor <dan.mcgregor@usask.ca>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We add new pages for the layer importing, layer details,
showing project builds and project configuration.
The pages are in read-only mode, but they're needed as
to be able to verify the quality of data in the system.
Write capabilities will be added in a subsequent patch.
[YOCTO #6595]
[YOCTO #6590]
[YOCTO #6591]
[YOCTO #6588]
[YOCTO #6589]
(Bitbake rev: eed9ae5c2a2bd7567e12ae9a4f02a5a966a1e1a3)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We add new pages for the all-machines and all-targets
project-related views.
We update the existing template structure to create
a base project view, similar to a base build view, that includes
a breadcrumb.
Updating existing all layers view to use the new structure.
We update methods in the models to provide corrent
information display.
[YOCTO #6592]
[YOCTO #6593]
(Bitbake rev: 973f582a19441c1ec67061160e4c50ce03ed7b68)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We move the code to calculate build progress as percent
and the ETA of the build to the model, so that they can be
reused across different pages.
(Bitbake rev: c2ced09e7ea4a1762d2788bb12a761734d20fd8e)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We update pages and queries to match the new project
models.
(Bitbake rev: 8f47ec259106da714260c7388cee75a6c1f4622d)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We update the build controller application to make proper
use of the bitbake specification in project settings.
Added heuristic to detect when the meta* layers and bitbake
are checked out from Yocto Project poky, and use a single
git checkout.
Building without a proper oe-init-build-env is not yet supported.
(Bitbake rev: 9eafe14956013f5af39b68fc93e1b03e7ea1f5c2)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We enhance the checksettings command to try to automatically
detect settings for running on localhost.
The checksettings will look for a nearby poky layer source,
for a nearby build directory, and will try to import settings
from "toasterconf.json" files found in the local layer.
On new configuration, it will also perform updates from the
layer source.
(Bitbake rev: 2aab77dfccb538e2b09829841ea6c464d40cafb1)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We add a page where the user can browse
and import layers from all the layers known
by Toaster.
[YOCTO #6590]
(Bitbake rev: 59f4a9750a6c4f5360a91e3a4d1c03ceb42da086)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We create the navigation structure for the project section.
This includes adding URLs for configuration, builds, all layers,
layer details and all targets pages.
Changes to existing pages to exemplify navigation links.
(Bitbake rev: 6f0cb9d106129eb496a4c009d95b0727378e97c1)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We enable the admin interface in Toaster.
We add admin models for Build Environments (where the
sysadmin can configure where the builds take place)
and for Layer Sources (marking the upstream provider
for layer information).
The admin interface and associated data are enabled
only for the MANAGED version.
(Bitbake rev: 6618613c9210fb44d36d90f5f2404b435f10dfc8)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We add a ToasterSettings table that will keep
installation-wide settings.
We update the models for the layer-related data storage to
make them compatible with the layerindex application API.
We add a LayerSource class that can update local data from
a LayerIndex-like compatible API.
Adding a command line option to perform information update
from all upstream layer sources.
Fair warning - there is no backward migration from 0013.
(Bitbake rev: 89e13579e1b44b738f10fadec8454aa0e6f073af)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We fix application discovery, tuples don't have .append().
Other minor fixes.
(Bitbake rev: a6f18aac3e6bb448d89a3425a2f756c6514ee595)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We refactor log saving to go through only one code path.
All logs that happened outside the build (i.e. before build
starting) now will be logged to either toaster_ui.log if the
build command ran in interactive mode, or to the build request
errors if the command ran in managed mode.
This enables proper display of error logs in project page.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently the output in the python task/function run files is rather
incomplete and effectively useless. This enhances the code to take
advantage of the bitbake's dependency tracking and extend the output to
include dependencies. This makes the files more usable for debugging
purposes. Since this only happens at python function execution time, the
overhead is minimal in the grand scheme of things.
(Bitbake rev: 02667e048c3e632f857c87177c0022eaf5481802)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently we get no profiling oversight into either the main bitbake worker
process, or the overall parsing before task execution. This adds in extra
profiling hooks so we can truly capture all parts of bitbake's execution
into the profile data.
To do this we modify the 'magic' value passed to bitbake-worker to trigger
the profiling, before the configuration data is sent over to the worker.
(Bitbake rev: 446e490bf485b712e5cee733dab5805254cdcad0)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This stream redirection of stdout is horrible. pstats takes a stream argument
so lets use that instead.
(Bitbake rev: 93d155f4766e27e7b004d13569aa03961fe89e3b)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We should only add user/password options if they're specified as in the
fetch case. Patch from Volker Vogelhuber <v.vogelhuber@digitalendoscopy.de>
(Bitbake rev: 303e6256947f4df4f283b75b7ccfdffa72864d67)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently, if you have hard dependencies between setscene tasks (like avahi on
base-passwd through useradd.bbclass), other dependencies may not be installed
even if these exist in sstate. For example, avahi -> expat -> pigz-native
(and avahi -> base-passwd) yet if you cleansstate base-passwd:
bitbake gzip-native:do_clean avahi:do_clean expat:do_clean pigz-native:do_clean base-passwd:do_cleansstate
bitbake avahi | tee
you will currently see pigz-native being rebuilt even though it was in
sstate. The fix for this is to continue to iterate dependency chains
around hard blocked dependencies as per this patch.
After this patch is applied, you will see pigz-native installed from sstate.
(Bitbake rev: f787957a224e8c2682a19e5c4a4d9c86bdce52ba)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(Bitbake rev: 587b144ee409d444494d8d7f2d1c53ede8f7c953)
Signed-off-by: Robert P. J. Day <rpjday@crashcourse.ca>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This fixes an issue when trying to use depexp without specifying a
recipe. Before change:
$ bitbake -g -udepexp
Traceback (most recent call last):
File ".../bitbake/bin/bitbake", line 382, in <module>
ret = main()
File ".../bitbake/bin/bitbake", line 370, in main
return ui_module.main(server_connection.connection, server_connection.events, configParams)
File ".../bitbake/lib/bb/ui/depexp.py", line 201, in main
logger.error(cmdline['msg'])
NameError: global name 'logger' is not defined
After change:
$ bitbake -g -udepexp
Please specify a package name for dependency graph generation.
(Bitbake rev: 984ad90b2f1e29634dc79803a4a0404ab0534039)
Signed-off-by: Olof Johansson <olof.johansson@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
since python-2.5 string.endswith() takes a tuple
(Bitbake rev: 86a67a1fd4244da9343dbf14deed1ad0d3003f32)
Signed-off-by: Bernhard Reutner-Fischer <rep.dot.nop@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
With a shared work task like gcc, the task can be run from a variety of
different recipes which may have different virtual extensions in place.
Depending on whether gcc-runtime or nativesdk-gcc-runtime's do_preconfigure
task is called for example will change the sorting of the task hashes due
to the way clean_basename currently works.
The correct thing to do here is sort on the base filename first, then any
extension when ordering the hashes. This means we do account for things
like recipes with both a native and non-native dependency but we also fix
the shared work case where we don't care whether it was a virtual version
or not.
(Bitbake rev: 2e80b5d10a5037ed6f0bc227a1f9b42529c87086)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently, if an exception occurs in an event handler, the server shuts
down but the UI simply hangs. This happens in two places, firstly waiting
for events and secondly, sending events to a server which no longer exists.
The latter does time out, the former does not. These patches improve
both code sections to check if the main server process is alive and if not,
trigger things to shut down gracefully. This avoids the timeout in the
command sending case too.
This resolves various cases where the UI would simply hang indefintely.
(Bitbake rev: ac418e1112ff5f9c3157569316902f7a27fba4b4)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Our usage of multitprocessing is problematic. In particular, there is a bug
in python 2.7 multiprocessing where signals are not handled until command
completion instead of immediately.
This adds a workaround into our wrapper function to deal with the issue.
(Bitbake rev: a16185e602b39b71475aa7e9ee80ad2b1f28d0f7)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If we trigger a shutdown whilst parsing for whatever reason, in some
cases we were not closing down the parser threads. This change
ensures we do so. The function names are not entirely intuitive
but the behaviour is more correct (and commented). The previous bug
with the stdout failure would trigger this one, if there was a cold
cache and parsing was required (but not otherwise).
(Bitbake rev: 25bfa2478f1c3a8eb695e1e5760e06db5be8f2fc)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When processes terminate, we really want all of the child processes to
terminate too. This was not happening for worker processes which spawned their
own multiprocessing pools, leading to build hangs. This change ensures any
sigterm gets passed to the whole process group. In local tests, this resolved
some hanging process workloads I could generate. It does rely on signals
being delivered in a timely fashion and there is a multiprocessing bug we have
to work around there.
(Bitbake rev: 96f8ea07ace1379380fab2d78eb592fa40c867d4)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently, IOErrors are just passed over due to the broken Exception
clause. A command like "bitbake X | <invalid command>" would break stdout
triggering a traceback. With these changes we print the exceptions, shut down
the server gracefully and exit which is a much nicer behaviour and is less
confusion to the user.
(Bitbake rev: 9544108f7b413038d871ce6ca88232de2f2434d9)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently if an exception occurs, we just run the idle handler again and
again, usually looping indefintely. Chances are the exception that occurred
will keep occuring and this is not a good place to be.
This was breaking the autobuilders with gigabytes of logs.
At least improve things so the cooker shuts down gracefully when this happens.
Some trace of the original problem may still be present on the console too!
(Bitbake rev: 1f28d8d3311262427938180435b68f0a35c2b330)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(Bitbake rev: 000fa81013205dd9bc907ff7a61f06f57637212d)
Signed-off-by: Robert P. J. Day <rpjday@crashcourse.ca>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(Bitbake rev: fd293c5fcc89bc7a541d09e88a6aec5e0b2a1db2)
Signed-off-by: Robert P. J. Day <rpjday@crashcourse.ca>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(Bitbake rev: 4e9aef14d747c37444a4fc683f9641906906afe9)
Signed-off-by: Robert P. J. Day <rpjday@crashcourse.ca>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We don't need the "PR = r0" in the bb, so remove it to avoid misleading
the user.
(Bitbake rev: 17fc0174f177b444815487ba67a5d623e47ee8b1)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Its useful to have an event emitted when all of the sceneQueue tasks
have completed since the metadata can hook this for processing.
Therefore add such an event.
(Bitbake rev: 38d4f65bf1cbcdd5a2d60dff0e1d2859c34ed62e)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The re module was imported at the top, no need to do it again.
(Bitbake rev: c29c2370635075949c75937c237602600f4f10dc)
Signed-off-by: Namhyung Kim <namhyung@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
so that it doesn't check it twice for non-empty events
(Bitbake rev: e304ace6d8f7d0808ee401b7c01146b2798a81b4)
Signed-off-by: Namhyung Kim <namhyung@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
I inadvertantly used a &YOCTO_* variable in the book to get
a link to the landing YP page. Of course this does not work.
I replaced it with the actual URL. Unfortunately, a previous
commit slipped through that would have prevented the manual
from "making" correctly.
(Bitbake rev: 63f5a4b06a575e1fb18805b45a01549bffdc4a03)
Signed-off-by: Scott Rifenbark <scott.m.rifenbark@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Reported-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(Bitbake rev: dc669c4beda04ee88a50c9b48facfbd8cf17a331)
Signed-off-by: Scott Rifenbark <scott.m.rifenbark@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
It turns out the codeparser cache is the bottleneck I've been observing when running
bitbake commands, particularly as it grows. There are some things we can do about
this:
* We were processing the cache with "intern()" at save time. Its actually much
more memory efficient to do this at creation time.
* Use hashable objects such as frozenset rather than set so that we can
compare objects
* De-duplicate the cache objects, link duplicates to the same object saving
memory and disk usage and improving speed
* Using custom setstate/getstate to avoid the overhead of object attribute names
in the cache file
To make this work, a global cache was needed for the list of set objects as
this was the only way I could find to get the data in at setstate object creation
time :(.
Parsing shows a modest improvement with these changes, cache load time is
significantly better, cache save time is reduced since there is now no need
to reprocess the data and cache is much smaller.
We can drop the compress_keys() code and internSet code from the shared cache
core since its no longer used and replaced by codeparser specific pieces.
(Bitbake rev: 4aaf56bfbad4aa626be8a2f7a5f70834c3311dd3)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If there is a corrupt/invalid cache file, we'd keep trying to reopen
it. This is pointless, simplify the code paths and delete the dead
file.
(Bitbake rev: c22441f7025be012ad2e62a51ccb993c3a0e16c9)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If we're writing out merged data to disk, its safe to assume that either
we loaded the data or couldn't. Loading it again is relatively pointless
and time consuming.
(Bitbake rev: 14c31e18e4a4a52ee54056aeefb09ff8c295b475)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The SIGNATURE_HANDLER can be defined by the user, so we can't use choices.
(Bitbake rev: 0036b36e333a68c6d30a6bf5c21bfd4fd3e8441b)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This patch fixes a name collision on the event.data
in the MetadataEvent class. event.data is used in the
event system as a copy of "d" in certain situations,
and this collision triggered a bug leading to data loss.
[YOCTO #6332]
(Bitbake rev: 3f191b7cfe95aea4d4e96babf001d62d45dd3aaa)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Since Toaster adds a number of specific requirements managed
by pip, we document these requirements in order to be able
to run bitbake and toaster under virtualenv. The target here
is to break the dependency on distro packages for specific
python libraries.
In order to start bitbake/Toaster in a distro-independent manner,
we use virtualenv and pip. We add venv to the .gitignore list
to make room for the virtualenv deployment in this directory.
Use this command sequence to setup the virtualenv:
$ virtualenv venv
$ . venv/bin/activate
$ cat toaster-requirements.txt | xargs pip install
bitbake and Toaster will then run normally under virtualenv.
We also fix bugs related to toaster starting in managed mode
through this script:
* toaster will not use bldcontrol application in interactive mode
* a proper message will be displayed if toaster detects a managed
start request in a build environment
(Bitbake rev: b5789c7ea7e6d1a4b67ec60fc0e4233ec543ba3a)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This patch enables the localhost build controller to
properly set the layers before the build runs.
It creates the checkout directories under BuildEnvironment
sourcedir directory, and runs the build in the buildir
directory.
Build launch errors are tracked in the newly added BRError table.
These are different from build errors, in the sense that the
build can't start due to these errors.
(Bitbake rev: 1868d5635b517e0fe1b874674ea7a78910b26e2e)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This is the first commit on the project main edit page.
At this point we have:
* the default settings for a newly created project
* the ability to add targets
* the ability to trigger a build command, and have
the build executed
Project layers now have an optional field, allowing for
removal. Default meta, meta-yocto and meta-yocto-bsp
layers cannot be optional.
We add XHR calls for interactivity in the main page.
(Bitbake rev: 4e438854120cbd10319df1b571ec93e334002325)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We move the initial_data fixture to a code migration,
in order to make sure that we don't overwrite modified data.
The data is inserted only if no data is present, instead
of always overwriting the entry.
The check settings returns 0 if the check was ok.
(Bitbake rev: 525c316ac95c06c52fbcfc836ca5599ee9dbfe8b)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
A layer may live in a subdirectory of a git repository,
so we add a field to track this setting in the Project layers.
We add the Project schedule_build function, which creates
a build request from the current project configuration.
We also fix an import problem with Projects in views.
(Bitbake rev: 1b5835e5d48cbfb7d38e38437c45d161052dfb37)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We add explicit absolute paths for a directory where
the layer sources will be checked out (sourcedir) and
where the build activities will take place.
Adding minimal checking when starting the application in
order to make sure that BuildEnvironment (BE) settings are
usable. This check is ran by the toaster script at startup.
Modify the localhost bbcontroller to use the BE settings
instead of trying to self-configure on checked out sources.
(Bitbake rev: d17500d3f73fdeeef5f11fb3773a65e927be3f02)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(Bitbake rev: 1cd369883469747a8158826bb8d67dcca2a8577f)
Signed-off-by: Robert P. J. Day <rpjday@crashcourse.ca>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Note also, that I have added a new option that has since appeared
in the output but was not in the manual's example. That option
added is '--token=XMLRPCTOKEN'.
Reported-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(Bitbake rev: 8fbffd25d42f1f2d9a85ae1d9cc4852e835730d4)
Signed-off-by: Scott Rifenbark <scott.m.rifenbark@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The clearcase fetcher is used to retrieve files from a ClearCase
(http://en.wikipedia.org/wiki/Rational_ClearCase) repository.
Usage in the recipe:
SRC_URI = "ccrc://cc.example.org/ccrc;vob=/example_vob;module=/example_module"
SRCREV = "EXAMPLE_CLEARCASE_TAG"
PV = "${@d.getVar("SRCREV").replace("/", "+")}"
The fetcher uses the rcleartool or cleartool remote client, depending on which one is available.
Supported SRC_URI options are:
- vob
(required) The name of the clearcase VOB (with prepending "/")
- module
The module in the selected VOB (with prepending "/")
The module and vob parameters are combined to create
the following load rule in the view config spec:
load <vob><module>
- proto
http or https
Related variables:
CCASE_CUSTOM_CONFIG_SPEC
Write a config spec to this variable in your recipe to use it instead
of the default config spec generated by this fetcher.
Please note that the SRCREV loses its functionality if you specify
this variable. SRCREV is still used to label the archive after a fetch,
but it doesn't define what's fetched.
User credentials:
When using cleartool:
The login of cleartool is handled by the system. No special steps needed.
When using rcleartool:
In order to use rcleartool with authenticated users an `rcleartool login` is
necessary before using the fetcher.
(Bitbake rev: 76cff0aaad4cf10b325bceadd8ea90b3d75edcc2)
Signed-off-by: Dennis Meier <meier.dennis@siemens.com>
Reviewed-by: Roger Meier <r.meier@siemens.com>
Reviewed-by: Christian Liechti <christian.liechti@siemens.com>
Reviewed-by: Henrique Mendonca <henrique.mendonca@siemens.com>
Reviewed-by: Pascal Bach <pascal.bach@siemens.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If you hit Ctrl+C at the right point, the system processes the request
but merrily continues building. It turns out finish_runqueue() is called
but this doesn't stop the later generation and execution of the
runqueue.
This patch adjusts some of the conditionals to ensure the build really
does stop.
(Bitbake rev: 39b08c604ba713100e174c136f81f18eca6ef33d)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The bitbake -S had been updated which always reuires an argument, so
update the help info:
- Add the two args in the help message: none and printdiff
- Use type="choice" so that we can get more friendly error messages,
for example:
bitbake: error: option -S: invalid choice: 'printdiffX' (choose from 'none', 'printdiff')
- dump-signatures=DUMP_SIGNATURES -> dump-signatures=SIGNATURE_HANDLER
(Bitbake rev: 021b778fa4685bdde39e1a0f6c7c57632dcf792a)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
I added some whitespace to bring the example to use 5 spaces
rather than for so that it follows the manual's convention.
(Bitbake rev: 4c6e7d88d715db18e61bb263dfd2672ebb6c6b09)
Signed-off-by: Scott Rifenbark <scott.m.rifenbark@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
I added text to separate this example code from the general
BitBake User Manual's spirit of standing alone and away from
YP or OE stuff. Also, added another space to move the examples
code so that 5 spaces are used for literal indentation.
(Bitbake rev: 17fcac8a7b21afa5c7d201e94cbf603ab07ba70c)
Signed-off-by: Scott Rifenbark <scott.m.rifenbark@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
I noticed that the examples from some patches were not indenting
by 5 spaces. I have fixed these to be consistent.
(Bitbake rev: c487341bd20b70aae6b430ee1c74a7f20345f81a)
Signed-off-by: Scott Rifenbark <scott.m.rifenbark@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
"fakeroot" is now specified as a function attribute, not as a task
flag.
(Bitbake rev: 749b39a8e5e8060222c748a349efc31f541da94e)
Signed-off-by: Scott Rifenbark <scott.m.rifenbark@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
I think having a couple real-life examples here would make a huge
difference.
Comment from Scott: I modified the example description to indicate
that it is from an OpenEmbedded metadata-based recipe (i.e from
YP). We have been trying very hard to keep the BitBake User
Manual as separate as possible from the YP docs.
(Bitbake rev: 834780772a08ecce7ed863e96d0674b47d0e589d)
Signed-off-by: Scott Rifenbark <scott.m.rifenbark@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
As long as i've understood everything correctly, this is just
throwing in some physical examples of a couple concepts.
(Bitbake rev: a18cc69c2fef6484a6acd78ea008d1da71198e68)
Signed-off-by: Scott Rifenbark <scott.m.rifenbark@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Added wording to clearly state immediate or delayed application
for various operators.
Reported-by: Robert P. J. Day <rpjday@crashcourse.ca>
(Bitbake rev: 877c4c9a466e26953abd6fe416c2cba092607dac)
Signed-off-by: Scott Rifenbark <scott.m.rifenbark@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Some applied as is, some not applied, some modified.
(Bitbake rev: af98b2ff4f44f892b63a6598e7fdfc144972ed3d)
Signed-off-by: Scott Rifenbark <scott.m.rifenbark@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There is no "populate_staging" any longer, should be
"populate_sysroot".
(Bitbake rev: dbe5902e64259e2143f09ab1aa9afa03aa02f4c4)
Signed-off-by: Scott Rifenbark <scott.m.rifenbark@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fixes [YOCTO #6475]
A new parameter has been added to make sure the user is not
passed as the username for the subversion commands with the
SVN fetcher.
(Bitbake rev: 366c70dbb0e9916f50ce1a4ff3ad4bd2757832a3)
Signed-off-by: Scott Rifenbark <scott.m.rifenbark@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This patches fixes the build - project identification when
running under managed mode. The build is assigned to the
project from which it was triggered, and to the
build request, as to simplify relationships queries
in the database.
(Bitbake rev: af1d3373706d365f9138caec110fcb20a5966b7b)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We add the logic to create a new project. This page
also serves as user registration and silent login
for users.
Once the project is added, the main project page is displayed.
(Bitbake rev: 8855daebe55917c4f5855413d02ae1f3f7f76571)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This patch is just a whitespace (end-of-line) fix.
(Bitbake rev: 5f33d1a240faa49bd8259a48eb15ef57a6b4ce55)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Toaster uses the Django authentication system to assign
user accounts to the projects that are being created.
In the current implementation, the user accounts are
created/authenticated automatically, on the fly, based
on the fields specified in the create new project page.
(Bitbake rev: a9062d9692525e24e59b5b2bb4dfdef90b41bf2a)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We add the new project and project page skeletons.
In the process, we add an identifier in the settings.py
to detect whenever Toaster is running in managed mode,
and a context processor to make this value available
to the template processor.
(Bitbake rev: 927a27c68e24cfe13f62ca5f0e60878b04fa4e24)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Issue: TA53702
It was observed that a sequence in a script such as
bitbake --server-only ..
bitbake --observe-only ..
bitbake <some target>
could generate events from the server to the observer before
the observer was ready to read the events, and the early events
of builds were consistently dropped. Adding a "ready" note in the
log allows the script to scan for that message before proceeding
to building a target.
(Bitbake rev: 9949948f92bbe2717a05a380d3df63a332d39c9a)
Signed-off-by: Dave Lerner <dave.lerner@windriver.com>
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We automatically enable local applications in INSTALLED_APPS
based on detecting a models.py or views.py file.
This allows Toaster extensibility by adding applications,
without having to edit configuration files.
(Bitbake rev: 05b246095d161143ca2ebc126d9e70d198ab8bb8)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Alexandru DAMIAN
Martin Hundebøll
Aníbal Limón
Stefan Müller-Klieser
Richard Purdie
Gary Thomas
Fredrik Svensson
Belen Barros Pena
Belen Barros
Michael Wood
Ben Shelton
Oscar Utbult
Koen Kooi
Alejandro Hernandez
Konrad Scherer
Roxana
Ross Burton
Chad Nelson
Paul Eggleton
Marius Avram
Dan McGregor
Robert P. J. Day
Olof Johansson
Bernhard Reutner-Fischer
Andre McCurdy
Robert Yang
Namhyung Kim
Scott Rifenbark
Dennis Meier
Dave Lerner