- Add new build tables to be tested
- Add required data into the fixture and clean up a few empty fields
- Fix the SoftwareRecipesTable specific test so as not to rely on two
particular defined recipes
(Bitbake rev: 7cf23671659666b27b5629fecd5f947f9bdb94e0)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
As in the old build tables it's useful to jump and highlight a
particular row in the table using the #hash in the URL.
(Bitbake rev: 927bfc26ffcb0eb6c0a5b3c0905b7efc2f19b9a2)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Port the Task based tables to ToasterTable. This is the Task, Time, CPU
usage and Disk I/O tables.
(Bitbake rev: bebcef7a4bf08b10e472475435ddc7a524364adb)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This is the table that displays all the packages built in the build.
Build -> Packages. Adds a template snippet for the git revision popover.
(Bitbake rev: df62f38ff4e634544c9b1e97c5f6ca45e84a4f1e)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Make sure we can create empty states for toaster tables that include
actions for users to get out of the empty state. Allows a template to be
used as an empty state.
(Bitbake rev: 9f21a1781345a2d2757616a9ab6f34ec48e9e93d)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Belen Barros Pena <belen.barros.pena@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Convert all the HTML templates, JS and CSS in the project parts of
toaster to use bootstrap 3.
(Bitbake rev: 69527a731eada699d3f604ff8f3ae9410981ba9b)
Signed-off-by: Belen Barros Pena <belen.barros.pena@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The Twitter typeahead.js library expects the developer to use
a source which does a local search for matching suggestions, then
falls back to a remote search if that doesn't return enough
results.
However, in Toaster, we don't do any caching of the suggestions
for a typeahead, so our source only works in asynchronous mode.
Consequently, we see fewer than the expected number of suggestions
if the typeahead has already shown suggestions matching a query.
For example, searching for "meta-n" in the layers typeahead will
show the results for this query; but when the query changes to
"meta-ne", a new set of results is fetched, which mostly overlaps
with the results for "meta-n". The typeahead assumes that the
overlapping items are locally cached and have been delivered
synchronously, and just appends the new results which don't
overlap with the previous query. But because we don't provide any
results synchronously, we just end up with the single
non-overlapping result in the drop-down.
This can be fixed by hacking typeahead.js so that instead of
appending asynchronous results, we always overwrite and redraw
the whole typeahead menu.
This is a temporary fix, and should be properly fixed (when we
have time), perhaps by using typeahead.js's associated Bloodhound
library.
Added a note about the hack to the license file as an explanation
of why the unminified JS file is included in Toaster.
(Bitbake rev: afbaf326e1123c92952fa71e0e820a4ff83488ca)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Migrate from Bootstrap 2's built-in typeahead to Twitter's
typeahead library.
This is to facilitate moving to Bootstrap 3, which doesn't have
a typeahead.
(Bitbake rev: 0748177b40188a6fb735fe1ba1c17294afa4a3d0)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a helper class for running build tests. Subclass this and call the
build method to get setup for running tests on the resulting data from a
build.
(Bitbake rev: 2cafb16033b2f38156e6e0a74c02120f87a4fa92)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* In the custom images table, show the recipe file and download icon
only when the recipe file exists and can be downloaded.
* Also in the custom images table, make sure the download icon tooltip
shows on hover.
* In the custom image details page, show the correct icon (download)
next to the recipe file in the right hand column.
* In the custom image details page, show the recipe file and download
icon in the right hand column only when the recipe file exists and can
be downloaded
* Also in the custom image details page, simplify the help text we show
when the 'download' button is disabled: it gave so much information about
what's actually happening under the hood that it was a bit hard to
follow.
(Bitbake rev: 5d4172bc005efe209de595f6a151ae9ed062d051)
Signed-off-by: Belen Barros Pena <belen.barros.pena@linux.intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When the tests are complete add some marker elements to the test page
DOM which can then looked at for a selenium browser test.
(Bitbake rev: 81798ab046b0717614e28e0a339172f3803e451b)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fixed:
DeprecationWarning: The 'warn' method is deprecated, use 'warning' instead
(Bitbake rev: a3f464d202dafef4538e66c008cdecb7b8709ed1)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If a line like:
foo=${@' '.join([d.getVar('D', True) + x for x in (' '.join([d.getVar('FILES_bash-' + p, True) or '' for p in ['lib', 'dev', 'staticdev', 'doc', 'locale', 'ptest']])).split()])}
is added to a function like do_install, it fails with Exception name 'd'
is not defined. This is due to a change of behaviour in python 3 compared
to python 2. Generator expressions, dict comprehensions and set comprehensions
are executed in a new scope but list comprehensions in python 2.x are not. In
python 3 they all use a new scope.
To allow these kinds of expressions to work, the easiest approach is
to add 'd' to the global context. To do this, an extra optional parameter
is added to better_eval and we use that to add 'd'.
(Bitbake rev: 8f74881037bb01013d3d439dc0c269909a198c1c)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The functionality of BBPOSTCONF and BBPRECONF was added in
commit 21b314d4d1 but there
was a typo in the variable name that raises an exception
in bitbake.
[YOCTO #9235]
(Bitbake rev: 6d1379c8818400e5cdc442e6142f08a110fd5b95)
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
"hash() is randomised by default each time you start a new instance of
recent
versions (Python3.3+) to prevent dictionary insertion DOS attacks"
which means we need to use hashlib.md5 to get consistent values for
the codeparser cache under python 3. Prior to this, the codeparser
cache was effectively useless under python3 as shown by performance
regressions.
(Bitbake rev: 12d43cf45ba48e3587392f15315d92a1a53482ef)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
No functionality change, just avoids function call overhead in a
function which loops heavily.
(Bitbake rev: 633c0c19f87a92497a7e9771811cdc953e1b7047)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
deb packages in modern Debian versions have the data tarball compressed
with xz rather than gzip, and thus explicitly extracting data.tar.gz
fails. Unfortunately ar doesn't support wildcards matching items to
extract, so we have to find out what the name of the file is first and
then extract it, relying on tar to figure out how to unpack it based on
the filename rather than doing it with pipes and making that
determination ourselves.
(Bitbake rev: 17ff08d225a8fa7faffd683c028369574954fba9)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Previously the code used to match a reference to its SHA-1 in
_latest_revision() used the Python "in" operator, which made it match
if the reference matched the beginning of an existing tag or
branch. This test, however, must be exact. I.e., either the reference
matches a tag or branch exactly, or it does not match at all.
(Bitbake rev: e5986c78a6108fd7578989c20efcbf0b81c97e03)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Makes it easier for user to identify problems, e.g. typos, in BBLAYERS.
[YOCTO #9507]
(Bitbake rev: 32c9689e4b492dc5821749e284e397d717af2a6c)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We don't need to parse all recipes just to show the list of layers,
since that comes straight from the configuration, so save a bit of time
by not doing so. (A minor regression that came in with the the
bitbake-layers refactoring).
(Bitbake rev: a609ad7d560260a8d50dfa197cd960f496c5da73)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Some functions have been moved from urllib to urllib.parse
in python 3. Modifying the code to import unquote, urlencode and
unquote_plus from urllib.parse if import from urllib fails should
make it working on both python 2 and python 3.
(Bitbake rev: b91aa29fa20befd9841678a727bb91100363518f)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
mru is a list. It has a count method, but it differs from the
one for the queryset. Using mru.count causes 'Latest project builds'
section to silently disappear when toaster runs on python 3.
(Bitbake rev: c3ff90c9f028bff5733ebac7b5e72a4688addd3e)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Division operator works differently in Python 3. It results in
float unlike in Python 2, where it results in int.
Explicitly used "floor division" operator instead of 'division'
operator. This should make the code to result in integer under
both pythons.
(Bitbake rev: 0c38441ed99b49dae8ef9613e320f0760853d6aa)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This commit causes buildinfohelper to crash when run on python 3
as python 3 doesn't have unicode builtin function.
This reverts commit 7a309d964a.
(Bitbake rev: 750ca5c8d5a25fc519b75c56352dec7823c7e240)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Added use_builtin_types parameter to XMLRPCProxyServer.__init__
to fix this error:
ERROR: Could not connect to server 0.0.0.0:37132
: __init__() got an unexpected keyword argument 'use_builtin_types'
(Bitbake rev: ceb6e5bd33a25c45c2afe1559b9394c466db8a92)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Using empty BITBAKE_UI environment variable causes bitbake server
to fail with the error:
FATAL: Unable to import extension module "" from bb.ui
Valid extension modules: knotty or toasterui\n'
Used BITBAKE_UI="knotty" when starting and stoping bitbake
server to solve above issue.
(Bitbake rev: b9a9ddfb6f53259be214032fb93812149d0cce19)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Moved import xmlrpc module to the place where it's used
to avoid toaster crashes when importing bitbake code.
NOTE: This patch is made to be able to partly test toaster
with bitbake from master. It can be removed as soon as bb.server.xmlrpc
is ported to python 3.
(Bitbake rev: 0c5d691746fb02807568a8a470969a0b68e21915)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Used 'rb' mode to open files to avoid unicode error when code
runs on python 3:
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x80
in position 44: invalid start byte
[YOCTO #9584]
(Bitbake rev: 1414866b84fe1fd674ea79500cd62eda3aa30b33)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
translate has different set of parameters in python 3 and
can't be used the way it's used in toaster api module.
Replacing it with re.sub() should make the code work in
both python 2 and python 3.
[YOCTO #9584]
(Bitbake rev: 7cc3a41139abc5b8d1eef4376fb19aba3910ce0d)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
map returns map object in python 3. Replaced calls of
map to equivalent list comprehensions in the code which
requires lists.
[YOCTO #9584]
(Bitbake rev: d41eb6d6c061cb35ec1ecd899f856876f9077167)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Used items() and range() APIs instead of iteritems() and
xrange() as latter don't exist in python 3
[YOCTO #9584]
(Bitbake rev: 372dd3abcb201bd9ac2c3189c5505d3578ce0dd0)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Decoded binary data to utf-8 to fix the following
error when runnign on python 3:
TypeError: Type str doesn't support the buffer API
[YOCTO #9584]
(Bitbake rev: 752ea00919ef054622a51ce097923309934eff2b)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Replaced local imports with absolute imports.
Used .module for local imports.
This should make the code to work on python 2 and python 3.
[YOCTO #9584]
(Bitbake rev: 3f1e68c783308dcb51242d0fdeef758e581ccc8c)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Some APIs have been moved to other modules in python 3:
getstatusoutput: moved from commands to subproces
urlopen: moved from urllib2 to urllib.request
urlparse: moved from urlparse to urllib.parse
Made the imports work for both python versions by
catching ImportError and importing APIs from different
modules.
[YOCTO #9584]
(Bitbake rev: 1abaa1c6a950b327e6468192dd910549643768bb)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Replaced compicated calls of reduce with more clear code.
As reduce was removed from python 3 this change is mandatory
for the code to work on both pythons.
Here is an example change for illustration purposes:
original code:
querydict = dict(zip(or_keys, or_values))
query = reduce(operator.or_, map(lambda x: __get_q_for_val(x, querydict[x]), [k for k in querydict])))
replaced with:
query = None
for key, val in zip(or_keys, or_values):
x = __get_q_for_val(k, val)
query = query | x if query else x
[YOCTO #9584]
(Bitbake rev: 249d0bc6094ec9f369a02b78d8ed634a239e5ee4)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Usage of force_bytes in BuildRequest.__str__ method caused
python 3 to throw "__str__ returned non-string (type bytes)"
error.
Replaced force_bytes with force_text to make the code working
on both python 2 and python 3.
[YOCTO #9584]
(Bitbake rev: 9dd9c1393a84d1110c647e84253af8e0bb6acc45)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The upgrade to python3 is the final nail in the coffin for image-writer
and the goggle UI. Neither seem used or recieve patches and are based
on old versions of GTK+ so drop them, and the remaining crumbs support
pieces.
(Bitbake rev: ee7df1ca00c76f755057c157c093294efb9078d8)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Disable the problematic gtk usage for use with pygtkcompat. The following
commit removes these tools/UIs entirely but we may as well leave this
piece in the history in case anyone does want a starting point for reusing
them.
(Bitbake rev: c53c7418d392452450352ca2175667dbdbd92401)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
get_context was added to mutliprocessing as part of 3.4.0
(Bitbake rev: 710351610e3ca4a1b61abc67564f84907e9b2f1c)
Signed-off-by: Jeremy Puhlman <jpuhlman@mvista.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This seemingly convoluted syntax doesn't work in python3. Instead
use the chained exception handling syntax which appears to make more
sense here.
(Bitbake rev: b19a4c5166303b1fa680582adf63e6a5564bfb4c)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Various misc changes to convert bitbake to python3 which don't warrant
separation into separate commits.
(Bitbake rev: d0f904d407f57998419bd9c305ce53e5eaa36b24)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Upgrade to 4.4.1 which has been run through 2to3 as per the maintainers
recommendation for v3 use.
(Bitbake rev: 2f4b98af93c971a8c466ffaf3c09cca0edb6e3ad)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Under python the type conversions can mean there are float values
used for triggering the parse progress events which then fails.
Add an explict int() conversion to ensure the parse events are
generated under python3.
(Bitbake rev: 138329c58e92744c56aae3ab70ceeef09613250c)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Task.SSTATE_NA and Task.SSTATE_MISS are both valid conditions for the
condition that a Task.OUTCOME_COVERED and Task.OUTCOME_PREBUILT.
(Bitbake rev: acac9bf660ac5939beb64d7b8d55541dea1c7b4e)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This is a port of the oe self test to the django test framework from
oe-core meta/lib/oeqa/selftest/_toaster.py
(Bitbake rev: 94418b1b2f9466d35461acdb982fd6b130b2331c)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a helper class for running build tests. Subclass this and call the
build method to get setup for running tests on the resulting data from a
build.
(Bitbake rev: d1882371cae6b9a8296ea739e6e46ca6a6dc9da8)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When running certain tests we want a particular database specified.
When bitbake toaster ui is being tested pass it these test settings so
that it uses the same database as the unit tests running.
(Bitbake rev: c52e34cac4362ba0a3cb3ea1fcb639e1d802aa85)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Move the execution sequence for a build out of the polling loop and into
it's own fuction. This means that we can call the function on it's own
if we just want to trigger one build rather than infinite polling.
This is something needed for the build tests.
(Bitbake rev: e8e059602c31394a4be5293e2ca0b8434fb4a8cf)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We don't need to pass the DATABASE_URL around and read it back if we
setup the django framework in the correct way.
We make the default sqlite database path a full path so that the
database isn't being assumed to be in CWD.
Also add some more useful comments on the database settings.
This is preparation work to migrate the build tests and be able to
trigger builds on differently configured databases.
(Bitbake rev: 973c740404ca6a09feea250d3433075995067fe0)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add basic tests to validate the value user types
in the text box for IMAGEFS_TYPES. Added a test
case to show the checkbox get automatically selected
when user types value available in the check list.
Added a test case to verify if the check box is enabled
then the text box should also get updated accordingly.
[YOCTO #7828]
(Bitbake rev: 62c74eb38f44d98b40427edf56e40785b076a938)
Signed-off-by: Sujith H <sujith.h@gmail.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a label to clarify the purpose of the first input field, replace <p>
with <label> for the second set of controls, and give some top margin to
the validation message.
(Bitbake rev: 99b1b52fe4f2b593d7835d2db64eb028ed42600a)
Signed-off-by: Belen Barros Pena <belen.barros.pena@linux.intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This functionality helps users to add custom
image fs types available other than the checkboxes
in the UI. User can add imagefs types in the text
box and use them in the build.
[YOCTO #7828]
(Bitbake rev: c14cbb026598be064f08f46bb513456c7a3089f5)
Signed-off-by: Sujith H <sujith.h@gmail.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Use open() instead of file() and close files when finished with them.
(Bitbake rev: 033c5a16ff19781ed793c2d97d285884017a2a4e)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Previously we only tracked the flags (minus excluded) of variables we depend
on, but not the flags we use explicitly.
(Bitbake rev: bdeb3dcd7c92e62a7c079e7b27048c4114f24a3a)
Signed-off-by: Christopher Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This variable is a regex-escaped version of LAYERDIR, for safer use in
BBFILE_PATTERN, so as to avoid issues with regex special characters in the
layer path.
[YOCTO #8402]
(Bitbake rev: 72900522778b6ff08b135bf8bb97dff3f1a20bd9)
Signed-off-by: Christopher Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
It's useful to see tracebacks for ExpansionErrors, but only if we skip the
leading bitbake-internal elements, otherwise we see elements of the expansion
process.
As one example:
Before:
ERROR: ExpansionError during parsing /scratch/yocto-new/external-as-needed/poky/meta/recipes-core/glibc/glibc-locale_2.23.bb: Failure expanding variable PV[:=], expression was ${@get_external_libc_version(d)} which triggered exception AttributeError: 'module' object has no attribute 'external'
After:
ERROR: ExpansionError during parsing /scratch/yocto-new/external-as-needed/poky/meta/recipes-core/glibc/glibc-locale_2.23.bb
Traceback (most recent call last):
File "PV[:=]", line 1, in <module>
File "/scratch/yocto-new/external-as-needed/meta-sourcery/recipes-external/glibc/glibc-external-version.inc", line 3, in get_external_libc_version(d=<bb.data_smart.DataSmart
object at 0x7f05d2566950>):
sopattern = os.path.join(d.getVar('base_libdir', True), 'libc-*.so')
> found_paths = oe.external.find_sysroot_files([sopattern], d)
if found_paths:
ExpansionError: Failure expanding variable PV[:=], expression was ${@get_external_libc_version(d)} which triggered exception AttributeError: 'module' object has no attribute 'external'
(Bitbake rev: 7ff5b9eed82b7f4fd138fc6d746a0b79efbea98a)
Signed-off-by: Christopher Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This uses bb.utils.load_plugins, based on the plugin handling in recipetool
and devtool in oe-core.
(Bitbake rev: 5e542df9b966a99b5a5b8aa7cf6100174aff54b2)
Signed-off-by: Christopher Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We need to flush the footer removal, else it may not be outputted until
the buffer is flushed as part of StreamHandler and this would lead to
it removing the ERROR output just printed which is extremely confusing.
Also ensure the footer is cleared before printing a summary as in
some cases it wasn't being removed, also leading to user confusion.
(Bitbake rev: 0e030c4d074c41859608dab5f3ad26b05f56b306)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The signature data file comparison functions are meant to be able to
handle data files containing just the base hash data. This had regressed
in some places so add fixes to allow these comparisons to be made. The
runtime components in the data files are optional.
(Bitbake rev: 2a6659fd748e255a02c2f9d047829d6edfe65317)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
knotty captures two levels of keyboard interrupt: a single interrupt
or two interrupts in a row. These then trigger stateShutdown
and stateForceShutdown respectively.
toasterui doesn't have an equivalent way of capturing interrupts and
using them to shut down bitbake. Now that we are no longer using
knotty + XMLRPCServer for our command line builds (since switching to
per-project build directories), we see some odd side effects of this,
such as builds continuing after they have been interrupted on the
command line.
Bring toasterui in line with knotty (copy-paste most of the code
in knotty.py which deals with interrupts) so that a keyboard
interrupt actually shuts down the bitbake server (if not in
observe only mode).
Additionally use the cancel_cli_build() method to set the Build
status to CANCELLED in Toaster's db when we get keyboard interrupts.
This means that builds interrupted on the command line show as
cancelled (same as if they'd been cancelled from the Toaster UI),
as specified in the UI designs.
[YOCTO #8515]
(Bitbake rev: d39d2edca95900da433074ee95a192d7bfe7090d)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This will be used from toasterui to cancel the current command-line
build when a keyboard interrupt is captured.
[YOCTO #8515]
(Bitbake rev: 1486c770327b53bb5e04baa5f3ea26d8154aed63)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
After clean ups remove api assumptions. Our table data often contains
html snippets to display certain things such as dependency pop overs or
simply links to other parts of the UI. Take these into account when
testing the values of the table data.
(Bitbake rev: 7f61a68f2caba538e302ca9c2727bb43a6c77c64)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
- Remove the unused 'computation' field
- Remove the code to try to make the tables behave like an api
- Remove custom JSON encoder in favour of DjangoJSONEncoder
- Simplify get_data and add comments
- Add exception type instead of using generic Exception
- Clean up python style warnings
(Bitbake rev: 16d8198e8f6668c5fa5467ff4bda86c5d66a6cad)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Used except 'except (<exception1>, <exception2>):' syntax as it's
supported by python 2 and pythone 3.
Old syntax 'except <exception1>, <exception2>:' is not supported
by python 3.
[YOCTO #9584]
(Bitbake rev: d19e305ffa44a848b02ede63dc5de8d2640089e6)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Dictionary method has_key is deprecated in python 2 and absent
in python 3.
Used '<key> in <dict>' statement to make the code working on
both python 2 and python 3.
[YOCTO #9584]
(Bitbake rev: 3d7ad7ba0d1a6f688ae885817c049f2a8ced11b5)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently if the server dies, its possible that log messages are never
displayed which is particularly problematic if one of those messages
is the exception and backtrace the server died with.
Rather than having the event queue exit as soon as the server disappears,
we should pop events from the queue until its empty before exiting.
This patch tweaks that code so that even if the server is dead and we're
going to exit, we return any events left in the pipe. This makes
debugging certain failures much easier.
(Bitbake rev: 29f6ade68fb2b506a23a7eb3a00cdcffa291b362)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Now the event is sent unconditionally we can drop this feature
as its no longer needed.
(Bitbake rev: 473deeb0fc6065693e1fcfcbb8b79753103db537)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The data included in the event is useful for implementing a pre-build
check that warns about unexpected components, for example because of
an incorrect configuration or changed dependencies.
Such a check can be done in a .bbclass that gets inherited
globally. But in contrast to a UI, such a class cannot request that
the event shall be emitted, and thus the event has to be emitted
whether there is a consumer or not.
This was done conditionally earlier out of concerns about the
performance impact. But now events are handled more efficiently, so
that concern no longer seems valid: in some simple testing (admittedly
on a fast build workstation), the two lines (generating the data and
emitting the event with it) only took about 0.05 seconds (measured
with timeit). That was for a build with roughly 500 recipes (from
pn-buildlist aka depgraph['pn']), triggered via the command line. That
was even with a consumer of the data active and doing some work, so it
should be even faster when there is no consumer.
(Bitbake rev: 5ddaf5b7ed1001d2dd3f67e7a6d704afa85479d2)
Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When using MySQL, the project builds info delivered by MySQL
differs from that delivered by SQLite: the former returns text
values from the enumeration for Build outcomes, while the latter
returns the integer value. This causes the progress bar JS to
break, as it is expecting outcome strings.
Modify the recent_build() method to include an outcomeText property
for each Build object, then use this in the conditionals in the
progress bar JS.
[YOCTO #9498]
(Bitbake rev: 7ac374adf1cc70173ff6cc492bc078bba1cf500b)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If the cooker fails to start, ensure a correct exception is displayed to the
user. After handling any queued events simply re-raise the original exception
else the output can be unclear.
(Bitbake rev: 9a4db1aa608c17d31bf5ea1cab5a99beb565dd83)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The string formatting wasn't correct and we should exit if we hit
errors here similar to the other exception handlers.
(Bitbake rev: b90a16408a5c45ce5312384f278e19d09f8dda4d)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
python3 can't cope with the previous approach we were using to pass
exceptions through the RPC. Avoid this by creating a formatted exception
on the sender side.
(Bitbake rev: d7db75020ed727677afbad07a90fb3eac0bf2c45)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Without this, the dict can reorder causing sanity test failures.
(Bitbake rev: ca8c91acc9396385834b266d4e8b84d917e5e298)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This avoids a deprecation warning in python 3.
(Bitbake rev: bf1a92d0c002d73e8a34472dced1343dc4a4251a)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Ensure we pass the string parameter correctly.
(Bitbake rev: 7ed82bd1fe7bdd93b0614119c42eb218dc5d83e6)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Under python 3, if we spawn python processes, we need to have a UTF-8
locale, else python's file access methods will use ascii. You can't
change that mode once the interpreter is started so we have to ensure
a locale is set. Ideally we'd use C.UTF-8 since OE already forces the
C locale but not all distros support that and we need to set something.
Was tempted to choose en_GB so colour gets spelt correctly :).
This is in some ways pretty nasty, forcing it into the environment
everywhere however we only have a limited number of ways of making
everything work correctly and this beats having to add utf-8 encoding
to every file access command.
A similar change will be needed to bitbake.conf in OE.
(Bitbake rev: 8902c29638411d312e6fc4a197707e5742652e15)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Some tweaks to use python 3 syntax in a python 2 compatible way.
(Bitbake rev: 322949c77dbaa4db01b5a43d85b39a2af67ba7b2)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If we don't close the console log file handle, python prints a warning
about unclosed file handles upon exit which is annoying.
(Bitbake rev: 624dd92952b2fc736fd86abe5f2390b87b3a7dd3)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
python3 cares more about invalid type comparisons. Add break statements
and better tests to make the code paths clearer and avoid type issues
in python3. No code functionality change.
(Bitbake rev: 2c39ebdd2762d027f007a6a769fdf023cdf3da2b)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add browser tests for creating a project and test validation of
duplicate project names.
(Bitbake rev: 8055fbe840db426d6859ee2248f86abd44244b30)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When already existing project name is typed by user,
the ui pops up message regarding the existance of the
project name. When an existing project is typed the save
button will be disabled. Else user can proceed ahead by
modifying the project name.
[YOCTO #7005]
(Bitbake rev: 05ddf48cda6690adab4c097b16387578523e751b)
Signed-off-by: Sujith H <sujith.h@gmail.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When already existing project name is typed by user,
the ui pops up message regarding the existance of the
project name.
[YOCTO #7005]
(Bitbake rev: 83e5be7e74850f1bb019668de07f3f745063fe38)
Signed-off-by: Sujith H <sujith.h@gmail.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The projectNameValidation API would help users
to validate if a project name exists or not. This
API is added to libtoaster.
[YOCTO #7005]
(Bitbake rev: 3b1843553f23d78f1ddfec9f7865895ee42356a3)
Signed-off-by: Sujith H <sujith.h@gmail.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We shouldn't try and use fakeworker when performing a dry_run. This
makes the core match the other fakeworker execution points.
(Bitbake rev: 49bea821a2edad5e19c3a566d1a80c23718dede9)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When we pass data into explode_dep_versions2(), we need to result to be
able to be processed in a deterministic way so that we end up with
consistent hash values. This means we need an ordered structure rather
than an unordered one.
To do this, return an OrderedDict() rather than a dict().
(Bitbake rev: 0737e003ca549d08a7dfe13452ae982f2e11fecd)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This updates the print "" syntax to print() and fixes some exception
handling syntax such that its compatible with python v2 and v3.
(Bitbake rev: 58304fcce9727fd89564436771356c033ecd22a3)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The preferred form is assertEqual, assertEquals is deprecated and
not present in python v3.
This is v2.7 safe.
(Bitbake rev: b60261bf8ade14aca31238b50c243c01adcabc59)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
python deprecated logger.warn() in favour of logger.warning(). This is only
used in bitbake code so we may as well just translate everything to avoid
warnings under python 3. Its safe for python 2.7.
(Bitbake rev: 676a5f592e8507e81b8f748d58acfea7572f8796)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This avoids a common issue where PACKAGECONFIG is emitted as a function in
bitbake -e when the 'python' flag exists. It isn't a python function unless
both 'func' and 'python' are set. This aligns with the behavior of
emit_func_python.
(Bitbake rev: c5e45063cb3ae17bbe3304ea5e712bd76e686c4a)
Signed-off-by: Christopher Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This lets us avoid treating the module like an object, so no globals are
needed, if one chooses to do so.
(Bitbake rev: 71bfd5beb0d0ed88c7c14bbfd5ca1a1b56122bc1)
Signed-off-by: Christopher Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Rather than hardcoding .py, use python's knowledge of its file extensions.
(Bitbake rev: 09f838dbaefdaedc01a1f4818ed38280b38db744)
Signed-off-by: Christopher Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Imported as of oe-core 184a256.
(Bitbake rev: 99db61bf816d9c735032caa762aae8e6a0803402)
Signed-off-by: Christopher Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
It seems the frozenset constructor in pypy runs len(), so we can't pass the
DataSmart instance directly to it, instead pass the iterator. Fixes pypy
support.
(Bitbake rev: b492836e08745e04bd9ba2fb0b56a680a5fdce79)
Signed-off-by: Christopher Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
BB_ORIGENV value on the datastore can be NoneType thus raising an AttributeError
exception when calling the getVar method. To avoid this, a check is done before
accesing it.
[YOCTO #9567]
(Bitbake rev: f368f5ae64a1681873f3d81f3cb8fb38650367b0)
Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Wrapped long lines to fix "Line too long" pylint warnings.
(Bitbake rev: e329a932e14d002a561245b5026f974897f64598)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fixed pylint warnings:
No space allowed around keyword argument assignment
No space allowed after bracket
No space allowed before bracket
(Bitbake rev: c39770239f7b61217501782b9c5e9d3211355d42)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Environment variables BBSERVER, BBTOKEN and BBEVENTLOG silently
overwrite bitbake command line arguments. This is confusing and
can cause issues that are difficult to debug. It's better to use
them as default values instead.
Used environment variables BBSERVER, BBTOKEN and BBEVENTLOG to set
default values for command line arguments.
Changed setting default value of --ui command line argument from
BITBAKE_UI to look similar way.
(Bitbake rev: 87040be4ff54cd460961318224deef8f2ea4c85a)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Bitbake uses set of environment variables to set command line
options, e.g. seeting BBTOKEN variable has the same effect
as using --token command line option.
Added new environment variables BBPRECONF and BBPOSTCONF that
are equivalents of --read and --postread command line options.
They can be used by high level scripts to append or prepend
configuration files to conf/local.conf
[YOCTO #9235]
(Bitbake rev: bf604ec1ca4eb4d0b22bcc72249963e6d7445f34)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If the based on recipe has not yet been checked out/cloned we cannot
generate the custom image recipe file that uses it. So disable/remove
the option to download it.
[YOCTO #9425]
(Bitbake rev: f08651a954358e9c6ce4d69de285e4cd9e1408ae)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This function returns the base recipe file path only if it currently
exists. This allows us to know whether we can proceed at this point with
generating a custom image recipe. It also enables us to call this
function from the templates to enable visual indication of this state.
Some whitespace fixes also added in generate_recipe_file_contents
(Bitbake rev: bc30d1b235b9ecacef5b2eaa851b9247d857f317)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Some users may want to use authenticated SSH connections with credentials stored
in a keyring, such as gnome-keyring. These typically need a DBus session bus
connection, so pass DBUS_SESSION_BUS_ADDRESS into the fetcher environment.
To avoid the user needing to set it in their local.conf (which wouldn't be
usable) or adding it to the environment-cleansing whitelist (which would
potentially impact builds) allow the variables being passed to the fetchers to
come from the data store (first) or the original environment (second).
(Bitbake rev: 20ad1ea87712d042bd5d89ce1957793f7ff71da0)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Convert existing tests to Selenium.
Add basic tests to check that the modal contains radio buttons to select
a custom image to edit when a build built multiple custom images, and
to create a new custom image from one of the images built during
the build.
[YOCTO #9123]
(Bitbake rev: c07f65feaba50b13a38635bd8149804c823d446a)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add functionality to the placeholder button on the build dashboard
to open a modal dialog displaying editable custom images, in cases
where multiple custom images were built by the build. Where there
is only one editable custom image, go direct to its edit page.
The images shown in the modal are custom recipes for the project
which were built during the build shown in the dashboard.
This also affects the new custom image dialog, as that also has
to show custom image recipes as well as image recipes built during
the build. Modify the API on the Build object to support both.
Also modify and rename the queryset_to_list template filter so that
it can deal with lists as well as querysets, as the new custom image
modal has to show a list of image recipes which is an amalgam of two
querysets.
[YOCTO #9123]
(Bitbake rev: 8c2aea3fa8e1071de60390e86e2536904fa9b7c0)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When a build is viewed in the dashboard, enable users to edit
a custom image which was built during that build, and/or create
a new custom image based on one of the image recipes built during
the build.
Add methods to the Build model to enable querying for the
set of image recipes built during a build.
Add buttons to the dashboard, with the "Edit custom image"
button opening a basic modal for now. The "New custom image"
button opens the existing new custom image modal, but is modified
to show a list of images available as a base for a new custom image.
Add a new function to the new custom image modal's script which
enables multiple potential custom images to be shown as radio
buttons in the dialog (if there is more than 1). Modify existing
code to use this new function.
Add a template filter which allows the queryset of recipes for
a build to be available to client-side scripts, and from there
be used to populate the new custom image modal.
[YOCTO #9123]
(Bitbake rev: 4c49ffd28e41c4597bdac34d5e54c125571a4b95)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The Selenium helper's enter_text() method doesn't cause
keyup events to trigger unless the element where text is
being entered has been clicked.
Prefix all text entry with a click() on the element to ensure
that keyup events fire.
(Bitbake rev: cea34880ad3847bd0e24c9b650eb816e1757cf2b)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Test adding a new custom image when:
1. No custom images are in the project yet.
2. User tries to add custom image which duplicates the name of
an existing custom image.
3. User tries to add custom image which duplicates the name
of a non-image recipe.
[YOCTO #9209]
(Bitbake rev: 21c1f8f8e30ef868ea6fd861eea1389f149f1049)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Project.release can be null. This causes an exception when calling
get_all_compatible_layer_versions(), as the query to fetch
the layer versions references release.branch_name.
Add a guard to the function so that an empty queryset is returned
if the release isn't set for a project.
(Bitbake rev: 6919a2b2e412a9e7b652a6bc191e7c1bed035222)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We currently prevent the same name being used for multiple custom
images, but make the check across all projects. This means that
custom image names have to be unique across all projects in
the Toaster installation.
Modify how we validate the name of a custom image so that we
only prevent duplication of custom image names within a project,
while ensuring that the name of a custom image doesn't duplicate
the name of a recipe which is not a custom image recipe.
[YOCTO #9209]
(Bitbake rev: 9abbb46e799c06757e03addd54e3f5d3c0fe2886)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In the import layer page, the "Add layer" button in the layer dependencies
section doesn't accurately reflect whether the layer name in the
corresponding input can be added. A partial or empty layer name can
leave the button active, such that when it is clicked, a
previously-selected layer can be accidentally added.
Fix by keeping track of the items currently available in the typeahead,
only activating the "Add layer" button when the input matches the name
of one of those items.
[YOCTO #8511]
(Bitbake rev: dbb4f0282ded361baf9e5a0346e134bece5314b9)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When hiding a column in the build tasks or build packages included
table, if the column is set as the current order by for the table, the
order by is not reset to the default. The result is that the table
stays sorted by the hidden column.
Set the default_orderby for these two tables correctly to ensure the
corresponding table is re-sorted when a column is hidden, if that column
was being used as the order by.
[YOCTO #9011]
(Bitbake rev: b99e1012f0ad1dc82a769df15a232280c8e57b9e)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If a ToasterTable is ordered by an optional column and that
column is subsequently hidden, the table ordering switches back
to the default ordering for the table. However, the table headings
don't update to reflect the new ordering. This is because the
code which sets the heading weight and hides/shows the caret symbols
only runs when the table is first loaded.
Store the default order by and re-apply it when the data is updated.
[YOCTO #9011]
(Bitbake rev: 820761e664cd2d62cc6c333a0e59580b0c4a034d)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fixed typo that caused incorrect processing of BBEVENTLOG
environment variable. Even if variable is set it was ignored
by bitbake.
(Bitbake rev: 2705b5f59aef4a070e2df2752d27bd04ea747057)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Sometimes you can end up in a situation where you need to specify that
a specific runtime entity should be provided by a specific entry.
An example of this is bluez where you could end up in a situation where
for example:
NOTE: multiple providers are available for runtime libasound-module-bluez (bluez4, bluez5)
NOTE: consider defining a PREFERRED_PROVIDER entry to match libasound-module-bluez
NOTE: multiple providers are available for runtime bluez-hcidump (bluez-hcidump, bluez5)
NOTE: consider defining a PREFERRED_PROVIDER entry to match bluez-hcidump
The only option here is to set something like PREFERRED_PROVIDER_bluez4 = "bluez4"
which is clearly not very informative.
I've actually held off adding RPROVIDER support for a long while as this
does have sigificant potential for misuse. It doesn't for example allow
multiple runtime providers of the same name to coexist, that simply isn't
supported. It therefore doesn't replace some of the name mappings such
as busybox verses coreutils that OE-Core faces as that is a different
problem with different constraints. This mechanism is simply to provide
bitbake with a hint to decide what the dependency tree should look like.
Also, this allows us to stop printing a confusing message telling the user
to set PREFERRED_PROVIDER when the setting needed would be rather ambiguous.
[YOCTO #5044]
(Bitbake rev: 62eb39d1474d024b204634689071700605c6095c)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Back in history the code did depend on previous build results. This was
bad for determinism and we no longer do that. Update comments to match
the current behaviour.
(Bitbake rev: c3fa7e561c22786d3ac57d04c367aa50f1b3b820)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We now prefix log messages coming from worker task context with the
PF and task info, however parsing messages all have to be manually
prefixed which is ugly and error prone. This change modifies the log
handler filter so this happens automatically, meaning we don't have
to change every message to include that information. This makes error
messages longer but more usable.
(Bitbake rev: 1af0ccaac81e182c4ca520037dda362d180e5605)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
An invalid task causes bitbake to exit incorrectly, firing a
CommandCompleted event rather than a CommandFailed one. This
means that clients listening for CommandFailed events are
unable to detect the build failure even though one occurred.
Passing an exception string to finishAsyncCommand when a task
fails causes the CommandFailed event to be fired correctly.
[YOCTO #9087]
(Bitbake rev: 98a2c37e077b16e3bc8bb102bd18b293130d15a4)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When multiple recipes which both provide something are being built, bitbake
informs us that most likely one of them provides something the other doesn't,
which is usually correct, but unfortunately it's rather painful to figure out
exactly what that is.
This patch dumps two sets of information, one is the provides information for
each recipe, filtered so only common components are removed. The other is a list
of dependees on the recipe, since sometimes this can easily identify why something
is being built.
Its not straightforward for bitbake to obtain the information but since the
warning/error code path isn't the normal one, we can afford to go through some
less than optimal processing to aid debugging.
Also provide the same information even if we're showing a warning since its still
useful.
[YOCTO #8032]
(Bitbake rev: 96fc889b8e62ba4463c71158c4b7286c48d68cd8)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
For a custom image, if a search for a package results in no packages
found, then additional information should be presented to the user.
This is different than a 'no results' found for a search in other
contexts, for example, a search for a package in a non-customised build.
For a custom image, a package search failure can happen because the
package was not added to the custom image. This commit presents more
information to the user, suggesting why the package was not found in the
custom image.
The generic table view handling js changes to handle a new div
element no-results-special-... such that, if present, that template
section is shown rather than the default no-results-... section.
[YOCTO #9154]
(Bitbake rev: 66b7c7ef61058b52031d71b10effcfe69afbd57b)
Signed-off-by: Dave Lerner <dave.lerner@windriver.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Bitbake variables don't include ":" characters so exclude these from the variable
expansion regexp.
This assists when parsing shell code which does A=${B:-C} as we don't want a
dependency on a variable called "B:-C".
(Bitbake rev: 14ed41a292123374d94f5c786a619881f2ddea42)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There were no tests that verified the value of origvalue in the callback
routines used by edit_metadata(). This patch adds one for a simple
multiline variable.
(Bitbake rev: ece3a4d02d8162dee78c2062c10291b5fd625c36)
Signed-off-by: Randy Witt <randy.e.witt@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
edit_metadata() would corrupt a variable that was multiline, but
had the ending quotes on the same line as the last value. For example:
TEST_VAR = " foo \
bar"
would become " foo ba" because the code would always delete the last
character on the line and then do it again if the line ended in the
quote. This however doesn't show up if you have:
TEST_VAR = " foo \
bar \
"
which is how all the test cases were written.
This patch fixes that bug and adds and fixes a test that matched the bugs
behavior rather than the expected behavior.
(Bitbake rev: 14f05cbdc2ad8d59a94af1c8816567d93c39c88c)
Signed-off-by: Randy Witt <randy.e.witt@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(Bitbake rev: 81ccbf243050a5a9245d2de4c1de342771c09a59)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add set a MACHINE if needed and add a layer. When we're running in the
context of the django unit tests we don't have these defaults setup for
the project so add them.
(Bitbake rev: a0c1432f32930a17e10d50c08c2aa84a0659514b)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(Bitbake rev: 6288a3bd6678ed1c7863cfde6eb33b7c28207777)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently we're using a hosted version of quint however this means that the
testing has to be online or have a cached version of quint. Add the
files to Toaster to be able to use offline.
(Bitbake rev: fc5024e2ed0e4ec2ee234a42fd0403c70c3f819e)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a column to the custom image pages that shows the reverse
dependencies in a format matching the dependencies column:
- either blank or a button showing the count of reverse dependencies,
- when the button is clicked, a popover appears showing the list
of reverse dependencies, with each package's size, and the total
size of all of the reverse dependencies.
The implementation adds a packages table method to retreive the reverse
dependency total size, and adds a separate 'popover' html template. Both
of these changes follow the pattern for the dependencies column.
[YOCTO #9163]
(Bitbake rev: 2f978dccaa1ec82c7ad350bdc1cd9500000984eb)
Signed-off-by: Dave Lerner <dave.lerner@windriver.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We have been seeing UnicodeDecodeErrors when handling the
ImagePkgList MetadataEvent in ORMWrapper's
save_target_file_information() if the event includes filenames
that include non-ASCII characters.
In the short term work around this by converting paths to the
unicode type when passing them to Django's ORM. This is a bit of
a hack but it's too late in the cycle to do anything more invasive.
[YOCTO #9142]
(Bitbake rev: f50fff03b3de02e73a3cc2eb9935f7c345dbddc4)
Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
buildinfohelper stores current Build object in its internal
state. Any changes to Build object will be lost if internal
state is not updated as current buildinfohelper code
saves Build object from internal state when build is
completed.
This bug causes incorrect build state when build is cancelled.
Updating internal state should fix it.
Note, that this commit updates internal state after status of
the build is changed to Build.CANCELLED. There are several other
places in the code where Build object is updated without updating
internal state. They should be carefully analyzed and fixed.
(Bitbake rev: d056cf40fc55530cb1736aedfb9a3c355884991e)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When bitbake doesn't need to build anything it still sends
ImagePkgList event with empty 'pkgdata', 'imgdata' and 'filedata'
fields. This causes crash in buildinfohelper code as it's assumed
that above mentioned fields always have data keyed by build target:
ERROR: u'core-image-minimal'
Traceback (most recent call last):
File "toasterui.py", line 423, in main
buildinfohelper.store_target_package_data(event)
File "buildinfohelper.py", line 1218, in store_target_package_data
imgdata = BuildInfoHelper._get_data_from_event(event)['imgdata'][target.target]
KeyError: u'core-image-minimal'
Fixed this by using dict.get method with empty dictionary as default
return value instead of trying to get value without checking if target
key is in the data.
(Bitbake rev: c39cc463e6d9594bf2c5ac8bb74e834f6f2cf7c8)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
It was incorrectly assumed in the current code that bitbake is in
../bitbake/bin/ directory. It's not always the case.
Using bitbake from $BBBASEDIR should be .
(Bitbake rev: 6c9e3375d278bee712c41f07428bc82108b9aaae)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Modified the projectconf.html to include DL_DIR and
SSTATE_DIR. Updated the script section in the html
to handle the changes made by the user on DL_DIR
and SSTATE_DIR. Included validation check for the
folder names.
[YOCTO #8422]
(Bitbake rev: bd9f8973d4c9c0722874a058466b1b911112500e)
Signed-off-by: Sujith H <sujith.h@gmail.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Update toaster's views.py to support DL_DIR and SSTATE_DIR for
page projectconf.html. Removed DL_DIR and SSTATE_DIR from
blacklist. Initial value of DL_DIR and SSTATE_DIR comes from
BuildEnvironment.
[YOCTO #8422]
(Bitbake rev: 9f672d7ba503d17175eef37ec03a5779e4c9f792)
Signed-off-by: Sujith H <sujith.h@gmail.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If client xmlrpc token is not provided in the command line, bitbake
generates random token. Server token in --server-only mode is always
empty. This doesn't allow clients with non-empty tokens to connect
to the server. Specifying empty token should stop generation of
random token and make it possible for clients to communicate with
the server.
(Bitbake rev: b54ec2d7cb9f2fb4ff1e8af11b6ecf97d181272c)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
- Organise the imports into logical groups
- Fix 80 col wrapping
- Remove catch all exceptions
- Log to the toaster log
- Use QuerySet functions such as .first() and Q()
(Bitbake rev: c382f550c62437954205c7dd9c7e1891d5d03945)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add handlers to make sure we remove the BuildEnvironment LOCK when we
have cancelled a build.
(Bitbake rev: 23d0a7f9664450a09c2610631b38590a09b33744)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Don't count CANCELLED builds when returning the number of builds.
(Bitbake rev: c3c29fd4eb5116b771e8e16281d4e3cdf4fae165)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add the cancel build button to the mrb section template and add the event
handlers to cancelABuild.
Also clean up the calls to startABuild to use the updated libtoaster methods
and to make the code consistent with it's cancelABuild counterpart.
Co-Author: Sujith H <sujith.h@gmail.com>
(Bitbake rev: 6b82ffca1aa9ca2d0feec64b15466bc8ba160011)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Exclude cancelled builds from showing in the builds table
[YOCTO #6787]
(Bitbake rev: a724d6aa7515e712a1d656e46e1d0f3bf7d4cea9)
Signed-off-by: Sujith H <sujith.h@gmail.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When a build is cancelled the build (action) is complete if it has been
caused the request being cancelled then update the build outcome
accordingly.
(Bitbake rev: d94d12914d351bf560b06d6f4e45c294b04ecaa3)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
To accurately reflect the state of a build request we also need a
cancelling state. This is set when we've started a build and then for
whatever reason cancel it, cancelling is not instantaneous so we have
this state to indicate that a cancel is in progress.
Also add a state transition guard. As the state of a BuildRequest can
currently be modified by three processes; Toastergui,
Runbuilds/bldcontrol and the buildinofhelper we cannot say for sure
which process will be running at the time of cancellation so in order to
avoid one of these processes making an incorrect transition only allow
transitions of state to increase.
e.g. CREATED -> QUEUED -> INPROGRESS
And to ignore such requested changes such as
INPROGRESS -> CREATED
(Bitbake rev: 449598c8e6be75bd0c9d59e7bdf859d1d6f83858)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
A new state CANCELLED is introduced to, distinguish
the state of build.
[YOCTO #6787]
(Bitbake rev: 404f406fecae879703bcfe96f3b65086b115fa8a)
Signed-off-by: Sujith H <sujith.h@gmail.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Remove getBBController function from BuildEnvironmentController.
The constructor of BitbakeController is updated appropriately so that
call can be made to connect to running server.
The call to startBBServer is removed from bbcontroller and handledin
localhostbecontroller.
[YOCTO #6787]
(Bitbake rev: e15151106aae21d3b164ca868be42bd63905f0a1)
Signed-off-by: Sujith H <sujith.h@gmail.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Update the implementation of startABuild and cancelAbuild to reflect
changes to the backend api. We now have a dedicated endpoint to make
calls into so add this url to libtoaster.ctx and allow passing null in
as a url value to indicate that we want to use the current project
Also:
- Fix some documentation comments
- Add the convenience of passing in an array of targets to startABuild
(Bitbake rev: 61a21d96abab113cbd13376cdb8b08a426b50538)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Update the implementation of the backend api for cancelling builds with
the new cancelling BuildRequest state and cancelled Build state.
Also added some docstring about general usage.
Co-Author: Sujith H <sujith.h@gmail.com>
(Bitbake rev: 0d76084f5d896e4199e1446e2d6d43190a4fcc3a)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Move the backend xhr implementation of the build request changes
into it's own file and out of the ToasterTable definition.
It used to live in the views.py but in a hope of starting to collate logical
groups of views move this to a new file called api.
(Bitbake rev: 29572f0e6bd3b5e8315f3b93d55bdb8967b86bc3)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add forceShutDown function to BitbakeController class. This function
provides a mechanism to cancel the build from toaster. An API which
can be used safely to cancel build and hence shutdown running bitbake
server.
[YOCTO #6787]
(Bitbake rev: 89dc2ee8f5b7255538082ce4f6cb5277839875a8)
Signed-off-by: Sujith H <sujith.h@gmail.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
toasterui exits event loop on one of the following events:
CommandCompleted, CommandFailed or CommandExit.
Unfortunately none of them come from bitbake when build fails.
This is normai if toasterui runs in observer mode. However, if it's
in build mode this causes toasterui to stuck in the infinite loop
waiting for new events.
The only event we can rely on is BuildCompleted as it always
comes from bitbake unlike 3 above mentioned events.
Modified the code to always shutdown toasterui in build mode
on BuildCompleted event.
(Bitbake rev: 9cd60f98b13cf7b1c518851a51e1cbaa596d8f81)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Toaster can't run builds on Ubuntu as default /bin/sh points
to dash there. The reason is that oe-init-build-env can't
be sourced under dash for various reasons. It can be fixed
or work arounded, but it wouldn't fix builds for older releases.
Explicitly using bash to start builds should fix the issue.
(Bitbake rev: e4352603b21d43a8b5d9ee417c036fff0a263d40)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The keys 'started', 'ended', 'cpu_time_user', 'disk_io_read' and
'disk_io_write' were added to the event recently, so they don't
exist in the events generated by bitbake server from older releases.
Checking if task_to_update structure has these keys before using
them should fix build of older releases.
(Bitbake rev: 79611d0ea742263074fbb0bf5f1e39df75fd9f55)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Bitbake server is used only during the build. There is no need
in keeping server running after the build.
Running bitbake -m in the subshell after the build should stop
the server.
(Bitbake rev: 062c68e3e7d00834118fe07ceb0899874be714ae)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Added 'nowait' parameter to _shellcmd method to support
running chain of commands in a subshell.
This is going to be used to stop bitbake server after
the build.
(Bitbake rev: 89fae3c8712bfaac48686c58b480e40c1abdcfdc)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Reimplemented triggerBuild method to support one build directory
per project:
- start bitbake server from the cloned repository
- don't run observer
- run bitbake build directly instead of triggering it
through xmlrpc
[YOCTO #7880]
[YOCTO #9058]
[YOCTO #8958]
(Bitbake rev: aba8e19bf609196f749e185d43571f706707e408)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Removed updating of bblayers.conf. It will be done in runBuild method.
Changed return value: return list of layers.
Removed _updateBBLayers method.
(Bitbake rev: 198bf7e6b8d7f847f2619b71c6bd86a9a76156c9)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Called triggerBuild with brbe parameter instead of adding TOASTER_BRBE
variable to the database and fetching it in triggerBuild.
(Bitbake rev: 669bb3f6865d8bb89cd94ef9d0eba3346c47d7b2)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This API is quite dangerous as it removes build directory.
It's not used anywhere and most probably will not be used in future
as toaster is going to have one build directory per project.
(Bitbake rev: ef9e126227d3bdee86c1592e2baa43d21e6a32df)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We still will have to run bitbake server, but it will be done
different way and the code will be in triggerBuild function.
Removed startBBServer API from BuildEnvironmentController and
LocalhostBEController classes.
(Bitbake rev: a8f986d73a84f28fa8b116b91fb30c5d2cd4edbe)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
buildinfohelper.brbe is lost when buildinfohelper is closed.
This causes incorrect report of brbe when build is done.
Saved brbe attribute before closing buildinfohelper and used
it to report correct brbe.
Got rid of useless and confusing 'ToasterUI build done 1'
log message.
(Bitbake rev: 5d7cce0d0ed70f6b3ebd6cbad300d86964a13398)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
DepTreeGenerated event doesn't contain 'providermap' data in jethro.
Modified buildinfohelper to handle events without this data. This
should make it possible to handle jethro events coming from jethro
bitbake server by the latest buildinfohelper.
(Bitbake rev: f6dcb1c9967f042beae024146781cb8235a9e1f2)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Return value of self.BBServer.registerEventHandler differs between
jethro and master. To be able to build jethro toaster should be
able to communicate with jethro bitbake server i.e. it must work
with both old and new registerEventHandler call.
(Bitbake rev: f356c154016c428a3b53af61a075de6f14d9d1d9)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In current toaster code BRBE(build request:build environment) value
is passed from toaster to buildinfohelper through the 'SetBRBE' event.
Passing it through environment variable is easier as it doesn't
involve rpc communication between toaster and bitbake server.
It also eliminates the need in running bitbake observer process.
Added parameter 'brbe' to BuildInfoHelper.__init__
Used environment variable TOASTER_BRBE to set brbe for
buildinfohelper object.
(Bitbake rev: a0c8e2b309055e5927a8ff729d292ccaa69d0575)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
It was used for workaround git 1.7.9.2 which was released in 2012 which
should not be existed on nowadays host, so remove it to avoid
confusions.
(Bitbake rev: 6140d0cc9aecf1029ca16fed47071dfcc92f4269)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Python 2's sqlite3 module defaults to returning Unicode strings for SQL
text queries, which could trickle down to other parts of bitbake code and
cause unexpected Unicode conversions. Using byte strings avoids this issue.
For example, the git fetcher's AUTOREV support caches HEAD SHA1's using
bb.persist_data, so sometimes the git command strings passed to fetch2's
runfetchcmd() were unicode, potentially causing UnicodeDecodeErrors when
it appended the values of environment variables containing non-ASCII chars.
[YOCTO #9382]
(Bitbake rev: 09623a0811c613a47a01ae465b822d8156faca30)
Signed-off-by: Daniel Klauer <daniel.klauer@gin.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
sstate.bbclass for example writes siginfo files to a separate location
but we need to read taint data from the standard path.
(Bitbake rev: da444c9761ee15a59ea8880e3f812a5d3f1a1aaa)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The taint values need to be passed from the server to the workers to
ensure they see the same stamp values. Also ensure that the "nostamp:"
prefix isn't included in the checksum value to match the server
calculation. This ensures the checksums are all consistent.
(Bitbake rev: f80ba20e90f3746f7faee3e0ff7f249025fec8ee)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In theory all the information to recalcuate the task signatures was written
into the siginfo/sigdata files. In reality, some of the information was
written into the filename.
Firstly this patch duplicates that info into the file itself just for easy
of use since its small.
Secondly, we abstract out the existing "calculate the checksum" code for
the taskhash, and add a function to calculate the bashhash based on the
informaiton within the file.
Finally, we call these functions when we're writing out the data to check
that the data we're writing is consistent. I've found a couple of places
it wasn't and its good to know about these in advance, rather than having
a siginfo/sigdata file which a given hash in its filename but a contents
which give a different result.
This should all combine to avoid a certain class of checksum bugs making
it into world, and identifying problems in advance.
(Bitbake rev: 0f50a18d7a0ea0d68edd8e5217e29111f4b1ea0b)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When I enabled debugging of the checksum code, I found the value calculated
from siginfo/sigdata files for do_fetch tasks never matched. This was due
to an error in the way the data was being stored for these, it wasn't ordered
correctly. This patch fixes things so the checksums calculated from
siginfo/sigdata files is correct when file checksums are present.
(Bitbake rev: 046c1be7594fae2eec3d1f242ba3e9a85f1a1880)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The real method is a few lines later, this one is incorrect and
just causing confusion. Remove it.
(Bitbake rev: a896f263300f069400eae533be0daf5dedf41c95)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The tests were skipped when running without network even though they
didn't require network. This commit also adds a test case for URLs with
ports in them (the ports should not be considered when doing trusted
network checks).
(Bitbake rev: 77747de6b20538063eef3b188489a35bef225359)
Signed-off-by: Olof Johansson <olof.johansson@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Bitbake would fail to classify the following URL as belonging to a
allowed network, because of the port number in the url.
BB_ALLOWED_NETWORKS = "*.example.com"
SRC_URI = "http://git.example.com:8080/foo.tar.gz"
Since protocols aren't specified in the BB_ALLOWED_NETWORKS variable,
it's reasonable to believe that this should work regardless of protocol
being used.
(Bitbake rev: ff603df23037e10fb2cfdf150429cba3f65072cd)
Signed-off-by: Olof Johansson <olof.johansson@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We were doing a string search for the literal string "require" and
assuming that a match meant the recipe uses the require keyword to
include the contents of another recipe. This test isn't very smart (it
should at least have matched on "require ") and triggers on the word
require anywhere in the recipe, i.e. the word "required" in the
DESCRIPTION of core-image-lsb.bb, and then breaks as the much smarter
regex fails to match anything.
Instead always run the regex search and only try to update the
require entry when the regex matches.
(Bitbake rev: a6add0e95d3d1e9a6a9fcabd73543bc5c278915f)
Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The help text said to append a semicolon and a task name to a target to
run a task other than do_build. What you need to append is a colon.
[YOCTO #9326]
(Bitbake rev: 691419e23a94a94129b177e71d2c728b12689139)
Signed-off-by: Belen Barros Pena <belen.barros.pena@linux.intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add additional metadata to the layer created for build history to be
able to identify the layer and recipe later on. Specifically this is the
branch and release to which the recipe and layer are associated with
enabling differentiation of two recipes which are local release and
master and 'master' release.
[YOCTO #8528]
[YOCTO #8545]
(Bitbake rev: 3deebd887bddbbd02fd9829a180aab494b1af7c4)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(Bitbake rev: 20ce1e1b39a9b602eb51ca0ba3954ea3e999c874)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(Bitbake rev: 0e5f45d68e423f8462937879eed3253db31b2bb5)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(Bitbake rev: 31204937f71a7e0aa08361c3e20d02d063788a86)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(Bitbake rev: 4fda6be831d10e6d266b11975a0e9a35a7f35a77)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Create a new folder for Selenium tests.
Add a new base Selenium testcase class and a helper which
instantiates a webdriver for a given browser.
Add a sample Selenium test case which can be used as a template
for creating new tests.
(Bitbake rev: b7a377aa2ab36390d619e2a0436ccb4b8d186c23)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There appears to have been a lot of copy and pasting of the code
which prints tracebacks upon failure and limits the stack trace to
5 entries. This obscures the real error and is very confusing to the user
it look me an age to work out why some tracebacks weren't useful.
This patch removes the limit, making tracebacks much more useful for
debugging.
[YOCTO #9230]
(Bitbake rev: 5549748a200b5df259fc7352477ec59471b87b2f)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
It's very possible that we added layer as:
BBLAYERS += "/path/to/meta/"
then there would be warning:
WARNING: No bb files matched BBFILE_PATTERN_core '^/path/to/meta//'
This patch can fix the problem.
(Bitbake rev: 2b1cb21d18fb18399e682021b866babeced9a4aa)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
With the code as it stands today it not possible to execute a python function
and get "normal" python exception handling behaviour. If a python function
raises an exception, it forces a traceback to be printed and the exception
becomes a FuncFailed exception.
This adds in a parameter 'pythonexception' which allows standard python
exceptions to be passed unchanged with no traceback. Ultimately we may want
to change to this convention in various places but at least it means we can
start to add sane functions now.
(Bitbake rev: 85cf22fd0ed26bb7dc7738ef2a10441891f38ae2)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There are some codepaths where the file checksum is verified and can
be found to mismatch but the 'rename' logic doesn't kick in. If code
relies on the presence of a file for the checksum having been checked
(e.g. uninative.bbclass) then it can be used when the checksum hasn't
matched.
Therefore rename the file whenever an invalid checksum is encountered.
(Bitbake rev: 69ef6c8a9db02bfa0e3fac72481ec26586a29a01)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Prevent a hang when shutdown() is called during parsing (e.g. after
SIGINT). We must not append 'None' to the jobs queue. Otherwise the
worker loop inside Parser.realrun() may break out at the wrong point,
causing the results queue thread blocking bitbake indefinitely.
[YOCTO #9319]
(Bitbake rev: 7ebea3e9a60232222efa8a546a0ff28a53029949)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently bbappend files in a layer are applied in the order they're
found on disk (as reported by glob) which means things are not
deterministic.
By sorting the glob results, the order becomes deterministic, the parsing
order for .bb files also should be deterministic as a result of this change.
[YOCTO #9138]
(Bitbake rev: 3f8febc4212fbd3485ac9bdd4ac71b8fb0a05693)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Before this patch, directory symlinks mathcing filename pattern (either
a file name or a glob pattern) were followed. However, directory
symlinks deeper in the search chain were omitted by os.walk(). Now
directory traversal behaves consistently, ignoring syminks on all
levels.
One reason for choosing not to "walk into" directory symlinks is that
dir symlinks in externalsrc.bbclass in oe-core are causing problems in
source tree checksumming.
[YOCTO #8853]
(Bitbake rev: 66dff37ebcd1dd14ebd6933d727df9cf0a641866)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If local.conf contains an invalid line, e.g.:
APPEND += " igor"
(note the leading space) then nasty tracebacks are shown which confuse the
user. Change so the parse error is simply shown without a traceback, improving
the user experience.
[YOCTO #9332]
(Bitbake rev: 148aa1fb45dcb37a756a08301a7daf270e753180)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When prefix is part of the version directory it need to ensure that
only version directory is used so remove previous directories if exists.
Example: pfx = '/dir1/dir2/v' and version = '2.5' the expected result
is 'v2.5' instead of '/dir1/dir2/v2.5'.
[YOCTO #8778]
(Bitbake rev: c760531c6dbf88135ab9f8e6f0784ccbf2cce1e4)
Signed-off-by: Aníbal Limón <limon.anibal@gmail.com>
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Little improvement for reference tokens by names instead of index.
(Bitbake rev: e8ea15eeb1857ed4bb6337836bd2fb1f5dbb1bdf)
Signed-off-by: Aníbal Limón <limon.anibal@gmail.com>
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We recently dropped lockfiles for file:// urls which in itself makes
sense.
If a file url redirects to something like an http:// mirror, we'd have
no lock taken for the original file and could race against others
trying to download the file. We therefore need to ensure there is a
lock taken in the mirror handling code.
This adds code to take such a lock, assuming it isn't the same lock
as the parent url.
(Bitbake rev: 913b6ce22cd50eac96e8937c5ffc704bfce2c023)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
For customised image package removal change behavior.
From:
Only display the immediate dependents of the requested package
to remove, not the full dependent list, that is dependents of
dependents ...
Do not remove the displayed dependents, just notify the user
of the list.
To:
Display the complete dependent tree, traversing all reverse
dependencies starting from the package to be removed and then it's
dependents.
Change the modal dialog to note that all of these dependents will
be removed automatically.
[YOCTO #9121]
(Bitbake rev: 1185a5bfe1b05a1b63a927c9583dfc031fdac8a9)
Signed-off-by: Dave Lerner <dave.lerner@windriver.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
For customised image package removal, show a different modal dialog that
lists ALL of the packages dependent on this package, with a Remove All
button - implying that all of the dependents will be removed.
[YOCTO #9121]
(Bitbake rev: 768e6bb90c433687b0d52f256b7115499ca3418b)
Signed-off-by: Dave Lerner <dave.lerner@windriver.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When a package is to be removed, show the full list of packages that are
dependent on that package, telling user that these packages will also be
removed.
[YOCTO #9121]
(Bitbake rev: f5cb59b6b10a714b18b1c00b9a8598dd855c84b5)
Signed-off-by: Dave Lerner <dave.lerner@windriver.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This fixes a problem when the repository contains multiple levels of submodules via a resursive submodule init.
(Bitbake rev: dbafbe229360ffe5908b106a9c10e274712b9b17)
Signed-off-by: Derek Straka <derek@asterius.io>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently xmlrpc server implicitly sets itself into single use mode
when bitbake server is started with anonymous port (0) or no port is
provided in command line. In this mode bitbake shuts down xmlrpc server
after build is done. This assumption is incorrect in some cases.
For example Toaster uses bitbake in this mode and expects xmlrpc server
to stay in memory.
Till recent changes single use mode was always unset due to the bug.
When the bug was fixed it broke toaster builds as Toaster couldn't
communicate with bitbake server in single use mode.
Reimplemented logic of setting single use mode. The mode is explicity
set when --server-only command line parameter is not provided to bitbake.
It doesn't depend on the port number anymore.
[YOCTO #9275]
[YOCTO #9240]
[YOCTO #9252]
(Bitbake rev: afc0dd5c532684f6201b1e12bbf4c226ea19062d)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
bb.event.ParseStarted event is not processed by toasterui, but
present in event list. This causes the following error:
WARNING: Unknown event: <bb.event.ParseStarted object at ...
and non-zero return code:
WARNING: Return value is 1
(Bitbake rev: 1cc102f3d83d9467a3a3c422254333796ba95605)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Remove the very verbose log dump from toasterui. This generates several
megabytes of not that useful debug information and actually hinders
finding the original exception.
(Bitbake rev: a21dc134bdce2c9eb5e47c770094660f0c45c398)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This makes it possible to prevent a recipe to be cached, and thus,
parsed every time.
Use with care.
[YOCTO #8853]
(Bitbake rev: 78335c1fbe5266116700c2413aac28b00423a75b)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Try a secondary file path if the first does not exist. When we get the
recipe paths and layer information from the layer index it is not a
complete path but we are usually able to reconstruct it. If the complete
real path has been discovered by building then use this instead.
[YOCTO #9206]
(Bitbake rev: 238db2e03405d259d48dfc477a276191e6a47698)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Move the generated layer for custom recipes into the build directory.
The build directory makes more sense as this layer/recipe is generated
for a particular build/project.
(Bitbake rev: 77f3728ee6ea379bffcf73f33c26e982cb0795f4)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We don't need to skip file:/// uri type locations for git repositories.
If you're using a file:/// uri you should know that it has to be a local
path.
[YOCTO #9200]
igned-off-by: Michael Wood <michael.g.wood@intel.com>
(Bitbake rev: 4d0e5804103c2d98d038b3c490cac37a73f3cc47)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Use a constant to define the name for the toaster custom images layer;
this constant is then used to identify this layer in various places.
(Bitbake rev: 2540969ec71612af7f9041cadcc401513e9b357b)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We may have a recipe which is based on a custom image recipe that has
already been built so keep the recipe file around so that it can be read
by the generate recipe function.
(Bitbake rev: 502148fe498fe0a41ae89c9f649d3cb1253b0487)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Update the upstream url used for testing cups versions after upstream website
changes.
(Bitbake rev: 5f06041d4936fc22297945bbbad7020bfa9083c6)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fixes [YOCTO #9231]
npm when given an invalid registry URL with --registry actually goes and
fetches from the default registry, but this commit makes sure it goes to the
specified one.
(Bitbake rev: 7c849be7c70a5db4f66fe3041486abb923b5e4ee)
Signed-off-by: Brendan Le Foll <brendan.le.foll@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Instead of trying one time with a timeout of 20 seconds try 4 times with
a timeout of 5 seconds, to account for a slow server start.
(Bitbake rev: 4a7fe63126dd8177baa5ad21e59e0bebeea8c596)
Signed-off-by: Lucas Dutra Nunes <ldnunes@ossystems.com.br>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When returning the compatible layers make sure that we are only
listing: All the layers which are for this release && configuration
layers (i.e. aren't part of the build history) and which aren't an
imported layer OR are this project's imported layer(s).
[YOCTO #8944]
(Bitbake rev: de8baedaccb451c12bc3f642449db3f64aed6bf7)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The data available from buildstats is now more fine grained than
previously, so take advantage of that to enrich the data we save
against tasks:
* Store the CPU usage for user and system separately, and display
them separately.
* Disk IO is now measured in bytes, not ms. Also store the
read/write bytes separately.
* Store started and ended times, as well as elapsed_time. This
will enable future features such as showing which tasks were
running at a particular point in the build.
There was also a problem with how we were looking up the Task
object, which meant that the buildstats were being added to
new tasks which weren't correctly associated with the build. Fix
how we look up the Task (only looking for tasks which match the
build, and the task and recipe names in the build stats data) so
the build stats are associated with the correct task.
[YOCTO #8842]
(Bitbake rev: efa6f915566b979bdbad233ae195b413cef1b8da)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When user enters a non-ascii character in the project
name of toaster, the build doesn't get triggered.
Use force_bytes to fix this.
Also deal with non-ascii project names when logging the
build request in runbuilds.
[YOCTO #9071]
(Bitbake rev: b6141c4d170885d3bdf63074afcb1e41fde0a8f0)
Signed-off-by: Sujith H <sujith.h@gmail.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
For some reason, the values for SRC_URI[md5sum] and SRC_URI[sha256sum]
were not being expanded. That lead to the following code not working
as expected:
SRC_URI = "http://.../${PN}-${PV}.tar.gz"
MD5SUM = "123abc..."
SHA256SUM = "abcd1234..."
SRC_URI[md5sum] = "${MD5SUM}"
SRC_URI[sha256sum] = "${SHA256SUM}"
(Bitbake rev: ba011470df0ea8bd89f01c0b02ec4b3969e60ce7)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
XMLRPCServer.single_use attribute was always set to False.
This caused xmlrpc server to keep running after build is done as
BitBakeServerCommands.removeClient only shuts down server if its
single_use attribute is set to True.
(Bitbake rev: 0a60b0928a0a746a60d2c2f294ff1903963c7086)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Without this you get a rather odd traceback instead of the proper
exception message.
(Bitbake rev: 2fe1826d3077eeda6cde433d3a1e6620f74e08dd)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The output of "npm view dependencies" isn't entirely JSON if there are
multiple results, but the code here was just discarding the output if
the entire thing didn't parse as JSON. Split the output into lines and
iterate over it, parsing JSON fragments as we find them; this way we end
up with the last package's dependencies since it'll be last in the
output.
Digging further, it seems that the dependencies field reported by "npm
view" also includes optional dependencies. That wouldn't be a problem
except some of these optional dependencies may be OS-specific; for
example the "chokidar" module has "fsevents" in its optional
dependencies, but fsevents only works on MacOS X (and is only needed
there). If we erroneously pull in fsevents, not only is it unnecessary
but it causes "npm shrinkwrap" to throw a tantrum. In the absence of a
better approach, look at the os field and discard the module (along with
any of its dependencies) if it isn't for Linux.
As part of this, we can reduce the calls to npm view to one per package
since we get the entire json output rather than querying twice for two
separate fields. Overall the time taken has probably increased since we
are being more thorough about dependencies, but it's not quite as bad as
it could have been.
(Bitbake rev: 436d67fe7af89ecfbd11749a6ae1bc20e81f2cc8)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
"2 || 3" is a valid version specification for a dependency in an npm
package.json file, but of course that looks like something else when
sent to a shell. Quote the version value to avoid this.
(Bitbake rev: bea0246831a46d943d2e27d6b38f6e498bd3413c)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Continue after processing BuildStarted event to fix
WARNING: Unknown event: <bb.event.BuildStarted object at 0x2554150>
(Bitbake rev: 12f1fb8c9b70fea0c9145f881bcceb8af32df6af)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Toasterui exits only if bitbake observer shuts down.
In build mode it should exit when build is done.
Made toasterui exit on bb.command.CommandCompleted,
bb.command.CommandFailed and bb.command.CommandExit events
when it's running in build mode.
(Bitbake rev: b11f9d6d3c2eb615335901e1dcea699daf3afb4c)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently toasterui works only in observer mode. This is
artificial limitation which was made to support current toaster
design. As we decided to stop using bitbake server we'll
need to run toasterui also in build mode.
[YOCTO #7880]
(Bitbake rev: d4b5796899c3ca5c7becd7322291afd8afb35a31)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently toasterui ignores return value of setEventMask
command, which created confusing difference between set of
events set by this command and the real set used in the code.
Checked if setEventMask succeeded. Print error message and
exit if it's not.
(Bitbake rev: 6e3f13ffb47102b5df2da91fbc3f5da3179245b2)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Executing setEventMask command when bitbake server is in readonly
mode causes runCommand to fail with the following error:
'Not able to execute not readonly commands in readonly mode'
Set readonly attribute for setEventMask command to make it working
for Toaster UI. This should not do any harm as this command doesn't
influence cooker state.
(Bitbake rev: 8a47d30b2555255fbf6049c5ed69b29664c32b17)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Removed events not used in the code from the list.
Added events that are used in the code.
(Bitbake rev: 16b14ec16049cc2040a60ad5fc95f6e19dda91a6)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Reformatted and reordered list of events to make changes
easily and see them clearly in the diffs.
(Bitbake rev: 42a2d1115f2b23dc063a3172285ca3be73cf70bb)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The code of this module is broken for a long time.
The functionality of it can be easily achieved by running
'manage.py runbuilds.py' on remote machine.
[YOCTO #8806]
(Bitbake rev: 975081eefdd7041a6b4bef6842c1bac9799a1b44)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Removed usage of sshbecontroller from bbcontroller, models, tests
and database schema.
(Bitbake rev: 3ee06eb7e96de5dba539ad52201867e77d06a53e)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Raised NotImplementedError instead of Exception to be
able to catch it.
This is a preparation for removing sshbecontroller module.
It has to be done as code in bldcontrol/tests.py imports custom
NotImplementedException from sshbecontroller.
(Bitbake rev: c243ab6c83fe12d84777e4c3a18fd393827b9327)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This patch e0fd96442a removed the
"use strict" directive from the projectpage.js file by mistake.
This patch makes amends.
(Bitbake rev: b8044ce60af3f0b064cfba76c577503cc896e358)
Signed-off-by: Belen Barros Pena <belen.barros.pena@linux.intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
I've received some feedback on the 'Revision' label we use in the import
layer page. It is not quite communicating that what's required is a Git
revision. Changing it to 'Git revision' to make it a bit more specific.
The change applies not only to the import layer page, but to all pages
showing revision information in the project configuration section.
For more on the feedback received, check
https://bugzilla.yoctoproject.org/show_bug.cgi?id=8429#c3
(Bitbake rev: 09392f36a4f115c2432302125e8cac48a9aa304f)
Signed-off-by: Belen Barros Pena <belen.barros.pena@linux.intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
As we do with the popover snippet which shows dependencies inline in the
table also show dependencies which have both TYPE_TRDEPENDS and
TYPE_RDEPENDS.
Also remove obsolete comment
(Bitbake rev: d3b5f3b7ba4550e7cd03a37ca19ccd2fc0042b2d)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
[YOCTO #9156]
For a customized image when adding a dependent package X that depends on
dependency package Y, in addition to adding X to appends_set and Y to
includes_set, make sure that Y is no longer in the excludes_set. Y may
have been added to the excludes_set by a prior package removal.
(Bitbake rev: 6b29d3297de2ae48a3ac5529ba0d22f895276b56)
Signed-off-by: Dave Lerner <dave.lerner@windriver.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Tarballs that are fetched down via npm repositories seem to often have
unknown headers. This doesn't affect our ability to extract the contents
though so we don't really care to see those warnings.
(Bitbake rev: b38975103e52a0c25e9ad9032c8cca1c47cbdcc2)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
npm allows you to specify a dependency as a Github username+path, or
omit the version entirely. You can hit these if you don't use a
shrinkwrap file, with the result that the code later fails due to the
output of "npm view" being empty; so handle this lazily by just ignoring
this part of the dependency if it's not really a version.
(Bitbake rev: 7b7a65c44dbdd5ba9366d4e2093f76df8758d546)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
No code changes, just fix to use four spaces.
(Bitbake rev: 66a9ee7d54ca9c25209f72da079f260ccdcc872a)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This ui does not work in master, nor has it been updated for several years.
[YOCTO #9178]
(Bitbake rev: 9fad1d13eed1f725971e6d12d3977cd31e07019a)
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We've been gearing up the Toaster web UI to replace the Hob (GTK+ based) UI
for some time now; Hob has basically been on life support for the past few
releases. As of late last month in master, Toaster has the capability to
select the packages in an image, removing the last thing that Hob could do
that Toaster couldn't.
To recap, the reasons why Hob is being removed include:
- The code is tightly woven into BitBake, making it fragile. This means it
needs significant QA and maintenance on an ongoing basis.
- Some of the implementation is not ideal; we'll be able to remove some cruft
from BitBake and OE-Core at the same time.
- It's GTK+ 2 based, not the current GTK+ 3.
- Toaster is now a much more capable UI and is being actively maintained
The discussion about removing hob can be found at:
http://lists.openembedded.org/pipermail/openembedded-architecture/2016-February/000082.html
(Bitbake rev: be2cceea159c6ca9111eff3df87b98513eab6d72)
Signed-off-by: bavery <brian.avery@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
npm-shrinkwrap will sometimes resolve a git URL which instead of a http url, in
this case go and grab the dist.tarball via npm instead of using the resolved
URL.
(Bitbake rev: eb53b927ff59aa19cf28bc46beb9f9a185a59990)
Signed-off-by: Brendan Le Foll <brendan.le.foll@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The previous commit breaks absolute pathnames in file:// urls, this
fixes it.
(Bitbake rev: b8113a1800687a37a26ac28deafdbafd74cc138e)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When in SRC_URI appears file://dir;subdir=foo unpacker copies 'dir' to ${WORKDIR}, not
${WORKDIR}/foo as it should be.
These changes are fixing following bugs as well:
Bug 6128 - Incorrect wildcard unpack behaviour in fetcher
Bug 6129 - Local directories unpack to a different location than local files
(Bitbake rev: e659a3b0c2771679057ee3e13cd42e6c62383ff2)
Signed-off-by: Alexander Shashkevich <alex@stunpix.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
For some reason the enablement piece of the patch went missing, add it.
(Bitbake rev: 0270b5a3873ed0aeca3a66198c87a6164fb644b8)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
PN can contain '_', e.g. gcc-cross-x86_64 and an override cannot
hence we do this manually rather than use OVERRIDES.
(Bitbake rev: 7a6baf02617d1edced4eaff235e73d746e2a3b68)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
For local files, there are no races with downloads, we don't need ".done"
stamps and we don't need lockfiles.
This considerably cleans up DL_DIR and all the pointless ".done" files
as well as removes stalls over local files with the same name.
(Bitbake rev: 48e903745db578d9b9b425a8d411c1369df0eb94)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Rather than create ".lock" and ".done" files with no name, error,
forcing us to fix the cases where this is a problem.
(Bitbake rev: 81158071508cc68c39db7d501370872f44d335cc)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If we specify urls such as npm://somehost;someparams the fetcher currently
does a poor job of handling mirrors of these urls due to deficiencies in the
way decodeurl works. This is because "somehost" is returned as a path, not
a host.
This tweaks the code so that unless its a file url, the host is returned
correctly.
This patch also adds test cases for these urls to the exist set of test
urls.
We need to tweak the URI() class since this thinks this is a relative url
which is clearly isn't. We also need to handle the fact that encodeurl will
error if passed a url of this form (it would want the path to be '/'.
(Bitbake rev: 83203cd2e677706e0111892a7843b83263cb8bd9)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If we wget something which looks directory like we end up with lock files
and done stamps without names, they also all use the same lockfile.
This change ensures that we use separate lock files based on the url
and avoid creating the mysterious ${DL_DIR}/.done files.
(Bitbake rev: 20bc82086018832e047345a672d74b6c1c113650)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
I have received quite a few complaints about the use of the word
'delete' for layer removal, so change it to 'remove'. That also matches
the language we use for packages in image customisation.
[YOCTO #9165]
(Bitbake rev: 3c5ac2ddfb3f5ecd3f3218de0d6564e5f3842b98)
Signed-off-by: Belen Barros Pena <belen.barros.pena@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
I quite dislike the 'Run again' label we use in the button that rebuilds
things. Changing it to 'Rebuild', which is shorter and more specific.
(Bitbake rev: 865a2015e86a1bc5cc7d63308f27c292d1ca98eb)
Signed-off-by: Belen Barros Pena <belen.barros.pena@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Correct formatting of the banner message after adding a package to a
custom image.
[YOCTO #9101]
(Bitbake rev: da233005eb8cfa7842cd1a768c16e42aaaa55fad)
Signed-off-by: Dave Lerner <dave.lerner@windriver.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The default "Command line builds" project does not have a Configuration
page. It therefore needs a custom breadcrumb where the project name goes
to the project builds page, instead of the project configuration page.
(Bitbake rev: 5545acf6703a25ee46776138bbbd804615add89c)
Signed-off-by: Belen Barros Pena <belen.barros.pena@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Returns a script (string) that reconstructs the state of the
entire database at the time this function is called. The script
language is defined by the backing database engine, which is a
function of server configuration.
Returns None if the database engine does not support dumping to
script or if some other error is encountered in processing.
The SQLite3 implementation in db.py calls iterdump() [1] to generate
a script. iterdump() is the library equivalent of the `sqlite3 .dump`
shell command, and the scripts are compatible. Execute the script in
an empty SQLite3 database using the sqlite3 utility to restore a backup
of prserv.
Use case: Backup a live PR server database in a non-racy way, such
that one could snapshot the entire database after a set of bitbake
builds all using a shared server. I.e. All changes made prior to
the start of a dump_db() operation should be committed and captured
in the script. Subsequent changes made during the backup process are
not guaranteed to be captured.
Testing: ~7MB database backs up in ~1s while PR server is under load
from 32 thread bitbake builds on two separate machines.
[1] https://docs.python.org/2/library/sqlite3.html#sqlite3.Connection.iterdump
(Bitbake rev: 004003daf6bd0f0233ce5c2d95f1d7d64ab91bb3)
Signed-off-by: Haris Okanovic <haris.okanovic@ni.com>
Reviewed-by: Ken Sharp <ken.sharp@ni.com>
Reviewed-by: Bill Pittman <bill.pittman@ni.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fix the view of 'Image Recipes' under 'Configuration' to only show
image recipes that are not customised since custom images have their
own page.
[YOCTO #9111]
(Bitbake rev: 18a93b360301a5497d5c8ef74ab71f374f2ad210)
Signed-off-by: Dave Lerner <dave.lerner@windriver.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The build dashboard doesn't show image and artifact files correctly,
as it shows the full filename for images and the filename plus
path relative to DEPLOY_DIR for artifacts.
Instead, show just the suffix for image files, and the basename
for artifact files.
(Bitbake rev: 8084dcdc283b4dc170f066c202f89d56ce1abbef)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In the 'New custom image' page, each image recipe name listed should
link to the corresponding image recipe details page, so that users can
look into what packages are installed by a certain image, and decide
based on that if they want to customise it or not.
This patch adds that missing link.
(Bitbake rev: a481af693bfef0171732c18c298e285986b82de3)
Signed-off-by: Belen Barros Pena <belen.barros.pena@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The search field at the top of our tables was using one of the Bootstrap
classes for text field sizing. Those classes are a bit rigid, resulting
in text fields too wide that made other table controls wrap.
Setting a maximum width to the search form using one of the span classes,
combined with a % width css declaration, make for text fields that
adapt a bit better to the horizontal space available in each table.
(Bitbake rev: 7833fab2e03f2d9a01ab9ad0a13c190382098b5e)
Signed-off-by: Belen Barros Pena <belen.barros.pena@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Our project pages have 4 tabs: builds, configuration, import layer and
new custom image. Even though we treat the 'configuration' as the
default tab, it comes second after the builds tab.
That's a bit strange: the default tab should be the first one listed.
This patch changes the tab order to put 'configuration' first.
(Bitbake rev: ccb90019489c2c324c2a5a60295e02280a2ec18f)
Signed-off-by: Belen Barros Pena <belen.barros.pena@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The existing breadcrumb does not always provide a link to the project
configuration page. When you are in the build history pages, you must go
back to the builds information first, and from there access the project
configuration. That feels very long.
Change the breadcrumb so that the project name item always provides a
link to the project configuration.
(Bitbake rev: 9910f3f292d35fc91215d550c5f123dcf18ab35d)
Signed-off-by: Belen Barros Pena <belen.barros.pena@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Now that we use modal dialogs to display dependency information for
packages, we are hitting their maximum height relatively often. It is set
by default to 400px, which makes it a bit tight at a 1280x800 viewport
size.
Reduce the maximum height to 300px to make things a bit more
comfortable.
(Bitbake rev: e36001d61768979d66cba0f3d4f5a2aaf4af2cb7)
Signed-off-by: Belen Barros Pena <belen.barros.pena@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The 'add layer' button in the project configuration page remains enabled
after you add a layer. If you click it again, the same layer you just
added is added again.
This patch disables the 'add layer' button on click, to avoid this bit
of weirdness.
[YOCTO #8905]
(Bitbake rev: 63705f60035884a810fdd36e5a3fe10e411f23c7)
Signed-off-by: Belen Barros Pena <belen.barros.pena@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The form for naming new custom images shows you an error message when
the name already exists or you include an invalid character in it. But
when an error appears, the input field was missing the red highlight.
This patch applies the right class to the form controls whenever an
error message is shown.
(Bitbake rev: df342e7662179410467c47cd870180ea75f863d4)
Signed-off-by: Belen Barros Pena <belen.barros.pena@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The placeholder text in the form where you name your new custom images
didn't display fully. This patch fixes the styles so that the text shows
properly. It also changes the text itself to make it a bit shorter.
[YOCTO #9122]
(Bitbake rev: 9df802182b0b96295b148a1681c2265e72d8306b)
Signed-off-by: Belen Barros Pena <belen.barros.pena@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Although the support for building more than one release and how we
handle the build directories is the subject of lively discussion, we all
seem to agree on removing the ability to change the release of a
project. The feature is currently not working but exposed to users,
which is not a happy state of affairs.
This patch comments out the controls that give access to the release
changing functionality to hide them from users, but does not touch
anything else. Once all moving pieces start to settle down, we can make
a final decision regarding this feature, and clean up the code
accordingly.
[YOCTO #8917]
(Bitbake rev: 3a8c6f7155517cd61a160595b81e7bed84ba4eaf)
Signed-off-by: Belen Barros Pena <belen.barros.pena@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
npm fetcher with support for shrinkwrap files and lockdown files to easily
download and install an npm package with strict dependency resolution.
The SRC_URI should be in the format of:
SRC_URI = "npm://registry.npmjs.org/;name=${PN};version=${PV}"
To add a shrinkwrap and lockdown file use:
NPM_SHRINKWRAP := "${THISDIR}/${PN}/npm-shrinkwrap.json"
NPM_LOCKDOWN := "${THISDIR}/${PN}/lockdown.json"
(Bitbake rev: dec75bbc5d075acb322dad8b1c40d6bd518dc9fd)
Signed-off-by: Brendan Le Foll <brendan.le.foll@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This is useful as npm-lockdown uses sha1 because npm releases the sha1 of
packages and whilst this is undocumented it seems no other algorithm is
supported
(Bitbake rev: fd5d9011f6dd7029895b64d8a02d33185b9aa8ae)
Signed-off-by: Brendan Le Foll <brendan.le.foll@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This test was passing but because the assertion was the wrong way round
and should have expected the first one to pass and second one to fail,
in reality both were failing as the method for checking the invalid char
was incorrect.
(Bitbake rev: 932a92b8130d4815656dc885f0c6e4afa4502022)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When showing the package list for the image recipe details only show the
packages which are installed in the image rather than all the packages
which are produced.
[YOCTO #9108]
(Bitbake rev: be7dca6c7607c0d13151c2d3f7ad7adcdf365076)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Instead of having to manually write up in the setUp function all the
different objects to create in the database, just specifiy them in a
fixture and load it. This has the advantage that it can be used on a
live server by using the ./manage.py loaddata command and can then
be inspected.
(Bitbake rev: 8a653ad8b075892d24d15696b3070c5a28a4fbe2)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Make sure the default orderby for the SoftwareRecipesTable is applied
(Bitbake rev: 1688608b537d8de840c6d1e4802ae41ca872e5bf)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We messed up the migrations by squashing some of the image
customisation model definitions into the initial migration which
has meant some irreversible operations on mysql took place.
This deletes, re-orders and fixes the migrations.
If your schema is up to date you may want to use ./manage migrate
with --fake or --fake-initial to avoid re-applying migrations.
[YOCTO #9116]
(Bitbake rev: 19bd63fc3a28dcbd0f531a5b06a037da34568bac)
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
At some point in the future, getVarFlag should expand by default. To
get there from the current position, we need a period of time where the
expand parameter is mandatory.
This patch starts that process. Clear errors will result from any code
which doesn't provide this. Layers can be fixed with an expression
like:
sed -e 's:\(\.getVarFlag([^,()]*, [^,()]*\)):\1, False):g' -i `grep -ril getVar *`
(Bitbake rev: aa3faebdf6af66ab34f74d328b2113de0b08c7ee)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
At some point in the future, getVar should expand by default. To get
there from the current position, we need a period of time where the
expand parameter is mandatory.
This patch starts that process. Clear errors will result from any code
which doesn't provide this. Layers can be fixed with an expression
like:
sed -e 's:\(\.getVar([^,()]*\)):\1, False):g' -i `grep -ril getVar *`
(Bitbake rev: fab717d303df0bcef737661f6917f275f35215a4)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Define a new bitbake configuration variable BB_HASH_CHECKSUM_CACHE_FILE
that can be used to define the cache file to use for file checksum
cache.
(Bitbake rev: a965b390d6240e279c190b92b17c0573e9bd604c)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If no cache file name is given a default from class variable is used,
like before.
(Bitbake rev: 2602a312818f564961de7dfa63c429d45ff9e5ac)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Move the local file checksum functionality from bb.fetch2 into
bb.checksum module.
(Bitbake rev: 4f60933283f377d68f191db849dac6c1dc7a0aed)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Before this patch the usage of cache was quite useless as the file
checksums were not actually cached on disk but re-calculated every time.
This patch utilises the new writeout_file_checksum_cache() method of the
SignatureGenerator class to do the job.
(Bitbake rev: 5ac9cbf405841ed3f65e6f99a3cee032567fb182)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Extend the API in order to be able to write out the file checksum cache
onto disk. SignatureGeneratorBasic class now implements a method that
update the fetcher local files checksum cache with the task file
dependency checksums.
(Bitbake rev: ecdabd321d48fa367b89ebffc00aa525b6eaa95c)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Drop unused 'd' argument from the cache save methods, simplifying the
API.
(Bitbake rev: 81bc1f20662c39ee8db1da45b1e8c7eb64abacf3)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The current behavior of Hob is that there is a "Run Image" button which becomes visible only for qemu images.
My suggested change is:
- if an image is selected and it is qemu-compatible, let the "Run image" button be named "Run qemu image"
- if an image is selected and it is not qemu-compatible, let the same button show up with the name "Run custom image", and besides that, an option shows-up to allow the selection of the custom script (by default it points out to runqemu script) to be used for launching this custom image
Note: in case there is more than one toggled image (qemu runnable or deployable), when the user clicks the "Run custom image" button, a dialog will be presented, allowing to choose between any of the existing images.
[YOCTO #8940]
(Bitbake rev: cc4cfc2370297b8feb2dc39d4262e73adf06c09a)
Signed-off-by: Mirela Rabulea <mirela.rabulea@nxp.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
An example prefix: `perl-5.22.1-r0 do_compile:`
(Bitbake rev: 792b759e59e31d2e43d525a6e50d866b4f51f072)
Signed-off-by: Christopher Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This lets us filter and use -l to show messages from that source specifically.
(Bitbake rev: 7946927156dec33364418988eb921ddb273660eb)
Signed-off-by: Christopher Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If the worker segfaults, we may never see a TaskFailed event from it, only
a runQueueTaskFailed event. In this case, return_value isn't getting set
leading to an incorrect exit code from bitbake. Fix by setting return_value
in both places.
(Bitbake rev: e5dd50e0d95d532fe31dde61f8c6b1a7a72321e9)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If there is a missing provider and we're using "-k" mode alongside "-w",
we could get a traceback since there was no provider. Add tests to avoid this.
(Bitbake rev: 90a4805e4e770a433b4394ea99792731e9a4b546)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We're seeing issues where the self test, which uses tinfoil doesn't
notice the changed contents of include files. The issue is
cached_statements in the parser being reused when the files have changed.
Whilst looking at this, I realised there were some other issues:
* We need to also invalidate the mtime cache when cooker restarts
* We should pass full filenames to the file invalidation code
* We should process cached_statements as part of inotify invalidation
With these fixes, the caching is more reliable for memory resident
bitbake too. It does raise some questions about cache validation and
lifecycles and indicates bitbake does need more work in the area,
preferably with the removal of the globals. This at least highlights
and works around some of the current issues.
(Bitbake rev: 3f507ff8bc467fba936cf3f31bb8ea8e02f168e8)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
CookerParser.shutdown code doesn't do all required work to shutdown
parser processes. As a result bitbake hangs if interrupted during
parsing. Putting None into the parser_quit queue should fix this issue
as it makes parsers to quit main loop.
(Bitbake rev: f67307977e8f089ce6d208d3e9de2a6a1768757e)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The brbe variable is set on the bitbake server when the Toaster
UI starts. This enables Toaster to associate events with the
correct build and build environment.
However, the brbe variable is also used when a build starts to
identify whether a new build needs to be created, or an existing
one looked up. This causes a bug for command-line builds which
happen after a Toaster-triggered build: because the brbe variable
is never unset on the server or the buildinfohelper, the new
command-line build events are treated as originating from the
previous build.
Ensure the brbe variable is reset when the buildinfohelper "closes"
a build, so that each build then either sets the brbe variable
(Toaster-triggered builds) or leaves it blank (command-line builds).
Also modify the localhostbecontroller so that the brbe variable
is not set on the server and not looked up from the server. This
ensures that it is only set when the triggerBuild() method is
called, and that it remains as None for command-line builds.
[YOCTO #9021]
(Bitbake rev: 4a6a8d0074f62208d843b06344be31ae73d9b745)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Remove symlinks in the UI tests folder as they are causing
problems for bitbake upstream.
[YOCTO #8787]
Signed-off-by: Mihail Stanciu <stanciux.mihail@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If something fails in a exec_func_python() the current stack trace shows
incorrect filenames and linenumbers. For example:
The stack trace of python calls that resulted in this exception/failure was:
File: '/media/build1/poky/meta/recipes-sato/images/core-image-sato.bb', lineno: 200, function: <module>
0196: chksum = bb.utils.sha256_file(fn)
0197: f.write('%s\t%s\n' % (chksum, os.path.relpath(fn, baseoutpath)))
0198:
0199:
*** 0200:copy_buildsystem(d)
0201:
File: '/media/build1/poky/meta/recipes-sato/images/core-image-sato.bb', lineno: 9, function: copy_buildsystem
0005:IMAGE_FEATURES += "splash package-management x11-base x11-sato ssh-server-dropbear hwcodecs"
0006:
0007:LICENSE = "MIT"
0008:
*** 0009:inherit core-image
0010:
0011:IMAGE_INSTALL += "packagegroup-core-x11-sato-games"
File: '/usr/lib/python2.7/subprocess.py', lineno: 535, function: check_call
0531: The arguments are the same as for the Popen constructor. Example:
0532:
0533: check_call(["ls", "-l"])
0534: """
*** 0535: retcode = call(*popenargs, **kwargs)
0536: if retcode:
0537: cmd = kwargs.get("args")
0538: if cmd is None:
0539: cmd = popenargs[0]
The problem is the use of "FILE" to obtain the current filename. Instead,
we therefore inject the function being executed into the methodpool which
allows us to correct its linenumber and filename information. We can then
clearly mark the initial piece as autogenerated and the rest of the linenumber
and filename information should be correct. Afterwards the trace starts:
The stack trace of python calls that resulted in this exception/failure was:
File: 'exec_python_func() autogenerated', lineno: 2, function: <module>
0001:
*** 0002:copy_buildsystem(d)
0003:
File: '/media/build1/poky/meta/classes/populate_sdk_ext.bbclass', lineno: 66, function: copy_buildsystem
0062: import glob
0063: import oe.copy_buildsystem
0064: import subprocess
0065:
*** 0066: subprocess.check_call("foo")
0067:
0068: oe_init_env_script = d.getVar('OE_INIT_ENV_SCRIPT', True)
0069:
0070: conf_bbpath = ''
File: '/usr/lib/python2.7/subprocess.py', lineno: 535, function: check_call
0531: The arguments are the same as for the Popen constructor. Example:
0532:
0533: check_call(["ls", "-l"])
0534: """
*** 0535: retcode = call(*popenargs, **kwargs)
0536: if retcode:
0537: cmd = kwargs.get("args")
0538: if cmd is None:
0539: cmd = popenargs[0]
We can't inject into methodpool at parsing time, since there may be
_append or other override operations against the function before its
execution.
(Bitbake rev: fae153095d23157dd7e72c29f683f86149ee33a8)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Right now, if you have some python code like:
X = "a"
def somefunction(d):
d.setVar("X", "b")
d.setVar("Y", "${X}")
then any sane person would expect that Y = "b" at the end of the
function. This is not the case, Y = "a".
This is due to the python function being expanded before execution, the
executed code would read d.setVar("Y", "a"). This understandably
confuses people, it also makes it near impossible to write ${} in a
python function without unintended things happening.
I think there is general agreement we should fix this and standardise
on non-expansion of python functions. We already don't expand anonymous
python (mostly).
I've checked OE-Core with buildhistory before and after this change and
there were a small number of issues this exposed which I've sent
patches for.
I propose we default to not expanding python code and then deal with
any consequences from that if/as/where identified. This will improve
new user understanding and usability of the system, it also allows
several long standing weird expansion issues to be fixed.
(Bitbake rev: 8bf33a8e92c0e188fa392030025756196c96fcbb)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We don't want to expand python functions since they aren't expanded
at execution time (e.g. anonymous python). They can also have side
effects.
This function is primarily used by toaster for variable dumps for later
display. The lack of expansion of python functions won't matter in this case
and actively helps some variable handling (e.g. SRCPV).
(Bitbake rev: 3f5520b4844a4bdd615046479ba08ed192bdc8cd)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Expanding python functions for variable dependencies doesn't really make sense,
not least since this causes execution of any inline python, it also makes it
impossible to write expressions like d.expand("${X}") of d.setVar("X", "${Y}")
which may have the wrong values if expanded now.
This starts to standardise the approach across bitbake for handling python code.
(Bitbake rev: 765a2480dbe288f64562a9611dd93b6b6dd0a64e)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We don't expand anonymous python before execution, so nor should
we do this when calculating checksums for them.
(Bitbake rev: 5f10987edda35b08970a6dd6ccf9febad271ce3e)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Revert "toaster: models.py do not use transactions".
This reverts commit e248a503ba
(Bitbake rev: 48d0d510816346073f135bb86fb7904fdb50bb27)
Manually managing the database transactions caused more
problems than it temporarily solved so we return control back to
Django's orm.
[YOCTO #8796]
(Bitbake rev: 25c531915b6f8f79a0822996ceb97f90483e359f)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* Display always the first target in alphabetical order to match what we
do in the breadcrumbs and the build dashboard heading
* Remove the extra space between the '+' and the additional number of
targets
* Make sure the tooltip with the full target list takes the Bootstrap
tooltip styles
* Replace the word 'targets' in the tooltip with 'recipes', since that's
how we call build targets everywhere else in Toaster
(Bitbake rev: 3b8747d0af4b9164e973940ed97751c951e74110)
Signed-off-by: Belen Barros Pena <belen.barros.pena@intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The pages in the build data section of Toaster showed different
breadcrumbs: in some pages the machine was displayed, but not in others.
For builds with more than one target, some pages showed the first
alphabetical target (the correct behaviour), others didn't.
This patch removes the inconsistencies, showing exactly the same
breacrumb across all pages in the section.
The patch also removes the extra space between the '+' and the number of
targets when the builds have more than one target.
Remove an unneeded debug message
(Bitbake rev: 9cdbb543311b6f4a8a88c27fc157d998242444ee)
Signed-off-by: Belen Barros Pena <belen.barros.pena@intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The functionality overlap between these two functions is significant and
its clearer to handle both things together since they are intimately
linked. There should be no behaviour change, just clearer code.
(Bitbake rev: 391aa4afc91be90d8d3ee47e1bf797d6ebe61a71)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When we use functions from the data store, they now have correct line number
and filename information. This function would attempt to correct line numbers
which doesn't need correcting, leading to misleading messages to the user.
Therefore remove this code as being obsoleted.
(Bitbake rev: 918bec86bc8ee94feb82380ff410d9fdcbe9e720)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Extended the dot styling of dependencies created by bitbake -g in dot syntax to differentiate between the various kinds.
depends: solid
rdepends: dashed
rrecommends: dotted
The change observed is that depends get an explicit style which is the same as dot default behavior and the runtime recommends get
dotted while before they were dashed. This helps to distinguish them graphically as well as eases post processing by script.
(Bitbake rev: 86e78e0ca7aa5452411f35239942ecee3d8824ec)
Signed-off-by: Henning Schroeder <henning.schroeder@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add the dependencies of packages which are added to the
CustomImageRecipe. Currently just handle the first tier of dependencies
as this is what we show in the UI.
(Bitbake rev: 5c44609a9bf9fb23241b7dd7c58b08901d75008d)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Allow package groups in our custom image recipe. Excluding them creates
more undefined behaviour than including them at this stage. Also update
to use convenience method for returning all packages.
(Bitbake rev: 8c2e8a13badacb816c4b1178b6661600008b38af)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add link to the last successful build if there is one and add
conditionals for the ancillary recipe metadata.
(Bitbake rev: 4660aaf1c6775270f8f3d0afbb7fa2ee7a2a1563)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a method to update the packages included list from the last
build, this effectively "synchronises" the package list from what
we think will happen at the Customise image stage with what
actually was produced with a build. It's not ideal to have this
function here but we also need to make sure that no race condition
of the user accessing this list and it being updated occurs.
(Bitbake rev: 8cf6e67a955574b33856a082bdadf3194f2b6ba4)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Field to keep track of when the package list for the CustomImageRecipe
was last updated from a build.
(Bitbake rev: 4bd4e49f13a7625997a43f3b2e67ed42c3c8e08b)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a convenience method to get the last successful build
target for a CustomImageRecipe.
(Bitbake rev: 4dde3d830cd38bbe306d83629dcb80da5fc9b027)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In the dependencies popover just show direct dependency in the list
rather than recommends, conflicts etc
(Bitbake rev: e69b00532b011327bc2495a6fb52cfe98f0f897d)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This convenience method returns just the Package_Dependency for
the package which are regular dependencies i.e. not RECOMMENDS or
any other types.
(Bitbake rev: bd76c22fe2aa06690b4ee25de69219ac0bf6b4d6)
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Instead of keeping the original dependency information for the pool of
CustomImagePackage reset it with each new build.
(Bitbake rev: a0b97ffc7a468bad081ce3276c74728bf6830250)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add front end handling of validation response from create new
CustomImageRecipe api.
(Bitbake rev: eff66b502df8e001cd0abc25bcbd742687169619)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Check that the name for a new CustomImageRecipe doesn't already exist in
the project or in the database of existing recipes (e.g. from the layer
index). Also restrict the characters entered for the recipe naming
convention.
(Bitbake rev: f290d428460a07e73050ff613bc222cc8c04f5ec)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add the file size of the packages and the total to the JSON response.
(Bitbake rev: bbbd304c49b0940a695d15273934edff95d70836)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Whem a m2m field changes we need to clear the ToasterTables cache as
this can affect the state of items in ToasterTables.
For example the CustomImagePackages being added or removed from a custom
image recipe.
(Bitbake rev: c9d7b68ee0186a71e8e75a5d87122a0328001515)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Update the states of the packages in the package selection UI to reflect
whether it's likely that 1st level dependencies for the package will be
also added.
(Bitbake rev: 119569d83c3fb1d1bd162624819b3f9c63a791c4)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Instead of defining this as a string it's sufficiently large enough to
warrant its own file.
(Bitbake rev: 6b39423fe5a3ed30289a8b303329a5725f7d273b)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If we're breaking up an image recipe's packages we will also need to
override IMAGE_FEATURES to make sure the customisation is not altered
(Bitbake rev: 9fd7b05dc0cf9240f7c8e3dc77b009064fd2b0cb)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When the packages associated with a custom image recipe are
shown in the customrecipe editing page, locale packages are
shown in the same way as all other packages. This gives the false
impression that these packages can be removed, when in fact
they are automatically added due to the IMAGE_LINGUAS build
variable.
Modify the customrecipe page so that locale packages cannot be
removed, and provide some help text explaining why.
[YOCTO #8927]
(Bitbake rev: b2208e53c00a67a7d0345e7378e6806b8ae40fb4)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The custom image editing page doesn't show locale and packagegroup
packages: they are filtered out of the queryset used to populate
the ToasterTable.
Rather than filtering these packages out, include them in the list
of packages which are shown.
(Bitbake rev: 38a753e7b2e9ede326856b830b25e13bdd6d0d9b)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The custom image package selection filters were using the old
ToasterTable filter approach, which caused the table filter to fail.
Amend the table to use the new ToasterTable filtering API to fix
this.
(Bitbake rev: 72a4cb30842fd053e46dc56df222729cbe735162)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The progress updater for the recent builds section makes a JSON
call to the project view URL to get progress for each build.
However, conversion of the builds pages to ToasterTable broke this,
as the JSON response no longer contained the data necessary to
populate the progress bars.
Move the recent builds query to the Build model, so that it is
accessible to the ToasterTables using it ("project builds" and
"all builds"), as well as to the "project" view.
Modify the code in the recent builds template to use the slightly
different objects returned by the recent builds query on Build.
(Bitbake rev: 5189252635ddc7b90c9a43aaed9f196c31e1dcad)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When applying migrations, Django shows this warning:
"Your models have changes that are not yet reflected in a migration,
and so won't be applied."
This is because the customimagerecipe model has changed, but those
changes are not covered by a migration.
Add the missing migration to clear this warning.
(Bitbake rev: df8185fcbd84061976d91b03b2a9268b319a6184)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Django detects a conflict between a migration added to support
image customisation and another migration which supports PROVIDES.
Add a merge migration to resolve the conflict (as suggested by
Django).
(Bitbake rev: a26bfd9d2490dc0fd90bf6d1690e63ac26001559)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a special case for when the recipe we have based a custom image
recipe on requires another recipe.
In this case we need to adjust the file location to be able to
require
the recipe when we're in the toaster-custom-images layer.
For example:
"require core-image-minimal.bb" is changed to:
"require recipes-core/images/core-image-minimal.bb"
(Bitbake rev: 26025e1ea49b3ebfcfd508d1608fa8c9e722ad03)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Poll the server for the project build progress value. This is something
that will need to be re-done once we have a proper API for this on the
server side.
[YOCTO 8328]
(Bitbake rev: ec467e43c39eadf02412b89db10c09ed78a5a9f5)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We need to know the location of the based_on recipe's layer on the
file system before we try and generate the custom image recipe. As
we read the recipe to make the custom version.
(Bitbake rev: e6a7cacbddd1df5bac0b79384199cf7264c5bbd5)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Make sure we send the current list of layers to the customise button to
be able to know whether it should be set as an add layer button or a
customise button on the New custom image page.
(Bitbake rev: 5ddb35c98b609d85f97d482b54cabe3a2812afe6)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add additional data to the setUp to be able to test all the tables for
Image Customisation. Also add the name of the table being tested to the
num of rows assertion.
(Bitbake rev: dfcbcf789cf3f0733ca26b0601fdf97ce4291674)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Rename the recipe_id to custrecipeid to avoid confusion about which type
of object we're going to be accessing. This means that in the unit tests
for tables we can pass a different kwargs for custom recipes vs normal
recipes.
(Bitbake rev: ae3301a1047b3efb4b340b50a10d5d585b7333da)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Update test for adding and removing a package from a CustomImageRecipe
so that it uses the CustomImagePackage and correct fields for the packages
included. Change the test for error condition to use an invalid package
id as ManyToMany remove() on package that isn't in the relationship does
not throw an error.
(Bitbake rev: daccb2978f833a9e7af270160331da3e9a158219)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This adds some basic package dependency hint modals when you add and
remove a package. It also makes sure that if the CustomImageRecipe has
no current included packages that we go and check this with the server
to see if a relevant build has taken place which will provide this
information.
[YOCTO #8082]
(Bitbake rev: 418f5509e74d46d36a8eb966a245083006e5f4ba)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add response for GET to the xhr_customrecipe_packages ReST API
/xhr_customrecipe/<recipe_id>/packages/<package_id>
Thie response includes the id, name, version and dependency information
for the package.
(Bitbake rev: c45791fc85d26c43b0a3b0a459111d2ff5583540)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Instead of doing a shallow copy of the package into the
CustomImageRecipe when we add packages we can now use the
CustomImagePackage as a M2M field on the Package to CustomImageRecipe.
Also switch to using Target_Installed_Package as the method to retrieve
the package list from the build.
(Bitbake rev: 4ebc81823b3aec6ecf38835acad5263a81eb41c5)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When we create a CustomImageRecipe we create a Layer_Version and
Recipe for that Recipe to be in, we only need one Layer_Version for our
Recipes so if that Layer_Version is updated by building it we need
a slightly more custom version of get_or_create to take into account the
fields which we expect can change but still mean that the object we want is
valid and doesn't need to be created.
In the Recipe case this is when we're updating an existing
CustomImageRecipe as we allow people to create a recipe even when the
based on recipe hasn't been built so we need to update it once a build
has happened.
(Bitbake rev: 0fe2c72ab82c6de2825a390fbb460b892a7a9cfc)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This changes the SelectPackagesTable to use the ProjectPackage table
instead of very large expensive queries to retrieve a list of currently
available packages for the project.
(Bitbake rev: 4b4b7e28d602ac5283659f806d695cc0451d292e)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add the download recipe link and fix the package count field.
(Bitbake rev: 85891e5320014f363dba093ac2db681d55375ee3)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Use libtoaster.createCustomRecipe rather than own implementation of this
function.
(Bitbake rev: 74fa98752b1cf1ad18d35ab6dd25fe7e409133c5)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This adds the function to call the ReSt API to create a custom image
recipe.
(Bitbake rev: 03e7949f538733f682a05d0c318cf2f4cd64cbf5)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>