Commit Graph

4087 Commits

Author SHA1 Message Date
Alexandru DAMIAN 07b58c9734 bitbake: toaster: fix updates on failed build requests
The patch to fix the original mistake is wrong, in the sense
that a constant from the BuildRequest class is used on the build
object.

Fixing the patch to bring in the correct constant in.

(Bitbake rev: a740c17efc0ccc9f89a428ead08f9739af3c6de4)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-19 18:05:54 +01:00
Alexandru DAMIAN 242603e9b4 bitbake: toaster: replace raising Exceptions in loadconf
This patch replaces throwing Exceptions in toaster loadconf
command with proper explicit error messages. This allows
the user to properly understand and debug what went wrong when
loading a config file.

Additionally we change a bit the logic around auto-detecting relative
giturl handling so the user gets proper error messages when
trying to load an invalid toasterconf.json file.

[YOCTO #7945]

(Bitbake rev: 3d14cc033a855bf5b20e799438548b6d8f29d9b8)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-19 18:05:54 +01:00
Alexandru DAMIAN c159b3c117 bitbake: toaster: do not stop data import on bad data
The layer data update in Layer source needlessly stops
processing by throwing an Exception when bad layer data cannot
be inserted in the database (e.g. invalid dependencies).

This patch changes the behaviour to ignore bad data, and
continue with data import after detailed logging through the
logging system.

[YOCTO #7955]

(Bitbake rev: e0911e408db4fa07086738fe11121b173b4d1cf5)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-19 18:05:54 +01:00
Alexandru DAMIAN be4636aef2 bitbake: toastergui: rename variables in views.py
Renaming various variables in the views.py file to make
the code more readable, and prevent overshadowing of global
variables with local variables that have a different name.

(Bitbake rev: 5fe0c42ca9d34fa34b65b9fe0ff474200b78efe6)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-19 18:05:53 +01:00
Alexandru DAMIAN 12c9cca5b1 bitbake: toaster: use loggers instead of prints
Switching debugging from using print statement to using
loggers, as it uses the logging infrastructure to process
the messages.

The messages are logged with the "toaster" logger which
is defined in the toastermain/settings.py.

(Bitbake rev: adf3bdcbe8b0b0bbe9de2800f2d20a53e8d88543)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-19 18:05:53 +01:00
Alexandru DAMIAN d18e6303a0 bitbake: toaster: models field initialization
As prompted by pylint, the object fields need to be
initialized properly, so this patch fixes that.

Also adds some casts to int, because, sometimes, the IntegerField
is not returning an int, but a string.

(Bitbake rev: cc9ac01d0726203fbe916cec51cc464eaeae9305)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-19 18:05:53 +01:00
Alexandru DAMIAN 504396f6a3 bitbake: toaster: improve exception handling
This patch improves exception handling, fixing issues
as prompted by pylint, including exception renaming.

(Bitbake rev: 652fad64864703cb1bbeede8d5bfc4208974be68)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-19 18:05:52 +01:00
Alexandru DAMIAN 99730d64d4 bitbake: toaster: checksettings: fix TEMPLATECONF detection
Improving the TEMPLATECONF detection by verifying the return
code and dumping extra debug information in case of exception.

(Bitbake rev: bdc00755993aa37e7669c3859ef4ea9b1fc3e680)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-19 18:05:52 +01:00
Alexandru DAMIAN 3a1af07177 bitbake: toaster: checksettings: fix guesspath initialization
Object fields need to be instantiated in the constructor.
Fixed prompted by pylint.

(Bitbake rev: 40db75c6117c3226c27d278d2acf0cd3ba61a167)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-19 18:05:51 +01:00
Alexandru DAMIAN 98dfa27050 bitbake: toaster: pylint fixes
This patch fixes imports, default parameters to functions,
incorrect overloading and exception handling issues
highlighted by pylint. There are no functional changes.

(Bitbake rev: 066e096b80dcb0e93e1a088acf5d914184361769)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-19 18:05:51 +01:00
Mihail Stanciu c8df84777b bitbake: toasteruitest: Updated UI tests cases to use dom IDs and 1.9 UI changes
Updated existing tests to use dom IDs where necessary.
Updated tests to use 1.9 specific UI changes:
    - removed deprecated columns(log and layer directory)
    - changed expected names for table header("Recipe" instead of "Target")
Added tests for TC 942 and 943 which were missing
Removed test for TC 959 which was deprecated under 1.9

(Bitbake rev: d8c37780609571e95edef9a220225927eb94b174)

Signed-off-by: Mihail Stanciu <stanciux.mihail@intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-19 18:05:51 +01:00
Michael Wood 55f13d4556 bitbake: toastergui: Call csrf function in all pages to ensure cookie is sent
Not all pages call the csrf token template function so django does not
bother to send the cookike, this meant that all subsequent ajax calls
fail. So add this to the base template.

[YOCTO #8175]

(Bitbake rev: baed115081f888e027f92f37e12dc2dd2b874bf0)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-19 18:05:50 +01:00
Paul Eggleton ae7d04832b bitbake: bitbake-layers: show-recipes: allow filtering by class inheritance
Add a -i/--inherits option to filter the output to include only recipes
inheriting the specified class(es).

Implements [YOCTO #7475].

(Bitbake rev: e4ec622d9dc5f882049f5e8027331ec1e17681de)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-19 18:05:50 +01:00
Paul Eggleton a3967e2ba5 bitbake: bitbake-layers: fix mapping files to layers
bitbake-layers needs to map recipe and class files to the layer they
came from within the show-recipes and show-overlayed commands. However,
it turns out that mapping a file to the layer it came from is not as
trivial as it might seem. To do it properly we need to match the path to
an entry in BBFILES then map that to the collection name using
BBFILE_PATTERN, then map that to the actual layer using variable
history. If it doesn't match any entry in BBFILES, then we can fall back
to BBFILE_PATTERN (to handle classes and conf files).

This fixes the layer name not showing up properly in the output of the
show-recipes and show-overlayed commands for recipes in layers such as
meta-intel that have subdirectories in BBFILE_PATTERN. It also fixes the
priority not showing up in show-layers for such layers.

As part of this I've added a function to VariableHistory which for a
space-separated list variable gives you a dict mapping the items added
to the files in which they were added. I've also fixed
bb.utils.get_file_layer() and reduced some of the duplication by using
this function in bitbake-layers. Also fixes the priority not showing up
for layers such as meta-intel

Fixes [YOCTO #8160].

(Bitbake rev: e852f6cabd7489585477ab567a1afeb2252377ac)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-19 18:05:49 +01:00
Paul Eggleton 22e8c6cbd3 bitbake: bitbake-layers: remove-layer: accept just layer directory
If the specified layer isn't a path, then just match on the directory.

Fixes [YOCTO #7839].

(Bitbake rev: b4c45bf6c42b4d319ba868f4ce77e86c8b585818)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-19 18:05:49 +01:00
Paul Eggleton c4989e7bf3 bitbake: bitbake-layers: ensure we exit if BBLAYERS_LAYERINDEX_URL is not set
We were printing an error here, but not exiting.

(Bitbake rev: ddcaf8950a0b1cc74806e1ad7b49a1de0ea0d2b1)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-19 18:05:49 +01:00
Paul Eggleton 5807ab3266 bitbake: lib/bb/main: consolidate UI/server extension listing and loading
Provide us with a means of showing the list of UIs / server choices for
the command line help, and do the processing in one place for both.

(Bitbake rev: 24035c1daad5a904c3372d21d44191ee8182338f)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-19 18:05:48 +01:00
Paul Eggleton 9b05ef581c bitbake: lib/bb/parse: properly handle OSError when updating mtime cache
If a file no longer exists, drop it from the cache silently instead of
generating a traceback. This was visible in some cases when a recipe was
deleted when bitbake was resident in memory.

(Bitbake rev: fe105b9042bdac4afd9f38fcf92bfdc2c04ec23f)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-19 18:05:48 +01:00
Paul Eggleton 97067755b9 bitbake: bitbake-layers: use "with open" consistently
It's best practice to use "with open..." rather than open & close where
practical.

(Bitbake rev: 1ef38549cae5639f2c8bcc2b270c5c82a5e29e3c)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-19 18:05:47 +01:00
Paul Eggleton 90d97f7a57 bitbake: bitbake-layers: refactor show-appends to stop using cooker bbappends list
(Bitbake rev: 60a253555a3ebadea775cfdc3331cba78ee3e71b)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-19 18:05:47 +01:00
Richard Purdie 67a6a4d6c0 bitbake: bitbake-layers: Convert flatten to use collections.bbappends
flatten support currently looks broken since it doesn't appear to
deal with handling "%" support in bbappend file names.

This patch converts it to use collections.get_file_appends() which
correctly handles "%" support.

(Bitbake rev: 0448714c52bc1e9584a5282cffdcaa404fb0618a)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-19 18:05:47 +01:00
Paul Eggleton 3c3ff92882 bitbake: cooker: drop appendlist
Now we have the bbappends list and all users have been converted over to
use it, we don't need this anymore.

(Bitbake rev: 279770c42d4c63aa2cebce331b55a92a564b50ac)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-19 18:05:46 +01:00
Paul Eggleton b84bf58c19 bitbake: cooker: drop appliedappendlist
Whilst collecting this list on the fly may be quicker, doing so within a
function that's meant to *query* the list of bbappends is poor practice.

(Bitbake rev: 5c12aa3b0010d7d1733e54a0ca7d0af465454210)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-19 18:05:46 +01:00
Paul Eggleton e2ddc18874 bitbake: Fix -m handling if cannot connect to server
If we can't connect to the server we should error out, because it might
not be that the server is actually dead - it might just be unable to
execute commands.

(Bitbake rev: d4b921676859d6ba4e1922fa4682ee941652f483)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-19 18:05:46 +01:00
Paul Eggleton 7ce27b9fce bitbake: cooker: ensure prefile/postfile can work in memory resident mode
The prefile/postfile options weren't working in memory resident mode
because they weren't being passed through to the server, so ensure that
they do get passed through and that the server is reset when the values
come through.

(Bitbake rev: a3f7dc042fc7b1c308bfd248431930eb8ba50326)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-19 18:05:45 +01:00
Richard Purdie 37a2c062a5 bitbake: command: ensure sync commands that read configuration see updates
Add a means of ensuring that synchronous commands that read the results
of the configuration trigger a reparse of the configuration if any
underlying files have changed.

(Bitbake rev: aaf3cc024315450c1674819edf2a4e5cdf293f35)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-19 18:05:45 +01:00
Paul Eggleton 6e5ac6ba8e bitbake: cooker: further limit inotify watches
Unfortunately we were acting on inotify notifications about any files
changing within the watched directories, not just the ones we actually
care about. In OE the build directory is in BBPATH and hence it gets
watched, and we write things like bitbake.lock and
bitbake-cookerdaemon.log to that directory, hence effectively
notifications were being tripped on every bitbake invocation. To avoid
this, record which file/subdirectory we're interested in against each
watched directory so we can ignore any events for files/subdirectories
we don't care about.

Additionally, if we move up to the parent dir, ensure we haven't already
seen it before adding a watch on it (we were previously calling
watcher.add_watch() on the same directory multiple times before in a
typical OE configuration).

(Bitbake rev: bc39b8da34c046b629c43fd0a8cac2efbf1c060f)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-19 18:05:44 +01:00
Paul Eggleton e0daf78408 bitbake: command: intercept SystemExit to avoid trashing the server
If sys.exit() is called within a command run over XMLRPC, the XMLRPC
server is effectively trashed (apparently listening but no longer able
to respond to commands). We need to intercept the SystemExit exception
and deal with it as we would any other exception.

(Bitbake rev: 95e391acbc3b4efd6c77637a1ce815012ae0f09b)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-19 18:05:44 +01:00
Paul Eggleton 715d857174 bitbake: Fix default function parameter assignment to a list
With python you should not assign a list as the default value of a
function parameter - because a list is mutable, the result will be that
the first time a value is passed it will actually modify the default.
Reference:

http://docs.python-guide.org/en/latest/writing/gotchas/#mutable-default-arguments

(Bitbake rev: 7859f7388f2e3f675d0e1527cfde18625f36f637)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-19 18:05:44 +01:00
Ed Bartosh c348fa5018 bitbake: toaster: move code from setup_lv_tests to setUp
As ProjectLVSelectionTestCase was removed there is no point
to keep this function. Moved code back to setUp method.

(Bitbake rev: 8e9bd559c8ef0ebc9e8babbada06e710908bae08)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-17 14:41:45 +01:00
Ed Bartosh e5956a6b85 bitbake: toaster: remove ProjectLVSelectionTestCase
Moved code from ProjectLVSelectionTestCase.test_single_layersource to
LayerVersionEquivalenceTestCase.test_compatible_layerversions.

Removed ProjectLVSelectionTestCase.

(Bitbake rev: b322cb6f5fec0b979a646605ed0d7646764c07a7)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-17 14:41:44 +01:00
Ed Bartosh 5638321f03 bitbake: toaster: add header to orm/test.py
Added header and module docstring.

(Bitbake rev: f393d3b8943375d18ebd6bc5e00dec5f288b46ec)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-17 14:41:44 +01:00
Ed Bartosh 50f9e203bb bitbake: toaster: simplify testcase code
Replaced 2 asserts with one in test_single_layersource method
of ProjectLVSelectionTestCase.

(Bitbake rev: 54131366ac2a8fc7866fb1293f5cd0e3da13ea46)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-17 14:41:43 +01:00
Ed Bartosh 552b1feeef bitbake: toaster: remove duplicated code
Removed test_dual_layersource method from ProjectLVSelectionTestCase
as identical method presents in LayerVersionEquivalenceTestCase.

(Bitbake rev: 5611c6b911e8a638f462e80e924ac9265a2199f5)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-17 14:41:43 +01:00
Ed Bartosh 1f7d7bf216 bitbake: toaster: reuse common code
Moved setup code of LayerVersion tests to a function
and call it from setUp methods of 2 classes.

(Bitbake rev: 65121e71078499ba585cc6a42018339d7884322e)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-17 14:41:43 +01:00
Ed Bartosh 958696f560 bitbake: toaster: reformat LayerVersionEquivalenceTestCase
Reformatted to increase readability and satisfy pylint.
Added docstings.
Shortened variable names.

(Bitbake rev: 01e6fb74b60b1a66873f33866a35bdf6b2f807bd)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-17 14:41:42 +01:00
Ed Bartosh d637b91639 bitbake: toaster: rewrite LILSUpdateTestCase
Reformatted code. Added docstings.

(Bitbake rev: 0d1a9038b0cface75d3c858543e411f519b4ef5e)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-17 14:41:42 +01:00
Ed Bartosh 344ad2e128 bitbake: toaster: rewrite test for LayerSource model
Rewritten LayerSourceVerifyInheritanceSaveLoad class from scratch.

(Bitbake rev: 34ea4c661ee48e1986fe2375b94e5b1c5c16c8ee)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-17 14:41:41 +01:00
Ed Bartosh 11668e24e7 bitbake: toaster: fix test_build_layerversion test case
The reason of test failure is that Django querysets are
different even if they contain the same objects.

Fixed by converting querysets into lists.

(Bitbake rev: c1abc6f0905e4321668a483a3d5be7cef3c25401)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-17 14:41:41 +01:00
Ed Bartosh 77ca551813 bitbake: toaster: remove prints from the test code
Removed prints as they make test output harder to understand.

(Bitbake rev: 922d37bfee7cc75aa7a3fd5b76abc33df57d3704)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-17 14:41:40 +01:00
Mihail Stanciu 645435a645 bitbake: toastergui: Added IDs to elements used in testing
Added static IDs to elements commonly used in the testing process so
that future UI modifications would not break tests as easily.

[YOCTO #7380]

(Bitbake rev: 04e896d7fb170271fb09dae5c2a42acb4b68f513)

Signed-off-by: Mihail Stanciu <stanciux.mihail@intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-17 08:48:28 +01:00
Sujith H 91d8f88bb7 bitbake: toastergui: Add tests for xhr_importlayer
Inorder to make sure if layer name which we import is already
available in toaster database, a test case has been included.
This would help user to identify if layer name which has been
provided in the import layer web page already exists or not.

(Bitbake rev: 26e2b4a997503b82dc5535ce087565a0d6418141)

Signed-off-by: Sujith Haridasan <sujith.h@gmail.com>
Signed-off-by: Sujith Haridasan <Sujith_Haridasan@mentor.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-17 08:48:28 +01:00
Alexandru DAMIAN c9d1cb299a bitbake: toaster: fix invalid fields in the package details page
This patch safeguards referencing invalid fields in the
project details template, and fixes errors thrown in the
page.

[YOCTO #7992]

(Bitbake rev: 64b30bc15e87f81eddac79872b2282c435827ad0)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Belen Barros Pena <belen.barros.pena@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-17 08:48:27 +01:00
Belen Barros Pena 8ab2101166 bitbake: toaster: Fix typo in set up script output
Removing an spurious 'it' when setting the build directory.
Thanks to Scott Rifenbark for finding the typo!

(Bitbake rev: dd644a2c87e651c3990520759bab55a56ba110d1)

Signed-off-by: Belen Barros Pena <belen.barros.pena@intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-17 08:48:27 +01:00
Ed Bartosh a16e0b4014 bitbake: toaster: reduced amount of instance attributes
Used local variables instead of instance attributes in ViewTests.setUp

Fixed pylint warning:
    Too many instance attributes (9/7) (too-many-instance-attributes)

(Bitbake rev: 0c419902e5f58dcee9b7fbe9af1928da37e18399)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-10 13:58:02 -07:00
Ed Bartosh 5fe1c19644 bitbake: toaster: fix pylint warning 'no space allowed'
Fixed pylint warning "No space allowed before/after bracket".

(Bitbake rev: 0b0f360b3660373716189995f3d33ffbc1b21db5)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-10 13:58:02 -07:00
Ed Bartosh 27a483dcb3 bitbake: toaster: fix test_get_json_call_returns_json again
Fixed this test case again after rebase. Improved it
to check for all returned keys. Previously it was checking
only 3 of them.

(Bitbake rev: 2deef2e86a09b6cb907bfeff56774f889cfe30a1)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-10 13:58:02 -07:00
Ed Bartosh 3ba3cf22d6 bitbake: toaster: don't catch json.load exceptions
It doesn't make sense to catch json.load exceptions in the
testing code. If API throws traceback it's better to see
it instead of error message.

(Bitbake rev: 207cb1d16ab089995f23b6070be358870506aa62)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-10 13:58:01 -07:00
Ed Bartosh b3ab9b1211 bitbake: toaster: add header and docstings
Added header and docstrings to toastergui tests module.

(Bitbake rev: d46bc24e95fa27cee686b84fbd8c6407efa2635b)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-10 13:58:01 -07:00
Ed Bartosh 8ee14d59ed bitbake: toaster: remove unused import
Fixed pylint warning:
    Unused Build imported from orm.models (unused-import)

(Bitbake rev: 30585f7e37966e827861c25a0cc3bf9e3bd2f667)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-10 13:58:01 -07:00
Ed Bartosh 91fea094ab bitbake: toaster: get rid of class attributes
LAYER_NAME and RECIPE_NAME attributes don't make sense as
they're used only once.

(Bitbake rev: 86936224f49810147daa91ea2a050b0e7a82f106)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-10 13:58:00 -07:00
Ed Bartosh 8af82c207d bitbake: toaster: put all test cases into one class
Moved all functionality (one setup method and one test case) from
ProvisionedLayersProjectTestCase and XHRDataTypeAheadTestCase to
AllProjectsViewTestCase.

Renamed AllProjectsViewTestCase -> ViewTests.

(Bitbake rev: d87dc9a03bb2d1c21551894d611a4312254f46ef)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-10 13:58:00 -07:00
Ed Bartosh 2f8c8e4c31 bitbake: toaster: get rid of AllProjectsViewTestCase
Removed AllProjectsViewTestCase class as it had one method 'setUp'
and used as a parent for another class AllProjectsViewTestCase.

Moved setUp method to AllProjectsViewTestCase.

(Bitbake rev: b68e4d994eb825c1039fd90d197724b77aeebd5d)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-10 13:58:00 -07:00
Ed Bartosh 13e04177da bitbake: toaster: wrap long lines
Fixed pylint warning Line too long (line-too-long).

(Bitbake rev: 37f7588df9759481080db56462a271ca808ba9ff)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-10 13:57:59 -07:00
Ed Bartosh 9e3c0ee1c5 bitbake: toaster: fix test_get_json_call_returns_json test
This test case used wrong field name.
Fixed by renaming field 'lists' -> 'rows'.

(Bitbake rev: 534a302a413319d2573c894b1c1077ac0578bbb1)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-10 13:57:59 -07:00
Mariano Lopez 0b80ef5f8a bitbake: runqueue.py: Add provides to taskdepdata
Currently bitbake is the only one that knows the relation
between PN and PROVIDES. In some cases it is needed to have
this relation in the data store (the bootloader it's a good
case).

This adds the PROVIDES to the taskdata, so it would be in
the data store as one field of BB_TASKDEPDATA.

(Bitbake rev: a660787311d2961c66c0443bf0e2e094c9baef1b)

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-10 13:57:59 -07:00
Michael Wood 58197f345b bitbake: toastergui: layerBtn Fix build trigger mechanism
The new project page is no longer responsible for triggering the build so
add a handler for this in layerBtn which is used in the layerdetails and
layers pages. This also removes the conflicting and  redundant handler
for this in the layerdetails.

(Bitbake rev: b4c389443bdd121121fd1d1a9006a9f1f63f186f)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-06 16:04:50 -05:00
Michael Wood a65099eec3 bitbake: toastergui: Fix typo on layers in project filter condition
Should be & rather than "and" which was returning the truth'd value of
the and condition.

[YOCTO #8114]

(Bitbake rev: 58035667e433f2b77f5cd48240dd051c35476284)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-06 16:04:50 -05:00
Michael Wood 5194f0401e bitbake: toastergui: Move click disabled check after table ready event
We were attaching this handler before the elements were fully ready in
the dom. Which allowed these links although disabled to take users to
the change machine page.

(Bitbake rev: 8245a7d679914ca7d4548ab7e83b5047ff8a1ff5)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-06 16:04:49 -05:00
Michael Wood c1993d08cc bitbake: toastergui: Machine select button url parameters change
Update the url for changing the machine in the project page.

[YOCTO #8113]

(Bitbake rev: 6e46ad4aa404d80fded27db1fb072d35250cfcfb)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-06 16:04:49 -05:00
Michael Wood b12e51c352 bitbake: toastermain: Add a longer default database timeout
When using sqlite we sometimes see Database Locked exceptions when we
fire off database calls asynchronously from the UI. We need sqlite to
wait a bit longer for the lock to be released.
n.b In production setup we hopefully wouldn't be using sqlite.

docs.djangoproject.com/en/1.6/ref/databases/#database-is-locked-errors

(Bitbake rev: 7f3a4e0330f4de3ba8e092bc1c15a53d8c2be073)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-06 16:04:49 -05:00
Michael Wood af72d0fbb2 bitbake: toastergui: Move layerdetails view definition to the views.py
The layerdetails view definition was moved to tables though it isn't
a table. We have a mechanism for the JSON response for this page so use
this instead of a custom class.

(Bitbake rev: b5100bfd4cb32a9b59b0554c6dbc3a4295da7d8e)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-06 16:04:48 -05:00
Michael Wood da4c66385d bitbake: toastergui: Fix toastertable table header reference
A header id was mistakenly added to the table template which was not
also added to the simple version of the toaster template. We don't need
this id so remove it.

(Bitbake rev: daf902e2a6f736b9c8ef6492143fdbf856451559)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-06 16:04:48 -05:00
Michael Wood e616c201a5 bitbake: toastergui: libtoaster typeahead Add in results highlighter
Add a highlighter for the results in the typeahead, this highlights the
part of the results which is currently being matched. This is a modified
version of the bootstrap stock function with added escaping and the
addition of the details information.

(Bitbake rev: 3a6bad55a84d8d374a23f488ce42fed9b927c4f2)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-06 16:04:48 -05:00
Michael Wood aad380028f bitbake: toastergui: libtoaster: typeahead resiliency for slow server
When we have a slow request we don't want to see the typeahead results
changing in the middle of typing the item. To do this we make sure that
requests to the typeahead can only happen sequentially and that if
results have been retrieved but the input is now empty we don't bother
showing the results.

(Bitbake rev: 0b508e8b4a9e25c223c6c11ecd6d0e1aab727e8c)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-06 16:04:47 -05:00
Michael Wood 0494a2769b bitbake: toastergui: tests Fix and more comprehensive typeahead unittest
Fix the currently broken unit test for typeaheads and add some
additional fields to check for in layers, projects, recipes and machines
typeaheads.

(Bitbake rev: 5700cd269e3a4fd517e16a3a19e61a23091df465)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-06 16:04:47 -05:00
Michael Wood 4d7944d6ad bitbake: toastergui: views Standardise the fields project layer response
We need to keep consistent the field names of these objects so that we
can use the object response from different calls. e.g. layer dependences
or listing layers in the project.

(Bitbake rev: 5c7004908f6916c0d279299c47b9dca3237148ec)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-06 16:04:46 -05:00
Michael Wood 951e40dd7d bitbake: toasterui: views Remove unused xhr_typeahead view definition
The one thing left being used in this definition was a response which
contains the list of layers which would be deleted if you change the
project release. This patch moves that to it's own url/endpoint and
updates the frontend reference which is using it.

(Bitbake rev: 1cc19c84ee97182f39eae0338c712f7a2b40a18d)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-06 16:04:46 -05:00
Michael Wood 1e7707b63f bitbake: toastergui: libtoaster Throw an exception no url is specified
At a minimum for this typeahead to work we need a url parameter to call
for a JSON reponse of items to filter on.

(Bitbake rev: 80b214d93c5df63a251b807ca93f81f00c6bfeb2)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-06 16:04:46 -05:00
Michael Wood aff29d2cd8 bitbake: toastergui: Switch to using the new toaster typeahead widget
Switch the existing typeahead inputs to use the new typeahead widget.
This means we have a defined mechanism and end point for typeaheads
which meets the design specification.

(Bitbake rev: 31a8ae7909347f7b6edde5bbdf02b86dc1b32ed0)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-06 16:04:45 -05:00
Michael Wood 73367c2ca8 bitbake: toastergui: Add typeaheads layers, machines, projects, recipes
Implementation of typeahead widgets for layers machines and recipes
typeahead.

[YOCTO #7152]

(Bitbake rev: 913d01758564db2f5fae4451bf0fdca38a1b3d61)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-06 16:04:45 -05:00
Michael Wood f4e51fff4a bitbake: toastergui: widgets Add a typeahead widget
The typeahead behaviour is significantly different from searching in a
table so we need a separate widget to do this.

[YOCTO #7152]

(Bitbake rev: 195c5407a9de29d97f2525b9ae6c827afb934e37)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-06 16:04:45 -05:00
Michael Wood 68128ad2f1 bitbake: toastergui: tables Use util functions for the common querysets
Use the new utils functions on the project object to get the common querysets
that we also use in tables.

(Bitbake rev: 5e013a9bc3deb03bd6bac357b9ecb81c3c77b255)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-06 16:04:44 -05:00
Michael Wood c4e7113dd0 bitbake: toaster: orm Add util functions to return common querysets
We use these querysets when creating tables of results and also when we
want to have a typeahead search. These can also form the basis of future
API endpoints.

(Bitbake rev: 2a10fecd985343802f0e99c6fff25c28980eee20)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-06 16:04:44 -05:00
Michael Wood d3b9927a3c bitbake: toaster: orm Add util function to return the url to layerversion
Save duplicating this call and make a utility function for it on the
Layer_Version object.

(Bitbake rev: 32d382315bad37cebbe67ca3287085f3d856a9ed)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-06 16:04:43 -05:00
Belen Barros Pena c8bc7ee6ed bitbake: toastergui: add action to empty "most built" recipes
It is not very good to just tell people: "this is empty because you
haven't done anything". We should politely invite them to do something
about it. Thus adding a "Choose a recipe to build" link to the all
compatible recipes page whenever the 'most built recipes' section is
empty.

(Bitbake rev: 8d2ed32cd09cf6cb96521fd42789faae99656fad)

Signed-off-by: Belen Barros Pena <belen.barros.pena@intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-06 16:04:43 -05:00
Belen Barros Pena 1a0fa6cbfb bitbake: toastergui: fix spacing in the import layers page
Adjust the space between the first fieldset and the tabs,
and the help text and the first form field.

(Bitbake rev: 68086d260842c12e5459c719f383b045fb0b3c2d)

Signed-off-by: Belen Barros Pena <belen.barros.pena@intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-06 16:04:43 -05:00
Belen Barros Pena 0378aadd58 bitbake: toastergui: remove project name from layer change notification
The project name is always a link to the basic
configuration page, which looks quite silly when
you change layers from that page.

Since I guess the idea is reusing those notifications
everywhere, I've replaced the project name with a
generic 'to your project' / 'from your project', so
that they read right no matter where you see them.

(Bitbake rev: 59bcafa712b9a16b647bd26b3bd738ae6fc21eaa)

Signed-off-by: Belen Barros Pena <belen.barros.pena@intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-06 16:04:42 -05:00
Belen Barros Pena 69c6f1fd13 bitbake: toastergui: change link styles in alert-info
Make them a bit less bold: it was a bit in your face
before.

(Bitbake rev: fd9e6d262aa3fdb2377c30ebac54096453a3b9f9)

Signed-off-by: Belen Barros Pena <belen.barros.pena@intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-06 16:04:41 -05:00
Belen Barros Pena 569567d3b3 bitbake: toastergui: breadcrumb fixes
Make sure that we use the correct html entity
for the breadcrumb dividers, and remove the
'all builds' item from the history pages: all
breadcrumbs should now have the project name as
their first element, since the 'all builds' page
can be reached using the global naviation in the
top bar.

(Bitbake rev: c97f658f4b2131f280c364c2209efd28878570dc)

Signed-off-by: Belen Barros Pena <belen.barros.pena@intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-06 16:04:41 -05:00
Belen Barros Pena d47358fc95 bitbake: toastergui: spacing and top nav style change
Remove the margin for the .section class and the
.row-fluid class in the page header div to
reduce the vertical white space a little
bit in the project pages.

Also, change the style of the Toaster manual
link to make it look like the main navigation.

(Bitbake rev: 7b539bbf25786d313fd8092dc96b186388df58aa)

Signed-off-by: Belen Barros Pena <belen.barros.pena@intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-06 16:04:40 -05:00
Richard Purdie a8b723498c bitbake: cooker: Resolve file monitoring race issues when using memres bitbake
If you load memory resident bitbake, then change bblayers.conf or other
configuration files, then execute your first command, bitbake will not
notice the changes. This is because it adds the file watches during
inital parsing, which happens at the time the first command is run.

To fix this, we move the addition of the file watches to earlier in
the process, so bitbake then does track them correctly. This also
avoids some issues Paul was seeing with tinfoil2.

(Bitbake rev: b9375c73e736003e66575969c9ea244403e47aeb)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-03 07:36:25 +01:00
Christopher Larson 22078d5e53 bitbake: bb.parse: properly error out on filesystem errors
We've had a long-standing bug where a legitimate error reading a file (IOError
or OSError) is always suppressed as though it was a 'file not found' case. As
a concrete example, if you do a `chmod 000 conf/local.conf`, it'll silently
not parse local.conf, rather than erroring to let the user know about the
problem.

Fix this by handling the ENOENT case specifically.

(Bitbake rev: e691312a3add222b04e7b2f52f8df6abcb9068bf)

Signed-off-by: Christopher Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-01 22:24:20 +01:00
Christopher Larson 75fc6bf0a0 bitbake: bb.cookerdata: don't show traceback for ParseError/ExpansionError
Tracebacks are of extremely limited usefulness in this context. The exceptions
carry the necessary context already, and the user doesn't care about the calls
in bitbake internals that led to an expansion or parse failure.

(Bitbake rev: 9b95fa94eaae452ac7814f1e67c2f7a6314c52f1)

Signed-off-by: Christopher Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-01 22:24:19 +01:00
Juro Bystricky e4a4c9b1ac bitbake: bitbake-user-manual-fetching.xml: note about URLs with semi-colons
Some URLs contain semi-colons. Unfortunatelly, bitbake uses semi-colons in SRC_URI
as delimiters for various parameters.
It may still be possible to use such URLs, providing the user replaces semi-colons ';'
with '&' characters.

For example:

   http://abc123.com/git/?p=gcc/gcc.git;a=snapshot;h=a5dd47

 and

   http://abc123.com/git/?p=gcc/gcc.git&a=snapshot&h=a5dd47

should be equivalent, as W3C recommends treating ';' in queries identically to '&'.

See http://www.w3.org/TR/1999/REC-html401-19991224/appendix/notes.html#h-B.2.2

Kudos to Olof Johansson

(Bitbake rev: d8953c16c52c6e5de879f4a32ec8aeee396bb410)

Signed-off-by: Juro Bystricky <juro.bystricky@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-01 11:31:00 +01:00
Ed Bartosh 7f4eb55d70 bitbake: bitbake: toaster: Add is_image field to the Recipe model
Added new field is_image to the Recipe model.
Made sure is_image is populated when layer sources are imported.
Made sure the change doesn't break loading old style layer sources.

Tested by loading data from layers.pythonanywhere.com and
from layers.openembedded.org.

[YOCTO: #7571]

(Bitbake rev: 3ac0ba73790b8f68567daa2de313c78053411fbb)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-01 11:29:35 +01:00
Ed Bartosh 9be8ea632c bitbake: bitbake: toaster: Uncomment logging messages
Uncommented debug and warning messages in _shellcmd method of
LocalhostBEController as they seem to be useful for debugging.

(Bitbake rev: 9446f02520a3bee4417908d8da2ab1848f4759c3)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-01 11:29:35 +01:00
Ed Bartosh 79c8ffa5d3 bitbake: bitbake: toaster: get rid of _createdirpath function
Replaced call of recursive _createdirpath method with simpler
call of os.makedirs.

(Bitbake rev: 018442e2645390342d43a95a8685e51b29ea868a)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-01 11:29:35 +01:00
Ed Bartosh 3b6e5df2aa bitbake: bitbake: toaster: Fix usage of wrong variables
Replaced nonexistent variable 'be' with self in
models.py/BuildEnvironment methods.

Fixed typo: BuildRequest.TYPE_LOCAL -> BuildEnvironment.TYPE_LOCAL

(Bitbake rev: ed6094f78c75aab776a82967101d9c57e38e2c4d)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-01 11:29:35 +01:00
Ed Bartosh a403ddbbba bitbake: bitbake: toaster: Wait for toaster gui to come
Set logging level for the Toaster logger to get messages
to toaster_ui.log

Reverted previous workaround.

[YOCTO: #7965]

(Bitbake rev: f1d8e0ff7233f17cff3a56db10d2443b392b407d)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-01 11:29:35 +01:00
Michael Wood e68370cbf1 bitbake: bitbake: toastergui: Toastertables don't replace dash in cache key
The dash character was being stripped from the cache key when we sanitise
it for the memcache backend. This meant that we were getting a false hit
on the cache and returning non descending results which are indicated by
the dash prefix on the field name.

(Bitbake rev: 18743274a94966d2cd8d17d163e4dbab501a8a52)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-01 11:29:35 +01:00
Alexandru DAMIAN 6520feb4ab bitbake: toaster: toastergui: run pylint on the toaster files
This patch enables pylint running on the toaster files, showing
only the errors. This enables spotting common mistakes in the
toaster python files.

(Bitbake rev: d2e69721233ab3cbbc3bdd6a1c198fde8a1f287e)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-01 11:27:55 +01:00
Alexandru DAMIAN 1da5e0a793 bitbake: toaster: tts: execute tests in numeric order
As the tests are verifying different type of the functionality,
it is usually the case that a failing early test will completely
make the subsequent tests failing, e.g. if the system cannot
start due to a bug, there is little point in testing other
functions.

In order to prevent uneeded test runs, and to generate repeatable
test patterns, the test cases have now a numeric order in the
class name (e.g. Test01XXX). The tests are executed in this order,
and the first test failing will stop the test run.

(Bitbake rev: 639c46a08e524902018e28367fcb4e26362cd3e3)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-01 11:27:55 +01:00
Alexandru DAMIAN e6419ccd93 bitbake: toaster: tts: make sure to import data
Importing data from the layer index should be part of the setup
because otherwise we have no reliable database configuration.

This patch selects the first option for importing the database
configuration file.

(Bitbake rev: 86a69d294ace41bd109db97b753554ccc8f3dac0)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-01 11:27:54 +01:00
Alexandru DAMIAN ada494d3dd bitbake: toaster: tts: delete the database only if created
To allow the tests to run on different databases (in case of
manual runs), the HTML5 test will only delete the database
file if it was created in the setUp.

(Bitbake rev: 2b04165f6a70fad5beb8e4c7053a2b2053a51cde)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-01 11:27:54 +01:00
Alexandru DAMIAN ef70e137b2 bitbake: toaster: tts: delete pyc files before the tests start
There may be leftover *.pyc files from previous code runs in the
current checkout directory, so we delete everything before
starting the tests, to make sure that we actually testing the
current checkout.

(Bitbake rev: db29ef9ab23c9f083e9aa175b28430f122f35a84)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-01 11:27:54 +01:00
Alexandru DAMIAN f9eb958a8a bitbake: toaster: tts: fix startup and cleanup for managed mode
This patch fixes the toaster startup and database cleanup
for managed mode. It is needed because the toaster script
thinks it has been called with "source" instead of being
executed as independent script if not called through bash.

(Bitbake rev: f08ce41f1137f268049ef8345462aa58b233e50b)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-01 11:27:54 +01:00
Alexandru DAMIAN ba667a0908 bitbake: toaster: tts: improve logging in urlcheck.py
This patch improves logging in the urlcheck.py.
It allows properly running just HTML5 tests outside the
test suite.

(Bitbake rev: 91566d2ea8ca5a696ce742b9e5e3b7b6c10c200c)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-01 11:27:54 +01:00
Alexandru DAMIAN faaa5d45d8 bitbake: toaster: tts: improve debugging information
We add debug information, and exception handling as to make
it easy to figure out what is going on when tests fail on
automatic start.

(Bitbake rev: d7c5989b795566f8611208b26851871abccf9578)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-01 11:27:54 +01:00
Alexandru DAMIAN f169733738 bitbake: toaster tests: gitignore and use absolute path for log file
gitignore the cache directory created by the http client

the log file for tests is already set up as an absolute path,
so no need to recompute the path

(Bitbake rev: 80f525e5cbe83e0407ecddf84401d68213c6d5cf)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-01 11:27:52 +01:00
Alexandru DAMIAN 73f14975c5 bitbake: toaster: tts: clean up HTML5 compatible url list
This patch cleans up the list of URLs that are tested for
HTML5 compatibility using the w3c-validator. It removes
the URLs that are no longer in use, and the AJAX-only URLs
that return JSON content.

(Bitbake rev: c85a35e37150ee8be091bd142cac8a1885f1dff0)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-01 11:26:13 +01:00
Alexandru DAMIAN 1640a65091 bitbake: toaster: tts: fix pylint warnings
This patch brings TTS to the pylint coding standards.
Pylint was run with some disables:

disable=logging-too-many-args,line-too-long,missing-docstring

and achieved

Your code has been rated at 10.00/10

There are no functional changes.

(Bitbake rev: 2b40b412ff6a7e3fd4cc32707bd3cd713bc09ddb)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-01 11:26:13 +01:00
Michael Wood 160d610604 bitbake: toastergui: Remove angularJS and projectapp based on angular
AngularJS proved unsuitable for the new project page implementation.

(Bitbake rev: 6f25846dbcaf007f58f9a32e58dcd564f6430fe9)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-01 11:26:13 +01:00
Michael Wood 829fe32621 bitbake: toastergui: project page Add inline project name change feature
This feature allows the project name to be changed from where ever it is
displayed in the project pages.

[YOCTO #7329]

(Bitbake rev: 81cb1cd1aa900f29a25d14dce93d301cd6b6fdb7)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-01 11:26:13 +01:00
Michael Wood c76137e25b bitbake: toastergui: layerBtn use libtoaster for change notification
Use the common functionality for change notifications.

(Bitbake rev: 79af72e1b80b033a37992095b2e97449de5ebd8b)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-01 11:26:12 +01:00
Michael Wood b90a253fa6 bitbake: toastergui: newbuildbutton Fix removed elements
All the elements for the new build button need to be present for the
build button to work. Now that the template variables all have to have
values we have to make sure that the elements are still added to the dom
for the js to manipulate them.

(Bitbake rev: e2ab67eaf76da9ee2009e8420d5584c3daa97ac1)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-01 11:26:12 +01:00
Michael Wood 11627fc10f bitbake: bitbake: toastergui: newbuildbutton Fix typo on project save function
project.id rather than project.pk.

(Bitbake rev: 047df808203f3779243eb88c069757f7a7b7bbed)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-01 11:26:12 +01:00
Michael Wood c385f1551b bitbake: toastergui: Change build button popover to a build-button
To share the build-button mechanism from the pop over and project
topbar build button we now use a build-button class to reference them
and share the event handlers.

(Bitbake rev: e3c6a00b848a6f147e93c8da46b8a3516499fde8)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-01 11:26:12 +01:00
Michael Wood 10ffe1b8cc bitbake: toastergui: base Use removeAttr / attr
Use removeAttr and attr calls to enable and disable the build button and
build input field.

(Bitbake rev: c3753948bd00c3498ca3e76cb218eb6cf423c3e3)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-01 11:26:12 +01:00
Michael Wood be2b4239f6 bitbake: toastergui: Fix Duplicate layer importlayer api calls
As the JSON response changed the duplicate layer functionality wasn't
wasn't working as the field names in the returned data changed. This
patch brings the field names back in sync with the response.

(Bitbake rev: bdea86ae8720ee5792a5617c6073d51922a224ff)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-01 11:26:12 +01:00
Michael Wood 1a2f12099c bitbake: toastergui: libtoaster: Add guard against missing name property
Add a debug warning if a JSON payload isn't compatible. i.e. without a
name field.

(Bitbake rev: e65de693670203ddcbf5f23ad2a91c523df879d8)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-01 11:26:12 +01:00
Michael Wood a9549e839d bitbake: toastergui: libtoaster Make sure we always pass format=json
The new API for typeahead requires that we pass this parameter in to make
sure we get a JSON response.

(Bitbake rev: 2f8951d6e640d86f605b082aab4a950dab9065ad)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-01 11:26:12 +01:00
Michael Wood 691d547178 bitbake: toastergui: css Remove redundant css rules
The machine change notification animation is no longer required in
favour of the global overlay notification.

(Bitbake rev: aef36f2674d3d82b1757c02ffbf8b72fd4f32a4e)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-01 11:26:12 +01:00
Michael Wood 80c1586bbb bitbake: toastergui: Add new project page and navigation
This brings in the new project page design and improved navigation. As
this also removes the dependency on Angular it also required that the
entry points to the project page such as machine-change notifications
are also updated.

[YOCTO #7329]

(Bitbake rev: 6489e6eb5c3b0d59063b6d60521fc33fe563e707)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-01 11:26:12 +01:00
Michael Wood 258c929973 bitbake: toastergui: css Add new rules for new project page
Add new css rules needed for new project page.

(Bitbake rev: 6457d8fa30ac2b9b7413b8e9a7c5806bace10141)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-01 11:26:11 +01:00
Michael Wood 9af7ce513c bitbake: toastergui: http api Add guard against incomplete change requests
If we get a request to the project change api which either contains the
current configuration (i.e no change) or empty configuration handle this
gracefully.

(Bitbake rev: cc6c606c07baadc849c3290b23c56ad4e719fba2)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-01 11:26:11 +01:00
Michael Wood 7a589c0b20 bitbake: toastergui: tables Add name field to layers table
This field is required by the typeahead in the new project page to do
the name matching on.

(Bitbake rev: dbce3b43094b0a123b0d63aa07cc4f9547630094)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-01 11:26:11 +01:00
Michael Wood 8ff56b0fb3 bitbake: toastergui: libtoaster Add show change notification function
Now that we have a change notification that is global move this
functionality to libtoaster for shared use.

(Bitbake rev: 9d760462de99cb8729b7e0db773e77d4cd508733)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-01 11:26:11 +01:00
Michael Wood 1b13222e1c bitbake: toastergui: Standardise the layer object property layerdetailurl
To be able to use the utility functions in libtoaster  we need to have a
single property name for the value that represents the url for the
layer details in the layer object.

(Bitbake rev: 4d64fd0d1236d342b29537f601a68cd23a8255ea)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-01 11:26:11 +01:00
Michael Wood df999d2ac1 bitbake: toastergui: Remove erroneous CSS property
Remove a CSS property value which is missing a corresponding key.

(Bitbake rev: 770008f8ff3f8aa4f90636621a4162d676a71da7)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-01 11:26:11 +01:00
Michael Wood dd8c78b07a bitbake: toastergui: importlayer Add format type to JSON calls
After API was changed we now need to pass the parameter "format=json" as
a get parameter when requesting a json payload. Otherwise we get the
html version of the page instead of the data we need.

(Bitbake rev: 4545c21b70998bd240c3a21de05e8d4642044119)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-01 11:26:11 +01:00
Michael Wood d826d4e800 bitbake: toastergui: base Fix html indentation
(Bitbake rev: 37364e671ad1897bef818239a8e289b3d4c93925)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-01 11:26:11 +01:00
Michael Wood b06a633f25 bitbake: toastergui: Implement new project navigation
Change the structure of the project page to include a navigation menu
and top tab navigation. Remove old breadcrumb method.

[YOCTO #7329]

(Bitbake rev: 66fa0dd988e01ec79e74be7a5697eaa3b4f017d8)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-01 11:26:11 +01:00
Michael Wood 3d3a2dbf5f bitbake: toastergui: Implement new top navigation design
Additional items for top level navigation. This is part of a wider
navigation redesign.

[YOCTO #7329]

(Bitbake rev: 456a4a6782d3098cc9abb39659a619fdcf52eb01)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-01 11:26:10 +01:00
Alexander Kanavin 8acab25832 bitbake: wget.py: fix incorrect regexes
[\.-_] means "any character between . and _"
What was meant here is certainly "any character from the three characters .-_"

(Bitbake rev: af13eaba627f199f91c048c435b9dbe19c79527f)

Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-01 07:34:09 +01:00
Richard Purdie 4c2f28bf48 bitbake: cooker: properly fix bitbake.lock handling
If the PR server or indeed any other child process takes some time to
exit (which it sometimes does when saving its database), it can end up
holding bitbake.lock after the UI exits, which led to errors if you ran
bitbake commands successively - we saw this when running the PR server
oe-selftest tests in OE-Core. The recent attempt to fix this wasn't
quite right and ended up breaking memory resident bitbake. This time we
close the lock file when cooker shuts down (inside the UI process)
instead of unlocking it, and this is done in the cooker code rather than
the actual UI code so it doesn't matter which UI is in use. Additionally
we report that we're waiting for the lock to be released, using lsof or
fuser if available to list the processes with the lock open.

The 'magic' in the locking is due to all spawned subprocesses of bitbake
holding an open file descriptor to the bitbake.lock. It is automatically
unlocked when all those fds close the file (as all the processes terminate).
We close the UI copy of the lock explicitly, then close the server process
copy, any remaining open copy is therefore some proess exiting.

(The reproducer for the problem is to set PRSERV_HOST = "localhost:0"
and add a call to time.sleep(20) after self.server_close() in
lib/prserv/serv.py, then run "bitbake -p; bitbake -p" ).

Cleanup work done by Paul Eggleton <paul.eggleton@linux.intel.com>.

This reverts bitbake commit 69ecd15aece54753154950c55d7af42f85ad8606 and
e97a9f1528d77503b5c93e48e3de9933fbb9f3cd.

(Bitbake rev: a29780bd43f74b7326fe788dbd65177b86806fcf)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-01 07:34:09 +01:00
Andre McCurdy 1b88b7426a bitbake: build: Exit scripts with result of last shell function
Since shell scripts run with 'set -e' the final exit at the end of the
script can only be returning 0. However, for correctness and to follow
the original intention of the 'cleanup' commands, let's fix the typo
and return the success of the last shell function rather than the
success of unhooking the exit trap handler.

(Bitbake rev: bef724057f1ea81571dd3ac5a9cf862f818434b5)

Signed-off-by: Andre McCurdy <armccurdy@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-27 23:29:16 +01:00
Richard Purdie f5f6748b75 bitbake: data_smart: Improve override history logging
Calling record() for each override alteration is slow. Since we now expand
overrides dynamically we don't have to record the log data at each alteration,
we can instead print it directly from the existing data stores at variable
history print time using the exact same data stores.

This massively improves performance of the data store when parsing
with bitbake -e for example, it will improve memory overhead as well.

The only downside is that VariableHistory has to poke into the datastore
for some of its data but that seems an acceptable tradeoff rather than
double caching.

(Bitbake rev: 100b447a161ef20fa559e39516cd32fa78e38262)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-24 23:28:35 +01:00
Richard Purdie 9901415ecd bitbake: data_smart: Improve performance of infer_caller_details()
As things stand now, bitbake -e (which turns on all the caller tracking)
of OE-Core generates around 9.5 million stat calls which is slow and the
largest single thing on the profile data.

This is because infer_caller_details() calls traceback.extract_stack()
which adds line contents to the traceback. This in turn calls python's
internal linecache code which calls stat on every file for every callback.
We don't even use that info. We only even want a single frame of the stack.

Instead, open code for the pieces of information we need. Also, only
obtain the stack once for both halves of the infer_caller_details()
code.

This reduces the number of stat calls to around 0.5 million and significantly
improves parsing with bitbake -e.

(Bitbake rev: 7be76d8f79ea92fd4bd36146eb9a4b86551b526d)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-24 23:28:35 +01:00
Richard Purdie 67c6cc854f bitbake: data: Clean up datastore accesses and True/False values
We should use the d.xxxVar syntax rather than the older function style.

Also replace 0/1 with the more pythonic True/False.

No functionality changes.

(Bitbake rev: 90fdd69cca951f8bd2ff634f3b42fccd4fc03095)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-24 23:28:34 +01:00
Andre McCurdy fdfb3fdc5b bitbake: data: Add newline after the last export in emit_func
Minor formatting improvement in generated shell scripts.

Add a newline after the last export emitted by emit_func. Also, remove
chance of putting an extra newline in the middle of the exports, since
we never want to do that.

(Bitbake rev: 5117d9b5d32d5d81adf70fa3e3feac9cef654240)

Signed-off-by: Andre McCurdy <armccurdy@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-24 23:28:34 +01:00
Richard Purdie ada930055d bitbake: tinfoil: Add shutdown method
One drawback to tinfoil is that once a cooker is created, it will hold
the cooker lock and stop any other bitbake execution against a directory.

Add a shutdown method to tinfoil, allowing other users to use
the build directory after the tinfoil usage is no longer needed.

(Bitbake rev: e44ce85fe551677fc0dcc1da4f789a0c13093ff1)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-24 23:28:34 +01:00
Richard Purdie 901be2cb69 bitbake: bitbake-selftest: Add -v option for verbosity
Also document BB_SKIP_NETTESTS=yes parameter in --help output.

(Bitbake rev: 5196bfa9639eed2b1e6452f45775551203f8eeb4)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-23 08:48:41 +01:00
Richard Purdie aaea533e70 bitbake: tests/parse: Add BBCLASSEXTEND multiple data store corruption reproducer
One data store changing a variable poked through into a different data
store. This test case replicates that issue where the value 'B' would
become unset/disappear.

We also enhance parsehelper to generate files with an optional suffix
such as bbclass.

(Bitbake rev: 5c4179f58a4e04f1c354df5f17d1860eb403f0ac)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-23 08:48:41 +01:00
Richard Purdie 19f290f223 bitbake: data_smart: Add CoW approach for overridedata
Using deepcopy() caused a major performance regression. Turns
out we can work with a shallow copy as long as we force the
recreation of any list that we change, effectively a poor mans
CoW. This isn't too invasive and avoids the huge overhead of
deepcopy so this seems like the best way to have performance and
a working data store.

(Bitbake rev: 32dff4749c32f32e947c42c86f8357b8b607354b)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-23 08:48:41 +01:00
Aníbal Limón 205c39bf78 bitbake: tests/fetch.py: Updated test name FetchMethodTest -> FetchLatestVersionTest
Change the test name to be more specific on what is tested.

(Bitbake rev: 3e39156bc330c4c726058a5b9c13d33e2daad89f)

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-22 08:23:50 +01:00
Aníbal Limón 22d2f7b1ca bitbake: fetch2/wget.py: latest_versionstring now returns (version, revision)
Now latest_versionstring method returns (version, revision) for comply
the new return convention needed by SCM's like git get the current
revision.

bb/tests/fetch.py: Updated wget latest_versionstring test for comply new
convention.

[YOCTO #7605]

(Bitbake rev: 8d454646cbe1b04758ca178d8c6fcfd02b818b7b)

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-22 08:23:50 +01:00
Aníbal Limón bd5167d154 bitbake: fetch2/git.py: latest_versionstring now returns (version, revision)
We need to know the revision associated with the upstream version in
SCM like git, this change broke return convention, oe-core recipeutils
get_recipe_upstream_version need to be updated.

tests/fetch.py: Updated git latest_versionstring test for comply new
convention.

[YOCTO #7605]

(Bitbake rev: fd175dc90024c503134c11cbd83e77d88c406ac8)

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-22 08:23:49 +01:00
Aníbal Limón 4bbc5dd6e7 bitbake: fetch2/git.py: latest_versionstring search in all tags
Don't limit the tag search for only tags end with ^{}.

(Bitbake rev: 7006ab313766344cf33481228465082ed5977d28)

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-22 08:23:49 +01:00
Richard Purdie f5206e85b3 bitbake: data_smart: Use deepcopy for overridedata
dbus sets RDEPENDS_dbus-ptest_class-target = "X". nativesdk as
BBCLASSEXTEND was expanding and setting various RDEPENDS variables
which were clearing overrides queued against RDEPENDS. The problem
was that this was leaking into other data contexts such as the
target case. This was because overridedata was a shallow copy.

Replacing the shallow copy with a deep copy avoids this problem of
leakage between the data stores at a small performance penalty.

(Bitbake rev: 83e2923cd3c8f2e25987132f85b06ce72bc941d7)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-21 23:37:22 +01:00
Belen Barros Pena 9a92500306 bitbake: toastergui: make our tables Chrome-happy
Make sure that Toaster tables adapt nicely to the
viewport width in Chrome without squashing the
buttons in the right column (when they are there).

(Bitbake rev: 61307693a25a4a2eeeab5c450df330229638ddad)

Signed-off-by: Belen Barros Pena <belen.barros.pena@intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-21 15:36:14 +01:00
Brian Avery 0c0ec86110 bitbake: toaster: continue processing layer dependencies despite errors
Sometimes, a layer dependency will not exist,
  or not exist for a particualr branch. In this case
  we need to continue processing and hope for the best.

[YOCTO #7955]

(Bitbake rev: 2961bf2612fce111a7be6cef285595a075200780)

Signed-off-by: Brian Avery <brian.avery@intel.com>
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-21 15:36:14 +01:00
Paul Eggleton d3809b77c4 bitbake: lib/bb: provide mechanism to bypass UI log suppression
The recent change to connect through the shell logging functions had an
unexpected side-effect - bb.error() and bb.fatal() cause a flag to be
set internally such that BitBake's UI will not print the full task log
on failure; unfortunately we have in places within the OpenEmbedded
metadata called these shell logging functions under error situations
where we still want to see the full log (i.e., the message we're sending
doesn't include the full error). Thus, provide a mechanism to fatally
exit with an error but unset the flag, utilising the built-in python
logging functionality that allows extra values to be passed in the log
record.

(Bitbake rev: e561b997c55e8537d82aa1339adfff4505cc38b7)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-16 15:09:26 +01:00
Paul Eggleton 5261c5b5bd bitbake: lib/bb/process: check output of select() before reading extrafiles
We're calling select() to find the fds that have data available, so we
really ought to check the return to see if the extra file is in there
before trying to read from it. This is part of the fix for the
performance regression that this code introduced (5-10 minutes extra in
a reasonable size OE build); the rest is down to an issue in the way
that pseudo currently handles FIFOs and will need to be addressed there,
see: https://bugzilla.yoctoproject.org/show_bug.cgi?id=7993

Solution suggested by Richard Purdie <richard.purdie@linuxfoundation.org>

(Bitbake rev: 3e2db8d7eaa0f14813213d1c95d3ee9efd97dc9d)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-16 15:09:26 +01:00
Richard Purdie a730981b75 bitbake: tests/data: Add new data tests
Add a variety of tests which were found to be useful when working
on the data store recently.

(Bitbake rev: 5c5f8da509f6bbc1fad263462142519ef3d54a35)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-12 22:50:46 +01:00
Richard Purdie 4c386e1dd5 bitbake: data_smart: Fix appendVar/prependVar
Now that overrides get expanded 'on the fly', change appendVar
and prependVar to work using _append and _prepend, else we'd have
to re-implement pieces of getVar and the timing of expansions
becomes problematic.

Using _append/_prepend equivalence gives the behaviour users likley
expect from these functions.

(Bitbake rev: 40d661aaf7a563c6447b073310c5f2fdae6ca3d0)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-12 22:50:46 +01:00
Richard Purdie 75a58555ab bitbake: data_smart: Fix data expansion cache issues
The expand cache can be cleared in the middle of getVar now
due to the use of operations like delVar. We therefore need
to check if variables are in the cache before accessing in
case it was cleared.

(Bitbake rev: a97ce216cfe44136f742383542954bfce027831e)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-12 22:50:46 +01:00
Richard Purdie a259968611 bitbake: data_smart: Fix keys() missing overrides
d.keys() was not returning items which had no base content but the
variable came into existance through overrides. We have to process
self.overridedata to find these other variables.

(Bitbake rev: 77fc57c610d7f65f7e328ad555f525b2f8bdf1d7)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-12 22:50:46 +01:00
Richard Purdie 816acb6ee8 bitbake: data_smart: Cache set(self.overrides)
This performs better than continually regeneratiing the set().

Also only use set comparisions when its necessary to save some overhead
as issubset() is slower in the single item case.

(Bitbake rev: db9a4eedcf78969ec50bf0e4a8defba8ff0daa4a)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-12 22:50:46 +01:00
Richard Purdie 0ac9c80e08 bitbake: data_smart: Improve override recursion handling
When expanding OVERRIDES, its possible someone might try and override a variable
that is used in OVERRIDES. This could lead to infinite recursion. Add in
guards against this.

(Bitbake rev: 07d773369f571028c2cf82dd1f65d9731af6d00e)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-12 22:50:46 +01:00
Richard Purdie 2dbb067eba bitbake: data_smart: Tweak OVERRIDES value cache for performance
Updating the value of OVERRIDES whenever it changes turns out to be
extremely expensve/pointless. Instead, clear its value and re-establish
the value when we're going to use it.

This gives significant speed back.

(Bitbake rev: 41cf8d0c92d2d8a33fdad0921e424a0024914be1)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-12 22:50:45 +01:00
Richard Purdie d8ebb4708b bitbake: data_smart: Seperate out the override cache
Using an internal flag for override mapping turns out to be slower
than is optimal, not least as we don't want the keys list to list
variables that have no value other than a potential override expansion.

Maintinaing a seperate cache is therefore more optimal from a speed
perspective.

(Bitbake rev: 1a39ccf66b30b54e1274669cb0e1dc8fc72dba49)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-12 22:50:45 +01:00
Richard Purdie c690d5b818 bitbake: data_smart: VariableHistory: Ignore override duplicates and overlap with CoW functions
Trying to look up a variable called 'copy' in COW is problematic due
to internal implmentation details, at least avoid tracebacks from this.

Also don't duplicate override history (which is an atrefact of changed
override behaviour) as otherwise the bitbake -e output is convoluted.

(Bitbake rev: dddff5b7b8e6c18515b43389cef35503468b843d)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-12 22:50:45 +01:00
Richard Purdie 933e19697b bitbake: parse/ast/data_smart: Add parsing flag to getVar/setVar
When parsing we find problems if we clear prepends/appends when
setting variables during the initial parsing phases. Later, we actively
want to do this (in what would be post finalisation previously).

To handle this, pass a parsing flag to the operations to control
the correct behaviour for the context.

(Bitbake rev: ae87f5b8bf16191b3201cfb445062938eab992a0)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-12 22:50:45 +01:00
Richard Purdie 95be16f428 bitbake: data_smart: Move override handling to getVar
Compeltely remove the replaces functionality and move all overrides
handling to getVar time. We can move the cookie cache into a hidden
flag within the variables themselves.

This removes the need for any of the internal_finalize steps.

This obsolete the need for the _seen_overrides COW cache.

(Bitbake rev: 2a0b73110bede91777ada54d1d89b45fb6034b6c)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-12 22:50:45 +01:00
Richard Purdie 7fc6f6c98d bitbake: data_smart: Cache overrides and fix data store
Rather than repeatedly expanding the value of OVERRIDES, cache
the value and update it when things change.

There were also some bugs introduced in the replaces functionality
which this approach fixes by ensuring the replaces data is updated
at the correct points.

(Bitbake rev: f3b7c3e054ce230ea5c2db5813390383e8dfd6db)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-12 22:50:45 +01:00
Richard Purdie 35a6f7d21f bitbake: data_smart: Fix cache clearance problems
These write operations should clear the expand cache since they can
influence returned variable values but currently do not. Fix this.

(Bitbake rev: a075332c9e13be35f1ae84adc6b29e9137a487ff)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-12 22:50:45 +01:00
Richard Purdie 577f160400 bitbake: data_smart: Improve override handling
Rather than the heavy lifting in internal_finalize, move the bulk of
the functionality to getVar and rely on a new internal replaces variable
to keep track of any mappings that are needed. This removes the need
for the COW _special_values cache.

This change in functionality also implies we need to track any changes
not only to OVERRIDES but also any variable which OVERIDES depends upon.
Add code to handle this.

Explicitly list FILE as a value OVERRIDES depends upon since it doesn't
automatically get detected and is of key importance to OVERRIDES,
otherwise PN doesn't update when FILE changes.

(Bitbake rev: a6f1377ce3386d274882072d1ae6da3b1834149b)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-12 22:50:45 +01:00
Richard Purdie e7ccd90712 bitbake: data_smart: Remove need for update_data calls
Move the update_data functionality into internal data store operations
so the main finalize (update_data) call is a nop.

To make this work we need to call the internal finalization function
whenever OVERRIDES is changed to ensure values get updated correctly.

This has performance issues but the subsequant patches look into this.

(Bitbake rev: 546d9932d6c4413824319a9d780c0d77d2725f4a)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-12 22:50:44 +01:00
Richard Purdie 42a59961fa bitbake: data_smart: Defer append/prepend handling
(Bitbake rev: b1ce9975ef96f2506042832f4518cde73f6be917)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-12 22:50:44 +01:00
Aníbal Limón 5ae52eec3a bitbake: fetch2/wget.py: checkstatus disable SSL cert validation.
Since Python 2.7.9 ssl cert validation is enabled by default
see PEP-0476, this causes verification errors on some https
servers so disable by default.

(Bitbake rev: e177170200ece76b36e3f7d5597651fdef67736f)

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-12 22:50:44 +01:00
Aníbal Limón 73b9cd9764 bitbake: tests/fetch.py: FetchCheckStatusTest add cases for ftp and https.
(Bitbake rev: 36f2577d075f87090766877473f9030e44a941a2)

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-12 22:50:43 +01:00
Aníbal Limón 804f1a2312 bitbake: fetch2/wget.py: checkstatus fix using proxy handler
Only use ProxyHandler opener when exists proxies in env.

(Bitbake rev: 0f062c5d99e12ce20c0e46a8e602448032144ff6)

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-12 22:50:43 +01:00
Aníbal Limón 913631c466 bitbake: tests/fetch.py: Add FetchCheckStatusTest tests.
Add tests for checkstatus method using http with/without
connection cache.

[YOCTO #7796]

(Bitbake rev: b1f2d3edefb7dd274174eb983666213b0f49c994)

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-12 22:50:42 +01:00
Aníbal Limón 3b186fba59 bitbake: fetch2/wget.py: Add support of connection cache in checkstatus.
fetch2/__init__.py: Add connection_cache param in Fetch __init__.

In order to pass connection cache object to checkstatus method.

[YOCTO #7796]

(Bitbake rev: 9fa6407e6cefe66c77467419a8040d6957a6bb01)

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-12 22:50:42 +01:00
Olof Johansson 97c5ecffaa bitbake: fetch2/sftp: Set BatchMode=yes as argument to sftp
Set BatchMode=yes instead of PasswordAuthentication=no. This will make
sftp fail immediately, not only when SSH requires interactive
authentication, but also on errors related to host key verifcation.

(Bitbake rev: 31305853a177735cc9c4553ea8905cd0acfcb100)

Signed-off-by: Olof Johansson <olof.johansson@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-12 22:50:41 +01:00
Robert Yang 518abb731c bitbake: git.py: create a branch when checkout
* Create a branch and named as upstream branch when checkout source
* Set the branch to track remote branch.

(Bitbake rev: 1ba20e4fe9c884515b200589fe379ad5eeda10bd)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-12 22:50:41 +01:00
Richard Purdie 61fbff2cd4 bitbake: cooker/taskdata: Make NoProvider errors non-fatal to -e/-g options
If you have a situation where you are getting a "Nothing PROVIDES" error
(for example when something you request to build DEPENDS on something
that has been skipped or doesn't exist) it would be useful to be able to
use bitbake -g or bitbake -e to debug it, but currently both of those
are blocked by the error.

This patch adds an "allowincomplete" option to taskdata and uses this
for the -e/-g bitbake options. The NoProvider errors are still
printed and bitbake does return an error exist code but the environment
and task graph files are generated.

[YOCTO #7623]

(Bitbake rev: 2ca36a9f088438a8d1db44119c704f9480b04298)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-12 22:50:41 +01:00
Richard Purdie 625a6bf6e8 bitbake: prserv/db: Document history modes
I keep having to dig into the archives to remember this information.
Add it as a comment to the file instead.

(Bitbake rev: 21dce82056887d8d28edde61b1b82f78bdf7613c)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-12 22:50:40 +01:00
Paul Eggleton 3b50312307 bitbake: lib/bb: set up infrastructure for shell message functions
Create a fifo under ${T} for each running task that can be used by the
task to send events back to BitBake. The purpose of this is to allow
OpenEmbedded's logging.bbclass (which provides shell function
equivalents for bb.warn(), bb.note() etc.) to have those functions
trigger the appropriate events within BitBake so that messages are shown
through the BitBake UI rather than just in the task log; to do that we
just call the python functions.

Part of the fix for [YOCTO #5275].

(Bitbake rev: bae330a1f8a59a912eca71177b3c6ba7c92a2f38)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-12 22:50:40 +01:00
Aníbal Limón f9ea480e6c bitbake: fetch2: Add fetch parameter to checkstatus
In order to pass connection cache object to checkstatus function
add fetch parameter.

(Bitbake rev: fbb9c6f5538084e125b58118a86968908e6f895b)

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-09 18:00:21 +01:00
Aníbal Limón bf6c21c38b bitbake: fetch2/__init__.py: Add FetchConnectionCache class
FetchConnectionCache class acts as a container for socket connections
useful when implement connection cache into fetcher modules.

(Bitbake rev: 454da2cd17539ceb9caad6d76f034757e44ee12f)

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-09 18:00:21 +01:00
Leonardo Sandoval 3b5b74b430 bitbake: knotty.py: Make sure bitbake.lock is unlocked before exiting
Before exiting the UI, unlocks the bitbake.lock owned by cooker; this
way consecutive bitbake executions can lock it again without trouble.

[Yocto #7941]

(Bitbake rev: 69ecd15aece54753154950c55d7af42f85ad8606)

Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-09 18:00:21 +01:00
Leonardo Sandoval 67eaefee29 bitbake: command.py: Unlock function included into CommandsSync class
Function which calls cooker's unlock method, which in turn unlocks bitbake.lock
file.

(Bitbake rev: e97a9f1528d77503b5c93e48e3de9933fbb9f3cd)

Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-09 18:00:21 +01:00
Leonardo Sandoval 7bccf59a24 bitbake: cooker.py: Lock/Unlock members function into BBCooker
(Bitbake rev: d66dccf9f9a33bfef5c28cc1c767bfc89faee532)

Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-09 18:00:21 +01:00
Richard Purdie 25702f3a2f bitbake: cooker: Improve DATE/TIME handling
Currently, once set, DATE and TIME in the metadata remain unchanged.
This is suboptimal for cases where cooker is reused. This ties setting
the variables into the BuildStarted event which seems like a more
appropriate time to do so.

It also changes BUILDNAME to be based off DATE/TIME by default if not
already set so that the data is more consistent. We therefore need to
expand the value rather than the previous default of not doing so.

This change does mean the date/time values are in sync across all
variables too.

It does mean bitbake now has special knowledge of DATE/TIME but that
would seen unavoidable (other than doing this in event handlers which
has its own set of downsides).

[YOCTO #5187]

(Bitbake rev: f883cf240266ee7be2cbd8971a8164cf4df9e372)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-08 00:06:08 +01:00
Ross Burton 18d71ed16e bitbake: main: add skeleton documentation for the recipe:do_task syntax
The incredibly useful recipe:do_task syntax on the command line isn't documented
at all.  This isn't much but it's better than nothing.

(Bitbake rev: 7f4c07886ecff4ac77fdd2165bedd179099fcf19)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-07 16:18:34 +01:00
Felipe F. Tonello 62caaa4d6d bitbake: fetch2: Checkout to correct ref begore init and update submodules
This is nessary when specified branch with submodules is different then
default (master) branch.

[YOCTO #7771]

(Bitbake rev: f7b0b5e33e00f3ce0744322eee93835ee76bf184)

Signed-off-by: Felipe F. Tonello <eu@felipetonello.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-07 13:47:32 +01:00
Michael Wood c86a0d290f bitbake: toaster: Fix build execution regression
Make sure the error value is initialised and update the string match
that we have to identify bitbake's current startup status.

Patch contribution from Eduard Bartosh

(Bitbake rev: 915ba08a8a3013e9787e564f2ffd8698c948f433)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-02 22:56:39 +01:00
Michael Wood 56066872ae bitbake: toaster: runbuilds Fix incorrect variable name
The recent refactoring moved this enum value to BuildRequests table
rather than Build.

(Bitbake rev: 4338ab03826df3353e1222f0bd825dfdd04dd933)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-02 22:56:39 +01:00
Michael Wood 483b21b0cb bitbake: toaster: Remove erroneously committed template include
Partial revert of bb commit 6e46e1e3882b9770872d8a0bb459bc7d5d6bfed3

The non managed mrb_section defined the title 'All builds' which is already
defined correctly for the projects page as 'All projects'

(Bitbake rev: 3500b59cbfd006a83bab9431f434f98c1944a0ac)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-02 22:56:39 +01:00
Michael Wood cfc4f1b343 bitbake: toaster: Revert the addition of analysis project mode selection
Partial revert of bb commit 2efc338cefd6e6e097af83d7dff63e9ba177d021

This feature was not ready to go live as the surrounding implementation
work to make this usable has not yet been done.

(Bitbake rev: 40c8829369628927944c8a950524d482a663913e)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-02 22:56:39 +01:00
Michael Wood 3b3ef9cee2 bitbake: toaster: Partial Revert "fixes after html5 compliance testing"
Partial revert of bb commit 7aecb974d44d9bc711ffba5cc65e770811397fba

There were changes in this commit which broke the compatible layers
mechanism meaning that no layers, recipes or machines were showing.

(Bitbake rev: 124157b567469639c6c839233f42049a093ba083)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-02 22:56:38 +01:00
Richard Purdie 2578ff3dfe bitbake: codeparser: Add repr() methods
These aid when debugging parts of the codeparser cache since the
object contents becomes identifiable.

(Bitbake rev: 344b098c7eafc2bcc5c6b44ea47985bc0cb446b5)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-27 22:51:21 +01:00
Richard Purdie dcbbe545bf bitbake: codeparser: Allow empty functions
The main parser and other code copes with empty python functions but
the python codeparser does not. Fix this to avoid errors with code like:

python do_build () {
}

which currently results in the obtuse:
"Failure expanding variable do_build: IndexError: string index out of range"

[YOCTO #7829]

(Bitbake rev: e4f594c670189e04d58ce7d160fc1d86123620af)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-27 22:51:20 +01:00
Robert Yang 43c6e8c0dd bitbake: bitbake: fetch2/hg.py: fix unpack error and mirror tarball
Fixed:
* do_unpack error:
  abort: repository DL_DIR/hg/vim.googlecode.com/hg/vim not found!

* The mirror tarball doesn't work
  - Add the build_mirror_data to create the tarball
  - Unpack the mirror tarball when needed

* The hg files will put in the dir like git: DL_DIR/hg, it was
  DL_DIR/hg/path/to/src/uri/path in the past.

(Bitbake rev: 378647281ae883dd726f1e1b775dd35ef6a0e8d1)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-27 22:51:20 +01:00
Richard Purdie 3836d9b4ab bitbake: fetch/perforce: Fix single file checkouts
Fetching directories with p4 worked but single files did not. This
patch from Helmut Auer (helmut.auer@harman.com) fixes that issue.

[YOCTO #7891]

(Bitbake rev: 39da6579901c62a83ed9319c2016c58fbbc108fe)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-27 22:51:20 +01:00
Richard Purdie 8c3dd2d9eb bitbake: event: Handle recursive events and the data store better
Events can call each other recursively, e.g. an event handler can call
bb.note which in turn generates another event. If these loop, it
can lead to multiple deletions of 'd' from __builtins__ which
can fail since __builtins__ is global scope.

Add handling to only remove 'd' when we added it and it wasn't already
present.

(Bitbake rev: b45952650ce8f470f124df36185b79e0d3a1783a)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-26 09:27:34 +01:00
Richard Purdie 00ca43be05 bitbake: cooker: Fire BuildCompleted before finishing the command
Some handlers hook on BuildComplete so it avoids certain event races
to finish the command after the BuildComplete event is sent out.

This means the UI is available to handle events until the command
completes.

What appears to be a race on one of the sanity tests for event handlers
triggered this change although the failure is hard to reproduce.

[YOCTO #7921]

(Bitbake rev: e42d7c47a06fbb5981e0313478c8e3656b99f4e7)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-26 09:27:33 +01:00
Michael Wood d93bc4cf48 bitbake: toaster: Add url pattern for backward compatibility
This adds an url to match the old orm application.

(Bitbake rev: 0a8e740e18333da981b24a76db4c891845e5df78)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-26 09:27:33 +01:00
Michael Wood df48243970 bitbake: toaster: split orm app into it's own module and app
The orm application was also the django application to collect the build
information. Splitting this module up into it's functional parts. orm
for the data module and bldcollector for build collection data.

(Bitbake rev: 8ca10764ffd6cfec12cbfeabf240d81213a07845)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-26 09:27:33 +01:00
Michael Wood 62288a6689 bitbake: toaster: bldcontrol Ignore toasterconf files in own directories
If toaster has previously cloned poky or a layer with a toasterconf
file we don't want this to be picked up as a choice as these clones are
"internal" to toaster and may have undesired effects on toaster's setup.

[YOCTO #7741]

(Bitbake rev: 2f74ff59f2873c1cf121f7a385f3ab971c793a9c)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-26 09:27:33 +01:00
Michael Wood 10e35ab742 bitbake: toaster: Restore 'in project' filters to main tables
Restores the previously removed filters from recipes, machines and layers
table. These filters allow filtering down the results to just displaying
the current added layers, machines or recipes in the project.

[YOCTO #7851]

(Bitbake rev: 04a3e4614aae8ba794a3b8ac9083e723de7ca522)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-26 09:27:33 +01:00
Michael Wood 4013784b45 bitbake: toaster: table.js fix filter visual indicator and interaction
- Fix the id of the filter modal dialog.
- Fix the visual indicator toggle button
- Add check to see if the number of items to filter on is !=0 if it is
  then don't allow applying this filter.

(Bitbake rev: a73f08c79077b6ded0c08ea7d36032f4568d12e3)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-26 09:27:33 +01:00
Michael Wood d21e1749c8 bitbake: toaster: toastertable Pass up the kwargs for setup_filter
This allows us to setup_filter in tables using args such as the current
project.

(Bitbake rev: e0d59c32a7cff76d732a90cc6d507c7a9b68cb84)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-26 09:27:33 +01:00
Michael Wood 93c54c2f9d bitbake: toaster: Enable toastertable cache
Enable the cache and fix warning on the toastertable cache which happens
on the memcache backend "CacheKeyWarning: Cache key contains characters that
will cause errors if used with memcached"

(Bitbake rev: 330692e4e2e4dea59f70caa03001905ffff4860a)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-26 09:27:33 +01:00
Alexandru DAMIAN f28116ec10 bitbake: toasterui: fixes after html5 compliance testing
This patch brings fixes for issues highlighted by
HTML5 compliance testing.

(Bitbake rev: 7aecb974d44d9bc711ffba5cc65e770811397fba)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-26 09:27:32 +01:00
Alexandru DAMIAN 583e320764 bitbake: toasterui: verify variable before usage
This patch verifies that BRBE is set before trying to use
it to read the checkout paths. This is needed for builds
ran outside Toaster control.

(Bitbake rev: e04807cd3135c9de96cc7f79245f329c24618b85)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-26 09:27:32 +01:00
Alexandru DAMIAN 8ef5165b5a bitbake: toaster: delete multiple builds
This patch fixes the build deletion on unmigrated databases,
and enhances it to delete multiple builds in a single run.

[YOCTO #7726]

(Bitbake rev: d5468d84c1ef83c780de5974c8e3a11eab762489)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-26 09:27:32 +01:00
Alexandru DAMIAN 5d0abf197a bitbake: toastergui: select project types
This brings in project types in the New Project page.

The "analisys" projects are the projects with no "release"
set, and have read-only properties in the project page.

(Bitbake rev: 2efc338cefd6e6e097af83d7dff63e9ba177d021)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-26 09:27:32 +01:00
Alexandru DAMIAN 70c4eb8d3a bitbake: toaster: refactor build model
We remove the "timespent", "errors_no" and "warnings_no" fields
in favor of computing the needed values at runtime. This prevents
inconsistencies in the UI.

Also removeing all references to BuildRequests from the interface -
all build details now display in the build dashboard.

Minor fixes related to data logging.

(Bitbake rev: 44f37394ed3e4ca02f940be172fe4395b0ee0f7d)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-26 09:27:32 +01:00
Alexandru DAMIAN 287b49a35b bitbake: toaster: bring django-aggregate-if into the project
We bring the django-aggregate-if module (with minor changes to import)
into the project.

(Bitbake rev: 4b92add34167304b45c66b9726cbc64bc0f74d9f)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-26 09:27:32 +01:00
Alexandru DAMIAN 35fe3117ab bitbake: toaster: add django-aggregate-if
Import library that provides conditional aggregates
in Django. The tests are removed as they conflict
with the Django startup.

The sources are
https://github.com/henriquebastos/django-aggregate-if

Licence is MIT

(Bitbake rev: 016b416fd0eaeab648a588053a2ee1f41bc02a84)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-26 09:27:32 +01:00
Alexandru DAMIAN 335c975650 bitbake: toaster: fixes after replacing BuildRequest with Build
This is a set of fixes that repair the interface after
we switched from displaying BuildRequest data to Build data
in the formerly "managed" mode.

(Bitbake rev: 57f790b0c56297af8c83d5def8461bd5d61fe4af)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-26 09:27:32 +01:00
Alexandru DAMIAN d7c8d9558c bitbake: toaster: fill in build data from buildrequest
This patch adds logic to complete changing the interface
from showing BuildRequests to showing Build data.

The BuildRequest data is now transformed in Build data with
proper Toaster exceptions being recorded instead of listing
problems during startup as build errors.

(Bitbake rev: 51a41172d0b390370f9a38696b1ac65666ada4d2)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-26 09:27:32 +01:00
Alexandru DAMIAN 160563532f bitbake: toaster: refactor the builds pages
Taking out the managed mode-specific bits in build-related
pages, as there is always only one mode available.

Also refactors the build pages in order to always display
Build objects instead of BuildRequest objects.

(Bitbake rev: 6e46e1e3882b9770872d8a0bb459bc7d5d6bfed3)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-26 09:27:31 +01:00
Alexandru DAMIAN 2c7ed96b56 bitbake: toaster: remove BuildRequest references
In the toastergui application we should not display
implementation details about how the builds are run.
This patch removes the references to BuildRequest on the
majority of the views (except Builds page itself, as
that is more complicated).

(Bitbake rev: ead45a600813a1d8594e604875803ded781f83d8)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-26 09:27:31 +01:00
Alexandru DAMIAN c362e61ee2 bitbake: toaster: remove MANAGED references
We conflate the managed and analysis modes by
deleting alternative code paths, favouring the MANAGED mode,
always considering the MANAGED variable True.

(Bitbake rev: 0ac02d3775106b485e29496b62f31e91fd3f9387)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-26 09:27:31 +01:00
Alexandru DAMIAN b98a2fcab6 bitbake: toaster: fixing undefined variables
This patchset fixes usage of undefined variables in
the base page.

(Bitbake rev: a9e889af41e1e59c3b426c78ba80e4d49c6c122b)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-26 09:27:31 +01:00
Alexandru DAMIAN 933e4dbb50 bitbake: toastergui: enable strict variable checking
In order to make sure we don't use undefined variables in the
templates, we enforce strict variable checking in the templating
engine.

(Bitbake rev: 3928dc93188880386588fe3f440cd0aaa83d22de)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-26 09:27:31 +01:00
Alexandru DAMIAN 0b2e6442a6 bitbake: toaster: improve the buildenvironment API
We improve the buildenvironment API by reducing it to a single
command: triggerBuild.

This command is specifically implemented in each BE controller
type, so the runbuilds management command is only concerned
with scheduling the next build, and not with the details
of how a build is actually started.

(Bitbake rev: 7ee0f1da0a8fcac37419ffdddbe35a9268a1ded4)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-26 09:27:31 +01:00
Alexandru DAMIAN aad93dd3b7 bitbake: toastergui: fix angular error
Due to invalid identification of builds in the array update
code (by id and status), we could end up with multiple entries
sharing the same primary id, visible in the UI as an angular error.

We modify the code to identify the builds exclusively by id.

[YOCTO #7611]

(Bitbake rev: 77d3f7c6cdc7f06807024896369f8f8ae94263fc)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-26 09:27:31 +01:00
Richard Purdie 6f4304e36d bitbake: runqueue: Sanity check BB_NUMBER_THREADS
Bitbake does really weird things with negative or zero numbers of threads
which is confusing to the user. Add a sanity check for this.

When you have code doing arithmetic on the values and a VM reconfigures
to only a single thread, negative numbers are easier than you'd think.

(Bitbake rev: 32166ac3c85ff3c04081580ae76bd63590d6ff3e)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-23 11:57:58 +01:00
Richard Purdie 810bf0093a bitbake: runqueue: Handle BBHandledException correctly
If we see a BBHandledException in runqueue, the understanding is the system
handled it, printing a log and traceback is just confusing.

Therefore only print these in the cases where its an unknown/unhandled
exception.

(Bitbake rev: 29d28e22ce431c3d3aabdb88ff4d8cca67a1cfad)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-23 11:57:58 +01:00
Richard Purdie 2ce9055e4f bitbake: server/process: Don't log BBHandledException
If we see a BBHandledException in the idle handler, the understanding
is the system handled it, printing a log and traceback is just confusing.

Therefore only print these in the cases where its an unknown/unhandled
exception.

(Bitbake rev: d88ecc2bc44dce8fd92ca3a2c0fd4124a5e464fa)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-23 11:57:58 +01:00
Richard Purdie c6918b62fd bitbake: runqueue: Improve handling of fakeworker failing to start
Currently if the fakeworker failes to start the output from bitbake is confusing.
Improve the error handling to give a clear indication of what failed.

Patch from Chris Larson.

(Bitbake rev: ad286d6fed7a580bec36a92c7b7e205322ac407b)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-23 11:57:57 +01:00
Richard Purdie df123ddab4 bitbake: event: Inject 'd' into event handlers
To quote Chris Larson:

"""
e.data.getVar() gets a bit old in a large event handler, and it means a
simple handler has to be modified if switching between an event handler
(e.g. RecipeParsed) and anonymous python. I think it would make sense
to restore the 'd' convention here to align with python elsewhere.
It'd just be a convenience, d==e.data, to avoid the common pattern of
setting it at the top of the event handler.
"""

I couldn't find a way to inject 'd' via locals/globals due to the use
of a function parameter so this left __builtins__ as the only way
I could find to make this work.

[YOCTO #7668]

(Bitbake rev: 44ac81e5281fb62ad00e2f79a9d754118ea62526)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-23 11:57:57 +01:00
Richard Purdie 69b6919341 bitbake: bitbake: Add explict getVar param for (non) expansion
Rather than just use d.getVar(X), use the more explict d.getVar(X, False)
since at some point in the future, having the default of expansion would
be nice. This is the first step towards that.

This patch was mostly made using the command:

sed -e 's:\(getVar([^,()]*\)\s*):\1, False):g' -i `grep -ril getVar *`

(Bitbake rev: 659ef95c9b8aced3c4ded81c48bcc0fbde4d429f)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-23 11:57:53 +01:00
Trevor Woerner a06a5ae684 bitbake: bitbake-user-manual-ref-variables: Added BBLAYERS_FETCH_DIR glossary item.
Add BBLAYERS_FETCH_DIR to the Variables Glossary.

(Bitbake rev: 5ecd6a671c088f270d5f49093f8da83278fc0aa9)

Signed-off-by: Scott Rifenbark <scott.m.rifenbark@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-23 11:47:40 +01:00
Scott Rifenbark 7a98387e29 bitbake: bitbake-user-manual-ref-variables.xml: Added BB_ALLOWED_NETWORKS desc.
Added a new variable description for the BB_ALLOWED_NETWORKS
variable.

(Bitbake rev: 1408654e6d8f5ae962744c6cc2d8a49df34c5101)

Signed-off-by: Scott Rifenbark <scott.m.rifenbark@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-16 23:58:16 +01:00
Alexandru DAMIAN 379a030702 bitbake: toastergui: prevent contamination of libtoaster context
This patch modifies the New Build button to use a local
copy of the default libtoaster project context in order
to prevent page contamination when a different project is
selected in the drop-down menu.

(Bitbake rev: 937665bd57dad04dd6bb46d06488b699f5c54d29)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-12 00:01:50 +01:00
Michael Wood 4f459fca79 bitbake: toaster: Add global ajaxError handler
If any ajax calls fail and debug is enabled log the error to the console.

(Bitbake rev: 978ab17033ec48ee0a82016b7e4d6a2fe5d21dbb)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-12 00:01:50 +01:00
Michael Wood 0573e2d31c bitbake: toaster: Add ajax loading spinner
This adds an ajax loading spinner to provide feedback when in-page
loading is happening. It will show after 1.2 seconds of initial loading.

[YOCTO #7790]

(Bitbake rev: ecb70b0bc996529f1a6e58e5716b31e273395c98)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-12 00:01:49 +01:00
Vlad Fulgeanu eb28534423 bitbake: toaster: tables: Remove obsolete field_name for select column
field_name is no longer used for this kind of column. So no need to specify
it here.

(Bitbake rev: 058e9db05767f2d46d46c02dd5043bb0f6b8e460)

Signed-off-by: Vlad Fulgeanu <andrei-vlad.fulgeanu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-12 00:01:49 +01:00
Ed Bartosh 5a013b562e bitbake: toaster: Make toaster script to work in dash
Made it working in dash. Note, that due to dash limitations
script will not work if sourced.

(Bitbake rev: febf8bedf8f9d37659831cdde208d14ece2fb322)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-12 00:01:49 +01:00
Ed Bartosh 0eaf070337 bitbake: toaster: Code cleanup: bashisms
Fixed the following bashisms:
 replaced echo -e -> printf
 removed 'function' from function definitions
 replaced $(< ${file}) -> `cat ${file}`

(Bitbake rev: bd95425c35c7d8386c57329e425aa7802537b479)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-12 00:01:49 +01:00
Alexandru DAMIAN 58cd4a14ea bitbake: toaster: fixes after refactoring
This patch fixes issues brought in by refactoring:

* the New Build button is working with pre-set projects
* the xhr_datatypeahead is exposed for calls that are not
mapable to the REST objects
* a new table returing recipes provided by layers currently
selected in the project is used to provide recipe suggestions
* the field names in json are switched from "list" to "rows" as
to maintain consistency with the ToasterTables
* the "value" field in xhr_ calls is now named "search" to maintain
consistency

(Bitbake rev: a5bc29083d4f85a5695f3f62d5badb783c6f7224)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-12 00:01:49 +01:00
Alexandru DAMIAN 27f5137cd6 bitbake: toastergui: remove xhr_datatypeahaed layerdeps call
This patch removes the url-constructing calls to get the layer details
in favor of embedding the look-up URL in the JSON data on the
layer list page.

This allows further removal of the XHR-specific code for layer dependencies
in favor of REST calls to layer details data.

(Bitbake rev: 33d2b87aca667d72262a3928deaf35414b46a7c1)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-12 00:01:49 +01:00
Alexandru DAMIAN a8be6d4bb1 bitbake: toastergui: remove xhr_datatypeahead and xhr_XXXbuild
We remove the endpoints for XHR on the toastergui application.

The endpoints are now replaced with calls to the respective
REST endpoints (i.e. projectlayers, projecttargets, projectmachines).

(Bitbake rev: 8e7a2c3b125a34fd9d6fa0442ab13290137ecc51)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-12 00:01:49 +01:00
Alexandru DAMIAN 88dca45a70 bitbake: toaster: eliminate duplicate querysets
The Layer_Version.get_equivalents_wpriority performs the same
function as Project.compatible_layerversions, but in memory and
with worse performance.

Replace the code in get_equivalents_wpriority with a call to
the project compatible_layerversions, which also returns a queryset
instead of a list (can be used to further enhance queries)

(Bitbake rev: fb5eb1d7759222573565936a964d602c148df139)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-12 00:01:48 +01:00
Alexandru DAMIAN 7c2b86625b bitbake: toaster: add class template view for single-object pages
Adding ToasterTemplateView as prototype for all class-based
views that display single objects; equivalent to ToasterTable
for object lists.

(Bitbake rev: d3edea773000a663f5883e04f477d853bff64cf6)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-12 00:01:48 +01:00
Alexandru DAMIAN 51ae1de5b7 bitbake: toaster: toastertables raise errors
Modifies the widget code to raise the Exceptions to the user
instead of printing then to stdout - making the programming
errors much more visible.

(Bitbake rev: 26dc19284e06a7ae35f75a243b2062f61e30f2ca)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-12 00:01:48 +01:00
Alexandru DAMIAN c941743c9a bitbake: toaster: ToasterTables add computational fields
This patch adds the ability to pass a function to be computed
for generating a field value in setting up a column in
ToasterTables.

Also adding "displayable" property that can be turned False for
columns that are present in JSON data but are not part of the UI.

Add the "id" column by default for all rows.

(Bitbake rev: fb683135348b074412da154585c75865aad1eab0)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-12 00:01:48 +01:00
Alexandru DAMIAN 58600cf8e7 bitbake: toaster: toaster table add raw data
We add in a JSON response both the raw data and the rendered
version for display. The rendered fields start with "static:"
to mark a different "namespace".

The toaster.js is updated to always display the "static:" version
of a field, if it exists (and ignore the raw data unless the
static rendering is missing).

(Bitbake rev: 928ee3fd4b52ea14b7eb704f1f27351362a9d27a)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-12 00:01:48 +01:00
Alexandru DAMIAN 4a2a057130 bitbake: toastergui: remove xhr_projectedit and xhr_projectinfo URLs
This patch removes the xhr_projectedit and xhr_projectinfo URLs
in favour of REST calls to the Project page.

The project page takes now the POST requests to modify project
settings. All usages of removed URLs are now changed to point to the
project page, using the json format.

The interface call specs have not modified.

(Bitbake rev: 6ad3078bd2be1a8cda99040acaa9bb81d77f0013)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-12 00:01:48 +01:00
Alexandru DAMIAN b1c91d06e0 bitbake: toastergui: use defaults on xhr_datatypeahead parameters
This patch prevents crashes when xhr_datatypeahead is called
without proper parameters.

(Bitbake rev: 4528490cf663790a455815dc48fbf3dda264098c)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-12 00:01:48 +01:00
Alexandru DAMIAN d9341d1a77 bitbake: toaster: toastertables REST refactoring
This patch refactors the ToasterTables to bring them in line
with REST principles -

- all table pages now support the "format=json" GET parameter
that returns the data in JSON format
- the tables themselves

This cleans up the URL namespace by aleviating the need to
have two URLS for each table (one for the template and one for
the data loading), and fixes minor things in the ToasterTable
implementation.

(Bitbake rev: 1778dac9fd39dae75c55bf2cf836cdd488dbc265)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-12 00:01:47 +01:00
Alexandru DAMIAN 751e9182ac bitbake: convert all project-based view to JSON APIs
This patch converts all-project views that are REST-style URLs
to support JSON encoding if the "format=json" parameter is supplied.

The formatting code is enhanced to prevent following Django foreign
keys, to convert enum-values from int to string, and support for
timedelta.

(Bitbake rev: 7a6eb36b82c5f2e342777e479d9c401ded42453d)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-12 00:01:47 +01:00
Alexandru DAMIAN afe06e313e bitbake: toaster: move project data typeahead to the REST API
This patch enables JSON requests on the project REST endpoint,
and replaces the universal queries "xhr_datatypeahead" with the
`project` type to the REST project endpoint.

The patch adds a decorator that takes a context returned by a view
and either renders the template specified as the decorator argument,
or converts the context to JSON.

Normal "search", "filter" and "order" options for view work as normal
on the JSON API format. To enable the JSON return, set the "format"
GET parameter to "json".

In order to demonstrate the functionality, the "New build" button
is switched from using the xhr_datatypeahead to the project
REST API with JSON formatting. Additionally, the XHR APIs that
perform actions with the project id passed as parameter are removed,
and the needed URLs are populated from the project JSON API returns after
the project has been selected.

(Bitbake rev: 15a2274eba13d19b864f337057d61c75ff7849cc)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-12 00:01:47 +01:00
Richard Purdie 926de56da9 bitbake: bitbake: Bump version to 1.27.1
(Bitbake rev: 430f7a288b4446600b3a943c51f6711ffcf9e619)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-09 13:30:09 +01:00
Richard Purdie 4bedfeffa2 bitbake: cooker/event: Add an event which lists all stamps reachable after parsing
The metadata can potentially use such an event to clean up any
"unreachable" data, solving several problems we currently have
where obsolete data may continue to exist in the shared areas.

(Bitbake rev: c5e6f929f3d5eeb7954660dea62611c58b795ff8)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-09 13:30:09 +01:00
Richard Purdie ffaa0f1b8a bitbake: parse/BBHandler: Avoid repeatedly resetting FILE
If we're not going to change the value of FILE, or we know it isn't
going to have changed (ext == bbclass), don't set FILE.

This avoids messy looking history of the variable as well as optimises
parsing speed slightly.

(Bitbake rev: 88e4600aa66dda2e6c807f9d97af8982bcd8817b)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-05 16:42:01 +01:00
Alexandru DAMIAN 6e23a3deb9 bitbake: toaster: more HTML5 fixes
More HTML5 fixes highlighted by tests on new data.

(Bitbake rev: a52242bb4d928b9abba8a3489e350e4615a9da29)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-29 11:59:47 +01:00
Alexandru DAMIAN 225daf45a4 bitbake: toaster tests: enable url check test
Integrate the HTML5 validation as a test instead of
calling a separate script. This enables us to get the
HTML5 validation report as part of patch-level
testing.

gitignore the cache directory created by the http client

(Bitbake rev: 931caab56301876cb8632b289835c2545a096ef6)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-29 11:59:46 +01:00
Alexandru DAMIAN 29143915ae bitbake: toaster/tts: Fix tests
(Bitbake rev: f4257f8df3be0627bdc7209fe34d674c42baf8d2)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-29 11:59:46 +01:00
Alexandru DAMIAN 356809ff40 bitbake: toaster: enable no browser start
We enable a "nobrowser" parameter that inhibits
the launch of a browser when toaster starts.

This is useful for integration with automated startup scripts
and enables headless testing.

(Bitbake rev: ccf7f39d470fe6d743b58b1140c19cb8da31ecaf)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-29 11:59:46 +01:00
Michael Wood b3189f3071 bitbake: orm: Fix all failing unit test
This fixes all the unit tests for the orm. Also added is the ability to
set a custom Layer index if you want to avoid using the public one by
specifying TTS_LAYER_INDEX

(Bitbake rev: dfbcbe116d0b987b850f67056f02f489ac0b8360)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-29 11:59:46 +01:00
Michael Wood 3480be741e bitbake: bldcontrol: Fix all failing unit tests
This fixes the unit tests for the bldcontrol it requires the
implementation of a new Exception type so that a known Exception can be
handled. Also fixed is the path to the toaster conf files so that the
test doesn't need to be run from the top level directory and the ability
to specify the values of TTS_SOURCE_DIR and TTS_BUILD_DIR and
TTS_TEST_ADDRESS used for testing.

Edited by Alex Damian to correct the rebasing of the localhostbecontroller.py
file.

(Bitbake rev: c17933271cd273a346115c2ee0b6695ff3f981ce)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-29 11:59:46 +01:00
Alexandru DAMIAN 22a0d8aab2 bitbake: toastergui: add tests for xhr_datatypeahead calls
In order to make sure we replace the xhr_datatypeahead calls
with similar APIs, we add tests that validate the data returned
by the xhr_datatypeahead. A copy of these tests will be enabled
for each type of datatypeahead that will be replaced.

(Bitbake rev: 9a3197fba46b38e5863e7ded11e4bf9530cfcc9b)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-29 11:59:46 +01:00
Alexandru DAMIAN 65c805eb6a bitbake: toastergui: tests for the all-projects API point
This patch adds Django tests that verify that the 'all-projects'
page returns a valid HTML page when invoked normally, containing
the project name; and valid JSON containing API-needed fields
if the GET parameter `format` is set to "json"

(Bitbake rev: 9edd61fe7afaf273ed31d214af9251462182ad4f)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-29 11:59:46 +01:00