Commit Graph

4187 Commits

Author SHA1 Message Date
Ed Bartosh 8b7a548e30 bitbake: toaster: get rid of complicated heuristics
Removed buildinfohelper code which was trying to guess layer version of
the recipe using build request information. The code caused creation of
duplicated recipes as it resulted in layer version from layer index
instead of returning build layer version. As a result of this Toaster
UI was not showing any information about recipes.

Default approach used to find layer version seems to work much better as
it finds proper layer version. Now toaster will use it as the only
way to find layer version.

(Bitbake rev: 101690bda7ad55dc0657483233c90c374713755b)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-07 17:37:11 +00:00
Ed Bartosh 556b8b6156 bitbake: toaster: remove SDKMACHINE from project variables
Removed SDKMACHINE from predefined set of variables
for the project as it causes bitbake build error:
   SDKMACHINE is set, but SDK_ARCH has not been changed as a result

This variable does not need to be predefined as it's not used by
toaster.

It's still possible to specify it in project configuration
if needed. SDK_ARCH variable should be set too to avoid above
mentioned build failure.

(Bitbake rev: a6ab2a38ab666376b4a5e796d3a88929e8cbfb22)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-07 17:37:11 +00:00
Ed Bartosh 4186f5b04e bitbake: toaster: stop using toaster-pre.conf
This file is not needed anymore as toaster UI doesn't
write variables to it. It sets them directly on bitbake
server.

(Bitbake rev: 624240460ec32af918dcea8c04a913e3b94e012b)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-07 17:37:10 +00:00
Ed Bartosh 361faa3e6c bitbake: toaster: remove writeConfFile API
This API is not used anymore as toaster doesn't write variables
to configuration files anymore. It sets variables through its
connection to bitbake server.

(Bitbake rev: 6dda2916b75d688874d208192d5c7cdb302eec35)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-07 17:37:10 +00:00
Ed Bartosh fcbba5a89b bitbake: toaster: set varibales on bitbake server
Set project variables using connection to bitbake server
instead of writing them to toaster configuration files.

This is a way to avoid rewriting configuration files before
and after every build, triggered by toaster UI.

(Bitbake rev: 3f77c69b867735b5d27c6ab7c62e39ac8d5cb2b5)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-07 17:37:10 +00:00
Ed Bartosh 993bc7eacd bitbake: toaster: implement BitbakeController.getVariable
Added new API to get value of bitbake variable from bitbake server.
The API will be used to update INHERIT variable instead of writing
it to the toaster configuration files.

(Bitbake rev: effc371e3a4a2262e53e9e560c7c08cffad195da)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-07 17:37:10 +00:00
Michael Wood 53e981e583 bitbake: toaster: buildinfohelper Broaden the toaster created recipe data case
When build information is the 'original' source of the information we
need to return the recipe that was created rather than the copy of the
recipe that is taken for keeping build history. We do this already for
command line triggered builds, but we also have this case for custom
images. We can simply check if the built_recipe exists instead of
special casing this.

(Bitbake rev: 9a8653bf602b2111dee7ee6a459682a68a695b22)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-07 17:37:10 +00:00
Ed Bartosh 57e5f24e57 bitbake: toaster: do not create duplicate HelpText objects
buildinfohelper code expects only one HelpText object per
build/variable/description.
Current code creates more than one such an object, which causes
toastergui to crash with this exception:
 MultipleObjectsReturned: get() returned more than one HelpText -- it returned 2!

Used git_or_create API to ensure that only one HelpText object is
created.

(Bitbake rev: e9b46803eb6f1f4044919abf90c8aeb3536e73ed)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-07 17:37:10 +00:00
Ed Bartosh 4c1e5ec29a bitbake: toaster: remove usage of BUILD_MODE variable
As both modes can be used at the same time we can't have any
difference in UI between modes.

Removed all conditional statements that used BUILD_MODE.

(Bitbake rev: 4ce1559c2558bd0fd278ff02a1a93bec03c4156b)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-07 17:37:10 +00:00
Ed Bartosh 990289511b bitbake: toaster: do not terminate bb server
Toaster needs bb server to be running all the time due
to merged analysis and managed modes. Server gets restarted
before every build triggered by UI, but it shouldn't be
terminated as it will influence command line builds.

[YOCTO #8279]

(Bitbake rev: e69c87c7842e796ffcd7193ecde22c8f688498f5)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-07 17:37:09 +00:00
Ed Bartosh 58765a8719 bitbake: toaster: remove stopBBServer API
Removed stopBBServer API from build controller as toaster
doesn't stop bitbake server anymore. It's reused for both
types of builds: triggered by UI and started manually.

(Bitbake rev: 0cad802da171d51814f22cc9383f496a63cd0c75)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-07 17:37:09 +00:00
Ed Bartosh 95a3cf7549 bitbake: toaster: reimplemented startBBServer method
Rewritten LocalhostBEController.startBBServer to use
'toaster restart-bitbake' and read bitbake port number from
bitbake.lock.

Removed complicated logic of running oe-init-memres and looking for
bitbake port number in the logs.

(Bitbake rev: 4b065353c3454923a1ef88e9f0a8702e5626060e)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-07 17:37:09 +00:00
Ed Bartosh 76d53b5856 bitbake: toaster: remove _setupBE function
This function is not needed as build environment is always created
because of the new way to run Toaster. It can be only sourced after
oe-init-build-env is sourced.

(Bitbake rev: 27a59b4373a449f17e46ab93e0eba15cf75967ae)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-07 17:37:09 +00:00
Ed Bartosh 87b2f954b9 bitbake: toaster: implement 'toaster restart-bitbake'
This implementation allows to have functionality of restarting
bitbake in toaster script. It can be used by toaster script and
build controllers.

[YOCTO #8279]

(Bitbake rev: 24cd26b39014419d883b7cf45e4943c301d585cc)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-07 17:37:09 +00:00
Ed Bartosh 891484ada9 bitbake: toaster: implement start_bitbake function
Separated functionality of starting bitbake server and observer
processes.

This functionality will be used by build controllers to restart
bitbake processes.

(Bitbake rev: d8e9ce7ba3f5b89c51e769bd4a46c2026a110d40)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-07 17:37:09 +00:00
Ed Bartosh bf25471341 bitbake: toaster: implement stop_bitbake function
Separated functionality of stopping bitbake server and observer
processes.

This functionality will be used by build controllers to restart
bitbake processes.

(Bitbake rev: dd40d70b707a3ff59edfa69923d4d9b671928a66)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-07 17:37:08 +00:00
Ed Bartosh 7c2b22556b bitbake: toaster: update brbe and project attributes
Updated attributes of buildinfohelper object as they can
be changed for every build. For example brbe is set by
runbuilds for every build triggered by Toaster UI.

(Bitbake rev: ea3bc8d01704dc64f6cb7b4f5fe66c312a575174)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-07 17:37:08 +00:00
Ed Bartosh de812d086e bitbake: toaster: start 'manage.py runbuilds' in the script
Running runbuilds machinery in background allows Toaster to
start builds initiated by toaster UI.

Added runbuilds pid file to the list in webserverKillAll to
kill runbuilds the same way as runserver process.

(Bitbake rev: 91bf323e873c31d883b01ea993c005f45382e83f)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-07 17:37:08 +00:00
Ed Bartosh 28e8ccf437 bitbake: toaster: make runbuilds to loop
Made runbuilds to loop to avoid having a loop in shell code and
initialize heavy Django init machinery every second.

Ignored exceptions to prevent exiting a loop.

(Bitbake rev: e04da15556ca0936de652b8c085e4199e5551457)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-07 17:37:08 +00:00
Ed Bartosh a3871a37de bitbake: toaster: use parent of the build dir
Assigned TOASTER_DIR to the parent of the build directory.
This should fix local controller crash as it assumes that
TOASTER_DIR is a root of local poky and tries to clone it.

(Bitbake rev: 124eb2bb15de18e4d2d6483c3549e1d887234736)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-07 17:37:08 +00:00
Ed Bartosh 2a96d35893 bitbake: toaster: check for toaster configuration later
Moved check for toasterconf.json after check of build environment.
We'll need some variables from build environment to find toasterconf.json
better way.

(Bitbake rev: a4f6809237dffb1abff78c6a8c82bcc4852aedfa)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-07 17:37:08 +00:00
Ed Bartosh d87a5346cf bitbake: toaster: remove unused variable
Removed variable RUNNING as it's not used anymore
in the toaster script.

(Bitbake rev: 187c31a8789c37723b9288afd35f1785f1993895)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-07 17:37:07 +00:00
brian avery dc6a489c65 bitbake: toaster: change toasterconf.json logic to use TEMPLATECONF, like oe-setup-builddir
This changes the way we pick a toasterconf.json file so that it
matches the way oe-setup-builddir does it. We now base it on TEMPLATECONF
found in the .templateconf file.

(Bitbake rev: 8476db82ea80edf2b70bc2082bcecee09037556d)

Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-07 17:37:07 +00:00
Ed Bartosh 5a42c2d050 bitbake: toaster: run bitbake the same way
Run bitbake in toaster script the same way as it's run
by localbuildcontroller.

(Bitbake rev: c5d5da095d5eace03f8a50910f5a60ab35acbb1a)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-07 17:37:07 +00:00
Ed Bartosh cac91db01d bitbake: toaster: set DATABASE_URL in toaster script
Set environment variable DATABASE_URL in bitbake/bit/toaster
Variable value is obtained by running 'manage.py get-dburl'

(Bitbake rev: ac4800cc8b765b270f27ea9a5938f3f2a2e0ced4)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-07 17:37:07 +00:00
Ed Bartosh a464bf23b4 bitbake: toaster: implement get-dburl command
Implemented management command to call getDATABASE_URL API.
It will be used to get database url from toaster shell script
by running 'manage.py get-dburl'

(Bitbake rev: fabe9c8f14fd7b8ab204a2b610c64ac5b62135ac)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-07 17:37:07 +00:00
Ed Bartosh e4731514f8 bitbake: toaster: don't allow to run toaster as a script
Removed support of starting toaster as a script.

Sourcing a toaster script becomes the only way to start it.
It's consistent with the way oe build system is started by sourcing
oe-init-build-env. It also returns user back to shell, so user can
continue running builds without having to open new terminal window.

[YOCTO #8279]

(Bitbake rev: 7e8632c0527245c1474e487dd9a972948e794471)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-07 17:37:07 +00:00
Paul Eggleton 4de214f752 bitbake: lib/bb/utils: improve edit_bblayers_conf() handling of bblayers.conf formatting
Make the following improvements to edit_bblayers_conf():

* Support ~ in BBLAYERS entries
* Handle where BBLAYERS items are added over multiple lines with +=
  instead of one single long item

Also add some comments documenting the function arguments and return
values as well as a set of bitbake-selftest tests.

(This function is used by the bitbake-layers add, remove and
layerindex-fetch subcommands, as well as devtool when adding the
workspace layer).

(Bitbake rev: e9a0858023c7671e30cc8ebb08496304b7f26b31)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-07 17:01:21 +00:00
Paul Eggleton 0debb11883 bitbake: lib/bb/utils: fix error in edit_metadata() when deleting first line
If you tried to delete the variable on the first line passed to
edit_metadata() this failed because the logic for trimming extra blank
lines didn't expect the list to be empty at that point - fix that bad
assumption.

(Bitbake rev: 8bce6fefdc5c046b916588962a2b429c0f648133)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-07 17:01:21 +00:00
Alexander Kanavin 9d19dd9bd7 bitbake: wget.py: parse only <a> tags
For two reasons:
1) The important one: we hit the following bug when doing upstream version checks
on some webpages:
https://bugs.launchpad.net/beautifulsoup/+bug/1471755

2) Also, documentation for beautifulsoup states that memory usage and
speed is improved that way.

(Bitbake rev: 7546d4aeb3ba8fda9832081b84d93138dc5e58d6)

Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-07 17:01:21 +00:00
Michael Wood 71ede7b689 bitbake: toaster: toastergui tests Add generic test for ToasterTables widget
For each of the ToasterTables defined test:
 - Data returned
 - Search
 - Order by
 - Limit
 - Paginate
 - Filter

These tests test that the functions on all tables work, it does not
validate the content of the data in the tables themselves, this needs
to be done in table specific tests.

(Bitbake rev: 649d563f95ffc57d0fe3dbf494e7dbffcc99a1b4)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-07 17:01:20 +00:00
Michael Wood 34b22cf897 bitbake: toaster: tables Fix invalid field name on NewCustomImagesTable
Correct the field name in the NewCustomImagesTable as it is a Recipe
object it's self and not a container.

(Bitbake rev: ebd1c493d8b7fb9ee7b3e40c17165dc9c22ca795)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-07 17:01:20 +00:00
Michael Wood 1c59846fc0 bitbake: toaster: tables Add default_orderby field where it was missing or unset
This value is used to set the default ordering of the model that is used
for ToasterTables, it is picked up client side to set the ordering
indicator.

[YOCTO #8695]

(Bitbake rev: f1c91bff810a579b169c46b7710e22f5553b484e)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-07 17:01:20 +00:00
Michael Wood d82c541147 bitbake: toaster: CustomImageRecipe add search_allowed_fields to this model
In order to search the model from the UI some fields must be nominated
as searchable.

(Bitbake rev: 2558729b4c248a2fa5a11d877bc42cb05a30602e)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-07 17:01:20 +00:00
Michael Wood bdf62414f2 bitbake: toaster: machines table Fix missing layers information needed for filter
The current layers information wasn't being passed to the template for
the Select/Add button for the Compatible machines filter.

(Bitbake rev: 559102984d3f62a7675899e058166bdf4d552c46)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-07 17:01:20 +00:00
Michael Wood b90a8dca99 bitbake: toaster: tablejs Make sure click handlers consume click event
Avoid the click event from propagating and causing strange side effects
in toaster tables.

[YOCTO #8527]
[YOCTO #8148]

(Bitbake rev: f23e54292a673b2fd3f17d317c21984c753727c7)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-07 17:01:20 +00:00
Michael Wood c075bcf545 bitbake: toaster: projectpage Make sure build targets are space separated
Make sure the build targets are space separated when building multiple
targets. Also fix error path now that YOCTO #7995 is resolved.

[YOCTO #8450]

(Bitbake rev: e9719eb2e8249f0d10b39bfdd4aef563368b5ffd)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-07 17:01:20 +00:00
Belen Barros Pena 88f9310ce0 bitbake: toaster: builds pages Fix the download cooker log link
The 'all builds' page was missing the download icon next to the outcome
icon, which allows you to download a build log from the 'all builds' page.

This patch brings it back. It also adds a check in the project builds page
to make sure the download icon only appears if the build generates a cooker
log, since builds that fail at the build request stage do not generate a
cooker log.

(Bitbake rev: 50d7ffa697764d84d96f26a3ae58a9dacde7b31e)

Signed-off-by: Belen Barros Pena <belen.barros.pena@intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-11-16 11:55:08 +00:00
Belen Barros Pena d04af8b571 bitbake: toaster: project pages Link to image recipes table in notifications
The project created notifications and the 'Choosea recipe to build' link in
the project page are linking to the software recipes table. Changes them to
link to the image recipes table instead, which is bound to be more useful.

(Bitbake rev: 10e1fbee0d79dbbf15b89ebc155bb2c0092a71f2)

Signed-off-by: Belen Barros Pena <belen.barros.pena@intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-11-16 11:55:08 +00:00
Ke Zou 70465c717a bitbake: toaster: tests: Re-write some cases to make them more maintainable
(Bitbake rev: c3c7e94cadb3a3a2d8ef33ed456f439c106018c1)

Signed-off-by: Ke Zou <ke.zou@windriver.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-11-16 11:55:08 +00:00
Richard Purdie 536b73f55f bitbake: data_smart: Only support lowercase OVERRIDES
Our current OVERRIDES handling means we end up caching and checking for
a lot of possible override combinations which turn out to very unlikely.
A typical example is the SRC_URI variable where we have to check if
"URI" is an override. Having spent many hours working in this code, I've
realised all the actual overrides we use are lower case and our standard
variables are mostly uppercase.

This means we could gain quite some speed advantage if we write this
into the code, that overrides only consist of lowercase characters. This
patch shows how simple this is and the resulting speed gains are
significant. This is a significant change but tests show we don't appear
to have any users of capitals in overrides in any OE-Core metadata.

Before "time bitbake -p":

real	2m4.224s
user	7m32.312s
sys	0m7.116s

After "time bitbake -p":

real	1m26.009s
user	5m10.484s
sys	0m4.640s

This check could also be made conditional however I'm not seeing a need
to do that at present.

(Bitbake rev: c9b9443faa76ee7366b1400a56f826f3f9dec1be)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-11-16 11:55:08 +00:00
Richard Purdie fb01a66f48 bitbake: fetch2: Remove crazy code in unpack
This looks reasonable until you realise self.localpath is a function. Data
expansion of something which isn't a string is the original value so this
code just wastes CPU cycles and makes no sense. Remove it.

(Bitbake rev: 37214ea9bf484998b75dbc1200d53f1afc5257ed)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-11-16 11:55:08 +00:00
Richard Purdie 7db88aa67d bitbake: parse: Don't try to expand __base_depends/__depends
Trying to expand a variable which isn't a string doesn't make sense.

(Bitbake rev: 62367cca1f1793eb9827406bcdd5980fdeb80a60)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-11-16 11:55:07 +00:00
Richard Purdie 4c04ce0576 bitbake: cache: Don't try to expand __inherit_data
Trying to expand a variable which isn't a string doesn't make sense.

(Bitbake rev: 54d0ddd166a6707b4f8c8535639e3055b783363b)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-11-16 11:55:07 +00:00
Elliot Smith 9d8e36a6c9 bitbake: toaster: localhostbectrl Pass DATABASE_URL in via the process environment
Instead of putting the DATABASE_URL as part of the command for launching
the bitbake observer process set it as part of environment.

This fixes two issues 1. Where the value isn't quoted and therefore will be
interpreted in the shell and 2. Anyone being able to see the value of
DATABASE_URL in the process tree.

[YOCTO #8669]

(Bitbake rev: 832a8523067606b180c02f0d1544e8a23219bb08)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-11-16 11:55:07 +00:00
Michael Wood 4677d8b4fb bitbake: toaster: Remove the new-build-input button widget
The button required a lot of state maintenance to make sure it
showed up when the project was configured properly, showed correctly
according to the projects known to Toaster, displayed correctly
according to the mode Toaster was in, and was able to be
used to change the current project.

(Bitbake rev: b644514a96f3947ad3f307a26301c064c8ae18f8)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-11-16 11:55:07 +00:00
Michael Wood 55f44943d7 bitbake: toaster: projecttopbar Use the project in context to get num builds
Use the project in the template context rather than ajax request to get
the number of ended builds.

(Bitbake rev: 1f6bfa876b17a600fc83c789af8aaa2639a463b2)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-11-16 11:55:07 +00:00
Michael Wood e9d4962a0c bitbake: toaster: projectpage Disable/Enable build input if we have 0 layers
If we've removed all the layers in the configuration, disable the build
button and build input.

(Bitbake rev: 872bd5ccf58236f5146b1640cc1c465b58371e44)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-11-16 11:55:07 +00:00
Michael Wood 5fa4c73603 bitbake: toaster: orm Fix get_number_of_builds to count all apart from IN_PROGRESS
The count of a project's builds should not include those which are
currently in progress.

(Bitbake rev: a981700701c41c7bbb6a9778e95f691278c5c294)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-11-16 11:55:06 +00:00
Richard Purdie c4032f479e bitbake: codeparser: Only load the codeparser cache once
The server state gets reset multiple times during startup and currently
we reload the codeparser cache each time. This is pointless and causes
unnecessary interaction time with bitbake.

(Bitbake rev: 4f700316933e8e7b2d27366e5ce6176895b913e7)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-11-16 11:55:06 +00:00
Elliot Smith a0791c19db bitbake: toasterui: Create per-build logs
Rather than relying on bug 8411, which is conveniently creating
separate log files for each of our builds, create our own
log file for each build.

The log files are created in the same tmp directories that
bitbake users, but are timestamped to the millisecond to avoid
name collisions.

Each log file is opened on a ParseStarted event (for builds
triggered by Toaster) or BuildStarted event (for builds on the
command line: Toaster doesn't get the ParseStarted event
for command-line builds).

The log file is closed on the BuildCompleted event, or if the
build fails.

Because we start logging on ParseStarted for Toaster builds,
we're able to capture the "Build Configuration" section which
bitbake writes to output.

[YOCTO #8373]

(Bitbake rev: 7974203cd8bc66dff1fcc55f8723dedefaf72840)

Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-10-29 22:05:36 +00:00
Richard Purdie 290534d3ff bitbake: build/utils: Add BB_TASK_IONICE_LEVEL support
Similarly to BB_TASK_NICE_LEVEL, add BB_TASK_IONICE_LEVEL which allows the ioprio
of tasks to be adjusted. This is in response to various qemu runtime timeouts
which have been witnessed on the autobuilder, seemingly due to IO starvation (we
already use NICE_LEVEL to adjust tasks). This has a fairly urgent need to deal
with certain 'random' failures we're seeing on the autobuilders in testing.

The format of the data in the variable is BB_TASK_IONICE_LEVEL = "<class>.<prio>".

For <class>, 2 is best effort (the default), 1 is real time and 3 is idle. You'd
need superuser privileges to use realtime. The <prio> value is a default of 4,
and can be set between 0 and 7 with 7 being lowest priority and 0 the highest.
The user can set this freely with normal privileges

Note that in order for this to take effect, you need the cfq scheduler selected
for the backing block device.

We could use nice wrapper functions for ioprio from modules like psutil however
that would complicate bitbake dependencies. This version has some magic numbers
but works on the main 32 and 64 bit x86 build architectures and can easily be
extended if ever needed. When we move to python 3.x, we can likely replace this
with standard calls.

(Bitbake rev: b9471ad147b102c45d65f5ffd9521864df7ff9c1)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-10-29 07:35:52 +00:00
Richard Purdie 3ebf7617d6 bitbake: cooker: Ensure BB_CONSOLE remains correct over server resets
The console log data is written to is created at console initialisation
time and does not change over reset events. This ensures the
BB_CONSOLELOG value is correct over such resets by preserving it.

(Bitbake rev: 335eb2db228f7543a49de71f063ac72b865c947a)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-10-29 07:35:52 +00:00
Richard Purdie 5b19b71252 bitbake: bb/ui: Use getSetVariable command for BB_CONSOLELOG
Metadata can define BB_CONSOLELOG as containing ${DATETIME} and
this can get expanded to a different value each time the variable
is read. In the case of BB_CONSOLELOG, this behaviour is not
desireable.

The values of DATE/TIME are locked down at build time but this is too
late for the purposes of ensuring the system can figure out the real
value of BB_CONSOLELOG.

The best way to do this is to set the variable into the datastore, thereby
preserving its value.

[YOCTO #8411]

(Bitbake rev: 021f2eb55ab5863b57ed1b3f19f1b329bc1ad477)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-10-29 07:35:51 +00:00
Richard Purdie acc7b4d931 bitbake: command: Add getSetVariable command
There are some use cases where we want to read a variable but also
set the variable to the value read, effectively locking in any
expansion of it. This adds such a command.

(Bitbake rev: 0c0c524691e3d2ffd9953a106fcc06262cbde910)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-10-29 07:35:51 +00:00
Scott Rifenbark c8051c52a4 bitbake: bitbake-user-manual: Added new description for BB_TASK_IONICE_LEVEL
Added a new variable description here for this variable.

(Bitbake rev: 4e0da1ef4a58a7ba4b92d4e07ef79cf7bd90abf0)

Signed-off-by: Scott Rifenbark <srifenbark@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-10-29 07:35:51 +00:00
Scott Rifenbark 183290a9d4 bitbake: bitbake-user-manual: Added BBTARGETS variable description.
(Bitbake rev: 0f0c205b9d6c24bee334eabb359973e0368ce931)

Signed-off-by: Scott Rifenbark <srifenbark@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-10-29 07:35:51 +00:00
Belen Barros Pena 66d3c353f8 bitbake: toaster: templates Add meaningful title tags
Our title tags are all over the place, and have no relation to the page
content. This commit adds a meaningful title tag to each Toaster page.

(Bitbake rev: 1ab8827d684a19a70f3b788aed2327bf30edffe2)

Signed-off-by: Belen Barros Pena <belen.barros.pena@intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-10-29 07:32:10 +00:00
Michael Wood 62e200e3e3 bitbake: toaster: orm Fix restrictive LogMessage message length
Log messages can be quite long so use a TextField rather than a char
field with max length of 240. mySQL is especially picky about field
lengths and will cause an exception if the log is too long.

[YOCTO #8475]

(Bitbake rev: 349119f44cdad240dda7f4db45992e2539e1416f)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-10-27 07:24:31 +00:00
Elliot Smith 78f935df8f bitbake: toaster: Remove all navigation when not in build mode
The user is redirected to the all builds page or all projects
page from the landing page, regardless of mode.

In build mode, this makes sense; but in analysis mode, we are
restricting the view to just the cli builds project. This means
that "all projects" and "all builds" only contains items relating
to this one project.

Modify the landing page so it redirects to the project builds page
for the cli builds project when not in build mode. Also remove
navigation elements which are irrelevant when not in build mode.

[YOCTO #8514]

(Bitbake rev: ae754655fa1bc5168f43e8821773e7b7b9440a5d)

Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-10-27 07:24:31 +00:00
Elliot Smith c5f147b4ab bitbake: toaster: Run tests in build mode
Now that the UI content is conditional on whether BUILD_MODE
is active, modify the existing tests so that they run in
this mode by default.

[YOCTO #8514]

(Bitbake rev: 991712f3f88759e2d68af5a3fe83c1395c4f67f6)

Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-10-27 07:24:31 +00:00
Elliot Smith 1d17109d62 bitbake: toaster: Hide builds for non-cli projects in analysis mode
The "latest builds" sections of the "all builds" page
show builds for all projects. This is not appropriate
for analysis mode, where we can only affect the command-line
builds project.

Modify the "latest builds" section to only show builds for
the command line builds project if in analysis mode.

Also rationalise where we get the queryset of latest builds from:
we have a _get_latest_builds() function which was being used
to get the latest builds in most places, but not all.

Also modify _get_latest_builds() to sort by started_on, rather
than primary key, as assuming that a higher primary key value equates
with later start time is incorrect.

[YOCTO #8514]

(Bitbake rev: 52de2c8cd11fa743f8fd7cb0cf776eea622ac474)

Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-10-27 07:24:31 +00:00
Elliot Smith a580479452 bitbake: toaster: Hide top bar buttons in analysis mode
The "new build" and "new project" buttons are irrelevant in
analysis mode, as you can't start a build or a project.

Hide these buttons if not in BUILD_MODE.

[YOCTO #8514]

(Bitbake rev: b1858653c22eb6d51fc23f75ccad69b76523eb41)

Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-10-27 07:24:31 +00:00
Elliot Smith 1ec2ec3429 bitbake: toaster: Show mode-appropriate landing page
The same landing page is shown for both analysis and
build modes. This means that users in analysis mode can see
options which are not available or broken in that mode.

Modify the landing page template to show a simple "run a build"
message if the user is in analysis mode and has no builds yet.

Also clean up the landing page HTML, because the indentation was
a mess and the HTML was invalid.

[YOCTO #8514]

(Bitbake rev: 86b4a10cc817abb5e3e63360dc8b3c550337f17c)

Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-10-27 07:24:31 +00:00
Elliot Smith bbac0f092a bitbake: toaster: Add BUILD_MODE flag to context
We set a TOASTER_MANAGED env variable in the startup script,
which has a value of "1" if Toaster should run in build mode.

Add a BUILD_MODE variable to settings.py which is True if
TOASTER_MANAGED is set to "1", False otherwise.

Add this to the context for every template, so we can use this
information to conditionally alter the content of pages
according to the mode we're in.

[YOCTO #8514]

(Bitbake rev: f9cf4739fc85c9760ce748323dc7c8de3fa7eaec)

Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-10-27 07:24:31 +00:00
Ed Bartosh 851f0d82d9 bitbake: toaster: add get_or_create_targets API
Target objects are created before the build if build is
started from UI in build mode. However, in analysis mode Target
objects don't exist and need to be created using information
from bitbake events.

Added new API call get_or_create_targets to retrive existing
target objects or create them if they don't exist yet.

(Bitbake rev: ef69be31d133696bde54605f5a18da660099734c)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-10-27 07:24:30 +00:00
Jens Rehsack dcd9cd0f9e bitbake: fetcher: svn: Add support for checkout to a custom path
Add support for the Subversion fetcher to checkout modules to a custom path
than the module name to avoid checkout is always module - svn is path
based and tag/branch-checkout might break builds because of invaid path specs.

(Bitbake rev: af88f538e61afa1b115be4d7afe00d8477f61750)

Signed-off-by: Jens Rehsack <sno@netbsd.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-10-27 07:24:30 +00:00
Ed Bartosh 4ab7202c31 bitbake: cooker: preserve pre and post configs
Additional config files passed to bitbake server with --read and
--postread options are rewritten by client bitbake even if
it doesn't use those options.

This is a show stopper for toaster as toaster command line
builds are based on the assumption that server is aware of
toster configs, provided by --postread option.

This behaviour is fixed by preserving values of --read and
--postread options when bitbake server starts and restoring
them if client bitbake doesn't explicitly specify them.

(Bitbake rev: 02c64f7487ca8ec5d32c440f5002c4b8f64b76a6)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-10-27 07:24:30 +00:00
Elliot Smith a9053ac082 bitbake: toaster: Add tests for error message display on the build dashboard
Add tests to ensure that log messages with CRITICAL level and
with EXCEPTION level are shown in the main errors display
on the build dashboard.

[YOCTO #8320]

(Bitbake rev: 7b3ed62800cb7352154897c1c5ad2440a54df198)

Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-10-16 15:00:00 +01:00
Elliot Smith 2517987f8d bitbake: toaster: Modify "New build" button behaviour for cli builds project
The "New build" button should only be displayed if there are
user-generated projects, not if there is only the command-line
builds project. (Toaster can't run builds on behalf of the
command-line builds project.)

The "New build" form should also display as if no project has
been set (i.e. show the project and recipe text entries).

Add a variable which tracks the number of non-command-line projects,
then use this to hide the button when appropriate.

Also track whether the current project is the default one, and
use this to add extra conditions to when the "New build" text
entries are shown.

[YOCTO #8231]

(Bitbake rev: 07e7bc29a7d976941eeae997f78a25ed6edddfc8)

Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-10-16 14:59:59 +01:00
Elliot Smith 56d4c84cec bitbake: toaster: Clean up template code
Some errors in the HTML template didn't show up until
they were rendering command line builds.

One useless conditional was lingering in another template.

Clean up these issues to display the latest builds section
correctly.

(Bitbake rev: c781d6be94a25f942946d3a406e07791385e4596)

Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-10-16 14:59:59 +01:00
Elliot Smith d96cedf51f bitbake: toaster: More linting of tests
Fix some more lint errors on the tests for toastergui.

(Bitbake rev: a31710db562a91da8ef37f5cd2231132474f46da)

Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-10-16 14:59:59 +01:00
Elliot Smith 7c8877efde bitbake: toaster: Show tooltip next to cli builds project name in all builds
In the all builds page, show an icon with tooltip next to the
command line builds project name.

[YOCTO #8231]

(Bitbake rev: 0240d92c6a508e0b22f37b3ea804171dd3dea5a6)

Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-10-16 14:59:59 +01:00
Elliot Smith 76702347a8 bitbake: toaster: Hide tabs and add info popups for command line builds
Command line builds don't have a configuration or
layers which can be modified through Toaster.

Change the project builds page for the command line builds project,
to hide the tabs and add some info popups in appropriate places on
that page.

[YOCTO #8231]

(Bitbake rev: 565611749d47c915035890db60d19ab2fca7c42e)

Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-10-16 14:59:59 +01:00
Elliot Smith da4c6144f1 bitbake: toaster: Make the builds view the project page for "command line builds"
Command line builds don't have configuration or layers which can
be manipulated in Toaster, so these pages shouldn't be visible.
However, the configuration page is the default page for the
project view (/project/X/), which isn't correct for the
command line builds project.

Modify all project page links across the application so that
the command line builds project (aka the "default" project)
always displays the builds tab.

Add a project_url tag for templates which contains the logic
determining where the URL for a project links to, based on
whether it is the default project or not.

[YOCTO #8231]

(Bitbake rev: 3ea10f4c16a557e94781251f6776b13acb8e9eba)

Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-10-16 14:59:59 +01:00
Elliot Smith ef6fc2bc1a bitbake: toaster: Replace "Run again" button with help text for cli builds
It's not possible to run a command-line build again, as Toaster
doesn't have access to the data used for that build.

Replace the "Run again" button with an icon which pops up some
help text to that effect.

Add test to check that the run again button is hidden and the
help icon displayed instead for command-line builds.

[YOCTO #8231]

(Bitbake rev: b67ac9e7cbab50951847dd1a63b12f41bb345dbb)

Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-10-16 14:59:59 +01:00
Elliot Smith 7467b6802d bitbake: toaster: Exclude "command line builds" project from projects typeahead
The "New build" drop-down provides autocomplete for Toaster projects.

However, it should not include the default project (for command
line builds), as it should not be possible for the user to
select this project as the container for a Toaster build.

[YOCTO #8231]

(Bitbake rev: d6210cd75b0cd8b6d24d5e99f607ba5cc45daf97)

Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-10-16 14:16:17 +01:00
Elliot Smith b5624c77ed bitbake: toaster: Show 'not applicable' for default project machine and release
The machine and release for the default project should show as
'not applicable' on the all projects page, as that information
isn't available for command-line builds.

Modify the templates with some conditionals to check for the
default project row, plus some data-* attributes to mark
where that data is to make testing possible.

Add some tests for the all projects page to ensure that
the correct machine/release are still shown for non-default projects,
and 'not applicable' for the default project.

[YOCTO #8231]

(Bitbake rev: ce27b3fd728f0373aa1adc0d47baace264529b45)

Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-10-16 14:16:17 +01:00
Elliot Smith 3c4c984e33 bitbake: toaster: Reorganise and lint tests
Get test suite ready to accommodate new tests for how command line
builds are shown on various pages.

(Bitbake rev: 1ee1fc5dcdbb26c9f6e04b7719d7196083212d4c)

Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-10-16 14:13:24 +01:00
Logan Buchy 3ba43f26ee bitbake: fetch2/hg: Include missing errno import
errno.ENOENT checked if deletion of the fullmirror fails.
Exception was thrown since module not imported

(Bitbake rev: d92ebfc34b69ad5df2d151e6b8299fbb5afa3e5f)

Signed-off-by: Logan Buchy <logan.buchy@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-10-16 14:13:24 +01:00
Ed Bartosh 6fa3fecb12 bitbake: cooker: normalize build targets
BuildStarted event not fully represents build tasks for
the targets. If -c option is used to specify default task
it's not included into the event.

Made build targets to always look as <target>:do_<task>.
Consider default task (do_build or specified by -c command
line option) when normalizing.

(Bitbake rev: 0b0e214e6f53c97ad3d48f622c7fc0ca149956f6)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-10-16 14:13:23 +01:00
Michael Wood 5effe8f632 bitbake: toaster: Allow any text input to machine configuration variable
Allow any text input to the machine variable; as we may not have discovered
all the available machines until after a build.

[YOCTO #8418]

(Bitbake rev: f44b34833f164daf34c57703429ed8f122888037)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-10-16 14:13:23 +01:00
Ed Bartosh 320d05ea9f bitbake: toaster: exit or return depending on the mode
When toaster script is sourced 'return' should be used as
using 'exit' will cause script to exit the shell and close terminal
window. If script is called 'exit' should be used. Otherwise shell
will throw and error:
  return: can only `return' from a function or sourced script

Used 'return' or 'exit' depending on the toaster mode using
TOASTER_MANAGED variable.

(Bitbake rev: 643c42069a2788e9f40fa1bc9845f32aed3d096c)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-10-16 14:13:23 +01:00
Ed Bartosh 2e2e40c2c4 bitbake: toaster: set TOASTER_MANAGED variable
Set TOASTER_MANAGED to 1 if toaster is run as a script.

(Bitbake rev: 9f68f317e33d617558fce3427a94e14ef65c7d88)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-10-16 14:13:23 +01:00
Ed Bartosh a73895e492 bitbake: toaster: get rid of SRCFILE
Used TOASTER variable instead of SRCFILE as they're
essentially the same.

(Bitbake rev: 4b8cb1098ad73b4e9dc65b5c6de7d11d2e2468c7)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-10-16 14:13:23 +01:00
Ed Bartosh 779539cfb9 bitbake: toaster: use path to the script to guess config path
Used $TOASTER variable(path to the toaster script) instead of
current directory name to get path to the toasterconf.json

This way script should work in both modes and it doesn't depend
on the current directory.

(Bitbake rev: 5368d9af4e2c121938fc2421a536fb7f20a987fd)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-10-16 14:13:23 +01:00
Elliot Smith eb8b2b9ec8 bitbake: toaster: Guard against builds with no targets
Although this probably can't happen in practice (builds without
targets), this constraint is not currently enforced in the data model.

In the unlikely event that a build has no target (e.g. in test cases),
this causes a template rendering error.

Rather than rework the data model, add a guard to the template
to prevent it from rendering the target name if there are no
targets associated with the build.

(Bitbake rev: 4f409d8c01bae898ea142bd1417db99e12067753)

Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-10-16 14:13:23 +01:00
Elliot Smith 65e8bde9f1 bitbake: toaster: Remove Toaster exceptions section of build dashboard
Fatal build errors were displayed as exceptions, and highlighted
with less severity than they deserved.

Roll back to treating Toaster exceptions as errors by removing
the toaster_exceptions member on Build objects and displaying
EXCEPTION events in the errors section on the dashboard.

[YOCTO #8320]

(Bitbake rev: 59197320bdcefddf06084e871f1b5b21b21cbb63)

Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-10-16 14:13:23 +01:00
Elliot Smith 93f0b61749 bitbake: toaster: Record critical errors
Critical errors (where a build failed for reasons of
misconfiguration, such as a machine being specified which is not
in a project's layers) were being ignored (only log records
up to ERROR level were being logged to Toaster's db). This meant that
the build would fail but would not correctly report why.

Add support for CRITICAL error levels to the LogMessage model,
include errors at this level in the errors property for a build,
and show errors at this level in the build dashboard.

[YOCTO #8320]

(Bitbake rev: b6eacbca9cacb607de864ab7d093deb296da8226)

Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-10-16 14:13:22 +01:00
Elliot Smith 069a611097 bitbake: toaster: Test that exception isn't thrown by project page
Add a test which checks that an exception is no longer thrown
for the /toastergui/project/X page for the default project.

Note that we still get a spinning dialogue box on this page
because the default project has no configuration to display,
but at least it doesn't fail altogether.

[YOCTO #8277]

(Bitbake rev: 8795667d03bd8705d7e13c5d3d6bb6da371fa91d)

Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-10-16 14:13:22 +01:00
Elliot Smith 026e981264 bitbake: toaster: Check whether buildrequest exists before using it
Builds initiated from the command line don't have a buildrequest
associated with them. The build.buildrequest association is
only added if a build is triggered from toaster.

Some of the code for displaying the status of a build refers
to build.buildrequest without checking whether it has been set,
which causes an error to be thrown.

Add a guard to check whether the buildrequest has been set.

[YOCTO #8277]

(Bitbake rev: af33409612139ab2347baf6b847b23faea19752d)

Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-10-16 14:13:22 +01:00
Elliot Smith 1feeb8e499 bitbake: toaster: Always run bldcontrol migrations
The toaster startup script conditionally migrates the database
tables depending on whether you are in managed mode or not. This
means that if you are in analysis mode, some of the bldcontrol*
database tables used by managed mode are not available.

As a consequence, some of the code in toaster which refers to
those tables can break in analysis mode, as there's no clean
isolation of the two modes.

To prevent this from happening, always run the migrations for
managed mode and create the bldcontrol* tables, even if in
analysis mode.

Also clean up the function which starts up toaster so the
logic is easier to follow.

[YOCTO #8277]

(Bitbake rev: b1fc592131286ebbede2693be8c86636f0039011)

Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-10-16 14:13:22 +01:00
Michael Wood ae82d778e6 bitbake: toaster: buildinfohelper Detect command line builds
When we're building using toaster as just a listener to bitbake
(analysis mode) we need to handle the case where the toaster configuration data
isn't present so we don't need to try and update the existing information.

(Bitbake rev: a22faae2c3a5948356ce3cbc73c34509de65d370)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-10-16 14:13:22 +01:00
Belen Barros Pena 596c219bf4 bitbake: toaster: Disable add layer button when input is empty
The 'add layer' button in the project configuration page
is enabled when you select a layer from the type ahead.
However, if you delete the layer name, the 'add layer'
button remains enabled, and if you click it, the last
selected layer from the type ahead will be added to the
project.

It is probably better to disable the 'add layer' button
when the input field is empty.

[YOCTO #8449]

(Bitbake rev: d4820f16c76398400ddd573db03c654c51d40c1b)

Signed-off-by: Belen Barros Pena <belen.barros.pena@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-10-16 14:13:22 +01:00
Belen Barros Pena 24e5a178e1 bitbake: toaster: Have 'Version' next to recipe name
In all Toaster tables we show the version next to the recipe
or package names, with one exception: the recipes table in
the layer details page, where we are showing the version
next to the recipe description.

This patch moves the version column next to the recipe name,
for consistency with all other Toaster tables.

(Bitbake rev: b5af7084b28158a8a9eaf78db463555ae8e0f620)

Signed-off-by: Belen Barros Pena <belen.barros.pena@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-10-16 14:13:22 +01:00
Belen Barros Pena c895838189 bitbake: toaster: Improve directory structure layout
File names in the directory structure often wrap
due to big indenting for nested files and directories,
unnecessary width in certain columns, and not using
colspan when the symlink and package cells are empty.
The wrapping makes the table harder to read.

This patch reduces the amount of indenting, limits the
width of the 'Size', 'Permissions', 'Owner' and 'Group'
columns, and sets colspan to use the white space of the
symlink and package names when empty.

(Bitbake rev: 6167ee5a7569d8f841c340e672645cc133ea5a31)

Signed-off-by: Belen Barros Pena <belen.barros.pena@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-10-16 14:13:21 +01:00
Ed Bartosh 2f52ef4b2f bitbake: toaster: importlayer Update property names for importlayer api calls
Update the property names used in checking for existing layers. The
server side API changed but not all references were updated.

[YOCTO #8300]

(Bitbake rev: 1cbf0cf77a638257e18066f911fe4c4e13c278c6)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-10-16 14:13:21 +01:00
Michael Wood d302c98822 bitbake: toaster: Fix missing tooltips from layers on project configuration page
Re-enable the layer tooltips on the project configuration page.
This adds the required fields to the API used for the layer dependencies
to be able to show the metadata needed.

Also fixes link hrefs which were missing in the add layers confirmation
dialog (bug 8251).

[YOCTO #8295]
[YOCTO #8251]

(Bitbake rev: c7cb8255d0ab1fd7715e878c1c83ef0cd920387e)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-10-13 15:35:49 +03:00
Ed Bartosh 7e5464b451 bitbake: toaster: Fix broken test case
Fixed test_show_tasks_in_projectbuilds broken by latest changes in
project build view.

(Bitbake rev: 3c72c7634ab69a5eb18aa20a5c6d16a3e2666f62)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-10-12 14:38:57 +01:00
Ed Bartosh 2e375e63bf bitbake: toaster: exclude recipes with empty names
Modified get_all_compatible_recipes function to exclude recipes
with empty names from the result queryset. This should stop UI
to show recipes with empty names in compatible recipes page.

[YOCTO #7969]

(Bitbake rev: f7c1bd49e2e28d12c6604f5ae54bc96e1e7b6946)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-10-12 14:38:57 +01:00