Commit Graph

257 Commits

Author SHA1 Message Date
David Reyna 372f72f2ee bitbake: toaster: recipe links broken for default layers
The default layers are missing the recipe link definitions in
the fixture files, and because they are predefined they do not
get the updated information from the Layer Index.

[YOCTO #12006]

(Bitbake rev: b408dfae3685494ae34417e72c586b9eb0ddace9)

Signed-off-by: David Reyna <David.Reyna@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-09-21 16:51:07 +01:00
David Reyna 4b35d0a012 bitbake: toaster: get_last_build_id not called correctly
The method "get_last_build_id" missing the "()" in several calls. This
has been failing silently with Django 1.8 but intermittently crash with
Django 1.9.

[YOCTO #11570]

(Bitbake rev: 98e424c2cd3ce8bc7df247263e44a6d6d9ede3e6)

Signed-off-by: David Reyna <David.Reyna@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-07-27 22:36:46 +01:00
David Reyna c519dc1cd4 bitbake: toaster: move release lookup from morty to pyro
Move the release name lookup into the layer index
from 'morty' to the 'pyro'.

Move the bitbake branch from 1.32 to 1.34.

[YOCTO #11377]

(Bitbake rev: 21d963149b5d97452420230a252101115b708d85)

Signed-off-by: David Reyna <David.Reyna@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-20 07:59:40 +01:00
Sujith H d4b5a70831 bitbake: bitbake: toaster: add wic.bmap support to toaster
Add wic.bmap file extension support for toaster

(Bitbake rev: 6a3f6f1b5cc9ec69b28bebf70daed7579c60cd9e)

Signed-off-by: Sujith H <sujith.h@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-02-15 20:08:15 -08:00
Michael Wood 90d995c637 bitbake: toaster: models Layer_Version get_detailspage_url optional project_id
Allow passing none as the project id, this is convenient for layers
which belong to projects already and therefore have their own project
field.

Add documentation string to function

(Bitbake rev: 78dc7b9f1801e7f4c266ba1369e5706f177ddaa1)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-12 20:44:53 +00:00
Michael Wood 9dbcc0fbb3 bitbake: toaster: models Layer fix whitespace
(Bitbake rev: 8116cb4f71de09eff124cd48d80cc7a7c64da5e1)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-12 20:44:53 +00:00
Michael Wood 6f8df05e6e bitbake: toaster: orm models Handle CustomImageRecipe BRLayer here
The schedule_build function on the project object is where the BRLayers
are created for the build. Instead of creating the BRLayer for the
CustomImageRecipe in the localhostbbcontroller create it here so that
all that mechanism is in one place.

Also fix a number of pyflake errors.

(Bitbake rev: f8d3ea784937b6e416d3e5a4feb1283c478e4caa)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-11-30 15:48:10 +00:00
Michael Wood b58b982eaf bitbake: toaster: orm models Project class Fix pyflake errors
(Bitbake rev: 69f33397083f54f977fa0cd4dd621731f32fd034)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-11-30 15:48:10 +00:00
Michael Wood 30a9f65dcc bitbake: toaster: buildinfohelper toaster-custom-images layer
This fixes the unidentified layers issue by making the
toaster-custom-images layer a local layer. By doing this we also fix the
git assumptions made for the local layers which stop recipes and other
meta data being associated with them. This also removed some of the
special casing previously needed when we didn't have the concept of a
local (non git) layer.

Also rename created flag var to a have a different var for each returned
value so that the same value isn't used multiple times.

[YOCTO #10220]

(Bitbake rev: ba5332d4960d7f4f79aef63136796e2fa67284e3)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-11-30 15:48:10 +00:00
Reyna, David 13c0ffc81c bitbake: toaster: orm gen_layerdeps Protect against circular Layer dependencies
Limit the recursion (to say 20 levels) when processing layer dependencies
so that circular dependecies do not cause infinite decent and an
out-of-memory failure. The duplicate found layers are already immediately
filtered in the code.

[YOCTO #10630]

(Bitbake rev: e9efef0bdb8068984c3013b87aac9e872ffb38ae)

Signed-off-by: David Reyna <David.Reyna@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-11-30 15:48:10 +00:00
Michael Wood 310a9e5d35 bitbake: toaster: runbuilds Write the pidfile in python rather than shell script
Write the pid file out in the start up of this management command. This
ensures this has happened instead of relying on the shell command having
been run which may or may not be the case. This also makes it simpler for
testing.

Couple of clean ups of runbuilds as identified by pyflake

(Bitbake rev: 999e980ee1a58d16f33ef6c0e41aecdcd0206f39)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-11-30 15:48:09 +00:00
Michael Wood dbf3a9fa8e bitbake: toaster: layerindex updater Take into account layers being predefined
As we can now provide layer definitions through fixtures we need to be
more clever how we update the metadata in the database to avoid
duplicate metadata being created. To do this we make more effort to
match existing data in the database and update only the fields which
will be better provided by the layer index.

This removes the need for us to special case layers which are provided
as part of poky such as openembedded-core or meta-poky which exist on
the layerindex but with different git urls.

(Bitbake rev: f981b68f66718d5b196684f4e378a5f195ff0337)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <brian.avery@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-11-23 10:59:56 +00:00
Michael Wood f59ac87b46 bitbake: toaster: orm/fixtures Add the master release and correct morty release
Add the master release option to base your project on and correct the
morty release so that for poky based setups we use the poky provided
version of the layer rather than checking out the layer from its own git
repository.

[YOCTO #10497]

(Bitbake rev: c83ab92362378b22d3f4d6119bf362f704577ca2)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <brian.avery@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-11-23 10:59:56 +00:00
Michael Wood 50cb0f596f bitbake: toaster: settings fixture Set default release to master
Now that morty has been released we now set the DEFAULT_RELEASE back to
master.

(Bitbake rev: 115cee16a9aecfcd1061bb106ebca4e861d9b296)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <brian.avery@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-11-23 10:59:56 +00:00
Ed Bartosh 98ac0b0a5b bitbake: toaster: use current directory if BUILDDIR is not set
If BUILDDIR environment variable is not set signal_runbuilds function
throws TypeError as os.getenv('BUILDDIR') returns None:

ERROR: unsupported operand type(s) for +=: 'NoneType' and 'str'
Traceback (most recent call last):
  File "bitbake/lib/bb/ui/toasterui.py", line
391, in main
    buildinfohelper.update_build_information(event, errors, warnings,
taskfailures)
  File "bitbake/lib/bb/ui/buildinfohelper.py",
line 1184, in update_build_information
    self.internal_state['build'], errors, warnings, taskfailures)
  File "bitbake/lib/bb/ui/buildinfohelper.py",
line 238, in update_build_stats_and_outcome
    signal_runbuilds()
  File "bitbake/lib/toaster/orm/models.py", line
1746, in signal_runbuilds
    '.runbuilds.pid')) as pidf:
  File "/usr/lib64/python3.4/posixpath.py", line 82, in join
    path += b
TypeError: unsupported operand type(s) for +=: 'NoneType' and 'str'

Used os.getenv('BUILDIR', '.') to make it always return meaningful
directory path. Current directory '.' will be used if BUILDDIR is
not set.

(Bitbake rev: da631152a0db3f432709a05ff15a268d784ca3ab)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-11-04 12:50:55 +00:00
Michael Wood 8e70fa1d79 bitbake: toaster: orm models Handle run builds process not yet running
During tests we may want to call the runbuilds process manually for
example when doing a "one shot" approach rather than a long running
process during tests.

(Bitbake rev: 60d3f93836da5523705b0b2e25567d1c9040ec89)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-11-04 12:50:55 +00:00
brian avery 4b94b498e2 bitbake: toaster: Update default release to Morty
Set Morty to be the default release in toaster for the Morty release
when creating new projects.

(Bitbake rev: 00f79096f639ce3a9c0b7c72cfb36f14e264733d)

Signed-off-by: brian avery <brian.avery@intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-10-15 13:21:54 +01:00
brian avery 33324267c1 bitbake: toaster: Update poky fixture for Morty release
Update the poky fixture to the Morty release.  This removes the
master branch from the release and limits it to the morty branch.
Normally, we would also support at least one past branch but the change
from Python 2.7 -> Python 3 makes that infeasible.

(Bitbake rev: 2674ca33b900f4f3f16be504d7c67d0fc69c1c2d)

Signed-off-by: brian avery <brian.avery@intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-10-15 13:21:54 +01:00
brian avery 7656dea6db bitbake: toaster: Update oe-core fixture for Morty release
Update the oe-core fixture to the Morty release.  This removes the
master branch from the release and limits it to the morty branch.
Normally, we would also support at least one past branch but the change
from Python 2.7 -> Python 3 makes that infeasible.

(Bitbake rev: 93f1e6c3d022b1f12a230879160efa941cb1e250)

Signed-off-by: brian avery <brian.avery@intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-10-15 13:21:54 +01:00
Ed Bartosh e9a12e5d37 bitbake: toaster: notify runbuilds when build status changes
Called signal_runbuilds API when build is scheduled, cancelled or
finished to notify runbuilds process about builds status change.

[YOCTO #8918]

(Bitbake rev: fe08f0fa4b328908e73695ebbceca87bc86a49f9)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-21 21:58:06 +01:00
Ed Bartosh 2478a6f208 bitbake: toaster: implement signal_runbuilds function
This function reads pid of runbuilds process from
BUILDDIR/.runbuilds.pid and sends SIGUSR1 to it. signal_runbuilds
function will be used in Toaster code to notify runbuilds when
build is scheduled, finished or cancelled.

(Bitbake rev: 62955224a6d99e9f581d2bef924058070bfa4c43)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-21 21:58:06 +01:00
Belen Barros Pena 87cf84fead bitbake: toaster: orm Update IMAGE_FSTYPES values
This patch fixes a typo in one of the IMAGE_FSTYPES values listed in
Toaster. It also updates the hardcoded list of values to match the
latest list in meta/classes/image_types.bbclass

[YOCTO #9447]

(Bitbake rev: 46db3279cb81b3ca6ce047204aee620f5ee51220)

Signed-off-by: Belen Barros Pena <belen.barros.pena@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-08 00:33:47 +01:00
Michael Wood 3b87f2895a bitbake: toaster: Fix oe-core fixture
Due to a copy paste error we managed to get some of the wrong
information in the oe fixture that provides a suggested default settings
for Toaster. This meant it tested correctly when it shouldn't have.
Fix:
 - The use of local bitbake
 - An incorrect call to realpath which didn't include its parent module.
 - The field used for the local_dir of an existing openembedded-core

(Bitbake rev: d57a9124650e5367919668dfccf6aad4962a77f1)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-02 18:09:50 +01:00
David Reyna 40b655db22 bitbake: toaster: update web urls for openembedded-core's special case
The layer index update command has a special case for the
updating 'openembedded-core' layer, and it was missing reading
and updating the git web URL fields.

[YOCTO #8037]

(Bitbake rev: ce2f990a366d2d939e93e01f67688f12740c5fee)

Signed-off-by: David Reyna <david.reyna@windriver.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-08-17 10:22:58 +01:00
Michael Wood 4ac57fb73e bitbake: toaster: bldcontrol model BRLayer Add corresponding local_source_dir
Sync the BRLayer object with the new field added to the Layer object.
The BRLayer (BuildRequest Layers) are snapshots of the layers in the
project at build time and therefore need to mirror the required fields
of the layer object.

(Bitbake rev: a3112c922f036425977abffa0137b9133f61fcd6)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-08-11 00:09:29 +01:00
Sujith H 8cad51965d bitbake: toaster: add local_source_dir field to model
Add a new field local_source_dir to model.
This will clearly differentiate us from the
vcs_url which is for git path.
Adding migration file 0010_layer_local_source_dir_path.py
along with this patch.

[YOCO #9911]

(Bitbake rev: 83763d89f9d0bc535e930a2094ba8201675d40be)

Signed-off-by: Sujith H <sujith.h@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-08-11 00:09:28 +01:00
Ed Bartosh 953ba7a221 bitbake: toaster: fix unhandled exception
manage.py lsupdates throws DataError exception if the recipe can't be
saved to the MySQL database:
    django.db.utils.DataError: (1406, "Data too long for column 'license' at row 1"

Adding DataError exception to the list of exceptions should make
lsupdates to print a warning message and skip the recipe.

(Bitbake rev: 01891c13ae8dcba64e5cf79956f1b64c2afdaae3)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-08-11 00:09:27 +01:00
Elliot Smith e387a7ab9d bitbake: toaster: add started property to Build
Add a property to the Build model which records whether
the BuildStarted event has occurred for the build.

The proxy for this event is the presence of variables recorded
against the Build: as the buildinfohelper only saves variables
when the BuildStarted event occurs (as the variables aren't
available on the bitbake server before that point), we can
tell whether BuildStarted has happened by counting Variable
objects on the Build.

This can then be used to determine whether a Build "properly"
started, enabling a different dashboard display (left-hand menu
hidden) if the build didn't record any useful information (e.g.
if it had a bad target).

[YOCTO #8443]

(Bitbake rev: aa151a4d2de4a54fe3075a8c56a4935158398a18)

Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-08-11 00:09:26 +01:00
Elliot Smith 82d998d6c6 bitbake: toaster: show "Tasks starting..." until the first task completes
To prevent showing a "0% of tasks complete" message for a long time,
don't show the progress bar until the first task has finished.

While waiting for that first task, show a message about tasks
starting instead.

[YOCTO #9631]

(Bitbake rev: 5529bcd860d2932b967a064ae28690ac5a725342)

Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-08-11 00:09:26 +01:00
Elliot Smith dd99cf957d bitbake: toaster: show progress of recipe parsing in recent builds area
Modify buildinfohelper and toasterui so that they record the
recipe parse progress (from ParseProgress events in bitbake)
on the Build object.

Note that because the Build object is now created at the
point when ParseStarted occurs, it is necessary to set the
build name to the empty string initially (hence the migration).
The build name can be set when the build properly starts,
i.e. at the BuildStarted event.

Then use this additional data to determine whether a Build
is in a "Parsing" state, and report this in the JSON API.
This enables the most recent builds area to show the recipe
parse progress.

Add additional logic to update the progress bar if the progress
for a build object changes.

[YOCTO #9631]

(Bitbake rev: f33d51d46d70e73e04e325807c1bc4eb68462f7b)

Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-08-11 00:09:26 +01:00
Elliot Smith 952ffb3e1f bitbake: toaster: move most recent builds templating to client
The most recent builds area of the all builds and project builds
table needs to update as a build progresses. It also needs
additional functionality to show other states (e.g. recipe parsing,
queued) which again needs to update on the client side.

Rather than add to the existing mix of server-side templating
with client-side DOM updating, translate all of the server-side
templates to client-side ones (jsrender), and add logic which
updates the most recent builds area as the state of a build changes.

Add a JSON API for mostrecentbuilds, which returns the state of
all "recent" builds. Fetch this via Ajax from the build dashboard
(rather than fetching the ad hoc API as in the previous version).

Then, as new states for builds are fetched via Ajax, determine
whether the build state has changed completely, or whether the progress
has just updated. If the state completely changed, re-render the
template on the client side for that build. If only the progress
changed, just update the progress bar. (NB this fixes the
task progress bar so it works for the project builds and all builds
pages.)

In cases where the builds table needs to update as the result of
a build finishing, reload the whole page.

This work highlighted a variety of other issues, such as
build requests not being able to change state as necessary. This
was one part of the cause of the "cancelling build..." state
being fragile and disappearing entirely when the page refreshed.
The cancelling state now persists between page reloads, as the
logic for determining whether a build is cancelling is now on
the Build object itself.

Note that jsrender is redistributed as part of Toaster, so
a note was added to LICENSE to that effect.

[YOCTO #9631]

(Bitbake rev: c868ea036aa34b387a72ec5116a66b2cd863995b)

Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-08-11 00:09:26 +01:00
Michael Wood f4e65d1cd0 bitbake: toaster: fixtures Add README
Add README to explain fixtures directory

(Bitbake rev: 4060776e408ec6baec3bafba79ac5db1bc380f01)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-08-11 00:09:26 +01:00
Michael Wood 49039829e1 bitbake: toaster: lsupdates Add spinner for parsing/http fetch
Adds a spinner so that you know that the parse and http fetch from the
layerindex is in progress.

(Bitbake rev: e1c1c8827f3892551084bf1c0909c1b33f0dca83)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-08-11 00:09:25 +01:00
Michael Wood 72fa18fb35 bitbake: toaster: Add poky and openembedded configuration fixtures
These fixtures provide a recommended default configuration of toaster for
either using bitbake and oe-core or as part of poky.
They can be used as the sample configuration for writing custom configurations.

(Bitbake rev: d2f4ffbf061e874a7731441f8346fdc568e9f07f)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-08-11 00:09:25 +01:00
Richard Purdie aa2d945423 bitbake: lib/toaster: Fix missing new files from previous commits
(Bitbake rev: f77e6f21a2cc57a3fcb5970437e55cfae39849a3)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-08-01 09:38:35 +01:00
Michael Wood 04d1ad5fe7 bitbake: toaster: admin Add Layer_Version to the admin-able models
If the migration didn't get the release conversion right for say, a
local or imported layer it would be handy to be able to edit this
in the django admin page.

Also useful for developers to be able to tweak layers on the fly.

(Bitbake rev: 0b23b6919ef1162a0c7fb5b5a961a24653c51eb2)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-26 08:10:36 +01:00
Michael Wood 97278fb51c bitbake: toaster: orm Remove the layerindex specific up_branch fields
We don't need to keep track of layerindex data in our database. And
using branch==release is very confusing in the schema. Instead use the
existing Release definition to keep track of which release a
layer_version is for.

Remove the Branch model and all references to it.

Create a migration path to convert from up_branches to their
corresponding releases.

(Bitbake rev: f8f4cffe6fd371f3a7e63690c68f3fcb5dc1f297)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-26 08:10:36 +01:00
Michael Wood 8b3146007f bitbake: toaster: lsupdates Add progress information and clean up logging
Adds basic progress % information and provides better description of
what is happening.

(Bitbake rev: 6393dbf97d450d2521c0bc9429da0987bb7720ec)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-26 08:10:36 +01:00
Michael Wood bb260e94a6 bitbake: toaster: tests: Remove references to LayerSource model
Remove and replace layersource model references in the tests and test
data. Remove the orm/test as this only tested LayerSource interactions
which have now been removed.

(Bitbake rev: 61a47cbc92c856690cb0e8da7102b2e669eaee0a)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-26 08:10:36 +01:00
Michael Wood ef627d0ab8 bitbake: toaster: Replace references to LayerSource models
Replace references to the now deprecated layersource models across
Toaster with the new enums for layer source types.

(Bitbake rev: 48c09c62eb979d840132e58144f0d81ffee675b1)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-26 08:10:36 +01:00
Michael Wood ffc78d329d bitbake: toaster: lsupdates Add layerindex fetcher
Move and refactor the layerindex layer source update mechanism so that
we don't have to track the layerindex objects in the toaster database.
Move this out of the orm and into the management command.

Paves the way for future improvement to allow you to specify a layer
index server as an argument to the command.

(Bitbake rev: f83527edc6d52a34cd73a9c3650ee484407e2e0c)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-26 08:10:36 +01:00
Michael Wood 853450befc bitbake: toaster: models Remove LayerSource models and replace with enum
We had a lot of complexity around different layer sources to allow for
multiple sources and different priorities for these source. This was
implemented using rigged abstract classes which represented the
different layer sources when in fact just an enum/flag on the
layer_version object is sufficient for our needs.

Remove the LayerSourcePriority object as this is not needed. We no longer
have a problem of multiple layers coming from multiple sources so this
is not needed. Two migrations are added to first remove the child models
which represented layersources. Then a second migration is needed to
remove the LayerSource model it's self as Django can't understand the
non-standard base class dependency. Triggering this issue:
https://docs.djangoproject.com/en/1.8/topics/migrations/#dependencies

Clean up a number of flake8 warnings in classes which were modified.

[YOCTO #9853]

(Bitbake rev: 26624740418de95adb1f4a970a30a5f1149ebb79)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-26 08:10:35 +01:00
Elliot Smith 9475a684c4 bitbake: toaster: the customise image button shouldn't rely on targets
The build dashboard customise image button (for creating a new
custom image based on an image recipe used by a build) shouldn't
rely on targets: whether a new custom image can be created or not
depends on whether any of the recipes used by the build are image
recipes.

Modify the method used to determine whether a build has customisable
images to look at the image recipes used during the build, rather
than whether the targets run by the build refer to image recipes.

(Bitbake rev: 6648876c91134bda8498b4f8d7ace9147ec0d985)

Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: bavery <brian.avery@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-19 08:56:52 +01:00
Elliot Smith 2db40e3771 bitbake: toaster: add package manifest path to Target objects
Store the path to the *.rootfs.manifest file for targets which
generate images.

A link to the package manifest is displayed in the build dashboard
for targets which produce image files.

Like the license manifest path, if a target would have produced
the package manifest (but didn't, because it already existed), that
path is copied from the target which did produce the package
manifest.

(Bitbake rev: 79b8e349a0da2ea6b97ad82daa5837e6dfffe0af)

Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: bavery <brian.avery@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-19 08:56:52 +01:00
Elliot Smith 07a58a8944 bitbake: toaster: only show "New custom image" button for builds with image targets
Add a has_image_targets() method to Build, and use that to hide
the "New custom image" button on the build dashboard if a build
has no targets which build images.

[YOCTO #9514]

(Bitbake rev: 3c4b053e44ea512ef2ced67289a7b0161db6ce9b)

Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: bavery <brian.avery@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-19 08:56:52 +01:00
Elliot Smith 587275eefd bitbake: toaster: use has_images() methods to display images correctly
In the build dashboard, we had issues with showing images correctly,
as we were using the is_image property of targets to determine
whether a target would have image files. This property can
be set to True if a target refers to an image recipe
(e.g. "core-image-minimal"), even if the task used in the build
didn't produce any image files.

By adding has_images() methods to the Target and Build objects,
which count associated Target_Image_File objects,
we can correctly determine whether a target has image files
associated with it, and if any of the targets for a build has
image files. This means that we can screen out the left-hand
"Images" menu options for builds which contained image-related
targets (e.g. "core-image-minimal") but which didn't produce
any images (e.g. "rootfs" task).

[YOCTO #9500]
[YOCTO #9784]

(Bitbake rev: f6bba0ff254d5ed3163151d4b938f3a43c9acb0a)

Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: bavery <brian.avery@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-19 08:56:51 +01:00
Elliot Smith 00c2c0be5e bitbake: toaster: improve scan for SDK artifacts
SDK artifacts were previously picked up by toaster.bbclass and
notified to buildinfohelper (via toasterui). The artifacts
were then added to the Build object, so that it wasn't clear
which artifact went with which target; we were also unable
to attach SDK artifacts to a Build if they had already been
attached to a previous build.

Now, toaster.bbclass just notifies the TOOLCHAIN_OUTPUTNAME when
a populate_sdk* target completes. The scan is moved to buildinfohelper,
where we search the SDK deploy directory for files matching
TOOLCHAIN_OUTPUTNAME and attach them to targets (not builds).

If an SDK file is not produced by a target, we now look for a
similar, previously-run target which did produce artifacts.
If there is one, we clone the SDK artifacts from that target
onto the current one.

This all means that we can show SDK artifacts by target, and should
always get artifacts associated with a target, regardless of whether
it really build them.

This requires an additional model, TargetSDKFile, which tracks
the size and path of SDK artifact files with respect to Target
objects.

[YOCTO #8556]

(Bitbake rev: 5e650c611605507e1e0d1588cd5eb6535c2d34fc)

Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: bavery <brian.avery@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-19 08:56:51 +01:00
Elliot Smith 4125da7763 bitbake: toaster: attach kernel artifacts to targets
The bzImage and modules files were previously attached to a build,
rather than to the target which produced them. This meant it was
not possible to determine which kernel artifact produced by a
build came from which target; which in turn made it difficult to
associate existing kernel artifact with targets when those
targets didn't produce artifacts (e.g. if the same machine + target
combination was built again and didn't produce a bzImage or modules
file because those files already existed).

By associating kernel artifacts with the target (via a new
TargetArtifactFile model), we make it possible to find all
the artifacts for a given machine + target combination. Then, in
cases where a build is completed but its targets don't produce
any artifacts, we can find a previous Target object with the same
machine + target and copy its artifacts to the targets for a
just-completed build.

Note that this doesn't cover SDK artifacts yet, which are still
retrieved in toaster.bbclass and show up as "Other artifacts",
lumped together for the whole build rather than by target.

[YOCTO #8556]

(Bitbake rev: 9b151416e428c2565a27d89116439f9a8d578e3d)

Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: bavery <brian.avery@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-19 08:56:51 +01:00
Elliot Smith e9808576da bitbake: toaster: improve image file suffix retrieval
Refactor retrieval of suffix from image file path, making it a
a method on Target_Image_File. This makes it easier to use this
in the build dashboard for individual images, plus reduces the
complexity of the code required to get all of the image file
suffixes for a build.

(Bitbake rev: 9c38de3dec74c122c2060cad37331bdafc6858ec)

Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: bavery <brian.avery@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-19 08:56:51 +01:00
Ed Bartosh aa6894a436 bitbake: toaster: fix wrong usage of print_exc and format_exc
First parameter of traceback.print_exc and traceback.format_exc APIs is
a 'limit' - a number of stracktraces to print.

Passing exception object to print_exc or format_exc is incorrect, but
it works in Python 2 and causes printing only one line of traceback.

In Python 3 comparison of integer and exception object throws exception:
TypeError: unorderable types: int() < <Exception type>()

As these APIs are usually used in except block of handling another
exception this can cause hard to find and debug bugs.

(Bitbake rev: c5a48931ac8db9e56f978c50861c19d0d0c808e3)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-06-15 08:35:06 +01:00