The build dashboard doesn't show image and artifact files correctly,
as it shows the full filename for images and the filename plus
path relative to DEPLOY_DIR for artifacts.
Instead, show just the suffix for image files, and the basename
for artifact files.
(Bitbake rev: 8084dcdc283b4dc170f066c202f89d56ce1abbef)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Revert "toaster: models.py do not use transactions".
This reverts commit e248a503ba
(Bitbake rev: 48d0d510816346073f135bb86fb7904fdb50bb27)
Manually managing the database transactions caused more
problems than it temporarily solved so we return control back to
Django's orm.
[YOCTO #8796]
(Bitbake rev: 25c531915b6f8f79a0822996ceb97f90483e359f)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Allow package groups in our custom image recipe. Excluding them creates
more undefined behaviour than including them at this stage. Also update
to use convenience method for returning all packages.
(Bitbake rev: 8c2e8a13badacb816c4b1178b6661600008b38af)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a method to update the packages included list from the last
build, this effectively "synchronises" the package list from what
we think will happen at the Customise image stage with what
actually was produced with a build. It's not ideal to have this
function here but we also need to make sure that no race condition
of the user accessing this list and it being updated occurs.
(Bitbake rev: 8cf6e67a955574b33856a082bdadf3194f2b6ba4)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Field to keep track of when the package list for the CustomImageRecipe
was last updated from a build.
(Bitbake rev: 4bd4e49f13a7625997a43f3b2e67ed42c3c8e08b)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a convenience method to get the last successful build
target for a CustomImageRecipe.
(Bitbake rev: 4dde3d830cd38bbe306d83629dcb80da5fc9b027)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This convenience method returns just the Package_Dependency for
the package which are regular dependencies i.e. not RECOMMENDS or
any other types.
(Bitbake rev: bd76c22fe2aa06690b4ee25de69219ac0bf6b4d6)
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Whem a m2m field changes we need to clear the ToasterTables cache as
this can affect the state of items in ToasterTables.
For example the CustomImagePackages being added or removed from a custom
image recipe.
(Bitbake rev: c9d7b68ee0186a71e8e75a5d87122a0328001515)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If we're breaking up an image recipe's packages we will also need to
override IMAGE_FEATURES to make sure the customisation is not altered
(Bitbake rev: 9fd7b05dc0cf9240f7c8e3dc77b009064fd2b0cb)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When the packages associated with a custom image recipe are
shown in the customrecipe editing page, locale packages are
shown in the same way as all other packages. This gives the false
impression that these packages can be removed, when in fact
they are automatically added due to the IMAGE_LINGUAS build
variable.
Modify the customrecipe page so that locale packages cannot be
removed, and provide some help text explaining why.
[YOCTO #8927]
(Bitbake rev: b2208e53c00a67a7d0345e7378e6806b8ae40fb4)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The progress updater for the recent builds section makes a JSON
call to the project view URL to get progress for each build.
However, conversion of the builds pages to ToasterTable broke this,
as the JSON response no longer contained the data necessary to
populate the progress bars.
Move the recent builds query to the Build model, so that it is
accessible to the ToasterTables using it ("project builds" and
"all builds"), as well as to the "project" view.
Modify the code in the recent builds template to use the slightly
different objects returned by the recent builds query on Build.
(Bitbake rev: 5189252635ddc7b90c9a43aaed9f196c31e1dcad)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a special case for when the recipe we have based a custom image
recipe on requires another recipe.
In this case we need to adjust the file location to be able to
require
the recipe when we're in the toaster-custom-images layer.
For example:
"require core-image-minimal.bb" is changed to:
"require recipes-core/images/core-image-minimal.bb"
(Bitbake rev: 26025e1ea49b3ebfcfd508d1608fa8c9e722ad03)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Returns a queryset of the all the packages that we expect to have in a
CustomImageRecipe.
(Bitbake rev: 8b03bbae12ec077151c97579e329d89667040a78)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Instead of returning layercommits return the actual Layer_Version objects
for the layercommit as these are the useful objects which contain the
metadata.
(Bitbake rev: 480f0d307a3183df16fd383c7a3f96f00d09fbd4)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add function generate_recipe_file_contents to dump the custom image
recipe instance to a string for use either to push to the user as a
downloaded version of their custom image recipe or to use to generate
the recipe that we build.
(Bitbake rev: 6863343c3434ce19aa4b609c83f48a06e6943366)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This table is used to track all the available packages in the current
toaster. Many of these packages belong to many CustomImageRecipes.
(Bitbake rev: c1bd4f760cd35535e44f488250e0a56b99cad680)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This allows us to re-use the properties of a recipe for the custom image
recipes as well as re-using the existing templates and logic that deals
with recipe objects.
(Bitbake rev: bb8120b56be7eee6ed2e4434d8477282a01e0c00)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add function that returns the Sum of the size of all the packages which depend on a package.
Access get_total_source_deps_size via a packages's dependency
manager.
(Bitbake rev: e4c86d2f6c2e86ad054b37d0a5bf7464a4de4f9a)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fall back to a 'n/a' string for the vcs reference, not all our source
has to be in a vcs and therefore it is legitimate for this to be none.
(Bitbake rev: 1739b509e9efc4b016fc73c2d4399f1f9d3d285f)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Minor fixes to code formatting and small improvements from
code review.
(Bitbake rev: 2c97f3a5c6ae37de910deb90390c5b856a600c5f)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
For better long-term maintainability, use ToasterTable instead
of Django template and view code to display the all builds page.
NB the builds.html template has been left in, as this will
otherwise cause conflicts when merging the new theme.
[YOCTO #8738]
(Bitbake rev: e0590fc8103afeb4c5e613a826057555c8193d59)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The algorithm for finding the suffix for image files produced by
the build doesn't reference a list of known file suffixes, so
could be prone to error.
Modify how file suffixes are parsed from the file path so that
they are compared against a list of known types; if this fails,
use the part of the basename of the file path after the first
'.' character.
Also rationalise the places in the views code where we
extract the file name extensions for builds, so they both use
the same algorithm (before, the same code was duplicated in
two places).
[YOCTO #8417]
(Bitbake rev: dd1c509696b8ab5e593cc64637060a58e95fcd1f)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Image file suffixes are used in the project configuration page to
show a list of available image file types. This list is stored
as a function in the views code.
However, this list is also needed when parsing image file paths,
so that the suffixes can be shown in the "all builds" and "project
builds" tables.
Move the list of valid image file suffixes to the Target_Image_File
class to make is accessible in other places where it may be needed.
[YOCTO #8738]
(Bitbake rev: c2f20232077917552623fd0726d0820e50b04cae)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Added new model Provider and a foreign key 'via' to link
Recipe_Dependency to it.
[YOCTO #6169]
(Bitbake rev: e45fff6314741d46e2549b2f72ed380cbbb95593)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fixed 'database is locked' issue by monkey patching django QuerySet
methods.
The actual patching places were found by bisecting Django codebase.
This commit should be removed after Django is fixed if it's fixed
at all.
(Bitbake rev: 175411bf05423b1892c7928c2b928843b39645f0)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When sqlite can not cope with a stream of 'insert' queries it throws
'database is locked' exception.
Wrapping model.save in transaction.atomic context and repeating the call
should solve this issue.
(Bitbake rev: eb305308ca8f6228c6f52dac1bd941f29c7e5eb6)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Printing id as a number breaks toaster if object is not saved
yet as id/pk is None in this case.
Changed format spec to print id as a string in __str__
and __unicode__ methods of Build, Task and LayerVersion models.
(Bitbake rev: d5ca2bae623db214764324da5c9e4dac6beef760)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Remove compatible_layerversions() and replace calls to it with
calls to get_all_compatible_layer_versions().
The sorting done in compatible_layerversions() is not relevant,
and the code can be simplified by using get_all_compatible_layer_versions().
[YOCTO #8364]
(Bitbake rev: 7bca51277314f7c0b6ee0e0d470327dfd60d37fe)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Renamed get_query_set to get_queryset as get_queryset causes
deprecation warning in Django 1.7 and is deprecated in Django 1.8
(Bitbake rev: ef8b399dccd413070e32ce05e013337ea01fa64d)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Rather than maintain data as part of the migrations (as was
done for the default project previously), create the default
(cli builds) project on demand as a by-product of getting
it from the database.
[YOCTO #8364]
(Bitbake rev: 5fd8e90ab9b81d1bd0d301bc1c91228ecbbea74b)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In order to search the model from the UI some fields must be nominated
as searchable.
(Bitbake rev: 2558729b4c248a2fa5a11d877bc42cb05a30602e)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The count of a project's builds should not include those which are
currently in progress.
(Bitbake rev: a981700701c41c7bbb6a9778e95f691278c5c294)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Log messages can be quite long so use a TextField rather than a char
field with max length of 240. mySQL is especially picky about field
lengths and will cause an exception if the log is too long.
[YOCTO #8475]
(Bitbake rev: 349119f44cdad240dda7f4db45992e2539e1416f)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fatal build errors were displayed as exceptions, and highlighted
with less severity than they deserved.
Roll back to treating Toaster exceptions as errors by removing
the toaster_exceptions member on Build objects and displaying
EXCEPTION events in the errors section on the dashboard.
[YOCTO #8320]
(Bitbake rev: 59197320bdcefddf06084e871f1b5b21b21cbb63)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Critical errors (where a build failed for reasons of
misconfiguration, such as a machine being specified which is not
in a project's layers) were being ignored (only log records
up to ERROR level were being logged to Toaster's db). This meant that
the build would fail but would not correctly report why.
Add support for CRITICAL error levels to the LogMessage model,
include errors at this level in the errors property for a build,
and show errors at this level in the build dashboard.
[YOCTO #8320]
(Bitbake rev: b6eacbca9cacb607de864ab7d093deb296da8226)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Builds initiated from the command line don't have a buildrequest
associated with them. The build.buildrequest association is
only added if a build is triggered from toaster.
Some of the code for displaying the status of a build refers
to build.buildrequest without checking whether it has been set,
which causes an error to be thrown.
Add a guard to check whether the buildrequest has been set.
[YOCTO #8277]
(Bitbake rev: af33409612139ab2347baf6b847b23faea19752d)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Modified get_all_compatible_recipes function to exclude recipes
with empty names from the result queryset. This should stop UI
to show recipes with empty names in compatible recipes page.
[YOCTO #7969]
(Bitbake rev: f7c1bd49e2e28d12c6604f5ae54bc96e1e7b6946)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Due to Recipe model constraints some recipes can't be
saved. However, they still can create incomplete records in
the database. This causes all sorts of errors when Toaster
operates with those objects. Removing them should fix those
issues.
[YOCTO #7969]
(Bitbake rev: 4d76a9e418fd98a7882aa29f974a7389f9689314)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fixed exception: NameError: global name 'DoesNotExist' is not defined
(Bitbake rev: eee5311b867d3c8c33e06d04e103bfd3647146f4)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Implemented Layer_Version.get_alldeps API to recursively get
full list of dependencies for the layer. Dependencies that are
already in the project are filtered out from the result.
Result list of Layer_Version objects is sorted by layer name
for UI to look consistent.
This API is going to be used to show amount and list of
dependencies for the layer in the list of compatible layers
for the project.
(Bitbake rev: 7d853a3054a9ae3d18eb6f5bc13ba27d2795c31a)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This was accidentally returning the primary key of the ProjectLayer
rather than the ProjectLayer.Layer_Version.pk
(Bitbake rev: b20f3626148e89af0e9fcfca911a5a1e4e355a41)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fix typo don't pass self in as a parameter, this evaluated to true
giving the wrong results meaning the machines typeahead did not return
valid results.
(Bitbake rev: 55ba889ef8900c95447861fa3985ca9cfe06afdf)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We no longer need to compute each layer_version and all the recipes
which belong to this.
[YOCTO #8147]
(Bitbake rev: 505979ab931e3a2a218d7030d6064987e8f9ff14)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Create a copy of the built layer and the recipes associated with it.
This is so that the user can view the historical information about a
build. i.e. a snapshot of the layer version and artifacts produced at
that build.
(Bitbake rev: 0683d9a2b15e3234a94437aaebac84bfcca1420b)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When we do a build we update the last commit value that the layer was built at
However in future builds we do want to use the named reference rather
than the commit sha, e.g. master/fido
(Bitbake rev: a3d47eb923dd2cb5259f387c79549822807eca49)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Previously this layer relationship was done by trying to match path
information that came back to the buildinfohelper with trying to query for
data in toaster's layers table. This rarely matched due to the lose coupling.
(Bitbake rev: 838e77c7c3c4006abd1327343a004590ab652de9)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If the openembedded-core layer is specified in the toasterconf we need
to treat it differently because we may also get this layer either from
the layerindex source or I can also be provided locally.
(Bitbake rev: 8e36b6848c14f2708e1f87d12fb533ca0b596c65)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This model lists custom image recipes for the project.
It is populated when new custom image is created.
It holds reference to the base recipe and list of packages
included into custom image.
For CustomImageRecipes the packages will be copied in and
therefore not associated with a build so Remove the requirement
for the package to have a Build.
co-author: Ed Bartosh <ed.bartosh@linux.intel.com>
(Bitbake rev: 33cbf4cd3b4ca47c4901501f5f1eafdfdfdae023)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
For a new style indexes 'inherits' field is used to determine if
recipe is an image recipe.
As old style indexes don't have 'inherits' field this can be
guessed from recipe name. Let's consider recipe an image recipe
if recipe name contains '-image-'.
(Bitbake rev: d769d3f54faaf1a3091b1eb6ed1d17a8a108210a)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Information about a task is not stored in Target objects.
This makes it impossible to correctly operate with the builds
where task is specified.
Storing taks name is an enabler for other fixes in UI and backend
related to restarting builds.
(Bitbake rev: 0a69a8a18075c976ed8681d9d75529f8c2f48514)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
