When returning the compatible layers make sure that we are only
listing: All the layers which are for this release && configuration
layers (i.e. aren't part of the build history) and which aren't an
imported layer OR are this project's imported layer(s).
[YOCTO #8944]
(Bitbake rev: de8baedaccb451c12bc3f642449db3f64aed6bf7)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The data available from buildstats is now more fine grained than
previously, so take advantage of that to enrich the data we save
against tasks:
* Store the CPU usage for user and system separately, and display
them separately.
* Disk IO is now measured in bytes, not ms. Also store the
read/write bytes separately.
* Store started and ended times, as well as elapsed_time. This
will enable future features such as showing which tasks were
running at a particular point in the build.
There was also a problem with how we were looking up the Task
object, which meant that the buildstats were being added to
new tasks which weren't correctly associated with the build. Fix
how we look up the Task (only looking for tasks which match the
build, and the task and recipe names in the build stats data) so
the build stats are associated with the correct task.
[YOCTO #8842]
(Bitbake rev: efa6f915566b979bdbad233ae195b413cef1b8da)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When user enters a non-ascii character in the project
name of toaster, the build doesn't get triggered.
Use force_bytes to fix this.
Also deal with non-ascii project names when logging the
build request in runbuilds.
[YOCTO #9071]
(Bitbake rev: b6141c4d170885d3bdf63074afcb1e41fde0a8f0)
Signed-off-by: Sujith H <sujith.h@gmail.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The build dashboard doesn't show image and artifact files correctly,
as it shows the full filename for images and the filename plus
path relative to DEPLOY_DIR for artifacts.
Instead, show just the suffix for image files, and the basename
for artifact files.
(Bitbake rev: 8084dcdc283b4dc170f066c202f89d56ce1abbef)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Revert "toaster: models.py do not use transactions".
This reverts commit e248a503ba
(Bitbake rev: 48d0d510816346073f135bb86fb7904fdb50bb27)
Manually managing the database transactions caused more
problems than it temporarily solved so we return control back to
Django's orm.
[YOCTO #8796]
(Bitbake rev: 25c531915b6f8f79a0822996ceb97f90483e359f)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Allow package groups in our custom image recipe. Excluding them creates
more undefined behaviour than including them at this stage. Also update
to use convenience method for returning all packages.
(Bitbake rev: 8c2e8a13badacb816c4b1178b6661600008b38af)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a method to update the packages included list from the last
build, this effectively "synchronises" the package list from what
we think will happen at the Customise image stage with what
actually was produced with a build. It's not ideal to have this
function here but we also need to make sure that no race condition
of the user accessing this list and it being updated occurs.
(Bitbake rev: 8cf6e67a955574b33856a082bdadf3194f2b6ba4)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Field to keep track of when the package list for the CustomImageRecipe
was last updated from a build.
(Bitbake rev: 4bd4e49f13a7625997a43f3b2e67ed42c3c8e08b)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a convenience method to get the last successful build
target for a CustomImageRecipe.
(Bitbake rev: 4dde3d830cd38bbe306d83629dcb80da5fc9b027)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This convenience method returns just the Package_Dependency for
the package which are regular dependencies i.e. not RECOMMENDS or
any other types.
(Bitbake rev: bd76c22fe2aa06690b4ee25de69219ac0bf6b4d6)
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Whem a m2m field changes we need to clear the ToasterTables cache as
this can affect the state of items in ToasterTables.
For example the CustomImagePackages being added or removed from a custom
image recipe.
(Bitbake rev: c9d7b68ee0186a71e8e75a5d87122a0328001515)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If we're breaking up an image recipe's packages we will also need to
override IMAGE_FEATURES to make sure the customisation is not altered
(Bitbake rev: 9fd7b05dc0cf9240f7c8e3dc77b009064fd2b0cb)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When the packages associated with a custom image recipe are
shown in the customrecipe editing page, locale packages are
shown in the same way as all other packages. This gives the false
impression that these packages can be removed, when in fact
they are automatically added due to the IMAGE_LINGUAS build
variable.
Modify the customrecipe page so that locale packages cannot be
removed, and provide some help text explaining why.
[YOCTO #8927]
(Bitbake rev: b2208e53c00a67a7d0345e7378e6806b8ae40fb4)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The progress updater for the recent builds section makes a JSON
call to the project view URL to get progress for each build.
However, conversion of the builds pages to ToasterTable broke this,
as the JSON response no longer contained the data necessary to
populate the progress bars.
Move the recent builds query to the Build model, so that it is
accessible to the ToasterTables using it ("project builds" and
"all builds"), as well as to the "project" view.
Modify the code in the recent builds template to use the slightly
different objects returned by the recent builds query on Build.
(Bitbake rev: 5189252635ddc7b90c9a43aaed9f196c31e1dcad)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a special case for when the recipe we have based a custom image
recipe on requires another recipe.
In this case we need to adjust the file location to be able to
require
the recipe when we're in the toaster-custom-images layer.
For example:
"require core-image-minimal.bb" is changed to:
"require recipes-core/images/core-image-minimal.bb"
(Bitbake rev: 26025e1ea49b3ebfcfd508d1608fa8c9e722ad03)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Returns a queryset of the all the packages that we expect to have in a
CustomImageRecipe.
(Bitbake rev: 8b03bbae12ec077151c97579e329d89667040a78)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Instead of returning layercommits return the actual Layer_Version objects
for the layercommit as these are the useful objects which contain the
metadata.
(Bitbake rev: 480f0d307a3183df16fd383c7a3f96f00d09fbd4)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add function generate_recipe_file_contents to dump the custom image
recipe instance to a string for use either to push to the user as a
downloaded version of their custom image recipe or to use to generate
the recipe that we build.
(Bitbake rev: 6863343c3434ce19aa4b609c83f48a06e6943366)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This table is used to track all the available packages in the current
toaster. Many of these packages belong to many CustomImageRecipes.
(Bitbake rev: c1bd4f760cd35535e44f488250e0a56b99cad680)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This allows us to re-use the properties of a recipe for the custom image
recipes as well as re-using the existing templates and logic that deals
with recipe objects.
(Bitbake rev: bb8120b56be7eee6ed2e4434d8477282a01e0c00)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add function that returns the Sum of the size of all the packages which depend on a package.
Access get_total_source_deps_size via a packages's dependency
manager.
(Bitbake rev: e4c86d2f6c2e86ad054b37d0a5bf7464a4de4f9a)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fall back to a 'n/a' string for the vcs reference, not all our source
has to be in a vcs and therefore it is legitimate for this to be none.
(Bitbake rev: 1739b509e9efc4b016fc73c2d4399f1f9d3d285f)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Minor fixes to code formatting and small improvements from
code review.
(Bitbake rev: 2c97f3a5c6ae37de910deb90390c5b856a600c5f)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
For better long-term maintainability, use ToasterTable instead
of Django template and view code to display the all builds page.
NB the builds.html template has been left in, as this will
otherwise cause conflicts when merging the new theme.
[YOCTO #8738]
(Bitbake rev: e0590fc8103afeb4c5e613a826057555c8193d59)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The algorithm for finding the suffix for image files produced by
the build doesn't reference a list of known file suffixes, so
could be prone to error.
Modify how file suffixes are parsed from the file path so that
they are compared against a list of known types; if this fails,
use the part of the basename of the file path after the first
'.' character.
Also rationalise the places in the views code where we
extract the file name extensions for builds, so they both use
the same algorithm (before, the same code was duplicated in
two places).
[YOCTO #8417]
(Bitbake rev: dd1c509696b8ab5e593cc64637060a58e95fcd1f)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Image file suffixes are used in the project configuration page to
show a list of available image file types. This list is stored
as a function in the views code.
However, this list is also needed when parsing image file paths,
so that the suffixes can be shown in the "all builds" and "project
builds" tables.
Move the list of valid image file suffixes to the Target_Image_File
class to make is accessible in other places where it may be needed.
[YOCTO #8738]
(Bitbake rev: c2f20232077917552623fd0726d0820e50b04cae)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Added new model Provider and a foreign key 'via' to link
Recipe_Dependency to it.
[YOCTO #6169]
(Bitbake rev: e45fff6314741d46e2549b2f72ed380cbbb95593)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fixed 'database is locked' issue by monkey patching django QuerySet
methods.
The actual patching places were found by bisecting Django codebase.
This commit should be removed after Django is fixed if it's fixed
at all.
(Bitbake rev: 175411bf05423b1892c7928c2b928843b39645f0)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When sqlite can not cope with a stream of 'insert' queries it throws
'database is locked' exception.
Wrapping model.save in transaction.atomic context and repeating the call
should solve this issue.
(Bitbake rev: eb305308ca8f6228c6f52dac1bd941f29c7e5eb6)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Printing id as a number breaks toaster if object is not saved
yet as id/pk is None in this case.
Changed format spec to print id as a string in __str__
and __unicode__ methods of Build, Task and LayerVersion models.
(Bitbake rev: d5ca2bae623db214764324da5c9e4dac6beef760)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Remove compatible_layerversions() and replace calls to it with
calls to get_all_compatible_layer_versions().
The sorting done in compatible_layerversions() is not relevant,
and the code can be simplified by using get_all_compatible_layer_versions().
[YOCTO #8364]
(Bitbake rev: 7bca51277314f7c0b6ee0e0d470327dfd60d37fe)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Renamed get_query_set to get_queryset as get_queryset causes
deprecation warning in Django 1.7 and is deprecated in Django 1.8
(Bitbake rev: ef8b399dccd413070e32ce05e013337ea01fa64d)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Rather than maintain data as part of the migrations (as was
done for the default project previously), create the default
(cli builds) project on demand as a by-product of getting
it from the database.
[YOCTO #8364]
(Bitbake rev: 5fd8e90ab9b81d1bd0d301bc1c91228ecbbea74b)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In order to search the model from the UI some fields must be nominated
as searchable.
(Bitbake rev: 2558729b4c248a2fa5a11d877bc42cb05a30602e)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The count of a project's builds should not include those which are
currently in progress.
(Bitbake rev: a981700701c41c7bbb6a9778e95f691278c5c294)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Log messages can be quite long so use a TextField rather than a char
field with max length of 240. mySQL is especially picky about field
lengths and will cause an exception if the log is too long.
[YOCTO #8475]
(Bitbake rev: 349119f44cdad240dda7f4db45992e2539e1416f)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fatal build errors were displayed as exceptions, and highlighted
with less severity than they deserved.
Roll back to treating Toaster exceptions as errors by removing
the toaster_exceptions member on Build objects and displaying
EXCEPTION events in the errors section on the dashboard.
[YOCTO #8320]
(Bitbake rev: 59197320bdcefddf06084e871f1b5b21b21cbb63)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Critical errors (where a build failed for reasons of
misconfiguration, such as a machine being specified which is not
in a project's layers) were being ignored (only log records
up to ERROR level were being logged to Toaster's db). This meant that
the build would fail but would not correctly report why.
Add support for CRITICAL error levels to the LogMessage model,
include errors at this level in the errors property for a build,
and show errors at this level in the build dashboard.
[YOCTO #8320]
(Bitbake rev: b6eacbca9cacb607de864ab7d093deb296da8226)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Builds initiated from the command line don't have a buildrequest
associated with them. The build.buildrequest association is
only added if a build is triggered from toaster.
Some of the code for displaying the status of a build refers
to build.buildrequest without checking whether it has been set,
which causes an error to be thrown.
Add a guard to check whether the buildrequest has been set.
[YOCTO #8277]
(Bitbake rev: af33409612139ab2347baf6b847b23faea19752d)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Modified get_all_compatible_recipes function to exclude recipes
with empty names from the result queryset. This should stop UI
to show recipes with empty names in compatible recipes page.
[YOCTO #7969]
(Bitbake rev: f7c1bd49e2e28d12c6604f5ae54bc96e1e7b6946)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Due to Recipe model constraints some recipes can't be
saved. However, they still can create incomplete records in
the database. This causes all sorts of errors when Toaster
operates with those objects. Removing them should fix those
issues.
[YOCTO #7969]
(Bitbake rev: 4d76a9e418fd98a7882aa29f974a7389f9689314)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fixed exception: NameError: global name 'DoesNotExist' is not defined
(Bitbake rev: eee5311b867d3c8c33e06d04e103bfd3647146f4)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Implemented Layer_Version.get_alldeps API to recursively get
full list of dependencies for the layer. Dependencies that are
already in the project are filtered out from the result.
Result list of Layer_Version objects is sorted by layer name
for UI to look consistent.
This API is going to be used to show amount and list of
dependencies for the layer in the list of compatible layers
for the project.
(Bitbake rev: 7d853a3054a9ae3d18eb6f5bc13ba27d2795c31a)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This was accidentally returning the primary key of the ProjectLayer
rather than the ProjectLayer.Layer_Version.pk
(Bitbake rev: b20f3626148e89af0e9fcfca911a5a1e4e355a41)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fix typo don't pass self in as a parameter, this evaluated to true
giving the wrong results meaning the machines typeahead did not return
valid results.
(Bitbake rev: 55ba889ef8900c95447861fa3985ca9cfe06afdf)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We no longer need to compute each layer_version and all the recipes
which belong to this.
[YOCTO #8147]
(Bitbake rev: 505979ab931e3a2a218d7030d6064987e8f9ff14)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Create a copy of the built layer and the recipes associated with it.
This is so that the user can view the historical information about a
build. i.e. a snapshot of the layer version and artifacts produced at
that build.
(Bitbake rev: 0683d9a2b15e3234a94437aaebac84bfcca1420b)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When we do a build we update the last commit value that the layer was built at
However in future builds we do want to use the named reference rather
than the commit sha, e.g. master/fido
(Bitbake rev: a3d47eb923dd2cb5259f387c79549822807eca49)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Previously this layer relationship was done by trying to match path
information that came back to the buildinfohelper with trying to query for
data in toaster's layers table. This rarely matched due to the lose coupling.
(Bitbake rev: 838e77c7c3c4006abd1327343a004590ab652de9)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If the openembedded-core layer is specified in the toasterconf we need
to treat it differently because we may also get this layer either from
the layerindex source or I can also be provided locally.
(Bitbake rev: 8e36b6848c14f2708e1f87d12fb533ca0b596c65)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This model lists custom image recipes for the project.
It is populated when new custom image is created.
It holds reference to the base recipe and list of packages
included into custom image.
For CustomImageRecipes the packages will be copied in and
therefore not associated with a build so Remove the requirement
for the package to have a Build.
co-author: Ed Bartosh <ed.bartosh@linux.intel.com>
(Bitbake rev: 33cbf4cd3b4ca47c4901501f5f1eafdfdfdae023)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
For a new style indexes 'inherits' field is used to determine if
recipe is an image recipe.
As old style indexes don't have 'inherits' field this can be
guessed from recipe name. Let's consider recipe an image recipe
if recipe name contains '-image-'.
(Bitbake rev: d769d3f54faaf1a3091b1eb6ed1d17a8a108210a)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Information about a task is not stored in Target objects.
This makes it impossible to correctly operate with the builds
where task is specified.
Storing taks name is an enabler for other fixes in UI and backend
related to restarting builds.
(Bitbake rev: 0a69a8a18075c976ed8681d9d75529f8c2f48514)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Command line builds are associated with a "default project"
(as we currently require a build to have a project). This
acts as a container for builds initiated outside Toaster.
Currently, this project is marked as the default by its ID
being 0. However, this doesn't work with MySQL, as MySQL
won't allow 0 in a foreign key which references an
autoincrement field.
Instead, use an is_default field to track the default Project
for builds initiated outside Toaster.
Add a method to fetch this default project, rather than fetching
a project with a magic ID.
Add this default project in a migration, rather than as a side
effect of a get_or_create() style method.
Also ensure that builds always have a project explicitly assigned
to avoid any magic with a build's project foreign key defaulting to
0 (as it no longer does).
[YOCTO #7932]
(Bitbake rev: 71b709a1bbc26d89d61873763b467d21e625b274)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In an environment with a proxy which requires authentication, e.g. with
$http_proxy = 'http://user:password@ip:port', the following error
occurs when running Toaster:
EE: Using proxy http://user:password@ip:port
EE: could not connect to 'url', skipping update: 'error message'
This prevents Toaster from fetching layer, recipe and machine information
from remote repositories.
This patch allows Toaster to use the proxy settings from the
environment for HTTP/HTTPS requests.
(Bitbake rev: e7a85031fd05a46ef60b380883da4cc372acf89b)
Signed-off-by: Bian Naimeng <biannm@cn.fujitsu.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Prompted by issues discovered during running pylint on
the toaster sources, this patch fixes:
* missing import in toaster ui
* improper call of function in toaster ui (logger.debug)
* improper function definitions in bbcontroller
* invalid references to objects in bldcontrol.models
* proper initialization of object fields in ToasterTables
Also inhibiting specific pylint errors in files where
the problems are mis-identified.
(Bitbake rev: 1c71955c416fb68455f7f70669aba4202c411807)
Signed-off-by: Alexandru Damian <alexandru.damian@intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The layer data update in Layer source needlessly stops
processing by throwing an Exception when bad layer data cannot
be inserted in the database (e.g. invalid dependencies).
This patch changes the behaviour to ignore bad data, and
continue with data import after detailed logging through the
logging system.
[YOCTO #7955]
(Bitbake rev: e0911e408db4fa07086738fe11121b173b4d1cf5)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Switching debugging from using print statement to using
loggers, as it uses the logging infrastructure to process
the messages.
The messages are logged with the "toaster" logger which
is defined in the toastermain/settings.py.
(Bitbake rev: adf3bdcbe8b0b0bbe9de2800f2d20a53e8d88543)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
As prompted by pylint, the object fields need to be
initialized properly, so this patch fixes that.
Also adds some casts to int, because, sometimes, the IntegerField
is not returning an int, but a string.
(Bitbake rev: cc9ac01d0726203fbe916cec51cc464eaeae9305)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This patch improves exception handling, fixing issues
as prompted by pylint, including exception renaming.
(Bitbake rev: 652fad64864703cb1bbeede8d5bfc4208974be68)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This patch fixes imports, default parameters to functions,
incorrect overloading and exception handling issues
highlighted by pylint. There are no functional changes.
(Bitbake rev: 066e096b80dcb0e93e1a088acf5d914184361769)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We use these querysets when creating tables of results and also when we
want to have a typeahead search. These can also form the basis of future
API endpoints.
(Bitbake rev: 2a10fecd985343802f0e99c6fff25c28980eee20)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Save duplicating this call and make a utility function for it on the
Layer_Version object.
(Bitbake rev: 32d382315bad37cebbe67ca3287085f3d856a9ed)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Added new field is_image to the Recipe model.
Made sure is_image is populated when layer sources are imported.
Made sure the change doesn't break loading old style layer sources.
Tested by loading data from layers.pythonanywhere.com and
from layers.openembedded.org.
[YOCTO: #7571]
(Bitbake rev: 3ac0ba73790b8f68567daa2de313c78053411fbb)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Sometimes, a layer dependency will not exist,
or not exist for a particualr branch. In this case
we need to continue processing and hope for the best.
[YOCTO #7955]
(Bitbake rev: 2961bf2612fce111a7be6cef285595a075200780)
Signed-off-by: Brian Avery <brian.avery@intel.com>
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Partial revert of bb commit 7aecb974d44d9bc711ffba5cc65e770811397fba
There were changes in this commit which broke the compatible layers
mechanism meaning that no layers, recipes or machines were showing.
(Bitbake rev: 124157b567469639c6c839233f42049a093ba083)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This patch brings fixes for issues highlighted by
HTML5 compliance testing.
(Bitbake rev: 7aecb974d44d9bc711ffba5cc65e770811397fba)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This brings in project types in the New Project page.
The "analisys" projects are the projects with no "release"
set, and have read-only properties in the project page.
(Bitbake rev: 2efc338cefd6e6e097af83d7dff63e9ba177d021)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We remove the "timespent", "errors_no" and "warnings_no" fields
in favor of computing the needed values at runtime. This prevents
inconsistencies in the UI.
Also removeing all references to BuildRequests from the interface -
all build details now display in the build dashboard.
Minor fixes related to data logging.
(Bitbake rev: 44f37394ed3e4ca02f940be172fe4395b0ee0f7d)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This is a set of fixes that repair the interface after
we switched from displaying BuildRequest data to Build data
in the formerly "managed" mode.
(Bitbake rev: 57f790b0c56297af8c83d5def8461bd5d61fe4af)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This patch adds logic to complete changing the interface
from showing BuildRequests to showing Build data.
The BuildRequest data is now transformed in Build data with
proper Toaster exceptions being recorded instead of listing
problems during startup as build errors.
(Bitbake rev: 51a41172d0b390370f9a38696b1ac65666ada4d2)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We conflate the managed and analysis modes by
deleting alternative code paths, favouring the MANAGED mode,
always considering the MANAGED variable True.
(Bitbake rev: 0ac02d3775106b485e29496b62f31e91fd3f9387)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This patch fixes issues brought in by refactoring:
* the New Build button is working with pre-set projects
* the xhr_datatypeahead is exposed for calls that are not
mapable to the REST objects
* a new table returing recipes provided by layers currently
selected in the project is used to provide recipe suggestions
* the field names in json are switched from "list" to "rows" as
to maintain consistency with the ToasterTables
* the "value" field in xhr_ calls is now named "search" to maintain
consistency
(Bitbake rev: a5bc29083d4f85a5695f3f62d5badb783c6f7224)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The Layer_Version.get_equivalents_wpriority performs the same
function as Project.compatible_layerversions, but in memory and
with worse performance.
Replace the code in get_equivalents_wpriority with a call to
the project compatible_layerversions, which also returns a queryset
instead of a list (can be used to further enhance queries)
(Bitbake rev: fb5eb1d7759222573565936a964d602c148df139)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This fixes all the unit tests for the orm. Also added is the ability to
set a custom Layer index if you want to avoid using the public one by
specifying TTS_LAYER_INDEX
(Bitbake rev: dfbcbe116d0b987b850f67056f02f489ac0b8360)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This patch refactors the checksetting command to prevent
early return from the handle function.
It also adds a check that marks IN PROGRESS builds at startup time
as FAILED. Minor changes to BuildRequest and Build classes
ensure useful string representation for the objects.
(Bitbake rev: adf67dd79dbf6b585bf8cd54f99c389409b88ecd)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Instead of relying on the cache expiring over an amount of time we can
listen to the changed notification on the models to invalidate the
cache. Also fixes overlapping cache names.
(Bitbake rev: eb0b1450e421cf65b407b1ac0336ac24ffc626e8)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Bitbake logger now sets a default project if the TOASTER_BRBE
or TOASTER_PROJECT Bitbake variables are not set.
This a necessary step in getting all builds under a project,
as to unify the MANAGED and interactive modes.
Other small fixes are included, related to the size of the
fields in the database.
(Bitbake rev: 5e0bf388f4e5c1cc493ac8264785e631bad2f672)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This refactoring brings the "local_path" of the
layer from the Layer object to the Layer_Version object, which
is more appropriate as different checkouts of the same
Layer may live in different directories.
This enables us to store Recipe file paths relative to a Layer_Version
at all times, aleviating the need to store full file paths in the
database. We also turn the prefix of the path (e.g. virtual:native
path name space) into a pathflag field.
In turn, this solves the problem of mis-identification of tasks based
on the recipe file paths, since we can also match the namespace of the
file paths on the recipe files.
[YOCTO #7594]
(Bitbake rev: ec43dc569e370767c709dec225cbee0c99151c19)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Port of the main tables to the new ToasterTable widget.
(Bitbake rev: 6de539d5953b2dca2a9ed75556a59764337a194c)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This patch brings needed changes in all views so that
each view passes the HTML5 compliance test by the
W3C Markup Service.
(Bitbake rev: 6e60ed8ab71e4300ab571f42b7af5011086df697)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fixed TypeError: not all arguments converted during string formatting
[YOCTO #7618]
(Bitbake rev: 308b5529d4e183d550221864bfdbefd31cc23705)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The recipe paths in managed mode are not processed correctly. They need
any prefix tags removed, and then any build path prefix removed only if
present.
[YOCTO #7523]
(Bitbake rev: ee5c569ebe21c065508cd816ae8cb8842baf0b7e)
Signed-off-by: David Reyna <David.Reyna@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Preset uninitialized object members to bootstrap new layer source creation.
[YOCTO #7437]
(Bitbake rev: 4de721aa75a4aaad186312443726e6daa764934b)
Signed-off-by: David Reyna <David.Reyna@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We do not want to throw an exception while handling exceptions
that may print information, so replace the exception throwing
with returning data.
(Bitbake rev: 624e67b0d142e5dac946ebb63bcfad0625b88799)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If the bitbake server exists after the initial process start,
the only output displayed shows proper server start, with
no debug info.
This patch also brings the server log into the error message,
so the user has better info about what's going on.
(Bitbake rev: 6b46f436296683f7a2f72ac978ef30d0a38ad7a4)
Signed-off-by: Belen Barros Pena <belen.barros.pena@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This patch brings in a new set of optimizations in the tables pages,
with the focus of reducing the number of SQL queries performed per
row.
(Bitbake rev: a1ad86febb9d8a4ee7f15aa726f267d64b8e0dc4)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We update the search fields for the machine suggestions
and recipes tables.
(Bitbake rev: b85bcc854264336c1aef1e5fd4ea9e56f5af94fd)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In build mode, instead of creating our own layer objects,
we identify the layer objects that the build system set up.
[YOCTO #7378]
(Bitbake rev: 22962b540ace6868cb357c0fd13f01ffd24449c4)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This is the counterpart patch of
do not use transactions if the database autocommits
fix on toasterui, applied to models.py
[YOCTO #7402]
(Bitbake rev: 48d0d510816346073f135bb86fb7904fdb50bb27)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Implement the 'last build' data methods, enhance variable display,
add empty page and empty sort support.
[YOCTO #6682]
(Bitbake rev: cc6ca17e80844ecb4a777276d5f5177ebbcd93f9)
Signed-off-by: David Reyna <David.Reyna@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When we have multiple layers with the same priority, we
need to order them by date, newest first. The layer version
ID is a complete ordering by recent-ness, so it is suitable.
(Bitbake rev: 09171a892935d46ed32162b2d0f4a536c546facc)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This patch fixes a problem where set sstate scene tasks
were not identified, causing cache attempt not being recorded.
[YOCTO #7223]
(Bitbake rev: 8a326a9a5a08981f1b7960e02fdb8a9436db16fb)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The typo caused the tasks table to break when a task has no
help text.
(Bitbake rev: f1b794e6abba5c7cdda49666a7ff27db7dd6ea72)
Signed-off-by: Belen Barros Pena <belen.barros.pena@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This patch fixes the ETA calculation.
[YOCTO 7349]
(Bitbake rev: 8a5c83c953c1e541704fe979e90d9d3be66e9ffd)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This patch brings better forward and reverse dependency computing in
tasks page (offloading python code to the database), as well as
replacing the recursive depth-first covered-by tree search with
breadth-first search.
(Bitbake rev: bc76da054570ae385c87d14c9f2337368e8311f3)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This patch disables autocommit for inserting build data,
effectively updating all build data in a single transaction.
This is a purely performance improvement patch, as the transaction
will always be commited.
Similar manual transaction handling in the layer source update
method. Added feedback messages during update method.
[YOCTO #7140]
(Bitbake rev: 3978c819e797f857235499a4b8ec238134f1c028)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We reduce the number of queries by using "select_related"
to bring in more data in a single query. Improvements in
project page refresh, recipes table and tasks table.
(Bitbake rev: eefdae12120f879b555ba0a353277a18675eecbc)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This patch fixes the time estimation to build completion
in project page. Additionally it fixes the Most Recent Builds
section used in various pages in managed mode, and proper
time to build estimation in all pages.
[YOCTO #7004]
(Bitbake rev: 5fecfda0e47c2ecba9b7c903c6d258eefa431aa0)
Signed-off-by: Alexandru Damian <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Toaster needs to properly prompt the user and display
actionable help texts when the database is void of
meaningful information. This patch brings in the
"empty" states for pages.
[YOCTO #6755]
(Bitbake rev: 7cfe279bf77b59d5cbd20d8a93e1d33279bebc20)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Michael Wood
Elliot Smith
Sujith H
Ed Bartosh
Bian Naimeng
Alexandru Damian
Brian Avery
David Reyna
Belen Barros Pena