Commit Graph

4460 Commits

Author SHA1 Message Date
Michael Wood 1c9ce1ca73 bitbake: toaster: newcustomimage_modal use libtoaster method for new CustomRecipe
Use libtoaster.createCustomRecipe rather than own implementation of this
function.

(Bitbake rev: 74fa98752b1cf1ad18d35ab6dd25fe7e409133c5)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-10 13:29:18 +00:00
Michael Wood 8b1d0438bc bitbake: toaster: libtoaster Add createCustomRecipe method
This adds the function to call the ReSt API to create a custom image
recipe.

(Bitbake rev: 03e7949f538733f682a05d0c318cf2f4cd64cbf5)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-10 13:29:18 +00:00
Michael Wood 32048fa71f bitbake: toaster: orm Add convenience method to get all pkgs in a CustomImageRecipe
Returns a queryset of the all the packages that we expect to have in a
CustomImageRecipe.

(Bitbake rev: 8b03bbae12ec077151c97579e329d89667040a78)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-10 13:29:18 +00:00
Michael Wood c80b7dfee2 bitbake: toaster: orm get_project_layer_versions to return layer_version objects
Instead of returning layercommits return the actual Layer_Version objects
for the layercommit as these are the useful objects which contain the
metadata.

(Bitbake rev: 480f0d307a3183df16fd383c7a3f96f00d09fbd4)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-10 13:29:18 +00:00
Michael Wood 796e3481ad bitbake: toaster: toastergui tests Add unit test for download custom recipe
Add unit test to test downloading end point and the basic content of the
custom recipe that is generated.

(Bitbake rev: 1e78d1ca2f5b02d80ab037c6f5f81d18195fc817)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-10 13:29:18 +00:00
Michael Wood 04d8c94564 bitbake: toaster: toastergui tests Update to reflect changes to CustomImageRecipe
Now that CustomImageRecipe inherits from Recipe make sure that the
accessors and the required values for Recipe are now setup correctly.

(Bitbake rev: e958921e15a3c3e5a6b7c27ebe37fdf1f551f198)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-10 13:29:18 +00:00
Michael Wood 4e8a0aa66e bitbake: toaster: views xhr_customrecipe_packages clean up API
- Fix generic variable names such as "object" and "values" when not
  needed.
- Use try catch instead of a queryset filter to return the custom recipe
  object
- Be explicit about the fields returned for the custom recipe info field
- Remove redundant new_package field

(Bitbake rev: a1a69903a94264377666730b1eb4599e6f3b4398)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-10 13:29:18 +00:00
Michael Wood 66b5608ffe bitbake: toaster: toastertable remove title from Show all in table
Title is often very long so this is not a great property to use here.

(Bitbake rev: 541ff2b5e9c6ebec0d0ced59b3f61cbd37bd37c2)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-10 13:29:18 +00:00
Michael Wood ce7289613b bitbake: toaster: Add recipe details page
This page provides the details of recipes, showing their packages and
additional metadata. It also allows you to build the recipe or customise
the recipe if it is an image recipe.

[YOCTO #8070]

(Bitbake rev: b6023619397f679791b7eff1faac06f4246133ce)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-10 13:29:17 +00:00
Michael Wood 5f52614a87 bitbake: toaster: newcustomimage Move modal dialog out of newcustomimage template
Move the modal template and JS out of the newcustomimage page so that it
can also be used by the image details page.

(Bitbake rev: c310bc6bab1a33124906dd57b3c63462a773ff25)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-10 13:29:17 +00:00
Michael Wood 2a3dd32d66 bitbake: toaster: Continue front end features to custom image recipe page.
Continuation of the work on the custom image recipe page, this brings
in:

- Basic notification of having added/removed a package.
- Connect up Build button
- Download recipe feature
- No packages states
- Project bread crumb
- Display additional recipe metadata
- Update accessors for recipe object inheritance changes

[YOCTO #8082]

(Bitbake rev: a656756a9255ec5882686ce9563d17f2eb3136e3)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-10 13:29:17 +00:00
Michael Wood d6e7e4ad43 bitbake: toaster: tables Add table for Packages and update SelectPackagesTable
Create a Packages table for use as the image details page.
Change the SelectPackagesTable table to inherit from the Packages table.
Remove the need for a separate view by adding the additional template
context items to the Table's page context.

(Bitbake rev: 336b1d8369d9e86ece78b63cb0e140e653216011)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-10 13:29:17 +00:00
Michael Wood 43f0a05fa4 bitbake: toaster: views Add view to download custom recipe
View to provide the custom recipe download feature. The recipe is
generated on-demand to make sure that it is the most current version of
the Custom recipe.

(Bitbake rev: 2101c854bb2d7ff1e3a4f00ad4d33d77859439ed)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-10 13:29:17 +00:00
Michael Wood 2cf55afb97 bitbake: toaster: move CustomImageRecipe generation to API entry point
Use the CustomImageRecipe generate_recipe_file_contents to generate the
recipe that we build from. Move creation of the dummy layer and recipe
object to the point of recipe creation as we need these objects before
the build time. Also update the methods to add and remove packages to
account for the CustomImageRecipe inheriting from Recipe.

(Bitbake rev: f3322567378d6038a00da0fab6c5641a1a8e5409)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-10 13:29:17 +00:00
Michael Wood c402ac2654 bitbake: toaster: orm add CustomImageRecipe generate contents function
Add function generate_recipe_file_contents to dump the custom image
recipe instance to a string for use either to push to the user as a
downloaded version of their custom image recipe or to use to generate
the recipe that we build.

(Bitbake rev: 6863343c3434ce19aa4b609c83f48a06e6943366)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-10 13:29:17 +00:00
Michael Wood a6e4f94b01 bitbake: toaster: buildinfohelper Add the concept of CustomImagePackage
This adds the concept of CustomImagePackage this is similar to the way
layers and recipes work in that we have a set of data which is part of
the build history and a set of data which is part of the configuration
data that toaster uses to guide people in configuring their project. We
create a set of  built_packages for every build but only create a package
for configuration purposes if we don't already have one, so that the
CustomImagePackage only ever contains a unique list of packages that are
available to be added and removed from a CustomImageRecipe.

(Bitbake rev: f81bb65883baa6c0f8a4d48a4de3291a10543992)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-10 13:29:17 +00:00
Michael Wood e1bfe1ceb6 bitbake: toaster: orm: Add db migration for new CustomImagePackage table
(Bitbake rev: c7da71fe8509439656f482c16ed081cf442f4030)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-10 13:29:17 +00:00
Michael Wood f760a78e3c bitbake: toaster: orm Add CustomImagePackage table
This table is used to track all the available packages in the current
toaster. Many of these packages belong to many CustomImageRecipes.

(Bitbake rev: c1bd4f760cd35535e44f488250e0a56b99cad680)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-10 13:29:17 +00:00
Michael Wood 4117af29c6 bitbake: toaster: orm: Add db migration for new CustomImageRecipe inheritance change
(Bitbake rev: 2831d74201abba68c301d85ee583d706a51d5a5f)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-10 13:29:16 +00:00
Michael Wood 1f102890d6 bitbake: toaster: orm make CustomImageRecipe inherit from Recipe
This allows us to re-use the properties of a recipe for the custom image
recipes as well as re-using the existing templates and logic that deals
with recipe objects.

(Bitbake rev: bb8120b56be7eee6ed2e4434d8477282a01e0c00)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-10 13:29:16 +00:00
Michael Wood 648753b30d bitbake: toaster: orm Add sum of dependencies size function to PackageDependencyManager
Add function that returns the Sum of the size of all the packages which depend on a package.
Access get_total_source_deps_size via a packages's dependency
manager.

(Bitbake rev: e4c86d2f6c2e86ad054b37d0a5bf7464a4de4f9a)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-10 13:29:16 +00:00
Michael Wood a92fc3025b bitbake: toaster: tablejs Add an event handler to manually trigger a data reload
Allow users of ToasterTable to manually trigger a refresh of the data.
This can be useful if an action has happened in-page and the data is now
invalid. Such as new data being added or removed from the model.

(Bitbake rev: 6e42070d8abc80dacd8094c4f5019577453a9d49)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-10 13:29:16 +00:00
Michael Wood 4c82878225 bitbake: toaster: ToasterTables simplify filter function move common part to widget
Move part of the functionality of the filter functions to the Table
widget. We don't need to implement it in each subclass.

(Bitbake rev: 16e48829f6fd96c1d21339253871f2a9b2446f87)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-10 13:29:16 +00:00
Michael Wood 3e1e8e6a99 bitbake: toaster: models fall back to a sensible string for no vcs reference
Fall back to a 'n/a' string for the vcs reference, not all our source
has to be in a vcs and therefore it is legitimate for this to be none.

(Bitbake rev: 1739b509e9efc4b016fc73c2d4399f1f9d3d285f)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-10 13:29:16 +00:00
Michael Wood 14d09c8b22 bitbake: toaster: localhostbecontroller CustomRecipe now base_recipe is Recipe
Update the reference to the base_recipe. It is now a Recipe object
rather than an intermediate AvailableRecipe object.
Therefore doesn't need an extra traverse down the object hierarchy.

(Bitbake rev: 8056ec65bd93005ecb7b0ed12dcb21b3b60df22c)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-10 13:29:16 +00:00
Aníbal Limón d0d85a4d6b bitbake: bb/fetch2: Move export_proxies function from wget to utils.
In order to use in other modules since is a common function
when needs to get proxies working.

(Bitbake rev: 85c529044381895556d603a3974de22392646a22)

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-08 10:52:40 +00:00
Elliot Smith 6228cf8166 bitbake: toaster: reinstate ID on edit columns button
The ID was omitted from the "edit columns" button on the
"projects", "all builds" and "project builds" tables when they
were converted to ToasterTable. This caused the QA tests to fail.

Reinstate the ID so the tests can identify the "edit columns" button
correctly.

[YOCTO #9051]

(Bitbake rev: 934b82badcf063c8ff252d806c2fb019f7a2e55f)

Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-06 23:13:47 +00:00
Ed Bartosh 916c73dfef bitbake: cooker: shutdown cooker parser on shutdown
Currently any not catched exception in cooker causes bitbake
to hang because of not terminated children of CookerParser.

Long term solution would be to reimplement Cooker as a context
manager and terminate parser children in its __exit__ method.

Partial fix is to call CookerParser.shutdown in Cooker.shutdown in
hope that all Cooker exceptions are caught and shutdown method is
called.

[YOCTO #8900]

(Bitbake rev: 3f67600dc3292bc8208644ce89e8bf7ab95cf2e7)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-06 23:13:47 +00:00
Richard Purdie 8857498ba4 bitbake: fetch2/osc: Clean up old variable syntax
Since people do copy and paste these things, clean up old syntax styles.

(Bitbake rev: 4fb028b0bd14d3e4b3fd7a89c643528728566476)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-06 23:13:47 +00:00
Richard Purdie 54da82972c bitbake: fetch2/osc: Remove hardcoded url
This shouldn't be in here, use a variable instead.

(Bitbake rev: 2e25d09a1ab62ccc3573d13114d59838cf4b07f2)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-06 23:13:47 +00:00
Richard Purdie 6ae662acdc bitbake: parse/ast: Mark anonymous functions as python functions
Anonymous functions are python functions, set the variable
flags as such so we can detect them and avoid expansion where
needed.

(Bitbake rev: 1b303785c578bbae3a89be8d751d80fba860f62e)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-04 23:47:49 +00:00
Richard Purdie 9913fd88d9 bitbake: codeparser: Improve handling of data.expand() dependencies
Currently bitbake doesn't parse into data.expand() expressions,
relying on high level expansion of python code to handle this.

One of the tests does however test this works.

We don't really want to be doing string expansion on python code,
so specifically parse into expand() function calls so that when
the high level behaviour is tweaked, the self tests continue to
pass and that we do continue to handle expand() function calls as
best we can.

(Bitbake rev: b12c17be5e4a74c9680876605c87f46501f78d28)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-04 23:47:49 +00:00
Richard Purdie 4628fe12e7 bitbake: lib/bb: Add expansion parameter to getVarFlag
This sets the scene for removing the default False for expansion from
getVarFlag. This would later allow True to become the expand default.

On the most part this is an automatic translation with:

sed -e 's:\(\.getVarFlag([^,()]*, [^,()]*\)):\1, False):g'  -i `grep -ril getVar *`

There should be no functional change from this patch.

(Bitbake rev: 7c3b99c6a716095af3ffce0b15110e91fb49c913)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-04 23:47:49 +00:00
Felipe F. Tonello b98866d003 bitbake: fetch2/gitsm: Fix when repository change submodules
This fix a problem when checking out a commit that changes the submodules
previously checkout.

Example:
Recipe uses branch A and then it updates to use branch B, but branch B has
different submodules dependencies then what branch A previously had.

(Bitbake rev: 54a3864246f2be0b62761f639a1d5c9407aded4f)

Signed-off-by: Felipe F. Tonello <eu@felipetonello.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-04 23:47:49 +00:00
Richard Purdie 390c2c1268 bitbake: data_smart: Add missing expand parameter to getVar call
(Bitbake rev: 2876019e696ff2af164961d5d4c8e3ea9dfefc23)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-04 23:47:49 +00:00
Diego Santa Cruz 56454f6fb2 bitbake: bitbake: prserv: do not clear umask when daemonizing
Clearing the umask when daemonizing is not the correct thing
to do, as it will create files writable by anyone by default.
For instance the pid file was being created with mode 777.
This could also potentially affect the sqlite database.
Better let the calling process decide on the umask.

[YOCTO #9036]

(Bitbake rev: ff6d3f53a4504eae7ec4c190b9f7595b09aed017)

Signed-off-by: Diego Santa Cruz <Diego.SantaCruz@spinetix.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-04 23:47:48 +00:00
Diego Santa Cruz abf8a8fbc3 bitbake: bitbake: prserv: SIGTERM handling hung process
The current SIGTERM handler hungs the process instead of
making it exit. The problem seems to be that the handler thread
is not signaled to quit, so it stays there doing its work, as
it is not a daemon thread. Setting the quit variable fixes this.

While at it, to not use the SystemExit exception to terminate
upon SIGTERM but instead left the quit flag do its job. This way
the PID file is properly removed.

[YOCTO #9035]

(Bitbake rev: 655ec800d54da581229f12efb6f0baf54975fed4)

Signed-off-by: Diego Santa Cruz <Diego.SantaCruz@spinetix.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-04 23:47:48 +00:00
Diego Santa Cruz be032fc40e bitbake: bitbake: prserv: -wal and -shm sqlite lost when daemonizing
When daemonizing the PR service the -wal and -shm sqlite files were
being deleted, although they should never be. While the daemonized
process keeps the file descriptors open and thus a clean exit does
not loose any data, a power outage would loose all data in the WAL.
Removing these files also breaks sqlite collaboration between
processes and furthermore prevents taking proper backups without
stopping the PR service.

The reason this happens is that the DB connection is opened in
the initial process, before forking for daemonization. When the
DB connection is closed by the exiting parent processes it can
delete the -wal and -shm files if it considers itself to be the
last connection to the DB. This is easily fixed by opening the
DB connection after all forking.

[YOCTO #9034]

(Bitbake rev: bc867c81e3894da5bdd2e45fa695bb5f5f1bb58b)

Signed-off-by: Diego Santa Cruz <Diego.SantaCruz@spinetix.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-04 23:47:48 +00:00
Phil Blundell a4a5d1f323 bitbake: cooker, bitbake-worker: Fix spelling of "received"
I before E, except after C...

(Bitbake rev: 14c9593265f7469cb8a205a46f845ac7491246df)

Signed-off-by: Phil Blundell <pb@pbcl.net>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-01 11:20:42 +00:00
Richard Purdie 8f6b9c75a7 bitbake: cooker: Only start as many parse threads as we need
If we're only going to parse one recipe, no point in starting
a large number of threads.

(Bitbake rev: b977faf59dc08050a44a16032fe52d1bbb80f2a1)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-01 11:20:42 +00:00
Richard Purdie 602da7c921 bitbake: knotty: Don't show errors for universe provider issues
When running universe builds, we don't expect an error exit code for
provider warnings. Change the error messages to warnings in this case.

This deals with errors causing problems on our autobuilders amongst
other issues.

(Bitbake rev: d4989fb0355476de172169f0698757f7360e9a1f)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-01 11:20:42 +00:00
Sujith H 3072361f86 bitbake: bitbake: BBUIHelper: Remove function findServerDetails
findServerDetails function can be removed safely
from the source tree. Couldn't find any files
calling this function.

(Bitbake rev: 46871f769db13ccd36deedc5b6f3dbc0a3d31c4b)

Signed-off-by: Sujith H <sujith.h@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-30 11:43:56 +00:00
Peter Kjellerstedt 28c041c1e7 bitbake: fetch2: Simplify logic in verify_checksum()
The recent change to verify_checksum() to only show checksum warnings
if no checksums are supplied made it possible to simplify the logic a
bit more.

(Bitbake rev: 1dc00b874acae44bbba9d8028d94f7bc97ddcd76)

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-30 11:43:56 +00:00
Richard Purdie 5375e6431c bitbake: bitbake: Set process names to be meaninful
This means that when you view the process tree, the processes
have meaningful names, aiding debugging:

$ pstree -p 30021
bash(30021)───KnottyUI(115579)───Cooker(115590)─┬─PRServ(115592)───{PRServ Handler}(115593)
                                                ├─Worker(115630)───bash:sleep(115631)───run.do_sleep.11(115633)───sleep(115634)
                                                └─{ProcessEQueue}(115591)

$ pstree -p 30021
bash(30021)───KnottyUI(117319)───Cooker(117330)─┬─Cooker(117335)
                                                ├─PRServ(117332)───{PRServ Handler}(117333)
                                                ├─Parser-1:2(117336)
                                                └─{ProcessEQueue}(117331)

Applies to parse threads, PR Server, cooker, the workers and execution
threads, working within the 16 character limit as best we can.

Needed to tweak the bitbake-worker magic values to tell the
workers apart.

(Bitbake rev: 539726a3b2202249a3f148d99e08909cb61902a5)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-30 11:43:56 +00:00
Richard Purdie 5b234d1539 bitbake: utils: Add ability to change the process name
Being able to tell the bitbake processes apart is useful for debugging.
Add a helper function which allows this without making it a hard
dependency. Errors are ignored, this is just nice to have.

(Bitbake rev: fd7f1a94d196b8a3c445e313d9e699b352b1da97)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-30 11:43:55 +00:00
Andre McCurdy 0b06924759 bitbake: data.py: avoid double newlines at the end of functions in emit_var()
A newline is always appended to the function body when it's written
out, so strip any trailing newlines which may be there already.

(Bitbake rev: 8a3f50936113e15d2f2822f6aee494204fa1c24f)

Signed-off-by: Andre McCurdy <armccurdy@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-30 11:43:55 +00:00
Andre McCurdy 68600aee2d bitbake: build.py: minor shell_trap_code() formatting tweaks
Fix quoting of $BASH_COMMAND and avoid wrapping at 80 columns (the
script which follows is likely to contain some very long lines, so
line wrapping in bb_exit_handler() looks somewhat out of place).

(Bitbake rev: 8e12c8f8441a7c6a03e603c5789d6037945704c1)

Signed-off-by: Andre McCurdy <armccurdy@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-30 11:43:55 +00:00
Peter Kjellerstedt d61d290ad7 bitbake: bitbake-user-manual-ref-variables: Update the help for BBMASK
Update the help for BBMASK to reflect that it is now allowed to
contain multiple regular expressions. Also changed the examples for
BBMASK to be a bit more diverse.

(Bitbake rev: 023a5b69df05adce0ed192b393b45eb879c096a0)

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-29 17:03:53 +00:00
Peter Kjellerstedt a948f5252a bitbake: cooker: Allow BBMASK to contain multiple regular expressions
Before, BBMASK was only permitted to contain one regular expression.
This made it hard to add to the BBMASK in multiple places as one was
supposed to separate the different regular expressions with a "|"
rather than with whitespace as is customary in BitBake variables.

Now one can specify any number of regular expressions in BBMASK. This
makes it possible to, e.g., mask out recipes in another layer from the
layer.conf file.

This also properly ignores any regular expressions that do not compile
(before an invalid regular expression would cause a ParseError in the
first bbappend file found stating that it was not a BitBake file...)

(Bitbake rev: 2c778ad50aceaffb855baf5f4aa0fed98c880870)

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-29 17:03:53 +00:00
Martin Samuelsson e82101a4ed bitbake: bitbake-user-manual-metadata: Updated 'dir' flag
Added more detail to the 'dir' flag to state the last
directory listed will be used as the work directory for the
task.

(Bitbake rev: 2fbd9e4e44f6b491cccca81fd34586fcfdeea3d3)

Signed-off-by: Scott Rifenbark <srifenbark@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-29 17:03:53 +00:00
Scott Rifenbark 100d6c2782 bitbake: bitbake-user-manual: Updated the example BitBake directory
This directory no longer has setup.py as part of the directory.
I removed that line from the example in the
"Obtaining BitBake" section.

Fixes YOCTO #8623

(Bitbake rev: 6a82f6432022debe48830db98b3fb16424411a87)

Signed-off-by: Scott Rifenbark <srifenbark@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-29 17:03:53 +00:00
Ross Burton 9f22898e0f bitbake: fetch2/wget: fallback to GET if HEAD is rejected in checkstatus()
The core change here is to fall back to GET requests if HEAD is rejected in the
checkstatus() method, as you can't do a HEAD on Amazon S3 (used by Github
archives).  This meant removing the monkey patch that the default method was GET
and adding a fixed redirect handler that doesn't reset to GET.

Also, change the way the opener is constructed from an if/elif cluster to a
conditionally constructed list.

(Bitbake rev: 6ec70d5d2e330b41b932b0a655b838a5f37df01e)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-26 22:34:45 +00:00
Daniel Istrate 72f98ba577 bitbake: toaster: Update UI test runner
Add new runner options:
    --run-all-tests: finds all tests, ignores config
    --run-suite <suite> (from cfg)

Without arguments, run tests from current os section (config), e.g.:
    1. ./run_toastertests
    2. ./run_toastertests --run-all-tests
    3. ./run_toastertests --run-suite darwin

Update toaster logging to meet QA CI requirements.

(Bitbake rev: 5685feb51fbb6d54fde6027cc765b9edd8eda65a)

Signed-off-by: Daniel Istrate <daniel.alexandrux.istrate@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-25 16:29:10 +00:00
Richard Purdie 4a5458f38f bitbake: fetch2: Don't show checksum warnings if a single checksum was supplied
If one checksum is supplied to a SRC_URI, we really don't want to show
warnings about the other type which isn't present as one checksum
is really good enough for most cases.

(Bitbake rev: 43358a9b595b2928458a5f463cf1949394160c3a)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-24 09:40:42 +00:00
Ed Bartosh bd1a534a11 bitbake: toaster: run bitbake server with --read option
Toaster script run bitbake with --postread conf/toaster.conf.
It turned out that variables set this way don't influence the
build. Changing --postread to --read makes bitbake to read
variables earlier and should fix this issue.

[YOCTO #8781]

(Bitbake rev: 76c9871740ef42ac35fdfdcb89a68478cca370cd)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-22 12:45:44 +00:00
Paul Eggleton 76a281c870 bitbake: taskdata: add the ability to access world targets list
In certain circumstances it can be useful to get access to the world
targets list from a recipe in order to add dependencies on some or all
of the items in it. If a special function, 'calculate_extra_depends' is
defined in the recipe, and the recipe is to be built, then call it at
the right point before we calculate which tasks should be run. The
function can append items to the "deps" list in order to add
dependencies. This is not as tidy a solution as I would have liked, but
it does at least do the job.

As part of this change, the buildWorldTargets function was moved to
bb.providers to make it possible to call from taskdata.

Part of the implementation of [YOCTO #8600].

(Bitbake rev: aba0dce57c889495ec5c13919991a060aeff65d2)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-22 12:45:44 +00:00
Robert Yang 11a1f49630 bitbake: cache.py: check existence before add to cachedata.rproviders
The rprovides maybe contain duplicated lines when parse again, we need
check it before add to cachedata.rproviders, similar to what we had done
to cachedata.providers.

(Bitbake rev: 6c488afb0fe30a9655ec62a1d22f9f388365f012)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-22 12:45:44 +00:00
Robert Yang 05c17750c3 bitbake: taskdata.py: add RuntimeProviders to close matches
This is useful for newbie, for example:
$ bitbake rpm-build
ERROR: Nothing PROVIDES 'rpm-build'. Close matches:
  pm-utils
  rpm RPROVIDES rpm-build

[YOCTO #8881]

(Bitbake rev: 4b59eb8cc2321fe72f2988b6c9c0fecd4883255b)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-22 12:45:44 +00:00
Richard Purdie cf9cb65eec bitbake: data_smart: Don't show exceptions for EOL literals
If variables are unset, the code simply doesn't expand them, there
aren't errors. If the code is a python expression, this can get a bit
messy, see the attached test case. The python expansion code sees the }
of the unexpanded value rather than the close of the python expression
and then raises a SyntaxError exception.

Ideally, we'd update the code to match pairs of brackets. I don't know
how to do that with the current regex and this is unfortunately a
performance sensitive piece of code. We also run the risk of breaking
existing code in OE-Core where there are "{" characters but not "}"
to close them (PKGE and PE).

Rather than raising the exception, matching the existing "just return
the expression" behaviour seems more consistent with the standard
variable behaviour.

This addresses an issue found in the recent image.bbclass code where
there are some variables we choose not to expand (TMPDIR/DATETIME).

This patch also adds a test case for this behaviour. It wouldn't preclude
improved bracket matching code in the future either.

(Bitbake rev: d80d39e73223a50fda0090784303d2c57167bb4c)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-22 12:01:24 +00:00
Olof Johansson c886a786cf bitbake: tests/codeparser.py: Add filename/lineno flags to test variable
A recent change in bitbake added filename/lineno information to the
parameters of bb.data.build_dependencies(). The codeparser tests
required a little adaption to the changes, adding the flags to the FOO
variable used in the tests.

The error seen when running the tests is a TypeError exception raised
in bb.codeparser:

  TypeError: int() argument must be a string or a number, not 'NoneType'

(Bitbake rev: f1fe674397ac5cd355696d5b4cc90b7cfa6c867f)

Signed-off-by: Olof Johansson <olof.johansson@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-19 17:38:51 +00:00
Ed Bartosh f130033856 bitbake: toaster: write variables to toaster.conf
With the resent change in rootfs creation code setVariable
rpc calls don't set variables for bitbake workers anymore.

Writen variables to toaster.conf should solve this issue.

[YOCTO #8910]

(Bitbake rev: d6dfe40320ff6ca420d9f9016b4d1d83d10f1d59)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-19 17:38:51 +00:00
Richard Purdie f0d47a6d9a bitbake: Revert "runqueue.py: Ensure one setscene function doesn't mask out another which needs to run"
This reverts commit b22592af81.

That commit isn't entirely clear about why this change is needed but
I do have a usecase where this breaks things. If for example you run
"bitbake X -c packagedata" and that packagedata is in sstate, you'd
expect this to work.

If sstate doesn't contain a do_populate_sysroot for a dependency, you
would still expect the command above to succeed and you would not
expect
it to rebuild that dependency. With the current code, this isn't what
happens. The code finds the sstate for do_populate_sysroot missing,
this makes the task "uncovered" and this in turn makes it unskippable.

The example I found with this was avahi-ui, where it would trigger
a build of libdaemon to obtain its populate_sysroot.

Since this behaviour seems completely incorrect, revert the older patch
and we'll address any issues that crop up as a result.

(Bitbake rev: 36a9840a5da17cc14561881fdd6a4f2cb0a75e49)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-19 17:26:15 +00:00
Elliot Smith f98e11c809 bitbake: toastergui: make artifact download more robust
When an artifact download is requested, Toaster goes through a
convoluted series of conditions to decide which file to push
to the response. In the case of build artifact downloads for
command line builds, this caused an ugly exception, as command
line builds don't have a build request.

To simplify and catch more corner cases, remove the code which
fetches files via the build environment (we only support the local
build environment anyway). Then push all requests along a single
path, catching any missing file errors, missing object errors
or poorly-formed URLs in a single except clause which always returns
a valid response.

Also modify the text on the "unavailable artifact" page so it
says that the artifact doesn't exist, rather than it "no longer"
exists (exceptions may occur because an invalid artifact was
requested, rather than an artifact which was removed).

[YOCTO #7603]

(Bitbake rev: 24e20db55c2933de5e58ca754b8fd5b624f47820)

Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-18 22:06:46 +00:00
Elliot Smith 68f3e1e067 bitbake: toasterui: log OSErrorException metadata events
OSErrors occurring in toaster.bbclass are converted to
OSErrorException metadata events. They were then being swallowed
as unprocessed events by toasterui, which made them difficult
to spot.

Explicitly catch OSErrorException events and log them so they
are easier to spot and debug.

(Bitbake rev: 69f2b2bc373ce114609600b59a6b6ccef20771c9)

Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-18 22:06:46 +00:00
Elliot Smith fb94754e83 bitbake: toasterui: listen for bb.event.MetadataEvent
The event mask for toasterui doesn't include MetadataEvents.
This means that we're missing the ArtifactFileSize event
(among others), which is the one we use to populate the SDK
artifact table.

Add that event type to the toasterui event mask so we can
record SDK artifacts as they are created.

[YOCTO #7603]

(Bitbake rev: d0276a831bb8cffd42c8367895633eaa1fa1ed30)

Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-18 22:06:46 +00:00
Ed Bartosh 0fe6e2dc7b bitbake: toaster: increase timeout
Sometimes Toaster connection to the bitbake server fails with
this error in the log:
   ERROR: Could not connect to server 0.0.0.0:46572
   : Could not register UI event handler
It happens more often on slow machines.

Increasing timeout after restarting bitbake should fix this issue.

[YOCTO #8776]

(Bitbake rev: b64a0adc1efd72aa4d46df3e043dd14543abae3d)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-16 10:36:50 +00:00
Elliot Smith 798e8b8d03 bitbake: toastergui: code formatting and clean-up
Minor fixes to code formatting and small improvements from
code review.

(Bitbake rev: 2c97f3a5c6ae37de910deb90390c5b856a600c5f)

Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-15 16:30:01 +00:00
Elliot Smith c4b50111e9 bitbake: toaster tests: fix Django tests for new ToasterTable pages
The Django command-line tests can no longer test the content
of the projects/, builds/ and projectbuilds/ pages, as
ToasterTable pages are populated by JavaScript.

Fix/remove affected tests by converting them to tests on the
JSON returned by the ToasterTable.

[YOCTO #8738]

(Bitbake rev: 85efa9530fa6181855e051bfd14de1c15db9c3b7)

Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-15 16:30:01 +00:00
Elliot Smith 88a262cbd2 bitbake: toastergui: remove unused views and template code
The code in views.py for setting up the template context for
old non-ToasterTable views is no longer necessary, as this
is now implemented in tables.py.

The template files for these views have also been removed.

[YOCTO #8738]

(Bitbake rev: 2b5a13afb068c85466436914d8d4ac3b31bc5c02)

Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-15 16:30:01 +00:00
Elliot Smith 059a274aa9 bitbake: toastergui: fix error and warning counts for builds
The error and warning counts displayed for builds were counts of
the errors and warnings objects associated with a build. Because these
values were being derived on the fly, it was not possible to sort by
them.

Previously, the 3rd party django-aggregate-if library was used to
add aggregate fields to Build objects and should then have been
used to populate the "all builds" and "project builds" tables. However,
at some point the templates had changed so that the error and warning
counts were coming from the properties on the Build model and not from
these aggregates. This meant that it was not possible to sort by these
fields.

Django 1.8 supports conditional aggregates in annotation fields on
querysets. This means we can remove django-aggregate-if, use the new Django
1.8 feature to derive errors_no and warnings_no fields as annotations,
then use those annotation fields in the templates. This makes the "builds"
tables sortable again.

[YOCTO #8738]

(Bitbake rev: 9be7c5c18b325f6ed40bc431ac408db242007eb1)

Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-15 16:30:01 +00:00
Elliot Smith 4103e0cb74 bitbake: toastergui: make "Apply" button state depend on filter range
If a range filter action had an empty from/to field, the range
filter could still be applied. This was confusing, as an invalid
filter range caused all records to display, even though a filter
appeared to have been applied (by the highlighted state of
the filter button).

Change the state of the "Apply" button, disabling it if the radio
button for a range filter action is selected but the range is
incomplete (from or to field is empty).

When a non-range filter is selected, the "Apply" button always
enable the "Apply" button.

[YOCTO #8738]

(Bitbake rev: 168184b28165d7aa354b9092b5986f91c58d550d)

Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-15 16:30:00 +00:00
Elliot Smith 6c2d88fda5 bitbake: toastergui: mute label for filter actions with no records
The radio button for a filter action is disabled if that filter
action has no associated records. However, the label retains
the normal font styling, so it's unclear that the action is
not available.

Add the "muted" class to the label for a filter action (and still
disable its radio button) if it has no records associated with it.

[YOCTO #8738]

(Bitbake rev: b7f7ff095c9c4c922e608f776713f17acc1f150d)

Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-15 16:30:00 +00:00
Elliot Smith f08730ab4d bitbake: toastergui: set default visible and hideable columns
Incorrect columns were shown by default in the "all builds",
"project builds" and "all projects" pages.

Set the "hidden" property on columns in these tables to hide the
correct columns.

Add a set_column_hidden() method to ToasterTable so that the
"hidden" property can be overridden for the machines column
in the project builds page (it shares a superclass with
all builds).

Make the time column on all builds page hideable.

[YOCTO #8738]

(Bitbake rev: be3982c71703eaa51e7f3352e0cb5b3af11c9ead)

Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-15 16:30:00 +00:00
Elliot Smith 112f3746cd bitbake: toastergui: serialise decimals correctly
The conversion of some ToasterTable Build object querysets to
JSON caused a serialisation error. This is because one of the
fields in the queryset was of type decimal.Decimal, and our
serialiser didn't know what to do with it.

Add a clause to check for decimal fields and serialise them
so that correct JSON can be generated.

(Bitbake rev: fa6229d4edf5904ccaa9dc323d0ab2318d1ef314)

Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-15 16:30:00 +00:00
Elliot Smith e024aab39c bitbake: toastergui: streamline construction of filter objects
In line with comments from review, remove the QuerysetFilter
class (redundant) and convert ProjectFilters into a class
with static methods.

[YOCTO #8738]

(Bitbake rev: 59379bf6467029223045c5ebef868729d8e02c86)

Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-15 16:30:00 +00:00
Elliot Smith fcb20f9dfd bitbake: toastergui: ensure filter_value updates
Clicking on the radio button for a date range filter action
populates the from and to fields for that action if they are empty.

However, because this doesn't fire "change" events, clicking on
the radio button doesn't update the filter_value hidden field. This
means that the date range action's filter_value parameter isn't
set correctly when the filter popup is submitted.

Manually call the changeHandler() to set the filter_value whenever
the radio for a date range filter is clicked.

[YOCTO #8738]

(Bitbake rev: 1a3038cf8d9b32532f1fe92cd3472b4473ffc0c4)

Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-15 16:30:00 +00:00
Elliot Smith f9c46f5e5e bitbake: toastergui: don't hide all elements with .col class
Disabling the "project" column in a ToasterTable for builds
causes the recent builds area to be hidden. This is because
the column hiding code hides all elements with a class matching
".<column>", regardless of where they occur on the page; and
the recent builds area was using the ".project-name" class,
which means it is included in the set of elements which are hidden.

Scope the element search to the table so that only elements
within the table are hidden or shown.

[YOCTO #8792]

(Bitbake rev: 151bc20daf51e0ab8baf11ea29244b7fec1f8a22)

Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-15 16:30:00 +00:00
Elliot Smith eaae82a19a bitbake: toastergui: convert project builds page to ToasterTable
Use the all builds ToasterTable as the basis for the project builds
ToasterTable.

[YOCTO #8738]

(Bitbake rev: 87bcfb740dd2d9944e35a2a1f71cbf8ff3b266e9)

Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-15 16:30:00 +00:00
Elliot Smith 33b011c158 bitbake: toastergui: implement "today" and "yesterday" filters
Add the "today" and "yesterday" filters to the started_on
and completed_on columns in the builds table.

During this work, some minor adjustments were made to the
behaviour of the builds table:

* Amend filter action variable names so they're more succinct.
* Retain order in which actions are added to a filter, as this
ordering is used in the UI when displaying the filter actions.
* Always show the table chrome, otherwise it's not possible
to edit the columns shown until there are 10 or more results.
* Because date range searches may return no results, make sure
that the search bar and "show all results" link are visible
when the query returns no results.

[YOCTO #8738]

(Bitbake rev: f17cfa009e58833e0e55884fa04de8abd522b6bc)

Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-15 16:30:00 +00:00
Elliot Smith f8d383d87f bitbake: toastergui: implement date range filters for builds
Implement the completed_on and started_on filtering for
builds.

Also separate the name of a filter ("filter" in the querystring)
from its value ("filter_value" in the querystring). This enables
filtering to be defined in the querystring more intuitively,
and also makes it easier to add other types of filter (e.g.
by day).

[YOCTO #8738]

(Bitbake rev: d47c32e88c2d4a423f4d94d49759e557f425a539)

Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-15 16:30:00 +00:00
Elliot Smith b929889cdd bitbake: toastergui: show recent builds on all builds page
The recent builds section was disabled while converting the
all builds page to ToasterTable.

Re-enable the recent builds area and add the data it requires
to the ToasterTable context.

[YOCTO #8738]

(Bitbake rev: d6df4545bd134a23c9bd3cd1ba3b61ddb26545e4)

Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-15 16:30:00 +00:00
Elliot Smith 1a4b203ca5 bitbake: toastergui: switch off filter highlights when inactive
In ToasterTables with multiple columns which allow filtering
(e.g. all builds), selecting one filter, then a second filter
(e.g. selecting "failed builds" then "outcome" for all builds),
would result in both filters being highlighted at the same time.

Fix this by removing the "active" highlight on all column filter
buttons when a new filter value is submitted (via the filter modal).

NB to enable this, added a data-filter-on attribute to all
column filter buttons to make them easy to select.

[YOCTO #8738]

(Bitbake rev: 7347ad0d4baace593751b44a86ab8e11a04a02b6)

Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-15 16:29:59 +00:00
Elliot Smith 809046c6fb bitbake: toastergui: refactor ToasterTable filtering
The filter code for ToasterTable was difficult to follow
and inflexible (not allowing different types of filter, for example).

Refactor to a set of filter classes to make the structure cleaner
and provide the flexibility needed for other filter types
(e.g. date range filter).

[YOCTO #8738]

(Bitbake rev: 94031bb30bdaf665d0c8c68b591fcb7a17b6674d)

Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-15 16:29:59 +00:00
Elliot Smith 294579b531 bitbake: toastergui: convert all builds page to ToasterTable
For better long-term maintainability, use ToasterTable instead
of Django template and view code to display the all builds page.

NB the builds.html template has been left in, as this will
otherwise cause conflicts when merging the new theme.

[YOCTO #8738]

(Bitbake rev: e0590fc8103afeb4c5e613a826057555c8193d59)

Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-15 16:29:59 +00:00
Elliot Smith 6c12ca7f93 bitbake: toastergui: use event delegates for hover help elements
libtoaster.js binds to hover help elements via their hover() and
mouseout() methods. However, any elements added to the DOM after
libtoaster has initialised will not have these bindings added.
This causes a problem for ToasterTables which have hover-help
elements (e.g. the builds/ table).

Use the on() method instead. This uses event delegation to bind
a handler to any th or td elements already in the DOM, or
which will be added to the DOM in future. ToasterTables can
now reconstruct the table DOM and still have the correct handlers
attached once the table is done.

[YOCTO #8738]

(Bitbake rev: 1f3ff01fed0b4de8721191f108033ad044cdc26a)

Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-15 16:29:59 +00:00
Elliot Smith ef93dce7e1 bitbake: toastergui: switch projects/ view to ToasterTable
Remove the old projects page and replace with the new
ToasterTable-based version.

NB although the projects.html template is no longer required,
it's been left in as there will be changes applied to it for
the new theme. These changes will have to then be transferred
from the projects.html template to projects-toastertable.html.

Similarly, the code for the projects page in views.py has been
retained.

[YOCTO #8738]

(Bitbake rev: ebe7831ef65e78a9d100f29a63311518577fc838)

Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-15 16:29:59 +00:00
Elliot Smith 417f1d3f53 bitbake: toaster: check inferred file suffixes against list of known types
The algorithm for finding the suffix for image files produced by
the build doesn't reference a list of known file suffixes, so
could be prone to error.

Modify how file suffixes are parsed from the file path so that
they are compared against a list of known types; if this fails,
use the part of the basename of the file path after the first
'.' character.

Also rationalise the places in the views code where we
extract the file name extensions for builds, so they both use
the same algorithm (before, the same code was duplicated in
two places).

[YOCTO #8417]

(Bitbake rev: dd1c509696b8ab5e593cc64637060a58e95fcd1f)

Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-15 16:29:59 +00:00
Elliot Smith c02ee05857 bitbake: toaster: move image file suffix list to model
Image file suffixes are used in the project configuration page to
show a list of available image file types. This list is stored
as a function in the views code.

However, this list is also needed when parsing image file paths,
so that the suffixes can be shown in the "all builds" and "project
builds" tables.

Move the list of valid image file suffixes to the Target_Image_File
class to make is accessible in other places where it may be needed.

[YOCTO #8738]

(Bitbake rev: c2f20232077917552623fd0726d0820e50b04cae)

Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-15 16:29:59 +00:00
Elliot Smith d29e4cd040 bitbake: toastergui: use ToasterTable for projects page
The projects page uses the old approach for showing tables,
which means a template for each table. This means that applying
changes to ToasterTable (which is used for most tables) has
no effect on the layout, styling and behaviour for these older
tables, and requires additional duplicated effort.

Move the projects page to use ToasterTable instead, to remove
the duplication of effort.

[YOCTO #8738]

(Bitbake rev: df56a35bc71639457329c4b58839976c5ee40106)

Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-15 16:29:59 +00:00
Richard Purdie f8508deb01 bitbake: Revert "fetch/git: Change to use clearer ssh url syntax for broken servers"
This reverts commit 4193e99adce8e88f12ac88d7578ad39575f7e346.

It seems the underlying issue was caused by ":" in the url which isn't
supported. The patch was therefore incorrect.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-15 15:02:40 +00:00
Richard Purdie da43a56d35 bitbake: Revert "fetch2/local.py: avoid using PREMIRROR"
This reverts commit e130dca85bac82bd4d88f94a6bf9fe36e8ad4d7c.

This is in fact a valid use case, for example the sstate.bbclass code
sets up SSTATE_MIRRORS as PREMIRRORS. Its quite common to map those
file:// urls to remote http:// urls and with the above change, this
no longer works.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-13 09:47:28 +00:00
Kristian Amlie 1724ffd32d bitbake: fetch2/git.py: Add missing "errno" module import.
This goes undetected most of the time, but when updating a repository,
if the ud.fullmirror file is not present, you end up getting an
exception instead of carrying on because the errno module is not
loaded (specifically "if exc.errno != errno.ENOENT").

(Bitbake rev: e6fca8480731ce817df9bee61438347a5e3d3017)

Signed-off-by: Kristian Amlie <kristian.amlie@mender.io>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-11 23:28:55 +00:00
Chen Qi 74fa824e6c bitbake: bitbake: clean up stamp-base related codes
The 'stamp-base' and 'stamp-base-clean' related codes are no longer useful,
clean them up.

[YOCTO #8468]

(Bitbake rev: 7b4c42b315d4a26dd8f2ceb874a94737bf9f183e)

Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-11 23:28:55 +00:00
Richard Purdie 43328feccc bitbake: runqueue: Fix setscene task dependencies
Debugging suggests that setscene tasks are being a little greedy about their
dependencies, for example, lsof is insisting that gcc-runtime's do_package
is installed. If it isn't, its requiring gcc to rebuild.

If gcc-runtime do_package_write_xxx and do_packagedata is available, there
is no reason do_package should be needed.

The reason this is happening appears to be from the batching up of task
dependencies code, rather than setscene tasks stopping when passing over
a setscene task, they were being carried forward. This patch fixes it
so the data is 'zeroed' when passing over a setscene task boundary,
which gives the dependency graph that is expected.

After this patch, lsof will rebuild quite happily without
gcc-runtime:do_package being present, as expected. This should lead to
less dependencies being installed for builds from sstate and generally
better performance in general.

(Bitbake rev: f8bcb0a1e3b008b71c9a7cd21f76d0906f2d8068)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-11 23:26:33 +00:00
Michael Wood 7b905caf60 bitbake: toaster: settings Add uid to the toaster cache dir
Make the default toaster cache dir unique to the user running
toaster. If we have multiple users running toaster we previously
got a permission denied exception on saving a cache file.

[YOCTO #8782]

(Bitbake rev: 5207abdf58019271bf92bff4bcce3911b8691508)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-11 23:26:33 +00:00
Ed Bartosh dff7a277e3 bitbake: toaster: show 'satisfied via' text for reverse deps
Showed '<dependency> satisfied via <provider>' text and
help tooltip for the reverse build dependencies provided
through 'PROVIDES' in the 'Reverse build dependencies' tab.

[YOCTO #6169]

(Bitbake rev: c7bb98e2e2111790ded86087b13c8b49462d6b75)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-11 23:26:33 +00:00
Ed Bartosh 89f493261a bitbake: toaster: show 'satisfied via' text for build deps
Showed '<dependency> satisfied via <provider>' text and
help tooltip for the build dependencies provided through
'PROVIDES' in the 'Build dependencies' tab.

[YOCTO #6169]

(Bitbake rev: de77e338fe70341fe98561e2e40b534f5c88db10)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-11 23:26:33 +00:00
Ed Bartosh febb898a06 bitbake: toaster: show list of provides for the recipe
Showed list of names that recipe provides.

(Bitbake rev: 60318c9a049292bd33322d8446a629d778337e8a)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-11 23:26:33 +00:00
Ed Bartosh 2ff4ccb13f bitbake: buildinfohelper: add provides info to the db
Added new entries to Provides model and link them to
Recipe_Dependency using 'via' field.

This data will be used by Toaster UI to show 'Provides:'
information for the recipes.

[YOCTO #6169]

(Bitbake rev: 336ddc8df611d4c8f1c3d3a06d0a85bb544c38bc)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-11 23:26:32 +00:00
Ed Bartosh 16a81fb97a bitbake: toaster: add Provider model
Added new model Provider and a foreign key 'via' to link
Recipe_Dependency to it.

[YOCTO #6169]

(Bitbake rev: e45fff6314741d46e2549b2f72ed380cbbb95593)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-11 23:26:32 +00:00
Ed Bartosh 6a28ed33b7 bitbake: buildinfohelper: use providermap
Used providermap in store_dependency_information function
to find virtual dependencies. This should fix annoying
warnings "stpd: KeyError saving recipe dependency"

[YOCTO #6169]

(Bitbake rev: 85c416ca338c886db6e79651e44727482df9fb07)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-11 23:26:32 +00:00
Ed Bartosh f2b7252190 bitbake: cooker: add providermap to dep_tree
Added providermap information to the result of buildDependTree API.

This will be used by Toaster to map virtual dependencies to recipes.

(Bitbake rev: d3e07368549f30265f59846a260efa8230a225ca)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-11 23:26:32 +00:00
Ed Bartosh 7e380d460c bitbake: taskdata: refactor get_providermap
Added optional parameter 'prefix' to filter out names that
don't start with specified prefix. Changed existing call
of get_providermap according to changed API.

Optimized the code: got rid of extra loop and temporary
list variable virts.

(Bitbake rev: df5a1392d6f91ccb44a99721c7d847da242121bb)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-11 23:26:32 +00:00
Richard Purdie 46731daa5f bitbake: main/runqueue: Add --setscene-only option to bitbake
Its turning out that we really need a way to have bitbake just run
the setscene tasks but not any real tasks, particularly for SDK
operations.

Add an option for this since its pretty straight forward. This allows
various nasty workarounds in OE-Core to be removed.

(Bitbake rev: e4a2aafa1650a227a04d92a8a0b31efaed2c310e)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-11 23:26:32 +00:00
Richard Purdie 221d864042 bitbake: fetch/git: Change to use clearer ssh url syntax for broken servers
Some servers, e.g. bitbucket.org can't cope with ssh:// as part of
the git url syntax. git itself is happy enough with this but you
get server side errors when using it.

This changes the git fetcher to use the more common ssh url format
which also means we need a : before the path.

Seems a shame to have to do this due to broken servers however
it should be safe enough since this other form is the one most people
use on the commandline so it should be safe enough.

[YOCTO #8864]

(Bitbake rev: 4193e99adce8e88f12ac88d7578ad39575f7e346)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-08 13:58:08 +00:00
Richard Purdie 46d62d076d bitbake: knotty: Use non-interactive mode as fallback for dumb terminals
TERM=dumb bitbake X

shows no output for task status which is suboptimal. Use the non-interactive
mode if the terminal doesn't support what we need for interactive mode giving
a better user experience. Also print a note to the console to say this has
happened.

[YOCTO #8768]

(Bitbake rev: 6f84cf4bd77f35fcd07e0b2f5149f1d6866a414d)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-08 13:58:08 +00:00
Ross Burton bfa7859ffa bitbake: cooker: fix findFilesMatchingInDir documentation
The documentation for findFilesMatchingInDir() was inconsistant with the
implementation: the regex was escaped before searching so effectively it's a
pure textual substring, and the machine example was broken.

(Bitbake rev: 6bef981488ec94b46dbe3797acfecf9c4b6ecbbc)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-08 13:58:08 +00:00
Ross Burton 3d4273716d bitbake: cooker: use in instead of count
No point counting all instances when we just want to know if there's any or not.

(Bitbake rev: 3369072efb653339da8dbd1ca864ff8e1ff899ca)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-08 13:58:08 +00:00
Richard Purdie f8185ff9f8 bitbake: ast: Add filename/lineno to mapped functions
Where we add in mappings for EXPORT_FUNCTIONS, add dummy filename
and lineno data so ensure the assumption that all python functions
have this is correct.

(Bitbake rev: 547128731e62b36d2271c4390b3fee2b16c535dc)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-06 15:27:35 +00:00
Ed Bartosh a178c5a163 bitbake: main: kill server without queue setup
Setup of event queue includes registering of UI handler.
This operation can fail when cooker is busy. However, there is
no need in registering UI handler for terminating the server.

Moved the call of connection.terminateServer before setting up
of the event queue. This should make terminating server to work
more reliably as it doesn't depend on setting up the event queue
and registering UI handler anymore.

This should also help Toaster backend to restart bitbake server
and observer without getting "Could not register UI event handler"
errors.

[YOCTO #8776]

(Bitbake rev: 0c5a9349f797d05c282c2ada1893e187e05f0576)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-06 15:27:35 +00:00
Ed Bartosh 773700dbc3 bitbake: xmplrpc: split connect method
Current code in connect method sets up event queue, which requires
registering UI handler. This functionality may not be needed for
some operations, e.g. for server termination.

Moved functionality of setting up event queue in from 'connect'
method to 'setupEventQueue' in BitBakeXMLRPCServerConnection class.

(Bitbake rev: 4429871da76d6bd29e023ff42740fe7daa6b40fa)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-06 15:27:35 +00:00
Ed Bartosh 05b4fbc947 bitbake: uievent: refactor retry loop
Replaced 'while' loop with 'for' loop.
Made the code more compact and hopefully more understandable.

(Bitbake rev: 4e1e497c8432536b3522295e5b1284844ccea056)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-06 15:27:35 +00:00
Ed Bartosh ebc169c360 bitbake: uievent: get rid of EventHandler attribute
This attribute was introduced by mistake. EventHandle is used in the
code for the same purpose.

(Bitbake rev: 8d505ec8913a7d51de48b4f52bb64c5d6a0bb08e)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-06 15:27:35 +00:00
Ed Bartosh 4e0de6eca2 bitbake: uievent: add error to registerEventHandler return
Current code throws Exception("Could not register UI event handler")
if event handler can't be registered. The real reason of this is that
cooker is in busy state. Error message lacks information about this.

Added error message to the return value of registerEventHandler.
Included returned error message into the log message and exception
text.

(Bitbake rev: 07de1ca7d57dcd0cc37406feae2949da12a3fa7a)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-06 15:27:34 +00:00
Ed Bartosh 01419d5565 bitbake: cooker: add state.get_name method
Added method to convert state code into the human readable name.
It will be used in logging and error reporting.

(Bitbake rev: 9ec6379b27d210214d0b3f2e55962f721b7f5f51)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-06 15:27:34 +00:00
Juro Bystricky 763506d49b bitbake: fetch2/__init__.py: Add support for 7-Zip
7-Zip is a file archiver claiming the highest compression ratio.
This patch allows using 7-Zip commpressed files in bitbake recipes.
Two common formats are supported:

SRC_URI = "file://abc.tar.7z"
SRC_URI = "file://abc.7z"

(Bitbake rev: 7120f5bfaae54e91bc95da5667831424724ce613)

Signed-off-by: Juro Bystricky <juro.bystricky@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-06 15:27:34 +00:00
Richard Purdie f5bfc1cc26 bitbake: utils: Remove double compile from better_compile
Poking around the ast to correct linenumbers works well for runtime failures
but not for parsing ones. We can use blank linefeeds to correct the line
numbers instead, with the advantage that we don't need to double compile.

(Bitbake rev: 10256ac3e7be7e691176ecc5d55856d88f1fe940)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-06 15:27:34 +00:00
Robert Yang b4141f6494 bitbake: fetch2/local.py: avoid using PREMIRROR
The PREMIRROR isn't useful for "file://", so avoid using it, this is
good for searching speed and can reduce useless lines in log.do_fetch.

(Bitbake rev: e130dca85bac82bd4d88f94a6bf9fe36e8ad4d7c)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-06 15:27:34 +00:00
Richard Purdie 1ad3595e8c bitbake: siggen: Change exception note into a warning
This makes no sense as just a note, its at least a warning and useful
to get an idea of which codepath is failing.

(Bitbake rev: 0194cf0da24dc72dab0612cd54aa5190e6cd92f2)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-06 15:27:34 +00:00
Richard Purdie 4ba49aca42 bitbake: data: Drop misleading ExpansionError exception
This codepath can be triggered by a python indentation error for example.
Showing it as an ExpansionError is misleading.

Change the code to add a warning about where the failure came from (in
particular giving the variable key name that triggered it) but raise the
proper exception.

(Bitbake rev: d49d46533704e8b4404e29abfb5a7383d704c91a)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-06 15:27:34 +00:00
Richard Purdie 2c9431135d bitbake: cooker: Drop useless parsing exception
The SyntaxError exception simply shows the recipe that failed to parse
which is pretty useless without the actual exception. We could make it
print more info, however we can just use one of the more generic handlers
instead and remove this one.

For a python indentation error, this leads to a much more readable error
message.

(Bitbake rev: 9241eb10847634e34c5ff8767ed8c114f66ff6cf)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-06 15:27:34 +00:00
Richard Purdie a16b543436 bitbake: data: Pass lineno/filename data from build_dependencies
If parse_python() fails, the output is confusing. Passing in the extra
file/line data isn't expensive and improves readability significantly.

(Bitbake rev: a4bb753488d322e0e31c31d6377ba780f2f824c4)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-06 15:27:34 +00:00
Richard Purdie 958f0ff56b bitbake: codeparser: Add support for correct linenumbers
Currently, if there is something like a python indentation error in a
python function, the linenumbers and even file aren't reported correctly.

This allows lineno and filename parameters to be passed in to correct this.

The lack of a lineno parameter to python's compile() function is worked
around by using empty linefeeds. Ugly, but effective and with minimal
performance overhead.

(Bitbake rev: 5796ed550d127853808f38257f8dcc8c1cf59342)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-06 15:27:34 +00:00
Richard Purdie 0381b78aa4 bitbake: event/utils/methodpool: Add a cache of compiled code objects
With the addition of function line number handling, the overhead of
the compile functions is no longer negligible. We tend to compile
the same pieces of code over and over again so wrapping a cache around
this is beneficial and removes the overhead of line numbered functions.

Life cycle of a cache using a global like this is in theory problematic
although in reality unlikely to be an issue. It can be dealt with
if/as/when we deal with the other global caches.

(Bitbake rev: 98d7002d1dca4b62042e1589fd5b9b3805d57f7a)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-22 00:02:05 +00:00
Richard Purdie c61c1eb26a bitbake: BBHandler: Improve IN_PYTHON_EOF handling
Now we're actively using the line numbers for other thins, having
magic values like IN_PYTHON_EOF causes problems, in particular, 32
bit overflow on 32 bit machines.

There is a neater way to signal eof to feeder(), just using an extra
parameter so use this instead and drop the IN_PYTHON_EOF magic values.

This has the added bonus that line numbers are then correct for
python functions at the end of files.

(Bitbake rev: e0f05871c2a6f1e86ae19ad343c7c6f822ddb67e)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-22 00:02:05 +00:00
Ed Bartosh 5f406915b5 bitbake: toaster: remove 2 confusing parameters
Removed nobrowser and brbe script parameters as both
are confusing and nobrowser is not used anywhere.

brbe parameter usage can only be justified if toaster
doesn't work properly and user has to manually connect
toaster to running bitbake server. Even in this scenario
it's very unlikely to achieve as toaster script is not
designed for this kind of usage.

(Bitbake rev: 0fd04ede3fda6894d97a5ef830b79dbbc9c6cf51)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-18 13:51:54 +00:00
Ed Bartosh 3960b6ec5c bitbake: toaster: move setting of default values
Set default values of script parameters just before
they are parsed to increase readability.

(Bitbake rev: 627f0d6adcfe281ef0487bf15a35151f1ceff194)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-18 13:51:54 +00:00
Ed Bartosh b194c0c00a bitbake: toaster: move startup checks to a better place
Moved addr:port, bitbake.lock and toastermain.pid checks
to the place where the rest of Toaster starting happens.

(Bitbake rev: 0e7812b5512e609ea815db30be1d7caca896ab60)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-18 13:51:54 +00:00
Ed Bartosh 064d2c7984 bitbake: toaster: remove 2 unused functions
Removed check_pidbyfile and notify_chldexit functions from
toaster script as they're not called in the script.

(Bitbake rev: ead2823e5457c9c4d0321c2191fb99982b282d26)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-18 13:51:54 +00:00
Ed Bartosh c505f24ebd bitbake: toaster: remove addtoConfiguration function
This function is useless as it's called just once
and makes code less readable.

(Bitbake rev: 96e85159905e44533882ad8290b1c9dd252ff3ea)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-18 13:51:54 +00:00
Ed Bartosh c7e4404273 bitbake: toaster: updated header of the toaster script
Updated GPL information, years of development and
usage information. Removed outdated information about
2 ways of starting Toaster.

(Bitbake rev: b4789c9e3093e1ec8e70f67305c88709443578e6)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-18 13:51:54 +00:00
Ed Bartosh af34920dff bitbake: toaster: add MANAGE variable
Used MANAGE variable to avoid repeating path to
manage.py in many places in toaster script.

(Bitbake rev: e8e19168ff43b72a2481771e7a40d9d7c155cfd2)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-18 13:51:54 +00:00
Ed Bartosh 563b786d4b bitbake: toaster: remove unused variable
Removed unused variable RUNNING from the toaster script.

(Bitbake rev: 8b3f0b2c6efa2962d9c16e453f4d2c1af6419c6c)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-18 13:51:54 +00:00
Ed Bartosh aa3cc12db9 bitbake: toaster: split long lines, add/remove whitespace
Made toaster script more readable by splitting long lines and
removing and adding whitespace.

(Bitbake rev: 6d61d7903eb4fb12464bfea54d3a225f99ad394f)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-18 13:51:53 +00:00
Ed Bartosh 8e4acacb53 bitbake: toaster: check if address:port is in use
Used new management command checksocket to check if
Toaster can listen on address:port.

[YOCTO #8775]

(Bitbake rev: 8a306ffe5a3642fe833c875362a183096a39f641)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-18 13:51:53 +00:00
Ed Bartosh 847b93596d bitbake: toaster: implement checksocket command
Implemented new management command to check if it's
possible to listen on specified address:port.

[YOCTO #8775]

(Bitbake rev: 0339b90842fd7c878c511b4b89ebcaee9a431bba)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-18 13:51:53 +00:00
Richard Purdie 522dcaa0cb bitbake: knotty: Improve exception error message
Instead of:
"""
can only concatenate tuple (not "int") to tuple
"""
we now see:

"""
Traceback (most recent call last):
  File "/media/build1/poky/bitbake/lib/bb/ui/knotty.py", line 324, in main
    termfilter.updateFooter()
  File "/media/build1/poky/bitbake/lib/bb/ui/knotty.py", line 210, in updateFooter
    lines = 1 + int(len(content) / (self.columns + 1))
TypeError: can only concatenate tuple (not "int") to tuple
"""

which makes tacking down and fixing the problem much easier.

Also ensure we set an error exit code.

(Bitbake rev: d965bcae6cfd268406a3bd1ef77c5bb6c6e1c6d7)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-18 12:18:19 +00:00
Richard Purdie 01d67bfa77 bitbake: knotty: Fix row/column function return value issue
When the row handling was introduced, one of the callbacks was
missed resulting in:

TypeError: can only concatenate tuple (not "int") to tuple

Fix it.

(Bitbake rev: 0b77cea2bf5b5f5704e2650fb0332f5d78037781)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-18 12:18:19 +00:00
Richard Purdie 6c12efa5c2 bitbake: buildinfohelper: Update for buildstats layout change
This updates buildinfo helper for the recent buildstats layout change

(Bitbake rev: 30311bbe667e9f22de17fae00ff58da06a7c3e23)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-18 12:18:19 +00:00
Christopher Larson 28ea1a1c95 bitbake: fetch: use orig localpath when calling orig method
When a mirror tarball is fetched, the original fetch method is called, which
unpacks the mirror tarball. After the original method is called, it checks the
localpath of the mirror tarball rather than the clone path, which isn't ideal,
particularly if the mirror tarball was removed due to being out of date. We
know the original fetch method will do what it needs to do to get its content
in the form it needs from the mirror tarball, so we can use its localpath
instead.

(Bitbake rev: 1732ad65d6c7d67b7d07cb30c074f5016adadbea)

Signed-off-by: Christopher Larson <chris_larson@mentor.com>
Signed-off-by: Awais Belal <awais_belal@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-18 12:18:19 +00:00
Richard Purdie 5cb6d83f17 bitbake: utils: Improve traceback from better_exec internal errors
If you break the internals of better_exec(), you get a very weird
error about tb_next not being a method of None. Fix this by checking
we can step back a trace level.

(Bitbake rev: 1d710ed484f68fca0789022dde7ba877b9a894f5)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-18 12:18:19 +00:00
Richard Purdie 0019edc818 bitbake: ast/event/utils: Improve tracebacks to include file and line numbers more correctly
Currently bitbake tracebacks can have places where the line numbers are
inaccurate and filenames may be missing. These changes start to try and
correct this.

The only way I could find to correct line numbers was to compile as a
python ast, tweak the line numbers then compile to bytecode. I'm open
to better ways of doing this if anyone knows of any.

This does mean passing a few more parameters into functions, and putting
more data into the data store about functions (i.e. their filenames
and line numbers) but the improvement in debugging is more than worthwhile).

Before:
----------------
ERROR: Execution of event handler 'run_buildstats' failed
Traceback (most recent call last):
  File "run_buildstats(e)", line 43, in run_buildstats(e=<bb.build.TaskStarted object at 0x7f7b7c57a590>)
NameError: global name 'notexist' is not defined

ERROR: Build of do_patch failed
ERROR: Traceback (most recent call last):
  File "/media/build1/poky/bitbake/lib/bb/build.py", line 560, in exec_task
    return _exec_task(fn, task, d, quieterr)
  File "/media/build1/poky/bitbake/lib/bb/build.py", line 497, in _exec_task
    event.fire(TaskStarted(task, logfn, flags, localdata), localdata)
  File "/media/build1/poky/bitbake/lib/bb/event.py", line 170, in fire
    fire_class_handlers(event, d)
  File "/media/build1/poky/bitbake/lib/bb/event.py", line 109, in fire_class_handlers
    execute_handler(name, handler, event, d)
  File "/media/build1/poky/bitbake/lib/bb/event.py", line 81, in execute_handler
    ret = handler(event)
  File "run_buildstats(e)", line 43, in run_buildstats
NameError: global name 'notexist' is not defined

----------------

After:
----------------
ERROR: Execution of event handler 'run_buildstats' failed
Traceback (most recent call last):
  File "/media/build1/poky/meta/classes/buildstats.bbclass", line 143, in run_buildstats(e=<bb.build.TaskStarted object at 0x7efe89284e10>):
         if isinstance(e, bb.build.TaskStarted):
    >        trigger = notexist
             pn = d.getVar("PN", True)
NameError: global name 'notexist' is not defined

ERROR: Build of do_package failed
ERROR: Traceback (most recent call last):
  File "/media/build1/poky/bitbake/lib/bb/build.py", line 560, in exec_task
    return _exec_task(fn, task, d, quieterr)
  File "/media/build1/poky/bitbake/lib/bb/build.py", line 497, in _exec_task
    event.fire(TaskStarted(task, logfn, flags, localdata), localdata)
  File "/media/build1/poky/bitbake/lib/bb/event.py", line 170, in fire
    fire_class_handlers(event, d)
  File "/media/build1/poky/bitbake/lib/bb/event.py", line 109, in fire_class_handlers
    execute_handler(name, handler, event, d)
  File "/media/build1/poky/bitbake/lib/bb/event.py", line 81, in execute_handler
    ret = handler(event)
  File "/media/build1/poky/meta/classes/buildstats.bbclass", line 143, in run_buildstats
    trigger = notexist
NameError: global name 'notexist' is not defined
----------------

(Bitbake rev: 1ff860960919ff6f8097138bc68de85bcb5f88b0)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-18 12:18:19 +00:00
Richard Purdie b14ccb2367 bitbake: runqueue: Add support for <task>- syntax
It can be useful to run all tasks up to but not including a specific task. The
main reason this was never added was the lack of a good syntax. This patch
uses the syntax <taskname>- to denote this behaviour which is simple, not
invasive and fits what we need from good syntax IMO, hence we can add this.

(Bitbake rev: 99ccfd411ab3f7baa111f9f3d50fae68816a9a83)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-18 12:18:19 +00:00
Belen Barros Pena 03d715e541 bitbake: toaster: tables Set a default order for the software recipes table
Add default order by recipe name, so that the table content is sorted
when the page loads.

[YOCTO #8791]

(Bitbake rev: 36cc814b64bcf3825ed096ade0b8c590e497259f)

Signed-off-by: Belen Barros Pena <belen.barros.pena@intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-14 23:16:13 +00:00
Ed Bartosh 4ff0d60ea8 bitbake: toaster: rework checking of Django version
Checked django version considering information from
toaster-requirements.txt, e.g. if requirements file contains
line "Django>1.8,<1.9" toaster should be able to check that
requirement correctly.

(Bitbake rev: 49976eca4a6e37e7653814c569badcd3e0fb719a)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-14 23:13:08 +00:00
Ed Bartosh 4a78416943 bitbake: toaster: monkey patch Queryset
Fixed 'database is locked' issue by monkey patching django QuerySet
methods.

The actual patching places were found by bisecting Django codebase.

This commit should be removed after Django is fixed if it's fixed
at all.

(Bitbake rev: 175411bf05423b1892c7928c2b928843b39645f0)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-14 23:13:07 +00:00
Ed Bartosh c1c8eff3f9 bitbake: toaster: removed extra calls of migrate
One manage.py migrate should be enough to apply migrations for
all applications. Extra calls just slow toaster script down.

Removed calls of migrate for orm and bldcontrol apps.

(Bitbake rev: 9299d7fa88e51294c4fd3f1354874d8253d25aff)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-14 23:13:07 +00:00
Ed Bartosh 507aafbd7a bitbake: toaster: work around 'database is locked' error
When sqlite can not cope with a stream of 'insert' queries it throws
'database is locked' exception.

Wrapping model.save in transaction.atomic context and repeating the call
should solve this issue.

(Bitbake rev: eb305308ca8f6228c6f52dac1bd941f29c7e5eb6)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-14 23:13:07 +00:00
Ed Bartosh 322b4709d8 bitbake: toaster: fixed format strings
Printing id as a number breaks toaster if object is not saved
yet as id/pk is None in this case.

Changed format spec to print id as a string in __str__
and __unicode__ methods of Build, Task and LayerVersion models.

(Bitbake rev: d5ca2bae623db214764324da5c9e4dac6beef760)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-14 23:13:07 +00:00
Ed Bartosh 84daa40bc7 bitbake: toaster: use OneToOneField instead of ForeignKey
Used OneToOneField to reference BuildRequest in BRBitbake model.

Fixed django warning:
WARNINGS: Setting unique=True on a ForeignKey has the same effect
          as using a OneToOneField.

(Bitbake rev: aaa4319ebbb06facb77b4ba936cf3aa2068ff238)

Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-14 23:13:07 +00:00
Elliot Smith c464f34d5b bitbake: toaster: Amend regex for MySQL database URLs
The database URL detection doesn't admit a MySQL URL without a port.

As this is a common case (e.g. you would set the HOST to a mysql.sock
path if pointing at a local MySQL server, with no port setting),
amend the regex so it will correctly recognise paths, as well
as HOST:PORT URLs.

(Bitbake rev: 89386aab888f806d5aa4a8083c06566e48d9445b)

Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-14 23:13:07 +00:00
Elliot Smith f001a4ac3c bitbake: toaster: Remove compatible_layerversions() method
Remove compatible_layerversions() and replace calls to it with
calls to get_all_compatible_layer_versions().

The sorting done in compatible_layerversions() is not relevant,
and the code can be simplified by using get_all_compatible_layer_versions().

[YOCTO #8364]

(Bitbake rev: 7bca51277314f7c0b6ee0e0d470327dfd60d37fe)

Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-14 23:13:07 +00:00
Elliot Smith 0adffdf790 bitbake: toaster: Check Django version against toaster-requirements.txt
Instead of hard-coding the required Django version in the start
script, look it up from the toaster-requirements.txt file.

[YOCTO #8364]

(Bitbake rev: 2149ff805424692f9fde29618f2ae0eb99f710d2)

Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-14 23:13:07 +00:00
Elliot Smith 8d058cfb8f bitbake: toaster: Update deprecated manage.py command
syncdb is deprecated in favour of migrate in Django 1.7:
https://docs.djangoproject.com/en/1.8/releases/1.7/#schema-migrations

Update to the "migrate" command in Toaster's start script.

[YOCTO #8364]

(Bitbake rev: 7ff1630574180e1895b90ecef1ea0caf51304446)

Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-14 23:13:07 +00:00
Elliot Smith 717c636d71 bitbake: toaster: Prevent deprecation warnings for RedirectView
The API for RedirectView is due to change in Django 1.9, which
means that Toaster generates deprecation warnings.

Set the "permanent" flag when constructing RedirectView instances
to prevent this warning.

[YOCTO #8364]

(Bitbake rev: 4aa09488bfe65cb365356b320cd9865643bb4fe5)

Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-14 23:13:07 +00:00
Elliot Smith 0f602c1a22 bitbake: toaster: Update API used to make runbuilds methods run in transactions
runbuilds previously had its methods decorated with
transaction.commit_on_success, which is now deprecated. transaction.atomic
is an (almost) drop-in replacement for this, so use this instead.

[YOCTO #8364]

(Bitbake rev: c4804b84eaaef6a81027bae5cf8bfe63d949c293)

Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-14 23:13:07 +00:00
Ed Bartosh 93f573836c bitbake: toaster: rename get_query_set -> get_queryset
Renamed get_query_set to get_queryset as get_queryset causes
deprecation warning in Django 1.7 and is deprecated in Django 1.8

(Bitbake rev: ef8b399dccd413070e32ce05e013337ea01fa64d)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-14 23:13:06 +00:00
Elliot Smith 23c48065d2 bitbake: toaster: Start Django machinery for database access
buildinfohelper, with the new import paths for our Django models
and Django 1.8, was not getting an active connection to the database.

In buildinfohelper, call django.setup() explicitly to make sure
that the database connection is ready and models can be queried
and saved.

[YOCTO #8364]

(Bitbake rev: 671aaab8cb7c494cd5c7621b45a6f41a203d8bb5)

Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-14 23:13:06 +00:00
Elliot Smith 7a0c45e478 bitbake: toaster: Create default project with get_or_create* method
Rather than maintain data as part of the migrations (as was
done for the default project previously), create the default
(cli builds) project on demand as a by-product of getting
it from the database.

[YOCTO #8364]

(Bitbake rev: 5fd8e90ab9b81d1bd0d301bc1c91228ecbbea74b)

Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-14 23:13:06 +00:00
Ed Bartosh 9de8dfa11a bitbake: toaster: Fix references to app paths
The buildinfohelper runs outside of Django, but still needs
access to our Django app classes.

Previously, the imports referenced the toaster.* app, which worked
fine. But in Django 1.8, this causes an error about the same
module being loaded multiple times from different paths.

Change the paths to our Django modules so they don't cause
this error to be thrown. We can do this as we've added our
application libraries to sys.path in the buildinfohelper anyway.

[YOCTO #8364]

(Bitbake rev: 070da64cf32c32b5ffc34d611b463c3a3960b419)

Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-14 23:13:06 +00:00
Ed Bartosh 535fc9b6fb bitbake: toaster: Remove South migrations
South has been deprecated in favour of Django's own migration framework,
so remove the old South migrations and replace them with Django ones.

[YOCTO #8364]

(Bitbake rev: 427d8bc02d1aa00a19057602d592d58334514804)

Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-14 23:13:06 +00:00
Elliot Smith 8ca46641ef bitbake: toaster: Upgrade to Django 1.8.6 and remove South
Upgrade Django to long-term support version.

Django now provides its own migration framework, so remove
requirement for South.

[YOCTO #8364]

(Bitbake rev: 648b62654c52116451c6a68a46d7264db3a34d09)

Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-14 23:13:06 +00:00
Ed Bartosh b322dec016 bitbake: toasterui: process SetBRBE event
Currently BRBE is known to toasterui only when build is
started. It's passed to it with BuildStarted event. This is
too late as if build fails earilier than build starts toasterui
can not inform Toaster about the failure.

Set BRBE as soon as it's provided by Toaster.
This should make toasterui to be able to inform Toaster
about early build failures, e.g. failures during recipe parsing.

(Bitbake rev: d7819508dac488a64be3caec88db285cda9599ab)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-14 23:13:06 +00:00
Ed Bartosh 0274b68450 bitbake: toaster: trigger SetBRBE event
Triggered MetadataEvent "SetBRBE" when TOASTER_BRBE variable
is set on bitbake server. This should make buildinfohelper
aware of the build request id, which is used to properly
report build status and failures back to Toaster.

(Bitbake rev: 8b136fb093020bc912a7b21d5163e1cd5fb12124)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-14 23:13:06 +00:00
Ed Bartosh fdb8e7433c bitbake: toaster: implement BitbakeController.triggerEvent
This API will be used to pass data from Toaster to
toaster_ui and buildinfohelper.

(Bitbake rev: 6dca6fd47c5e8d66687fb07177e98cd531cd971d)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-14 23:13:06 +00:00
Richard Purdie 5de3800c30 bitbake: event: Fix subprocess event error traceback failures
If subprocess raises a CalledProcessError() error, e.g. from a call
like subprocess.check_call("false"), bitbake would try and pass the
object over IPC and fail, leading to an unusual error:

('__init__() takes at least 3 arguments (1 given)', <class 'subprocess.CalledProcessError'>, ())%

To avoid this, we turn the value into a string which prevents the
issues the IPC has trying to deal with the object (for the same reason
we deal with tracebacks here too).

[YOCTO #8752]

(Bitbake rev: 05695424b918fc81b16cbac70d79d8271a0b6045)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-14 23:13:06 +00:00
brian avery 7e8ff7b9d7 bitbake: toaster: toasterui Add ParseStarted/ParseProgress events to mask
Toaster is not able to see ParseStarted and ParseProgress events
for command-line builds. This means it's not possible for Toaster
to detect failed builds, if the failure occurs at a point before
the BuildStarted event, as the build won't show up at all.

Add these events to the event mask, so that Toaster's toasterui
can detect and respond to them.

(Bitbake rev: 16bfd3e3d145705a2b3a05648ddbcacc7a338dfa)

Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-09 09:03:35 +00:00
Richard Purdie f9384b0669 bitbake: knotty: Enforce terminal line limit to stop crazy scrolling
If there are more tasks running than there are lines on the terminal, the data
scrolls in ways the UI wasn't designed for. This patch adjusts the UI just to
show the processes which fit onto the number of rows in the terminal window.
You can see the total number running from the counter in the top left as usual
and this makes warning and errors messages scrolling from the top of the window
work as designed.

Ultimately, scrolling would be nice but is for another time, this fixes the
biggest UI issue on highly parallel machines.

(Bitbake rev: 67b77658e2bfa849f6f55c9c262cb11d6bfdb399)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-08 10:31:48 +00:00
Alexander Kanavin 3745479596 bitbake: bitbake: rename REGEX, REGEX_URI, and GITTAGREGEX.
Rename REGEX to UPSTREAM_CHECK_REGEX, REGEX_URI to UPSTREAM_CHECK_URI, and
GITTAGREGEX to UPSTREAM_CHECK_GITTAGREGEX to better reflect their purpose
and to reflect a common namespace.

(Bitbake rev: f0a9e783f9969573fd74edfa241ef14f18ac684e)

Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-08 10:20:31 +00:00
Ed Bartosh dd282d475c bitbake: toaster: return back 'New project' button
The button was removed by mistake in merging modes patchset.

(Bitbake rev: f7b33a95b5d53973bc673373bdd2652f4d32f0c2)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-07 17:39:08 +00:00
Mihail Stanciu 2a8e970d42 bitbake: toaster: tests Update UI tests to work with 2.0 changes
Update tests that were referencing items removed in 2.0.

Update most xpath elements left to use IDs.

(Bitbake rev: c4dda67dd3773d02b760d96dd9c6f26bff93533d)

Signed-off-by: Mihail Stanciu <stanciux.mihail@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-07 17:39:08 +00:00
Mihail Stanciu fe8a0a3697 bitbake: toaster: tests Automated build-mode backend tests
Add tests to check various backend scenarios that
require UI interaction.

Add necessary files for above tests (the sqlite file
and the default config json).

(Bitbake rev: 8dcad75ad1d9b8adddeca57e996b8d904b209df5)

Signed-off-by: Mihail Stanciu <stanciux.mihail@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-07 17:39:08 +00:00
Ed Bartosh 0497b570e5 bitbake: toaster: unset environment variables
Toaster script sets environment variables when it starts.
It makes sense to unset them when toaster stops as they can
cause other programs to behave incorrectly.

(Bitbake rev: 7a2e9745ff3ef6d6360f74b222248d4bd35e6a08)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-07 17:37:11 +00:00
Ed Bartosh 8b7a548e30 bitbake: toaster: get rid of complicated heuristics
Removed buildinfohelper code which was trying to guess layer version of
the recipe using build request information. The code caused creation of
duplicated recipes as it resulted in layer version from layer index
instead of returning build layer version. As a result of this Toaster
UI was not showing any information about recipes.

Default approach used to find layer version seems to work much better as
it finds proper layer version. Now toaster will use it as the only
way to find layer version.

(Bitbake rev: 101690bda7ad55dc0657483233c90c374713755b)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-07 17:37:11 +00:00
Ed Bartosh 556b8b6156 bitbake: toaster: remove SDKMACHINE from project variables
Removed SDKMACHINE from predefined set of variables
for the project as it causes bitbake build error:
   SDKMACHINE is set, but SDK_ARCH has not been changed as a result

This variable does not need to be predefined as it's not used by
toaster.

It's still possible to specify it in project configuration
if needed. SDK_ARCH variable should be set too to avoid above
mentioned build failure.

(Bitbake rev: a6ab2a38ab666376b4a5e796d3a88929e8cbfb22)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-07 17:37:11 +00:00
Ed Bartosh 4186f5b04e bitbake: toaster: stop using toaster-pre.conf
This file is not needed anymore as toaster UI doesn't
write variables to it. It sets them directly on bitbake
server.

(Bitbake rev: 624240460ec32af918dcea8c04a913e3b94e012b)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-07 17:37:10 +00:00
Ed Bartosh 361faa3e6c bitbake: toaster: remove writeConfFile API
This API is not used anymore as toaster doesn't write variables
to configuration files anymore. It sets variables through its
connection to bitbake server.

(Bitbake rev: 6dda2916b75d688874d208192d5c7cdb302eec35)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-07 17:37:10 +00:00
Ed Bartosh fcbba5a89b bitbake: toaster: set varibales on bitbake server
Set project variables using connection to bitbake server
instead of writing them to toaster configuration files.

This is a way to avoid rewriting configuration files before
and after every build, triggered by toaster UI.

(Bitbake rev: 3f77c69b867735b5d27c6ab7c62e39ac8d5cb2b5)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-07 17:37:10 +00:00
Ed Bartosh 993bc7eacd bitbake: toaster: implement BitbakeController.getVariable
Added new API to get value of bitbake variable from bitbake server.
The API will be used to update INHERIT variable instead of writing
it to the toaster configuration files.

(Bitbake rev: effc371e3a4a2262e53e9e560c7c08cffad195da)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-07 17:37:10 +00:00
Michael Wood 53e981e583 bitbake: toaster: buildinfohelper Broaden the toaster created recipe data case
When build information is the 'original' source of the information we
need to return the recipe that was created rather than the copy of the
recipe that is taken for keeping build history. We do this already for
command line triggered builds, but we also have this case for custom
images. We can simply check if the built_recipe exists instead of
special casing this.

(Bitbake rev: 9a8653bf602b2111dee7ee6a459682a68a695b22)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-07 17:37:10 +00:00
Ed Bartosh 57e5f24e57 bitbake: toaster: do not create duplicate HelpText objects
buildinfohelper code expects only one HelpText object per
build/variable/description.
Current code creates more than one such an object, which causes
toastergui to crash with this exception:
 MultipleObjectsReturned: get() returned more than one HelpText -- it returned 2!

Used git_or_create API to ensure that only one HelpText object is
created.

(Bitbake rev: e9b46803eb6f1f4044919abf90c8aeb3536e73ed)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-07 17:37:10 +00:00
Ed Bartosh 4c1e5ec29a bitbake: toaster: remove usage of BUILD_MODE variable
As both modes can be used at the same time we can't have any
difference in UI between modes.

Removed all conditional statements that used BUILD_MODE.

(Bitbake rev: 4ce1559c2558bd0fd278ff02a1a93bec03c4156b)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-07 17:37:10 +00:00
Ed Bartosh 990289511b bitbake: toaster: do not terminate bb server
Toaster needs bb server to be running all the time due
to merged analysis and managed modes. Server gets restarted
before every build triggered by UI, but it shouldn't be
terminated as it will influence command line builds.

[YOCTO #8279]

(Bitbake rev: e69c87c7842e796ffcd7193ecde22c8f688498f5)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-07 17:37:09 +00:00
Ed Bartosh 58765a8719 bitbake: toaster: remove stopBBServer API
Removed stopBBServer API from build controller as toaster
doesn't stop bitbake server anymore. It's reused for both
types of builds: triggered by UI and started manually.

(Bitbake rev: 0cad802da171d51814f22cc9383f496a63cd0c75)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-07 17:37:09 +00:00
Ed Bartosh 95a3cf7549 bitbake: toaster: reimplemented startBBServer method
Rewritten LocalhostBEController.startBBServer to use
'toaster restart-bitbake' and read bitbake port number from
bitbake.lock.

Removed complicated logic of running oe-init-memres and looking for
bitbake port number in the logs.

(Bitbake rev: 4b065353c3454923a1ef88e9f0a8702e5626060e)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-07 17:37:09 +00:00
Ed Bartosh 76d53b5856 bitbake: toaster: remove _setupBE function
This function is not needed as build environment is always created
because of the new way to run Toaster. It can be only sourced after
oe-init-build-env is sourced.

(Bitbake rev: 27a59b4373a449f17e46ab93e0eba15cf75967ae)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-07 17:37:09 +00:00
Ed Bartosh 87b2f954b9 bitbake: toaster: implement 'toaster restart-bitbake'
This implementation allows to have functionality of restarting
bitbake in toaster script. It can be used by toaster script and
build controllers.

[YOCTO #8279]

(Bitbake rev: 24cd26b39014419d883b7cf45e4943c301d585cc)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-07 17:37:09 +00:00
Ed Bartosh 891484ada9 bitbake: toaster: implement start_bitbake function
Separated functionality of starting bitbake server and observer
processes.

This functionality will be used by build controllers to restart
bitbake processes.

(Bitbake rev: d8e9ce7ba3f5b89c51e769bd4a46c2026a110d40)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-07 17:37:09 +00:00
Ed Bartosh bf25471341 bitbake: toaster: implement stop_bitbake function
Separated functionality of stopping bitbake server and observer
processes.

This functionality will be used by build controllers to restart
bitbake processes.

(Bitbake rev: dd40d70b707a3ff59edfa69923d4d9b671928a66)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-07 17:37:08 +00:00
Ed Bartosh 7c2b22556b bitbake: toaster: update brbe and project attributes
Updated attributes of buildinfohelper object as they can
be changed for every build. For example brbe is set by
runbuilds for every build triggered by Toaster UI.

(Bitbake rev: ea3bc8d01704dc64f6cb7b4f5fe66c312a575174)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-07 17:37:08 +00:00
Ed Bartosh de812d086e bitbake: toaster: start 'manage.py runbuilds' in the script
Running runbuilds machinery in background allows Toaster to
start builds initiated by toaster UI.

Added runbuilds pid file to the list in webserverKillAll to
kill runbuilds the same way as runserver process.

(Bitbake rev: 91bf323e873c31d883b01ea993c005f45382e83f)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-07 17:37:08 +00:00
Ed Bartosh 28e8ccf437 bitbake: toaster: make runbuilds to loop
Made runbuilds to loop to avoid having a loop in shell code and
initialize heavy Django init machinery every second.

Ignored exceptions to prevent exiting a loop.

(Bitbake rev: e04da15556ca0936de652b8c085e4199e5551457)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-07 17:37:08 +00:00
Ed Bartosh a3871a37de bitbake: toaster: use parent of the build dir
Assigned TOASTER_DIR to the parent of the build directory.
This should fix local controller crash as it assumes that
TOASTER_DIR is a root of local poky and tries to clone it.

(Bitbake rev: 124eb2bb15de18e4d2d6483c3549e1d887234736)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-07 17:37:08 +00:00
Ed Bartosh 2a96d35893 bitbake: toaster: check for toaster configuration later
Moved check for toasterconf.json after check of build environment.
We'll need some variables from build environment to find toasterconf.json
better way.

(Bitbake rev: a4f6809237dffb1abff78c6a8c82bcc4852aedfa)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-07 17:37:08 +00:00
Ed Bartosh d87a5346cf bitbake: toaster: remove unused variable
Removed variable RUNNING as it's not used anymore
in the toaster script.

(Bitbake rev: 187c31a8789c37723b9288afd35f1785f1993895)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-07 17:37:07 +00:00
brian avery dc6a489c65 bitbake: toaster: change toasterconf.json logic to use TEMPLATECONF, like oe-setup-builddir
This changes the way we pick a toasterconf.json file so that it
matches the way oe-setup-builddir does it. We now base it on TEMPLATECONF
found in the .templateconf file.

(Bitbake rev: 8476db82ea80edf2b70bc2082bcecee09037556d)

Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-07 17:37:07 +00:00
Ed Bartosh 5a42c2d050 bitbake: toaster: run bitbake the same way
Run bitbake in toaster script the same way as it's run
by localbuildcontroller.

(Bitbake rev: c5d5da095d5eace03f8a50910f5a60ab35acbb1a)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-07 17:37:07 +00:00
Ed Bartosh cac91db01d bitbake: toaster: set DATABASE_URL in toaster script
Set environment variable DATABASE_URL in bitbake/bit/toaster
Variable value is obtained by running 'manage.py get-dburl'

(Bitbake rev: ac4800cc8b765b270f27ea9a5938f3f2a2e0ced4)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-07 17:37:07 +00:00
Ed Bartosh a464bf23b4 bitbake: toaster: implement get-dburl command
Implemented management command to call getDATABASE_URL API.
It will be used to get database url from toaster shell script
by running 'manage.py get-dburl'

(Bitbake rev: fabe9c8f14fd7b8ab204a2b610c64ac5b62135ac)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-07 17:37:07 +00:00
Ed Bartosh e4731514f8 bitbake: toaster: don't allow to run toaster as a script
Removed support of starting toaster as a script.

Sourcing a toaster script becomes the only way to start it.
It's consistent with the way oe build system is started by sourcing
oe-init-build-env. It also returns user back to shell, so user can
continue running builds without having to open new terminal window.

[YOCTO #8279]

(Bitbake rev: 7e8632c0527245c1474e487dd9a972948e794471)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-07 17:37:07 +00:00