Commit Graph

882 Commits

Author SHA1 Message Date
Ed Bartosh fa68ae0455 bitbake: toaster: use meaningful logging levels
Changed logging levels to more appropriate ones.

(Bitbake rev: 27d0360d13af0c698bf3a224b3f0d415f17bb678)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-10-11 05:33:01 +01:00
Ed Bartosh bd8b27b6a9 bitbake: toaster: ignore ReachableStamps event
Added ReachableStamps event to the list of known events to
ignore. This should stop toaster throwing error message:
ERROR: Unknown event: <bb.event.ReachableStamps>

(Bitbake rev: cd4137e13af6964858640b78aa7fe6f1612be251)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-10-11 05:33:01 +01:00
Michael Wood fe29297c6a bitbake: Revert "bitbake: toaster: don't re-create Target objects"
This delete followed up the foreign keys and deleted things that were
not expected to be deleted.

This reverts commit 08000eb27e.

(Bitbake rev: 46b119eb62a5a612fe4c0847862d34f408e556f7)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-29 14:11:38 +01:00
Michael Wood e6d967dba2 bitbake: toaster: buildinfohelper Create a copy of the built layer and recipe
Create a copy of the built layer and the recipes associated with it.
This is so that the user can view the historical information about a
build. i.e. a snapshot of the layer version and artifacts produced at
that build.

(Bitbake rev: 0683d9a2b15e3234a94437aaebac84bfcca1420b)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-29 14:11:38 +01:00
Michael Wood 922503f4c1 bitbake: toaster: Create a relationship between build information and toaster layers
Previously this layer relationship was done by trying to match path
information that came back to the buildinfohelper with trying to query for
data in toaster's layers table. This rarely matched due to the lose coupling.

(Bitbake rev: 838e77c7c3c4006abd1327343a004590ab652de9)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-29 14:11:38 +01:00
Elliot Smith 15b482b16b bitbake: toaster: Add fake entry to Target_File for filesystem root
The files-in-image.txt file is produced by bitbake after an
image is created, listing all the files in the image.
However, this list doesn't include the root directory ('/').

buildinfohelper.py then tries to construct the filesystem
tree from this file, assuming that every directory apart from
the root directory (which is special-cased) can be assigned
a parent. But because the root directory isn't listed in
files-in-image.txt, an object for the root directory is never
created.

The direct subdirectories of the root ('./bin', './usr' etc.)
then can't be assigned a parent directory, as the object
representing the root directory doesn't exist. This
results in a Target_File lookup error and causes the
directory listing page to fail.

Fix this by creating a fake entry for the root directory
in the Target_File table, so that the direct subdirectories
of / can be assigned a parent. Note that it doesn't matter
that the root is faked, as its properties are never shown
in the directory structure tree.

[YOCTO #8280]

(Bitbake rev: a4015768183e5a3fa39a6c2b4dea0088ca182d80)

Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-23 22:44:55 +01:00
Ed Bartosh 08000eb27e bitbake: toaster: don't re-create Target objects
Due to re-creating Target objects from bitbake events task information
stored in original objects is lost.

There is no valid reason to remove existing objects. It's safer to query
them instead of re-creating as original object contain more information
than events coming from bitbake.

(Bitbake rev: aab4aff75eefb31aa53885d7735feee5daa294aa)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-23 22:44:53 +01:00
Elliot Smith 8c63d6049d bitbake: toaster: Read correct cooker log path from toasterui
The BB_CONSOLELOG variable changes by the time we read it in
BuildInfoHelper. This means that the log file location we
are using is incorrect, so the links to the cooker logs don't
work.

Instead, read it at the point when the BuildStarted event occurs
in toasterui. The BB_CONSOLELOG variable has the correct value
here, so pass that to BuildInfoHelper.

[YOCTO #8209]

(Bitbake rev: 20609eebee0d2318806cf81913e7ce6dc1005507)

Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-18 09:05:33 +01:00
Elliot Smith 214ba674d0 bitbake: toaster: Improve how default project is identified and fetched
Command line builds are associated with a "default project"
(as we currently require a build to have a project). This
acts as a container for builds initiated outside Toaster.
Currently, this project is marked as the default by its ID
being 0. However, this doesn't work with MySQL, as MySQL
won't allow 0 in a foreign key which references an
autoincrement field.

Instead, use an is_default field to track the default Project
for builds initiated outside Toaster.

Add a method to fetch this default project, rather than fetching
a project with a magic ID.

Add this default project in a migration, rather than as a side
effect of a get_or_create() style method.

Also ensure that builds always have a project explicitly assigned
to avoid any magic with a build's project foreign key defaulting to
0 (as it no longer does).

[YOCTO #7932]

(Bitbake rev: 71b709a1bbc26d89d61873763b467d21e625b274)

Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-03 16:52:48 +01:00
Alexandru DAMIAN 25d4a35b03 bitbake: toaster logger: fix pylint issues
This patch fixes pylint issues in the toaster build data logger,
toasterui. The following types of warnings are touched here:

* fixing imports
* unused variables are set to _
* logger calls now use lazy evaluation instead of formatting
the string
* correct whitespace identation
* removes unneeded "pass" statements, and extra parantheses
* disable specific pylint warnings when decideing to override
them

(Bitbake rev: 947d47f15048baa967f88e03d80014e88ce152aa)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-19 18:05:55 +01:00
Alexandru Damian c9455a7484 bitbake: toaster: fix pylint errors
Prompted by issues discovered during running pylint on
the toaster sources, this patch fixes:

* missing import in toaster ui
* improper call of function in toaster ui (logger.debug)
* improper function definitions in bbcontroller
* invalid references to objects in bldcontrol.models
* proper initialization of object fields in ToasterTables

Also inhibiting specific pylint errors in files where
the problems are mis-identified.

(Bitbake rev: 1c71955c416fb68455f7f70669aba4202c411807)

Signed-off-by: Alexandru Damian <alexandru.damian@intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-19 18:05:55 +01:00
Paul Eggleton 715d857174 bitbake: Fix default function parameter assignment to a list
With python you should not assign a list as the default value of a
function parameter - because a list is mutable, the result will be that
the first time a value is passed it will actually modify the default.
Reference:

http://docs.python-guide.org/en/latest/writing/gotchas/#mutable-default-arguments

(Bitbake rev: 7859f7388f2e3f675d0e1527cfde18625f36f637)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-19 18:05:44 +01:00
Ed Bartosh a403ddbbba bitbake: bitbake: toaster: Wait for toaster gui to come
Set logging level for the Toaster logger to get messages
to toaster_ui.log

Reverted previous workaround.

[YOCTO: #7965]

(Bitbake rev: f1d8e0ff7233f17cff3a56db10d2443b392b407d)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-01 11:29:35 +01:00
Richard Purdie 4c2f28bf48 bitbake: cooker: properly fix bitbake.lock handling
If the PR server or indeed any other child process takes some time to
exit (which it sometimes does when saving its database), it can end up
holding bitbake.lock after the UI exits, which led to errors if you ran
bitbake commands successively - we saw this when running the PR server
oe-selftest tests in OE-Core. The recent attempt to fix this wasn't
quite right and ended up breaking memory resident bitbake. This time we
close the lock file when cooker shuts down (inside the UI process)
instead of unlocking it, and this is done in the cooker code rather than
the actual UI code so it doesn't matter which UI is in use. Additionally
we report that we're waiting for the lock to be released, using lsof or
fuser if available to list the processes with the lock open.

The 'magic' in the locking is due to all spawned subprocesses of bitbake
holding an open file descriptor to the bitbake.lock. It is automatically
unlocked when all those fds close the file (as all the processes terminate).
We close the UI copy of the lock explicitly, then close the server process
copy, any remaining open copy is therefore some proess exiting.

(The reproducer for the problem is to set PRSERV_HOST = "localhost:0"
and add a call to time.sleep(20) after self.server_close() in
lib/prserv/serv.py, then run "bitbake -p; bitbake -p" ).

Cleanup work done by Paul Eggleton <paul.eggleton@linux.intel.com>.

This reverts bitbake commit 69ecd15aece54753154950c55d7af42f85ad8606 and
e97a9f1528d77503b5c93e48e3de9933fbb9f3cd.

(Bitbake rev: a29780bd43f74b7326fe788dbd65177b86806fcf)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-01 07:34:09 +01:00
Leonardo Sandoval 3b5b74b430 bitbake: knotty.py: Make sure bitbake.lock is unlocked before exiting
Before exiting the UI, unlocks the bitbake.lock owned by cooker; this
way consecutive bitbake executions can lock it again without trouble.

[Yocto #7941]

(Bitbake rev: 69ecd15aece54753154950c55d7af42f85ad8606)

Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-09 18:00:21 +01:00
Michael Wood c86a0d290f bitbake: toaster: Fix build execution regression
Make sure the error value is initialised and update the string match
that we have to identify bitbake's current startup status.

Patch contribution from Eduard Bartosh

(Bitbake rev: 915ba08a8a3013e9787e564f2ffd8698c948f433)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-02 22:56:39 +01:00
Alexandru DAMIAN 583e320764 bitbake: toasterui: verify variable before usage
This patch verifies that BRBE is set before trying to use
it to read the checkout paths. This is needed for builds
ran outside Toaster control.

(Bitbake rev: e04807cd3135c9de96cc7f79245f329c24618b85)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-26 09:27:32 +01:00
Alexandru DAMIAN 70c4eb8d3a bitbake: toaster: refactor build model
We remove the "timespent", "errors_no" and "warnings_no" fields
in favor of computing the needed values at runtime. This prevents
inconsistencies in the UI.

Also removeing all references to BuildRequests from the interface -
all build details now display in the build dashboard.

Minor fixes related to data logging.

(Bitbake rev: 44f37394ed3e4ca02f940be172fe4395b0ee0f7d)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-26 09:27:32 +01:00
Alexandru DAMIAN 335c975650 bitbake: toaster: fixes after replacing BuildRequest with Build
This is a set of fixes that repair the interface after
we switched from displaying BuildRequest data to Build data
in the formerly "managed" mode.

(Bitbake rev: 57f790b0c56297af8c83d5def8461bd5d61fe4af)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-26 09:27:32 +01:00
Alexandru DAMIAN d7c8d9558c bitbake: toaster: fill in build data from buildrequest
This patch adds logic to complete changing the interface
from showing BuildRequests to showing Build data.

The BuildRequest data is now transformed in Build data with
proper Toaster exceptions being recorded instead of listing
problems during startup as build errors.

(Bitbake rev: 51a41172d0b390370f9a38696b1ac65666ada4d2)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-26 09:27:32 +01:00
Richard Purdie 69b6919341 bitbake: bitbake: Add explict getVar param for (non) expansion
Rather than just use d.getVar(X), use the more explict d.getVar(X, False)
since at some point in the future, having the default of expansion would
be nice. This is the first step towards that.

This patch was mostly made using the command:

sed -e 's:\(getVar([^,()]*\)\s*):\1, False):g' -i `grep -ril getVar *`

(Bitbake rev: 659ef95c9b8aced3c4ded81c48bcc0fbde4d429f)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-23 11:57:53 +01:00
Alexandru DAMIAN 56c5ff920c bitbake: toaster logger: assign projects to all builds
Bitbake logger now sets a default project if the TOASTER_BRBE
or TOASTER_PROJECT Bitbake variables are not set.

This a necessary step in getting all builds under a project,
as to unify the MANAGED and interactive modes.

Other small fixes are included, related to the size of the
fields in the database.

(Bitbake rev: 5e0bf388f4e5c1cc493ac8264785e631bad2f672)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-29 11:59:44 +01:00
Alexandru DAMIAN e3e85bdf71 bitbake: toaster logger: refactor recipe and layer file paths
This refactoring brings the "local_path" of the
layer from the Layer object to the Layer_Version object, which
is more appropriate as different checkouts of the same
Layer may live in different directories.

This enables us to store Recipe file paths relative to a Layer_Version
at all times, aleviating the need to store full file paths in the
database. We also turn the prefix of the path (e.g. virtual:native
path name space) into a pathflag field.

In turn, this solves the problem of mis-identification of tasks based
on the recipe file paths, since we can also match the namespace of the
file paths on the recipe files.

[YOCTO #7594]

(Bitbake rev: ec43dc569e370767c709dec225cbee0c99151c19)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-29 11:59:43 +01:00
Alexandru DAMIAN 4b59ba1bbb bitbake: toasterui: close build on BuildCompleted event
Moving the code to close the build and instantiate a fresh
buildinfohelper object to BuildCompleted event, as the
CommandCompleted/Failed/Exit events come in too early.

(Bitbake rev: af63abe88327fd5c1b3d7c00a84d9e408ef23285)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-08 17:42:05 +01:00
Alexandru DAMIAN 7759cd4931 bitbake: toasterui: proper exit code on toaster errors
This patch modifies the toasterui to properly return the exit
code based on the errors found in the toaster itself.

The upload event file API call will not delete event logs for which
toasterui showed an error. This will facilitate debugging.

Minor enhancement in the buildinfohelper to reduce the number
of lookups on unknown layer objects (prevented testing of the patch).

(Bitbake rev: 1ddd6a9e4280a4adf971132ff1fe7ec9b3252905)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-08 17:42:05 +01:00
Alexandru DAMIAN 87c4e58dda bitbake: toasterui: enable sanity checker
Since toasterui acts as the user-facing UI, we need
to run sanity checks in order to let the GUI display proper
warnings and stop the build if something is wrong.

(Bitbake rev: 260dd77fa771ae3b777134f4178d344e96b6f3d6)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-03-25 12:39:58 +00:00
Alexandru DAMIAN 1385f28770 bitbake: uievent: catch and log exceptions in receiving events
This patch prevents tracebacks and instead logs exceptions
that may happen during event processing.

[YOCTO #7216]

(Bitbake rev: 0412631fb4a15ff42bf5ee46a77920fa558ae358)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-03-25 12:39:50 +00:00
Alexandru DAMIAN 5252c459ac bitbake: toastergui: recipe and layer identification
This patch fixes the recipe and layer identification by path
when Toaster uses relative paths.

(Bitbake rev: a92bb33a3ceacab2bfee9df1c39a202832866970)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-03-21 00:00:26 +00:00
Alexandru DAMIAN 9752757da0 bitbake: toasterui: fix error message parameter
Fix the error message parameter.

(Bitbake rev: 64b0867108d03c7e9215b80c59c1bba919e82994)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-03-21 00:00:25 +00:00
Alexandru DAMIAN a273d8f151 bitbake: toasterui: improve info in the toaster_ui.log
We improve logging and signalling of errors in the
toaster_ui.log to facilitate debugging on remote systems.

(Bitbake rev: 3cd248f99b90367bd41aab81e255fc1912434890)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-03-21 00:00:23 +00:00
Robert Yang 0308f3101b bitbake: knotty.py: fix indent
It used 5 spaces as the indent.

(Bitbake rev: 162d35ed53d34b28b153adf643044e7f105fcff1)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-03-19 14:20:14 +00:00
Alexandru DAMIAN 887c1cb446 bitbake: toasterui: relative recipe paths
We modify the toasterui to log relative recipe paths
in order to maintain consistency with data fetched from
the layer sources.

(Bitbake rev: 253d69e88fd68729196ad43c15e8733527d76198)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-03-16 17:44:10 +00:00
Alexandru DAMIAN cca517056b bitbake: toasterui: identify proper layer in build mode
In build mode, instead of creating our own layer objects,
we identify the layer objects that the build system set up.

[YOCTO #7378]

(Bitbake rev: 22962b540ace6868cb357c0fd13f01ffd24449c4)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-03-16 17:44:07 +00:00
Rob Woolley 855fc21ae9 bitbake: knotty: Do not log show_versions output
Every time the bitbake show versions command (bitbake -s) is run it creates
a 100k log file.

The consolelogfile is disabled for show environment and disabling show
versions would make the behaviour match.

(Bitbake rev: dee0ba94e39ea49c1e9261a5e8932356e3bb7c57)

Signed-off-by: Rob Woolley <rob.woolley@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-03-16 17:41:28 +00:00
Rob Woolley 0b5ab4d987 bitbake: knotty: Catch exceptions on broken pipes
Any exceptions that occur in calls to logging methods are automatically
suppressed, including exceptions due to broken pipes.

However, the knotty summary messages are printed directly to stdout, which
means that any broken pipes will cause an exception traceback in python.

By wrapping the summary section in a try / catch block we can check for
IOError exceptions caused by broken pipes and let them pass.

(Bitbake rev: 146e7e157f97b676858ecff583dd53800d997253)

Signed-off-by: Rob Woolley <rob.woolley@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-03-16 17:41:28 +00:00
Alexandru DAMIAN ee1af30548 bitbake: toasterui: do not use transactions if the database autocommits
Some databases, notably the SQLite3 adapter, force autocommits even if
the autocommit is turned off. The behavious is tracked in this bug:
http://bugs.python.org/issue8145#msg109965

Django refuses to work with autocommit off in this case, so we have
to take the same precautions when using manual transaction support.

[YOCTO #7363]
[YOCTO #7365]

(Bitbake rev: 90231ab63a129fa344d461c2911898ea0f07f206)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-02-27 07:36:07 +00:00
Alexandru DAMIAN f116c32f2a bitbake: toasterui: fix sstate task identification
This patch fixes a problem where set sstate scene tasks
were not identified, causing cache attempt not being recorded.

[YOCTO #7223]

(Bitbake rev: 8a326a9a5a08981f1b7960e02fdb8a9436db16fb)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-02-27 07:36:05 +00:00
Alexandru DAMIAN cb23e3043b bitbake: toasterui: disable autocommit for build logging
This patch disables autocommit for inserting build data,
effectively updating all build data in a single transaction.

This is a purely performance improvement patch, as the transaction
will always be commited.

Similar manual transaction handling in the layer source update
method. Added feedback messages during update method.

[YOCTO #7140]

(Bitbake rev: 3978c819e797f857235499a4b8ec238134f1c028)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-02-24 23:35:45 +00:00
Alexandru DAMIAN c368d83bd6 bitbake: toaster: bitbake cooker log saving and downloading
This patch brings in cooker log saving and proper download links.

* toasterui will now write the cooker log file if running in managed
mode
* the BuildRequest has a new state, REQ_ARCHIVE, indicating that the
build is completed, and the artifacts are ready to be grabbed
* the runbuild test execution commands will gather needed artifacts,
and save them to a storage directory selected during Toaster setup.
* the build dashboard, project builds and all builds pages have
permanent links for the cooker log

[YOCTO #7220]
[YOCTO #7206]

(Bitbake rev: fad80e36c9da663b000cdf2cb3c75440c6431d84)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-02-20 12:58:19 +00:00
Maxin B. John b427750a5d bitbake: depexp.py: Fix segfault if DISPLAY is not set
bitbake -g -u depexp <package> segfaults when DISPLAY is not set
properly. Fix it with a proper check.

[YOCTO #7299]

(Bitbake rev: f35e9bd7b59c180fe9a3d9177efb57b92d9cd373)

Signed-off-by: Maxin B. John <maxin.john@enea.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-02-16 09:09:43 +00:00
Alexandru DAMIAN da8110a86a bitbake: toaster: improve logging facilities for toaster
This patch improves the logging facilities for toaster in order
to help diagnose bugs that happen on user machines.

The logs are stored now under "/tmp/toaster_$$" where $$ is a
PID-based unique identifier. On shutdown, toaster will automatically
erase all logs unless errors are listed in the log file.

On error, Toaster provides suggestions on what to do.

This patch includes a minor fix found as a result of logging
improvements.

(Bitbake rev: 8a8248f7b7e30469f592e2f8adbf6ce21e8685c5)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-02-10 23:07:48 +00:00
Alexandru Damian fefef50e54 bitbake: toasterui: do not filter images by extension
ToasterUI filters build artifacts by extension in order
to determine if a build artifact is an image or not.

Using IMAGE_FSTYPES for this purpose is not correct as
the varible value holding image extensions is just a coincidence.

So we just look if the filename contains the "rootfs" magic
string, which is a pretty good approximation.

[YOCTO #7213]

(Bitbake rev: b11e8bd626e0212ee72914529c3d92d1dd718674)

Signed-off-by: Alexandru Damian <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-01-29 22:11:35 +00:00
Alexandru DAMIAN c7af070b66 bitbake: toasterui: save event backlog to build
We add a call that saves any queued events to the build

[YOCTO #7021]

(Bitbake rev: 4f5b19d453da64749affc1c27ec51b013bedc71a)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-01-23 11:33:52 +00:00
Richard Purdie db45ddeeaf bitbake: command/cooker/knotty: Fix memres handling of command environment changes
If the environment changes, we need memory resident bitbake to adapt to those
changes. This adds in functionality to handle this alongside the configuration
option handling code. This means that the common usage:

MACHINE=X bitbake Y

now works with the memory resident server.

(Bitbake rev: 4d1343010da757a0c126bc22475354da44aaf8e3)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-01-21 14:37:39 +00:00
Alexandru DAMIAN 4e20211090 bitbake: toasterui: fix variable data error
A bug slipped in the toaster ui that prevented saving of
build configuration despite the data being retrieved
from the server. This patch fixes the shaming mistake.

[YOCTO #7117]

(Bitbake rev: 8118f465b9f87c66b2a741008f69198ac5fea901)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-01-16 08:25:31 +00:00
Andy Voltz 6b33e1a840 bitbake: ui/ncurses: Update ncurses ui to accept configParams
(Bitbake rev: 730b4c50364fba7173309278d670eda02e0ce38e)

Signed-off-by: Andy Voltz <andy.voltz@timesys.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-12-23 10:18:21 +00:00
Alexandru DAMIAN 85a17f86ea bitbake: add option to write offline event log file
This patch adds a "-w/--write-log" option to bitbake
that writes an event log file for the current build.

The name of the file is passed as a parameter to the "-w"
argument. If the parameter is the empty string '', the file
name is generated in the form bitbake_eventlog_DATE.json,
where DATE is the current date and time, with second precision.

The "-w" option can also be supplied as the BBEVENTLOG
environment variable.

We add a script, toater-eventreplay, that reads an event
log file and loads the data into a Toaster database, creating
a build entry.

We modify the toasterui to fix minor issues with reading
events from an event log file.

Performance impact is undetectable under no-task executed builds.

(Bitbake rev: 1befb4a783bb7b7b387d4b5ee08830d9516f1ac2)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-12-18 10:24:06 +00:00
Alexandru DAMIAN d086fa3aed bitbake: toasterui: add extra debug and development infos
We update and add logs throughout the code in order to help
with development. The extra logging is turned off by default,
but it can be enabled by using environment variables.

All logging happens through the Python logging facilities.

The toaster UI will save a log of all incoming events if the
TOASTER_EVENTLOG variable is set.

If TOASTER_SQLDEBUG is set all DB queries will be logged.

If TOASTER_DEVEL is set and the django-fresh module is available,
the module is enabled to allow auto-reload of pages when the
source is changed.

(Bitbake rev: 10c27450601b4d24bbb273bd0e053498807d1060)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-12-18 10:24:06 +00:00
Alexandru DAMIAN f99f2cdd69 bitbake: add build artifacts table and other improvements
We add a BuildArtifacts class to store data about files
discovered during the build process and not stored anywhere
else.

Small cosmetic changes in the toasterui.

Add model methods to return file path display data relative
to the build environment instead of absolute file paths.

[YOCTO #6834]

(Bitbake rev: bbe24d912869312d561be199b2c029b0c898e049)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-12-18 10:24:06 +00:00
Alexandru DAMIAN af51fb0674 bitbake: toasterui: Compatibility patch for daisy and dizzy
This patch brings in changes that allow a toasterUI coming in
from 'master' branch to record data from a 'daisy' or 'dizzy'
bitbake server.

This is needed to allow Toaster to record builds running
on older branch releases.

(Bitbake rev: 8d75e28e0688a6520311afce36543175f36910b3)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-11-28 14:03:00 +00:00