Commit Graph

3970 Commits

Author SHA1 Message Date
Michael Wood 8ff56b0fb3 bitbake: toastergui: libtoaster Add show change notification function
Now that we have a change notification that is global move this
functionality to libtoaster for shared use.

(Bitbake rev: 9d760462de99cb8729b7e0db773e77d4cd508733)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-01 11:26:11 +01:00
Michael Wood 1b13222e1c bitbake: toastergui: Standardise the layer object property layerdetailurl
To be able to use the utility functions in libtoaster  we need to have a
single property name for the value that represents the url for the
layer details in the layer object.

(Bitbake rev: 4d64fd0d1236d342b29537f601a68cd23a8255ea)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-01 11:26:11 +01:00
Michael Wood df999d2ac1 bitbake: toastergui: Remove erroneous CSS property
Remove a CSS property value which is missing a corresponding key.

(Bitbake rev: 770008f8ff3f8aa4f90636621a4162d676a71da7)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-01 11:26:11 +01:00
Michael Wood dd8c78b07a bitbake: toastergui: importlayer Add format type to JSON calls
After API was changed we now need to pass the parameter "format=json" as
a get parameter when requesting a json payload. Otherwise we get the
html version of the page instead of the data we need.

(Bitbake rev: 4545c21b70998bd240c3a21de05e8d4642044119)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-01 11:26:11 +01:00
Michael Wood d826d4e800 bitbake: toastergui: base Fix html indentation
(Bitbake rev: 37364e671ad1897bef818239a8e289b3d4c93925)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-01 11:26:11 +01:00
Michael Wood b06a633f25 bitbake: toastergui: Implement new project navigation
Change the structure of the project page to include a navigation menu
and top tab navigation. Remove old breadcrumb method.

[YOCTO #7329]

(Bitbake rev: 66fa0dd988e01ec79e74be7a5697eaa3b4f017d8)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-01 11:26:11 +01:00
Michael Wood 3d3a2dbf5f bitbake: toastergui: Implement new top navigation design
Additional items for top level navigation. This is part of a wider
navigation redesign.

[YOCTO #7329]

(Bitbake rev: 456a4a6782d3098cc9abb39659a619fdcf52eb01)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-01 11:26:10 +01:00
Alexander Kanavin 8acab25832 bitbake: wget.py: fix incorrect regexes
[\.-_] means "any character between . and _"
What was meant here is certainly "any character from the three characters .-_"

(Bitbake rev: af13eaba627f199f91c048c435b9dbe19c79527f)

Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-01 07:34:09 +01:00
Richard Purdie 4c2f28bf48 bitbake: cooker: properly fix bitbake.lock handling
If the PR server or indeed any other child process takes some time to
exit (which it sometimes does when saving its database), it can end up
holding bitbake.lock after the UI exits, which led to errors if you ran
bitbake commands successively - we saw this when running the PR server
oe-selftest tests in OE-Core. The recent attempt to fix this wasn't
quite right and ended up breaking memory resident bitbake. This time we
close the lock file when cooker shuts down (inside the UI process)
instead of unlocking it, and this is done in the cooker code rather than
the actual UI code so it doesn't matter which UI is in use. Additionally
we report that we're waiting for the lock to be released, using lsof or
fuser if available to list the processes with the lock open.

The 'magic' in the locking is due to all spawned subprocesses of bitbake
holding an open file descriptor to the bitbake.lock. It is automatically
unlocked when all those fds close the file (as all the processes terminate).
We close the UI copy of the lock explicitly, then close the server process
copy, any remaining open copy is therefore some proess exiting.

(The reproducer for the problem is to set PRSERV_HOST = "localhost:0"
and add a call to time.sleep(20) after self.server_close() in
lib/prserv/serv.py, then run "bitbake -p; bitbake -p" ).

Cleanup work done by Paul Eggleton <paul.eggleton@linux.intel.com>.

This reverts bitbake commit 69ecd15aece54753154950c55d7af42f85ad8606 and
e97a9f1528d77503b5c93e48e3de9933fbb9f3cd.

(Bitbake rev: a29780bd43f74b7326fe788dbd65177b86806fcf)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-01 07:34:09 +01:00
Andre McCurdy 1b88b7426a bitbake: build: Exit scripts with result of last shell function
Since shell scripts run with 'set -e' the final exit at the end of the
script can only be returning 0. However, for correctness and to follow
the original intention of the 'cleanup' commands, let's fix the typo
and return the success of the last shell function rather than the
success of unhooking the exit trap handler.

(Bitbake rev: bef724057f1ea81571dd3ac5a9cf862f818434b5)

Signed-off-by: Andre McCurdy <armccurdy@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-27 23:29:16 +01:00
Richard Purdie f5f6748b75 bitbake: data_smart: Improve override history logging
Calling record() for each override alteration is slow. Since we now expand
overrides dynamically we don't have to record the log data at each alteration,
we can instead print it directly from the existing data stores at variable
history print time using the exact same data stores.

This massively improves performance of the data store when parsing
with bitbake -e for example, it will improve memory overhead as well.

The only downside is that VariableHistory has to poke into the datastore
for some of its data but that seems an acceptable tradeoff rather than
double caching.

(Bitbake rev: 100b447a161ef20fa559e39516cd32fa78e38262)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-24 23:28:35 +01:00
Richard Purdie 9901415ecd bitbake: data_smart: Improve performance of infer_caller_details()
As things stand now, bitbake -e (which turns on all the caller tracking)
of OE-Core generates around 9.5 million stat calls which is slow and the
largest single thing on the profile data.

This is because infer_caller_details() calls traceback.extract_stack()
which adds line contents to the traceback. This in turn calls python's
internal linecache code which calls stat on every file for every callback.
We don't even use that info. We only even want a single frame of the stack.

Instead, open code for the pieces of information we need. Also, only
obtain the stack once for both halves of the infer_caller_details()
code.

This reduces the number of stat calls to around 0.5 million and significantly
improves parsing with bitbake -e.

(Bitbake rev: 7be76d8f79ea92fd4bd36146eb9a4b86551b526d)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-24 23:28:35 +01:00
Richard Purdie 67c6cc854f bitbake: data: Clean up datastore accesses and True/False values
We should use the d.xxxVar syntax rather than the older function style.

Also replace 0/1 with the more pythonic True/False.

No functionality changes.

(Bitbake rev: 90fdd69cca951f8bd2ff634f3b42fccd4fc03095)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-24 23:28:34 +01:00
Andre McCurdy fdfb3fdc5b bitbake: data: Add newline after the last export in emit_func
Minor formatting improvement in generated shell scripts.

Add a newline after the last export emitted by emit_func. Also, remove
chance of putting an extra newline in the middle of the exports, since
we never want to do that.

(Bitbake rev: 5117d9b5d32d5d81adf70fa3e3feac9cef654240)

Signed-off-by: Andre McCurdy <armccurdy@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-24 23:28:34 +01:00
Richard Purdie ada930055d bitbake: tinfoil: Add shutdown method
One drawback to tinfoil is that once a cooker is created, it will hold
the cooker lock and stop any other bitbake execution against a directory.

Add a shutdown method to tinfoil, allowing other users to use
the build directory after the tinfoil usage is no longer needed.

(Bitbake rev: e44ce85fe551677fc0dcc1da4f789a0c13093ff1)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-24 23:28:34 +01:00
Richard Purdie 901be2cb69 bitbake: bitbake-selftest: Add -v option for verbosity
Also document BB_SKIP_NETTESTS=yes parameter in --help output.

(Bitbake rev: 5196bfa9639eed2b1e6452f45775551203f8eeb4)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-23 08:48:41 +01:00
Richard Purdie aaea533e70 bitbake: tests/parse: Add BBCLASSEXTEND multiple data store corruption reproducer
One data store changing a variable poked through into a different data
store. This test case replicates that issue where the value 'B' would
become unset/disappear.

We also enhance parsehelper to generate files with an optional suffix
such as bbclass.

(Bitbake rev: 5c4179f58a4e04f1c354df5f17d1860eb403f0ac)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-23 08:48:41 +01:00
Richard Purdie 19f290f223 bitbake: data_smart: Add CoW approach for overridedata
Using deepcopy() caused a major performance regression. Turns
out we can work with a shallow copy as long as we force the
recreation of any list that we change, effectively a poor mans
CoW. This isn't too invasive and avoids the huge overhead of
deepcopy so this seems like the best way to have performance and
a working data store.

(Bitbake rev: 32dff4749c32f32e947c42c86f8357b8b607354b)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-23 08:48:41 +01:00
Aníbal Limón 205c39bf78 bitbake: tests/fetch.py: Updated test name FetchMethodTest -> FetchLatestVersionTest
Change the test name to be more specific on what is tested.

(Bitbake rev: 3e39156bc330c4c726058a5b9c13d33e2daad89f)

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-22 08:23:50 +01:00
Aníbal Limón 22d2f7b1ca bitbake: fetch2/wget.py: latest_versionstring now returns (version, revision)
Now latest_versionstring method returns (version, revision) for comply
the new return convention needed by SCM's like git get the current
revision.

bb/tests/fetch.py: Updated wget latest_versionstring test for comply new
convention.

[YOCTO #7605]

(Bitbake rev: 8d454646cbe1b04758ca178d8c6fcfd02b818b7b)

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-22 08:23:50 +01:00
Aníbal Limón bd5167d154 bitbake: fetch2/git.py: latest_versionstring now returns (version, revision)
We need to know the revision associated with the upstream version in
SCM like git, this change broke return convention, oe-core recipeutils
get_recipe_upstream_version need to be updated.

tests/fetch.py: Updated git latest_versionstring test for comply new
convention.

[YOCTO #7605]

(Bitbake rev: fd175dc90024c503134c11cbd83e77d88c406ac8)

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-22 08:23:49 +01:00
Aníbal Limón 4bbc5dd6e7 bitbake: fetch2/git.py: latest_versionstring search in all tags
Don't limit the tag search for only tags end with ^{}.

(Bitbake rev: 7006ab313766344cf33481228465082ed5977d28)

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-22 08:23:49 +01:00
Richard Purdie f5206e85b3 bitbake: data_smart: Use deepcopy for overridedata
dbus sets RDEPENDS_dbus-ptest_class-target = "X". nativesdk as
BBCLASSEXTEND was expanding and setting various RDEPENDS variables
which were clearing overrides queued against RDEPENDS. The problem
was that this was leaking into other data contexts such as the
target case. This was because overridedata was a shallow copy.

Replacing the shallow copy with a deep copy avoids this problem of
leakage between the data stores at a small performance penalty.

(Bitbake rev: 83e2923cd3c8f2e25987132f85b06ce72bc941d7)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-21 23:37:22 +01:00
Belen Barros Pena 9a92500306 bitbake: toastergui: make our tables Chrome-happy
Make sure that Toaster tables adapt nicely to the
viewport width in Chrome without squashing the
buttons in the right column (when they are there).

(Bitbake rev: 61307693a25a4a2eeeab5c450df330229638ddad)

Signed-off-by: Belen Barros Pena <belen.barros.pena@intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-21 15:36:14 +01:00
Brian Avery 0c0ec86110 bitbake: toaster: continue processing layer dependencies despite errors
Sometimes, a layer dependency will not exist,
  or not exist for a particualr branch. In this case
  we need to continue processing and hope for the best.

[YOCTO #7955]

(Bitbake rev: 2961bf2612fce111a7be6cef285595a075200780)

Signed-off-by: Brian Avery <brian.avery@intel.com>
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-21 15:36:14 +01:00
Paul Eggleton d3809b77c4 bitbake: lib/bb: provide mechanism to bypass UI log suppression
The recent change to connect through the shell logging functions had an
unexpected side-effect - bb.error() and bb.fatal() cause a flag to be
set internally such that BitBake's UI will not print the full task log
on failure; unfortunately we have in places within the OpenEmbedded
metadata called these shell logging functions under error situations
where we still want to see the full log (i.e., the message we're sending
doesn't include the full error). Thus, provide a mechanism to fatally
exit with an error but unset the flag, utilising the built-in python
logging functionality that allows extra values to be passed in the log
record.

(Bitbake rev: e561b997c55e8537d82aa1339adfff4505cc38b7)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-16 15:09:26 +01:00
Paul Eggleton 5261c5b5bd bitbake: lib/bb/process: check output of select() before reading extrafiles
We're calling select() to find the fds that have data available, so we
really ought to check the return to see if the extra file is in there
before trying to read from it. This is part of the fix for the
performance regression that this code introduced (5-10 minutes extra in
a reasonable size OE build); the rest is down to an issue in the way
that pseudo currently handles FIFOs and will need to be addressed there,
see: https://bugzilla.yoctoproject.org/show_bug.cgi?id=7993

Solution suggested by Richard Purdie <richard.purdie@linuxfoundation.org>

(Bitbake rev: 3e2db8d7eaa0f14813213d1c95d3ee9efd97dc9d)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-16 15:09:26 +01:00
Richard Purdie a730981b75 bitbake: tests/data: Add new data tests
Add a variety of tests which were found to be useful when working
on the data store recently.

(Bitbake rev: 5c5f8da509f6bbc1fad263462142519ef3d54a35)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-12 22:50:46 +01:00
Richard Purdie 4c386e1dd5 bitbake: data_smart: Fix appendVar/prependVar
Now that overrides get expanded 'on the fly', change appendVar
and prependVar to work using _append and _prepend, else we'd have
to re-implement pieces of getVar and the timing of expansions
becomes problematic.

Using _append/_prepend equivalence gives the behaviour users likley
expect from these functions.

(Bitbake rev: 40d661aaf7a563c6447b073310c5f2fdae6ca3d0)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-12 22:50:46 +01:00
Richard Purdie 75a58555ab bitbake: data_smart: Fix data expansion cache issues
The expand cache can be cleared in the middle of getVar now
due to the use of operations like delVar. We therefore need
to check if variables are in the cache before accessing in
case it was cleared.

(Bitbake rev: a97ce216cfe44136f742383542954bfce027831e)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-12 22:50:46 +01:00
Richard Purdie a259968611 bitbake: data_smart: Fix keys() missing overrides
d.keys() was not returning items which had no base content but the
variable came into existance through overrides. We have to process
self.overridedata to find these other variables.

(Bitbake rev: 77fc57c610d7f65f7e328ad555f525b2f8bdf1d7)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-12 22:50:46 +01:00
Richard Purdie 816acb6ee8 bitbake: data_smart: Cache set(self.overrides)
This performs better than continually regeneratiing the set().

Also only use set comparisions when its necessary to save some overhead
as issubset() is slower in the single item case.

(Bitbake rev: db9a4eedcf78969ec50bf0e4a8defba8ff0daa4a)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-12 22:50:46 +01:00
Richard Purdie 0ac9c80e08 bitbake: data_smart: Improve override recursion handling
When expanding OVERRIDES, its possible someone might try and override a variable
that is used in OVERRIDES. This could lead to infinite recursion. Add in
guards against this.

(Bitbake rev: 07d773369f571028c2cf82dd1f65d9731af6d00e)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-12 22:50:46 +01:00
Richard Purdie 2dbb067eba bitbake: data_smart: Tweak OVERRIDES value cache for performance
Updating the value of OVERRIDES whenever it changes turns out to be
extremely expensve/pointless. Instead, clear its value and re-establish
the value when we're going to use it.

This gives significant speed back.

(Bitbake rev: 41cf8d0c92d2d8a33fdad0921e424a0024914be1)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-12 22:50:45 +01:00
Richard Purdie d8ebb4708b bitbake: data_smart: Seperate out the override cache
Using an internal flag for override mapping turns out to be slower
than is optimal, not least as we don't want the keys list to list
variables that have no value other than a potential override expansion.

Maintinaing a seperate cache is therefore more optimal from a speed
perspective.

(Bitbake rev: 1a39ccf66b30b54e1274669cb0e1dc8fc72dba49)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-12 22:50:45 +01:00
Richard Purdie c690d5b818 bitbake: data_smart: VariableHistory: Ignore override duplicates and overlap with CoW functions
Trying to look up a variable called 'copy' in COW is problematic due
to internal implmentation details, at least avoid tracebacks from this.

Also don't duplicate override history (which is an atrefact of changed
override behaviour) as otherwise the bitbake -e output is convoluted.

(Bitbake rev: dddff5b7b8e6c18515b43389cef35503468b843d)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-12 22:50:45 +01:00
Richard Purdie 933e19697b bitbake: parse/ast/data_smart: Add parsing flag to getVar/setVar
When parsing we find problems if we clear prepends/appends when
setting variables during the initial parsing phases. Later, we actively
want to do this (in what would be post finalisation previously).

To handle this, pass a parsing flag to the operations to control
the correct behaviour for the context.

(Bitbake rev: ae87f5b8bf16191b3201cfb445062938eab992a0)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-12 22:50:45 +01:00
Richard Purdie 95be16f428 bitbake: data_smart: Move override handling to getVar
Compeltely remove the replaces functionality and move all overrides
handling to getVar time. We can move the cookie cache into a hidden
flag within the variables themselves.

This removes the need for any of the internal_finalize steps.

This obsolete the need for the _seen_overrides COW cache.

(Bitbake rev: 2a0b73110bede91777ada54d1d89b45fb6034b6c)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-12 22:50:45 +01:00
Richard Purdie 7fc6f6c98d bitbake: data_smart: Cache overrides and fix data store
Rather than repeatedly expanding the value of OVERRIDES, cache
the value and update it when things change.

There were also some bugs introduced in the replaces functionality
which this approach fixes by ensuring the replaces data is updated
at the correct points.

(Bitbake rev: f3b7c3e054ce230ea5c2db5813390383e8dfd6db)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-12 22:50:45 +01:00
Richard Purdie 35a6f7d21f bitbake: data_smart: Fix cache clearance problems
These write operations should clear the expand cache since they can
influence returned variable values but currently do not. Fix this.

(Bitbake rev: a075332c9e13be35f1ae84adc6b29e9137a487ff)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-12 22:50:45 +01:00
Richard Purdie 577f160400 bitbake: data_smart: Improve override handling
Rather than the heavy lifting in internal_finalize, move the bulk of
the functionality to getVar and rely on a new internal replaces variable
to keep track of any mappings that are needed. This removes the need
for the COW _special_values cache.

This change in functionality also implies we need to track any changes
not only to OVERRIDES but also any variable which OVERIDES depends upon.
Add code to handle this.

Explicitly list FILE as a value OVERRIDES depends upon since it doesn't
automatically get detected and is of key importance to OVERRIDES,
otherwise PN doesn't update when FILE changes.

(Bitbake rev: a6f1377ce3386d274882072d1ae6da3b1834149b)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-12 22:50:45 +01:00
Richard Purdie e7ccd90712 bitbake: data_smart: Remove need for update_data calls
Move the update_data functionality into internal data store operations
so the main finalize (update_data) call is a nop.

To make this work we need to call the internal finalization function
whenever OVERRIDES is changed to ensure values get updated correctly.

This has performance issues but the subsequant patches look into this.

(Bitbake rev: 546d9932d6c4413824319a9d780c0d77d2725f4a)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-12 22:50:44 +01:00
Richard Purdie 42a59961fa bitbake: data_smart: Defer append/prepend handling
(Bitbake rev: b1ce9975ef96f2506042832f4518cde73f6be917)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-12 22:50:44 +01:00
Aníbal Limón 5ae52eec3a bitbake: fetch2/wget.py: checkstatus disable SSL cert validation.
Since Python 2.7.9 ssl cert validation is enabled by default
see PEP-0476, this causes verification errors on some https
servers so disable by default.

(Bitbake rev: e177170200ece76b36e3f7d5597651fdef67736f)

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-12 22:50:44 +01:00
Aníbal Limón 73b9cd9764 bitbake: tests/fetch.py: FetchCheckStatusTest add cases for ftp and https.
(Bitbake rev: 36f2577d075f87090766877473f9030e44a941a2)

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-12 22:50:43 +01:00
Aníbal Limón 804f1a2312 bitbake: fetch2/wget.py: checkstatus fix using proxy handler
Only use ProxyHandler opener when exists proxies in env.

(Bitbake rev: 0f062c5d99e12ce20c0e46a8e602448032144ff6)

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-12 22:50:43 +01:00
Aníbal Limón 913631c466 bitbake: tests/fetch.py: Add FetchCheckStatusTest tests.
Add tests for checkstatus method using http with/without
connection cache.

[YOCTO #7796]

(Bitbake rev: b1f2d3edefb7dd274174eb983666213b0f49c994)

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-12 22:50:42 +01:00
Aníbal Limón 3b186fba59 bitbake: fetch2/wget.py: Add support of connection cache in checkstatus.
fetch2/__init__.py: Add connection_cache param in Fetch __init__.

In order to pass connection cache object to checkstatus method.

[YOCTO #7796]

(Bitbake rev: 9fa6407e6cefe66c77467419a8040d6957a6bb01)

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-12 22:50:42 +01:00
Olof Johansson 97c5ecffaa bitbake: fetch2/sftp: Set BatchMode=yes as argument to sftp
Set BatchMode=yes instead of PasswordAuthentication=no. This will make
sftp fail immediately, not only when SSH requires interactive
authentication, but also on errors related to host key verifcation.

(Bitbake rev: 31305853a177735cc9c4553ea8905cd0acfcb100)

Signed-off-by: Olof Johansson <olof.johansson@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-12 22:50:41 +01:00
Robert Yang 518abb731c bitbake: git.py: create a branch when checkout
* Create a branch and named as upstream branch when checkout source
* Set the branch to track remote branch.

(Bitbake rev: 1ba20e4fe9c884515b200589fe379ad5eeda10bd)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-12 22:50:41 +01:00
Richard Purdie 61fbff2cd4 bitbake: cooker/taskdata: Make NoProvider errors non-fatal to -e/-g options
If you have a situation where you are getting a "Nothing PROVIDES" error
(for example when something you request to build DEPENDS on something
that has been skipped or doesn't exist) it would be useful to be able to
use bitbake -g or bitbake -e to debug it, but currently both of those
are blocked by the error.

This patch adds an "allowincomplete" option to taskdata and uses this
for the -e/-g bitbake options. The NoProvider errors are still
printed and bitbake does return an error exist code but the environment
and task graph files are generated.

[YOCTO #7623]

(Bitbake rev: 2ca36a9f088438a8d1db44119c704f9480b04298)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-12 22:50:41 +01:00
Richard Purdie 625a6bf6e8 bitbake: prserv/db: Document history modes
I keep having to dig into the archives to remember this information.
Add it as a comment to the file instead.

(Bitbake rev: 21dce82056887d8d28edde61b1b82f78bdf7613c)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-12 22:50:40 +01:00
Paul Eggleton 3b50312307 bitbake: lib/bb: set up infrastructure for shell message functions
Create a fifo under ${T} for each running task that can be used by the
task to send events back to BitBake. The purpose of this is to allow
OpenEmbedded's logging.bbclass (which provides shell function
equivalents for bb.warn(), bb.note() etc.) to have those functions
trigger the appropriate events within BitBake so that messages are shown
through the BitBake UI rather than just in the task log; to do that we
just call the python functions.

Part of the fix for [YOCTO #5275].

(Bitbake rev: bae330a1f8a59a912eca71177b3c6ba7c92a2f38)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-12 22:50:40 +01:00
Aníbal Limón f9ea480e6c bitbake: fetch2: Add fetch parameter to checkstatus
In order to pass connection cache object to checkstatus function
add fetch parameter.

(Bitbake rev: fbb9c6f5538084e125b58118a86968908e6f895b)

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-09 18:00:21 +01:00
Aníbal Limón bf6c21c38b bitbake: fetch2/__init__.py: Add FetchConnectionCache class
FetchConnectionCache class acts as a container for socket connections
useful when implement connection cache into fetcher modules.

(Bitbake rev: 454da2cd17539ceb9caad6d76f034757e44ee12f)

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-09 18:00:21 +01:00
Leonardo Sandoval 3b5b74b430 bitbake: knotty.py: Make sure bitbake.lock is unlocked before exiting
Before exiting the UI, unlocks the bitbake.lock owned by cooker; this
way consecutive bitbake executions can lock it again without trouble.

[Yocto #7941]

(Bitbake rev: 69ecd15aece54753154950c55d7af42f85ad8606)

Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-09 18:00:21 +01:00
Leonardo Sandoval 67eaefee29 bitbake: command.py: Unlock function included into CommandsSync class
Function which calls cooker's unlock method, which in turn unlocks bitbake.lock
file.

(Bitbake rev: e97a9f1528d77503b5c93e48e3de9933fbb9f3cd)

Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-09 18:00:21 +01:00
Leonardo Sandoval 7bccf59a24 bitbake: cooker.py: Lock/Unlock members function into BBCooker
(Bitbake rev: d66dccf9f9a33bfef5c28cc1c767bfc89faee532)

Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-09 18:00:21 +01:00
Richard Purdie 25702f3a2f bitbake: cooker: Improve DATE/TIME handling
Currently, once set, DATE and TIME in the metadata remain unchanged.
This is suboptimal for cases where cooker is reused. This ties setting
the variables into the BuildStarted event which seems like a more
appropriate time to do so.

It also changes BUILDNAME to be based off DATE/TIME by default if not
already set so that the data is more consistent. We therefore need to
expand the value rather than the previous default of not doing so.

This change does mean the date/time values are in sync across all
variables too.

It does mean bitbake now has special knowledge of DATE/TIME but that
would seen unavoidable (other than doing this in event handlers which
has its own set of downsides).

[YOCTO #5187]

(Bitbake rev: f883cf240266ee7be2cbd8971a8164cf4df9e372)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-08 00:06:08 +01:00
Ross Burton 18d71ed16e bitbake: main: add skeleton documentation for the recipe:do_task syntax
The incredibly useful recipe:do_task syntax on the command line isn't documented
at all.  This isn't much but it's better than nothing.

(Bitbake rev: 7f4c07886ecff4ac77fdd2165bedd179099fcf19)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-07 16:18:34 +01:00
Felipe F. Tonello 62caaa4d6d bitbake: fetch2: Checkout to correct ref begore init and update submodules
This is nessary when specified branch with submodules is different then
default (master) branch.

[YOCTO #7771]

(Bitbake rev: f7b0b5e33e00f3ce0744322eee93835ee76bf184)

Signed-off-by: Felipe F. Tonello <eu@felipetonello.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-07 13:47:32 +01:00
Michael Wood c86a0d290f bitbake: toaster: Fix build execution regression
Make sure the error value is initialised and update the string match
that we have to identify bitbake's current startup status.

Patch contribution from Eduard Bartosh

(Bitbake rev: 915ba08a8a3013e9787e564f2ffd8698c948f433)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-02 22:56:39 +01:00
Michael Wood 56066872ae bitbake: toaster: runbuilds Fix incorrect variable name
The recent refactoring moved this enum value to BuildRequests table
rather than Build.

(Bitbake rev: 4338ab03826df3353e1222f0bd825dfdd04dd933)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-02 22:56:39 +01:00
Michael Wood 483b21b0cb bitbake: toaster: Remove erroneously committed template include
Partial revert of bb commit 6e46e1e3882b9770872d8a0bb459bc7d5d6bfed3

The non managed mrb_section defined the title 'All builds' which is already
defined correctly for the projects page as 'All projects'

(Bitbake rev: 3500b59cbfd006a83bab9431f434f98c1944a0ac)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-02 22:56:39 +01:00
Michael Wood cfc4f1b343 bitbake: toaster: Revert the addition of analysis project mode selection
Partial revert of bb commit 2efc338cefd6e6e097af83d7dff63e9ba177d021

This feature was not ready to go live as the surrounding implementation
work to make this usable has not yet been done.

(Bitbake rev: 40c8829369628927944c8a950524d482a663913e)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-02 22:56:39 +01:00
Michael Wood 3b3ef9cee2 bitbake: toaster: Partial Revert "fixes after html5 compliance testing"
Partial revert of bb commit 7aecb974d44d9bc711ffba5cc65e770811397fba

There were changes in this commit which broke the compatible layers
mechanism meaning that no layers, recipes or machines were showing.

(Bitbake rev: 124157b567469639c6c839233f42049a093ba083)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-02 22:56:38 +01:00
Richard Purdie 2578ff3dfe bitbake: codeparser: Add repr() methods
These aid when debugging parts of the codeparser cache since the
object contents becomes identifiable.

(Bitbake rev: 344b098c7eafc2bcc5c6b44ea47985bc0cb446b5)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-27 22:51:21 +01:00
Richard Purdie dcbbe545bf bitbake: codeparser: Allow empty functions
The main parser and other code copes with empty python functions but
the python codeparser does not. Fix this to avoid errors with code like:

python do_build () {
}

which currently results in the obtuse:
"Failure expanding variable do_build: IndexError: string index out of range"

[YOCTO #7829]

(Bitbake rev: e4f594c670189e04d58ce7d160fc1d86123620af)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-27 22:51:20 +01:00
Robert Yang 43c6e8c0dd bitbake: bitbake: fetch2/hg.py: fix unpack error and mirror tarball
Fixed:
* do_unpack error:
  abort: repository DL_DIR/hg/vim.googlecode.com/hg/vim not found!

* The mirror tarball doesn't work
  - Add the build_mirror_data to create the tarball
  - Unpack the mirror tarball when needed

* The hg files will put in the dir like git: DL_DIR/hg, it was
  DL_DIR/hg/path/to/src/uri/path in the past.

(Bitbake rev: 378647281ae883dd726f1e1b775dd35ef6a0e8d1)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-27 22:51:20 +01:00
Richard Purdie 3836d9b4ab bitbake: fetch/perforce: Fix single file checkouts
Fetching directories with p4 worked but single files did not. This
patch from Helmut Auer (helmut.auer@harman.com) fixes that issue.

[YOCTO #7891]

(Bitbake rev: 39da6579901c62a83ed9319c2016c58fbbc108fe)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-27 22:51:20 +01:00
Richard Purdie 8c3dd2d9eb bitbake: event: Handle recursive events and the data store better
Events can call each other recursively, e.g. an event handler can call
bb.note which in turn generates another event. If these loop, it
can lead to multiple deletions of 'd' from __builtins__ which
can fail since __builtins__ is global scope.

Add handling to only remove 'd' when we added it and it wasn't already
present.

(Bitbake rev: b45952650ce8f470f124df36185b79e0d3a1783a)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-26 09:27:34 +01:00
Richard Purdie 00ca43be05 bitbake: cooker: Fire BuildCompleted before finishing the command
Some handlers hook on BuildComplete so it avoids certain event races
to finish the command after the BuildComplete event is sent out.

This means the UI is available to handle events until the command
completes.

What appears to be a race on one of the sanity tests for event handlers
triggered this change although the failure is hard to reproduce.

[YOCTO #7921]

(Bitbake rev: e42d7c47a06fbb5981e0313478c8e3656b99f4e7)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-26 09:27:33 +01:00
Michael Wood d93bc4cf48 bitbake: toaster: Add url pattern for backward compatibility
This adds an url to match the old orm application.

(Bitbake rev: 0a8e740e18333da981b24a76db4c891845e5df78)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-26 09:27:33 +01:00
Michael Wood df48243970 bitbake: toaster: split orm app into it's own module and app
The orm application was also the django application to collect the build
information. Splitting this module up into it's functional parts. orm
for the data module and bldcollector for build collection data.

(Bitbake rev: 8ca10764ffd6cfec12cbfeabf240d81213a07845)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-26 09:27:33 +01:00
Michael Wood 62288a6689 bitbake: toaster: bldcontrol Ignore toasterconf files in own directories
If toaster has previously cloned poky or a layer with a toasterconf
file we don't want this to be picked up as a choice as these clones are
"internal" to toaster and may have undesired effects on toaster's setup.

[YOCTO #7741]

(Bitbake rev: 2f74ff59f2873c1cf121f7a385f3ab971c793a9c)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-26 09:27:33 +01:00
Michael Wood 10e35ab742 bitbake: toaster: Restore 'in project' filters to main tables
Restores the previously removed filters from recipes, machines and layers
table. These filters allow filtering down the results to just displaying
the current added layers, machines or recipes in the project.

[YOCTO #7851]

(Bitbake rev: 04a3e4614aae8ba794a3b8ac9083e723de7ca522)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-26 09:27:33 +01:00
Michael Wood 4013784b45 bitbake: toaster: table.js fix filter visual indicator and interaction
- Fix the id of the filter modal dialog.
- Fix the visual indicator toggle button
- Add check to see if the number of items to filter on is !=0 if it is
  then don't allow applying this filter.

(Bitbake rev: a73f08c79077b6ded0c08ea7d36032f4568d12e3)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-26 09:27:33 +01:00
Michael Wood d21e1749c8 bitbake: toaster: toastertable Pass up the kwargs for setup_filter
This allows us to setup_filter in tables using args such as the current
project.

(Bitbake rev: e0d59c32a7cff76d732a90cc6d507c7a9b68cb84)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-26 09:27:33 +01:00
Michael Wood 93c54c2f9d bitbake: toaster: Enable toastertable cache
Enable the cache and fix warning on the toastertable cache which happens
on the memcache backend "CacheKeyWarning: Cache key contains characters that
will cause errors if used with memcached"

(Bitbake rev: 330692e4e2e4dea59f70caa03001905ffff4860a)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-26 09:27:33 +01:00
Alexandru DAMIAN f28116ec10 bitbake: toasterui: fixes after html5 compliance testing
This patch brings fixes for issues highlighted by
HTML5 compliance testing.

(Bitbake rev: 7aecb974d44d9bc711ffba5cc65e770811397fba)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-26 09:27:32 +01:00
Alexandru DAMIAN 583e320764 bitbake: toasterui: verify variable before usage
This patch verifies that BRBE is set before trying to use
it to read the checkout paths. This is needed for builds
ran outside Toaster control.

(Bitbake rev: e04807cd3135c9de96cc7f79245f329c24618b85)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-26 09:27:32 +01:00
Alexandru DAMIAN 8ef5165b5a bitbake: toaster: delete multiple builds
This patch fixes the build deletion on unmigrated databases,
and enhances it to delete multiple builds in a single run.

[YOCTO #7726]

(Bitbake rev: d5468d84c1ef83c780de5974c8e3a11eab762489)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-26 09:27:32 +01:00
Alexandru DAMIAN 5d0abf197a bitbake: toastergui: select project types
This brings in project types in the New Project page.

The "analisys" projects are the projects with no "release"
set, and have read-only properties in the project page.

(Bitbake rev: 2efc338cefd6e6e097af83d7dff63e9ba177d021)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-26 09:27:32 +01:00
Alexandru DAMIAN 70c4eb8d3a bitbake: toaster: refactor build model
We remove the "timespent", "errors_no" and "warnings_no" fields
in favor of computing the needed values at runtime. This prevents
inconsistencies in the UI.

Also removeing all references to BuildRequests from the interface -
all build details now display in the build dashboard.

Minor fixes related to data logging.

(Bitbake rev: 44f37394ed3e4ca02f940be172fe4395b0ee0f7d)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-26 09:27:32 +01:00
Alexandru DAMIAN 287b49a35b bitbake: toaster: bring django-aggregate-if into the project
We bring the django-aggregate-if module (with minor changes to import)
into the project.

(Bitbake rev: 4b92add34167304b45c66b9726cbc64bc0f74d9f)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-26 09:27:32 +01:00
Alexandru DAMIAN 35fe3117ab bitbake: toaster: add django-aggregate-if
Import library that provides conditional aggregates
in Django. The tests are removed as they conflict
with the Django startup.

The sources are
https://github.com/henriquebastos/django-aggregate-if

Licence is MIT

(Bitbake rev: 016b416fd0eaeab648a588053a2ee1f41bc02a84)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-26 09:27:32 +01:00
Alexandru DAMIAN 335c975650 bitbake: toaster: fixes after replacing BuildRequest with Build
This is a set of fixes that repair the interface after
we switched from displaying BuildRequest data to Build data
in the formerly "managed" mode.

(Bitbake rev: 57f790b0c56297af8c83d5def8461bd5d61fe4af)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-26 09:27:32 +01:00
Alexandru DAMIAN d7c8d9558c bitbake: toaster: fill in build data from buildrequest
This patch adds logic to complete changing the interface
from showing BuildRequests to showing Build data.

The BuildRequest data is now transformed in Build data with
proper Toaster exceptions being recorded instead of listing
problems during startup as build errors.

(Bitbake rev: 51a41172d0b390370f9a38696b1ac65666ada4d2)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-26 09:27:32 +01:00
Alexandru DAMIAN 160563532f bitbake: toaster: refactor the builds pages
Taking out the managed mode-specific bits in build-related
pages, as there is always only one mode available.

Also refactors the build pages in order to always display
Build objects instead of BuildRequest objects.

(Bitbake rev: 6e46e1e3882b9770872d8a0bb459bc7d5d6bfed3)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-26 09:27:31 +01:00
Alexandru DAMIAN 2c7ed96b56 bitbake: toaster: remove BuildRequest references
In the toastergui application we should not display
implementation details about how the builds are run.
This patch removes the references to BuildRequest on the
majority of the views (except Builds page itself, as
that is more complicated).

(Bitbake rev: ead45a600813a1d8594e604875803ded781f83d8)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-26 09:27:31 +01:00
Alexandru DAMIAN c362e61ee2 bitbake: toaster: remove MANAGED references
We conflate the managed and analysis modes by
deleting alternative code paths, favouring the MANAGED mode,
always considering the MANAGED variable True.

(Bitbake rev: 0ac02d3775106b485e29496b62f31e91fd3f9387)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-26 09:27:31 +01:00
Alexandru DAMIAN b98a2fcab6 bitbake: toaster: fixing undefined variables
This patchset fixes usage of undefined variables in
the base page.

(Bitbake rev: a9e889af41e1e59c3b426c78ba80e4d49c6c122b)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-26 09:27:31 +01:00
Alexandru DAMIAN 933e4dbb50 bitbake: toastergui: enable strict variable checking
In order to make sure we don't use undefined variables in the
templates, we enforce strict variable checking in the templating
engine.

(Bitbake rev: 3928dc93188880386588fe3f440cd0aaa83d22de)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-26 09:27:31 +01:00
Alexandru DAMIAN 0b2e6442a6 bitbake: toaster: improve the buildenvironment API
We improve the buildenvironment API by reducing it to a single
command: triggerBuild.

This command is specifically implemented in each BE controller
type, so the runbuilds management command is only concerned
with scheduling the next build, and not with the details
of how a build is actually started.

(Bitbake rev: 7ee0f1da0a8fcac37419ffdddbe35a9268a1ded4)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-26 09:27:31 +01:00
Alexandru DAMIAN aad93dd3b7 bitbake: toastergui: fix angular error
Due to invalid identification of builds in the array update
code (by id and status), we could end up with multiple entries
sharing the same primary id, visible in the UI as an angular error.

We modify the code to identify the builds exclusively by id.

[YOCTO #7611]

(Bitbake rev: 77d3f7c6cdc7f06807024896369f8f8ae94263fc)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-26 09:27:31 +01:00
Richard Purdie 6f4304e36d bitbake: runqueue: Sanity check BB_NUMBER_THREADS
Bitbake does really weird things with negative or zero numbers of threads
which is confusing to the user. Add a sanity check for this.

When you have code doing arithmetic on the values and a VM reconfigures
to only a single thread, negative numbers are easier than you'd think.

(Bitbake rev: 32166ac3c85ff3c04081580ae76bd63590d6ff3e)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-23 11:57:58 +01:00
Richard Purdie 810bf0093a bitbake: runqueue: Handle BBHandledException correctly
If we see a BBHandledException in runqueue, the understanding is the system
handled it, printing a log and traceback is just confusing.

Therefore only print these in the cases where its an unknown/unhandled
exception.

(Bitbake rev: 29d28e22ce431c3d3aabdb88ff4d8cca67a1cfad)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-23 11:57:58 +01:00
Richard Purdie 2ce9055e4f bitbake: server/process: Don't log BBHandledException
If we see a BBHandledException in the idle handler, the understanding
is the system handled it, printing a log and traceback is just confusing.

Therefore only print these in the cases where its an unknown/unhandled
exception.

(Bitbake rev: d88ecc2bc44dce8fd92ca3a2c0fd4124a5e464fa)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-23 11:57:58 +01:00
Richard Purdie c6918b62fd bitbake: runqueue: Improve handling of fakeworker failing to start
Currently if the fakeworker failes to start the output from bitbake is confusing.
Improve the error handling to give a clear indication of what failed.

Patch from Chris Larson.

(Bitbake rev: ad286d6fed7a580bec36a92c7b7e205322ac407b)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-23 11:57:57 +01:00
Richard Purdie df123ddab4 bitbake: event: Inject 'd' into event handlers
To quote Chris Larson:

"""
e.data.getVar() gets a bit old in a large event handler, and it means a
simple handler has to be modified if switching between an event handler
(e.g. RecipeParsed) and anonymous python. I think it would make sense
to restore the 'd' convention here to align with python elsewhere.
It'd just be a convenience, d==e.data, to avoid the common pattern of
setting it at the top of the event handler.
"""

I couldn't find a way to inject 'd' via locals/globals due to the use
of a function parameter so this left __builtins__ as the only way
I could find to make this work.

[YOCTO #7668]

(Bitbake rev: 44ac81e5281fb62ad00e2f79a9d754118ea62526)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-23 11:57:57 +01:00
Richard Purdie 69b6919341 bitbake: bitbake: Add explict getVar param for (non) expansion
Rather than just use d.getVar(X), use the more explict d.getVar(X, False)
since at some point in the future, having the default of expansion would
be nice. This is the first step towards that.

This patch was mostly made using the command:

sed -e 's:\(getVar([^,()]*\)\s*):\1, False):g' -i `grep -ril getVar *`

(Bitbake rev: 659ef95c9b8aced3c4ded81c48bcc0fbde4d429f)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-23 11:57:53 +01:00
Trevor Woerner a06a5ae684 bitbake: bitbake-user-manual-ref-variables: Added BBLAYERS_FETCH_DIR glossary item.
Add BBLAYERS_FETCH_DIR to the Variables Glossary.

(Bitbake rev: 5ecd6a671c088f270d5f49093f8da83278fc0aa9)

Signed-off-by: Scott Rifenbark <scott.m.rifenbark@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-23 11:47:40 +01:00
Scott Rifenbark 7a98387e29 bitbake: bitbake-user-manual-ref-variables.xml: Added BB_ALLOWED_NETWORKS desc.
Added a new variable description for the BB_ALLOWED_NETWORKS
variable.

(Bitbake rev: 1408654e6d8f5ae962744c6cc2d8a49df34c5101)

Signed-off-by: Scott Rifenbark <scott.m.rifenbark@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-16 23:58:16 +01:00
Alexandru DAMIAN 379a030702 bitbake: toastergui: prevent contamination of libtoaster context
This patch modifies the New Build button to use a local
copy of the default libtoaster project context in order
to prevent page contamination when a different project is
selected in the drop-down menu.

(Bitbake rev: 937665bd57dad04dd6bb46d06488b699f5c54d29)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-12 00:01:50 +01:00
Michael Wood 4f459fca79 bitbake: toaster: Add global ajaxError handler
If any ajax calls fail and debug is enabled log the error to the console.

(Bitbake rev: 978ab17033ec48ee0a82016b7e4d6a2fe5d21dbb)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-12 00:01:50 +01:00
Michael Wood 0573e2d31c bitbake: toaster: Add ajax loading spinner
This adds an ajax loading spinner to provide feedback when in-page
loading is happening. It will show after 1.2 seconds of initial loading.

[YOCTO #7790]

(Bitbake rev: ecb70b0bc996529f1a6e58e5716b31e273395c98)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-12 00:01:49 +01:00
Vlad Fulgeanu eb28534423 bitbake: toaster: tables: Remove obsolete field_name for select column
field_name is no longer used for this kind of column. So no need to specify
it here.

(Bitbake rev: 058e9db05767f2d46d46c02dd5043bb0f6b8e460)

Signed-off-by: Vlad Fulgeanu <andrei-vlad.fulgeanu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-12 00:01:49 +01:00
Ed Bartosh 5a013b562e bitbake: toaster: Make toaster script to work in dash
Made it working in dash. Note, that due to dash limitations
script will not work if sourced.

(Bitbake rev: febf8bedf8f9d37659831cdde208d14ece2fb322)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-12 00:01:49 +01:00
Ed Bartosh 0eaf070337 bitbake: toaster: Code cleanup: bashisms
Fixed the following bashisms:
 replaced echo -e -> printf
 removed 'function' from function definitions
 replaced $(< ${file}) -> `cat ${file}`

(Bitbake rev: bd95425c35c7d8386c57329e425aa7802537b479)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-12 00:01:49 +01:00
Alexandru DAMIAN 58cd4a14ea bitbake: toaster: fixes after refactoring
This patch fixes issues brought in by refactoring:

* the New Build button is working with pre-set projects
* the xhr_datatypeahead is exposed for calls that are not
mapable to the REST objects
* a new table returing recipes provided by layers currently
selected in the project is used to provide recipe suggestions
* the field names in json are switched from "list" to "rows" as
to maintain consistency with the ToasterTables
* the "value" field in xhr_ calls is now named "search" to maintain
consistency

(Bitbake rev: a5bc29083d4f85a5695f3f62d5badb783c6f7224)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-12 00:01:49 +01:00
Alexandru DAMIAN 27f5137cd6 bitbake: toastergui: remove xhr_datatypeahaed layerdeps call
This patch removes the url-constructing calls to get the layer details
in favor of embedding the look-up URL in the JSON data on the
layer list page.

This allows further removal of the XHR-specific code for layer dependencies
in favor of REST calls to layer details data.

(Bitbake rev: 33d2b87aca667d72262a3928deaf35414b46a7c1)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-12 00:01:49 +01:00
Alexandru DAMIAN a8be6d4bb1 bitbake: toastergui: remove xhr_datatypeahead and xhr_XXXbuild
We remove the endpoints for XHR on the toastergui application.

The endpoints are now replaced with calls to the respective
REST endpoints (i.e. projectlayers, projecttargets, projectmachines).

(Bitbake rev: 8e7a2c3b125a34fd9d6fa0442ab13290137ecc51)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-12 00:01:49 +01:00
Alexandru DAMIAN 88dca45a70 bitbake: toaster: eliminate duplicate querysets
The Layer_Version.get_equivalents_wpriority performs the same
function as Project.compatible_layerversions, but in memory and
with worse performance.

Replace the code in get_equivalents_wpriority with a call to
the project compatible_layerversions, which also returns a queryset
instead of a list (can be used to further enhance queries)

(Bitbake rev: fb5eb1d7759222573565936a964d602c148df139)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-12 00:01:48 +01:00
Alexandru DAMIAN 7c2b86625b bitbake: toaster: add class template view for single-object pages
Adding ToasterTemplateView as prototype for all class-based
views that display single objects; equivalent to ToasterTable
for object lists.

(Bitbake rev: d3edea773000a663f5883e04f477d853bff64cf6)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-12 00:01:48 +01:00
Alexandru DAMIAN 51ae1de5b7 bitbake: toaster: toastertables raise errors
Modifies the widget code to raise the Exceptions to the user
instead of printing then to stdout - making the programming
errors much more visible.

(Bitbake rev: 26dc19284e06a7ae35f75a243b2062f61e30f2ca)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-12 00:01:48 +01:00
Alexandru DAMIAN c941743c9a bitbake: toaster: ToasterTables add computational fields
This patch adds the ability to pass a function to be computed
for generating a field value in setting up a column in
ToasterTables.

Also adding "displayable" property that can be turned False for
columns that are present in JSON data but are not part of the UI.

Add the "id" column by default for all rows.

(Bitbake rev: fb683135348b074412da154585c75865aad1eab0)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-12 00:01:48 +01:00
Alexandru DAMIAN 58600cf8e7 bitbake: toaster: toaster table add raw data
We add in a JSON response both the raw data and the rendered
version for display. The rendered fields start with "static:"
to mark a different "namespace".

The toaster.js is updated to always display the "static:" version
of a field, if it exists (and ignore the raw data unless the
static rendering is missing).

(Bitbake rev: 928ee3fd4b52ea14b7eb704f1f27351362a9d27a)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-12 00:01:48 +01:00
Alexandru DAMIAN 4a2a057130 bitbake: toastergui: remove xhr_projectedit and xhr_projectinfo URLs
This patch removes the xhr_projectedit and xhr_projectinfo URLs
in favour of REST calls to the Project page.

The project page takes now the POST requests to modify project
settings. All usages of removed URLs are now changed to point to the
project page, using the json format.

The interface call specs have not modified.

(Bitbake rev: 6ad3078bd2be1a8cda99040acaa9bb81d77f0013)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-12 00:01:48 +01:00
Alexandru DAMIAN b1c91d06e0 bitbake: toastergui: use defaults on xhr_datatypeahead parameters
This patch prevents crashes when xhr_datatypeahead is called
without proper parameters.

(Bitbake rev: 4528490cf663790a455815dc48fbf3dda264098c)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-12 00:01:48 +01:00
Alexandru DAMIAN d9341d1a77 bitbake: toaster: toastertables REST refactoring
This patch refactors the ToasterTables to bring them in line
with REST principles -

- all table pages now support the "format=json" GET parameter
that returns the data in JSON format
- the tables themselves

This cleans up the URL namespace by aleviating the need to
have two URLS for each table (one for the template and one for
the data loading), and fixes minor things in the ToasterTable
implementation.

(Bitbake rev: 1778dac9fd39dae75c55bf2cf836cdd488dbc265)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-12 00:01:47 +01:00
Alexandru DAMIAN 751e9182ac bitbake: convert all project-based view to JSON APIs
This patch converts all-project views that are REST-style URLs
to support JSON encoding if the "format=json" parameter is supplied.

The formatting code is enhanced to prevent following Django foreign
keys, to convert enum-values from int to string, and support for
timedelta.

(Bitbake rev: 7a6eb36b82c5f2e342777e479d9c401ded42453d)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-12 00:01:47 +01:00
Alexandru DAMIAN afe06e313e bitbake: toaster: move project data typeahead to the REST API
This patch enables JSON requests on the project REST endpoint,
and replaces the universal queries "xhr_datatypeahead" with the
`project` type to the REST project endpoint.

The patch adds a decorator that takes a context returned by a view
and either renders the template specified as the decorator argument,
or converts the context to JSON.

Normal "search", "filter" and "order" options for view work as normal
on the JSON API format. To enable the JSON return, set the "format"
GET parameter to "json".

In order to demonstrate the functionality, the "New build" button
is switched from using the xhr_datatypeahead to the project
REST API with JSON formatting. Additionally, the XHR APIs that
perform actions with the project id passed as parameter are removed,
and the needed URLs are populated from the project JSON API returns after
the project has been selected.

(Bitbake rev: 15a2274eba13d19b864f337057d61c75ff7849cc)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-12 00:01:47 +01:00
Richard Purdie 926de56da9 bitbake: bitbake: Bump version to 1.27.1
(Bitbake rev: 430f7a288b4446600b3a943c51f6711ffcf9e619)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-09 13:30:09 +01:00
Richard Purdie 4bedfeffa2 bitbake: cooker/event: Add an event which lists all stamps reachable after parsing
The metadata can potentially use such an event to clean up any
"unreachable" data, solving several problems we currently have
where obsolete data may continue to exist in the shared areas.

(Bitbake rev: c5e6f929f3d5eeb7954660dea62611c58b795ff8)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-09 13:30:09 +01:00
Richard Purdie ffaa0f1b8a bitbake: parse/BBHandler: Avoid repeatedly resetting FILE
If we're not going to change the value of FILE, or we know it isn't
going to have changed (ext == bbclass), don't set FILE.

This avoids messy looking history of the variable as well as optimises
parsing speed slightly.

(Bitbake rev: 88e4600aa66dda2e6c807f9d97af8982bcd8817b)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-05 16:42:01 +01:00
Alexandru DAMIAN 6e23a3deb9 bitbake: toaster: more HTML5 fixes
More HTML5 fixes highlighted by tests on new data.

(Bitbake rev: a52242bb4d928b9abba8a3489e350e4615a9da29)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-29 11:59:47 +01:00
Alexandru DAMIAN 225daf45a4 bitbake: toaster tests: enable url check test
Integrate the HTML5 validation as a test instead of
calling a separate script. This enables us to get the
HTML5 validation report as part of patch-level
testing.

gitignore the cache directory created by the http client

(Bitbake rev: 931caab56301876cb8632b289835c2545a096ef6)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-29 11:59:46 +01:00
Alexandru DAMIAN 29143915ae bitbake: toaster/tts: Fix tests
(Bitbake rev: f4257f8df3be0627bdc7209fe34d674c42baf8d2)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-29 11:59:46 +01:00
Alexandru DAMIAN 356809ff40 bitbake: toaster: enable no browser start
We enable a "nobrowser" parameter that inhibits
the launch of a browser when toaster starts.

This is useful for integration with automated startup scripts
and enables headless testing.

(Bitbake rev: ccf7f39d470fe6d743b58b1140c19cb8da31ecaf)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-29 11:59:46 +01:00
Michael Wood b3189f3071 bitbake: orm: Fix all failing unit test
This fixes all the unit tests for the orm. Also added is the ability to
set a custom Layer index if you want to avoid using the public one by
specifying TTS_LAYER_INDEX

(Bitbake rev: dfbcbe116d0b987b850f67056f02f489ac0b8360)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-29 11:59:46 +01:00
Michael Wood 3480be741e bitbake: bldcontrol: Fix all failing unit tests
This fixes the unit tests for the bldcontrol it requires the
implementation of a new Exception type so that a known Exception can be
handled. Also fixed is the path to the toaster conf files so that the
test doesn't need to be run from the top level directory and the ability
to specify the values of TTS_SOURCE_DIR and TTS_BUILD_DIR and
TTS_TEST_ADDRESS used for testing.

Edited by Alex Damian to correct the rebasing of the localhostbecontroller.py
file.

(Bitbake rev: c17933271cd273a346115c2ee0b6695ff3f981ce)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-29 11:59:46 +01:00
Alexandru DAMIAN 22a0d8aab2 bitbake: toastergui: add tests for xhr_datatypeahead calls
In order to make sure we replace the xhr_datatypeahead calls
with similar APIs, we add tests that validate the data returned
by the xhr_datatypeahead. A copy of these tests will be enabled
for each type of datatypeahead that will be replaced.

(Bitbake rev: 9a3197fba46b38e5863e7ded11e4bf9530cfcc9b)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-29 11:59:46 +01:00
Alexandru DAMIAN 65c805eb6a bitbake: toastergui: tests for the all-projects API point
This patch adds Django tests that verify that the 'all-projects'
page returns a valid HTML page when invoked normally, containing
the project name; and valid JSON containing API-needed fields
if the GET parameter `format` is set to "json"

(Bitbake rev: 9edd61fe7afaf273ed31d214af9251462182ad4f)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-29 11:59:46 +01:00
Alexandru DAMIAN fea4a82a2b bitbake: toasterui: replace cookie-based preferences with session data
We switch from storing the user preferences using cookies
to saving them in the server-side session.

Patch for "count/pagesize" and "orderby" fields in the table-based
pages.

This patch will solve two problems:
* the browser-side race between the GET header data and the cookie data
* page breakages when field names chances in orderby statements.

(Bitbake rev: 125d0e05805247450be0675e281a21bd6146d108)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-29 11:59:45 +01:00
Alexandru DAMIAN 1b6a50c6b2 bitbake: toaster: refactor checksettings command
This patch refactors the checksetting command to prevent
early return from the handle function.

It also adds a check that marks IN PROGRESS builds at startup time
as FAILED. Minor changes to BuildRequest and Build classes
ensure useful string representation for the objects.

(Bitbake rev: adf67dd79dbf6b585bf8cd54f99c389409b88ecd)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-29 11:59:45 +01:00
Ed Bartosh 52fe880f9e bitbake: toastergui: Consider task name when restarting a build in /builds
Previously the same issue was fixed for project view.
'Run again' button now restarts builds using target:task also in builds view.

[YOCTO #7442]

(Bitbake rev: ae6349a13f11b6fa90fe5603b000bef14ee0f2f0)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-29 11:59:45 +01:00
Ed Bartosh 6fab4cb701 bitbake: toaster-gui: Show task name for the builds in /builds
Task name was only shown for the latest builds in project view.
Now it should be shown also in builds view.

(Bitbake rev: 6492349d50a2048b1cbebab482bfd075bb48dd96)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-29 11:59:45 +01:00
Michael Wood e5aa569d47 bitbake: toaster: Add an invalidate cache method on data update signal
Instead of relying on the cache expiring over an amount of time we can
listen to the changed notification on the models to invalidate the
cache. Also fixes overlapping cache names.

(Bitbake rev: eb0b1450e421cf65b407b1ac0336ac24ffc626e8)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-29 11:59:45 +01:00
Michael Wood 2de01a68eb bitbake: toaster: Port layerdetails to ToasterTables
This ports the layerdetails page to using ToasterTables
Also some whitespace and strict clean ups in the existing layerdetails
js and html template.

(Bitbake rev: 8ce35f81631e31539aeb82f8a85abbb3312e5097)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-29 11:59:45 +01:00
Michael Wood a0ea663b83 bitbake: toaster: Add a simplified ToasterTable template
This is like the normal ToasterTable template but with a number of
features removed such as edit columns and has a lower profile compared
to the "main" tables.

(Bitbake rev: 4ce35c74d2b1ecd68c494fa9bab3c1133f9679c1)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-29 11:59:44 +01:00
Michael Wood e3a4905d08 bitbake: toaster: Fix toastertable header
Fixes regression introduced by 'tables updates for HTML5 compliance'

(Bitbake rev: 406a7f81420f7baa5cbfc034b573a4d692b98874)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-29 11:59:44 +01:00
Ed Bartosh 6a9efefbba bitbake: toastergui: show relative paths in configvars view
Reworked filtering of config paths.

Stripped clone paths, topdir and its parent directory from the paths
to config files in configvars view.

[YOCTO #7463]

(Bitbake rev: 873087b11653848ec2704d67de5680a265b71eaa)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-29 11:59:44 +01:00
Alexandru DAMIAN 9dcfa32cf1 bitbake: toastergui: table header bring back
This patch brings back the table header, which I mistakenly
took out when fixing HTML5 compliance.

(Bitbake rev: 9855e0b9735ebf0a6c622bee6ec787dfc1d9e474)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-29 11:59:44 +01:00
Alexandru DAMIAN 56c5ff920c bitbake: toaster logger: assign projects to all builds
Bitbake logger now sets a default project if the TOASTER_BRBE
or TOASTER_PROJECT Bitbake variables are not set.

This a necessary step in getting all builds under a project,
as to unify the MANAGED and interactive modes.

Other small fixes are included, related to the size of the
fields in the database.

(Bitbake rev: 5e0bf388f4e5c1cc493ac8264785e631bad2f672)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-29 11:59:44 +01:00
Alexandru DAMIAN e3e85bdf71 bitbake: toaster logger: refactor recipe and layer file paths
This refactoring brings the "local_path" of the
layer from the Layer object to the Layer_Version object, which
is more appropriate as different checkouts of the same
Layer may live in different directories.

This enables us to store Recipe file paths relative to a Layer_Version
at all times, aleviating the need to store full file paths in the
database. We also turn the prefix of the path (e.g. virtual:native
path name space) into a pathflag field.

In turn, this solves the problem of mis-identification of tasks based
on the recipe file paths, since we can also match the namespace of the
file paths on the recipe files.

[YOCTO #7594]

(Bitbake rev: ec43dc569e370767c709dec225cbee0c99151c19)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-29 11:59:43 +01:00
Alexandru DAMIAN a5236be47f bitbake: toaster: enable server-side sessions for all usages
In an effort to fix client-side session storage using
cookies, we enable server-side session support for all pages.

(Bitbake rev: ba10b6f89767c0dad8a2b064f42a84956764e9da)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-29 11:59:43 +01:00
Michael Wood 5da543c7b5 bitbake: toaster: Fix build button current project race
Make sure the current project value is set before we check to see if the
project is buildable. Also update the blacklist url patterns where we
aren't displaying the button.

[YOCTO #7739]

(Bitbake rev: e169ed5cf190af62586f3e1c6ed6db6120406e05)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-29 11:59:43 +01:00
Richard Purdie b8caddb57a bitbake: data: Make expandKeys deterministic
At least in theory, the order the keys are expanded in can make
a difference, particularly if there is key overlap.

We also want to ensure that any underlying base key is processed
before any overridden version of that variable (FOO before FOO_x)
which helps the update_data removal code I've been testing.

(Bitbake rev: 863b6add24c211d64ba7931647084321f2d65889)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-29 10:17:16 +01:00
Richard Purdie 3db9d6e02d bitbake: ConfHandler: Fix bogus dependency problems
Adding a dependency when errors occur accessing a file when calling
handle() is not the correct thing to do. THe handle() code calls
resolve_file() which can raise an exception without ever touching
"fn" itself, it has also already marked all the dependencies correctly.

This leads to bogus files being resolved to the local cwd and
hence triggers reparses for no good reason.

The solution is to simply remove the bogus dependency.

(Bitbake rev: 366af3be1cffd64e4a79c15990c1e05869022c14)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-29 10:17:16 +01:00
Richard Purdie 91c45c0163 bitbake: BBHandler: Drop cwd from search path
Whilst bitbake has done this for a long time, the behaviour of resolving
class files against cwd is not desirable. This can be seen during
base configuration parsing when looking for base.bbclass where a dependency
on cwd is added. If cwd then changes, the cache is invalid and triggers a
re-parse.

The only real option is to drop this entry and if files can't be found, we
fix BBPATH in the cases where it needs fixing. I didn't find any in the
random selection of layers I tested parsing locally.

(Bitbake rev: 508aad9d5db7e51328b1fd6ee53b4bc3720a30b7)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-29 10:17:16 +01:00
Richard Purdie 08b77c8cef bitbake: cooker/utils: Improve parsing profiling
Currently the cooker parsing processes each dump an individual profile
which is ok, but means absolute numbers of function calls for a given load
can be tricky to determine as parsing of recipes may go to different pool
threads on different runs.

This change collects up the individual thread parsing results and processes
them into one profile output. The profile processing function in utils
needed tweaks to allow this to work.

(Bitbake rev: d3d2541aacd1ea560da0d8b25a3ea3f0563dee70)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-29 10:17:16 +01:00
Richard Purdie 553267d8d9 bitbake: tests/data: Add key expansion unit test
This ensures basic key expansion works and that overlapping
keys generate a log message.

(Bitbake rev: ed5a8954ac923eda9750a636c5bb5b95ffce664f)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-29 10:17:16 +01:00
Richard Purdie bde0f60971 bitbake: tests/data: Add log parsing test code
This allows us to write tests which ensure a particular action generates
a particular log message.

(Bitbake rev: b30ee0aba51a35a194a4338b988f93ece1ed281c)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-29 10:17:16 +01:00
Richard Purdie 8d8b02aacd bitbake: data: Revert expandKeys warning move
This was incorrect, it *must* be done next to the rename as a previous
may overlap with the current one and we need to detect this case.

I'll add a test case to better catch this problem in future.

(Bitbake rev: 5e9d0911cd86e980ab310cc0d79c9383fbc2c844)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-29 10:17:16 +01:00
Richard Purdie 30cba374cc bitbake: data: Clean up expandKeys None value handling
This comparison is interesting even in the case of empty vales. Enabling
this warning actually found a bug in the metadata in avahi. Make the
code handle None specifically and also remove the dead code path in the
second if statement.

(Bitbake rev: a4cd4c56284812efb2a2bc0c8667ddad073f8e94)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-29 10:17:16 +01:00
Richard Purdie a32937c501 bitbake: data: Move warning code to the first loop for performance
By doing this we can take advantage of the expansion cache before
starting write operations on the data store.

(Bitbake rev: 702b42a47904f2378dd819e7463b3206883c2651)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-26 10:33:06 +01:00
Richard Purdie 9e9d05205a bitbake: siggen: Don't crash if number of task dependencies change
If the number of task dependencies change you currently get
a traceback when using diffsigs.

(Bitbake rev: c6798b431571aae18bb8699ac6e3ec75b731d719)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-26 10:33:06 +01:00
Richard Purdie 7fab819f7f bitbake: cache: Clean up getVar usage to modern syntax/style
No functional change.

(Bitbake rev: 0eb75a34bd9731e9de7bc9600a7418a927561fdb)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-26 10:33:06 +01:00
Jason Wessel 5ee122244a bitbake: bitbake-worker: Fix regression with unbuffered logs
I noticed that I was seeing loss of the log files when hitting
control-c while debugging a function in bitbake.  In fact if you
take a recipe and replace its compile function as shown below let
it run for a few seconds and hit control-c, you will see first
hand that log data is not there.

do_compile () {
        while [ 1 ] ; do
                echo -n "Output date: "
                date
                sleep 1
        done
}

It turns out there was a regression introduced by commit:
d0f0e5d9e6 which created the bitbake worker.  Since the bitbake
worker is started in its own process space, it needs the exact
same code added from commit: 88429f018b where the problem was
fixed the first time around.

(Bitbake rev: 8d1748f75763b4a66516cc46d5457ee6404b1b68)

Signed-off-by: Jason Wessel <jason.wessel@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-26 10:33:06 +01:00
Ash Charles 70c1d222c0 bitbake: fetch/hg: Disable checksums for archived downloads
Like the Git fetcher, the Mercurial fetcher shouldn't expect recipes to
provide a checksum.  As described [1], recipes using a mercurial
fetcher that don't provide a checksum will fail in a the repository has
previously been downloaded and archived.

Credit to Rafaël Carré for figuring out the bug.

[1] https://www.mail-archive.com/openembedded-devel@lists.openembedded.org/msg41328.html

(Bitbake rev: 2df35a25b4968f64adfa673d5b73442c1a30829d)

Signed-off-by: Ash Charles <ashcharles@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-26 10:33:06 +01:00
Mike Looijmans 5170177719 bitbake: fetch2/git.py: Add gitpkgv_revision alternative version information
gitpkgv_revision returns a sortable revision number that can be used
in the PKGV variable for example. To mimic meta-openembedded gitpkgv
behaviour to provide a sortable revision numner, one could set the
following:

PKGV = "1.0+${@bb.fetch2.get_srcrev(d, 'gitpkgv_revision')}"

This would yield a package version like "1.0+69+fb5eb80".

(Bitbake rev: 989c08f62aff7b707c25c692c23284f16506b7bc)

Signed-off-by: Mike Looijmans <mike.looijmans@topic.nl>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-24 07:20:56 +01:00
Mike Looijmans f03e108fb1 bitbake: fetch2/__init__.py: Make get_srcrev output configurable
The idea here is to support package version numbering similar to gitpkgv in
meta-openembedded. This commit is the first step towards such functionality.

The original plan was to add a "get_pretty_srcrev" method to the fetcher, as
per Richard's suggestion [1]. While writing this, I noticed that it would
become a copy of get_srcrev with only two lines changed. So to create something
more Pythonic than a boolean argument and conditionals around the calls to the
fetcher's sortable_revision, I just made the method to be called on the fetcher
an argument to the method. Defaulting to 'sortable_revision' prevents affecting
existing code.

Now if the git fetcher were to implement, say 'gitpkgv_revision' one could
set the following in a recipe:
  PKGV="1.2+${@bb.fetch2.get_srcrev(d, 'gitpkgv_revision')}"
and this would yield the same result as gitpkgv's GITPKGV variable.

See for the discussion leading to this change:
[1] http://lists.openembedded.org/pipermail/openembedded-core/2015-January/100345.html

(Bitbake rev: 2f1f4483493cc290f5d2c07f9906e90eaea2f4c1)

Signed-off-by: Mike Looijmans <mike.looijmans@topic.nl>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-24 07:20:56 +01:00
Scott Rifenbark 2f41470fd6 bitbake: bitbake-user-manual: Updated "Delete a Task" section for broken deps
Fixes [YOCTO #7803]

I added information for the scenario where deleting a task using the
deltask command could "break" implicit dependencies.

(Bitbake rev: 5ea3a206bb5beef4c12d56d9e42e29d324a08e16)

Signed-off-by: Scott Rifenbark <scott.m.rifenbark@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-22 16:52:33 +01:00
Scott Rifenbark 16caaabfcc bitbake: bitbake-user-manual-customization.xsl: Pointing to mirrors for XSL files.
Instead of pointing to the Sourceforge area, which seems to flake out
every so often, we are now pointing to a mirror to get the XSL files
needed to build the manual.

(Bitbake rev: d9811231b4c8211446bd9275084c26a4718cd175)

Signed-off-by: Scott Rifenbark <scott.m.rifenbark@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-19 22:09:36 +01:00
Marcin Smoczyński 7310c3c925 bitbake: fetch/git: Fix uri in git checkstatus
Pass proper repository url without arguments after a semicolon.

Executing checkuri on a rule with git repository in SRC_URI does
not report errors when working offline because wrong repository
url is passed to the ls-remote command. For example
"bitbake -c checkuri glibc" command executes:
"git -c core.fsyncobjectfiles=0 ls-remote git://sourceware.org/git/glibc.git;branch=release/2.21/master"
command in a shell subprocess to determine if url is valid.
Shell subprocess executes in fact 2 commands:
"git -c core.fsyncobjectfiles=0 ls-remote git://sourceware.org/git/glibc.git"
and
"branch=release/2.21/master"

First one returns 127 or 128 depending on error but second one
returns 0 because it is just env variable setup. Therefore we're not catching
connection error.

[YOCTO #7558]

(Bitbake rev: efa44d04137977f883db4a643b0f774e91514722)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-19 22:09:36 +01:00
Randy Witt 8b8859a8fe bitbake: toaster: Remove dependency on daemon application
The bitbake observer is now started using python subprocess. This should
allow for toaster to run without installing the "daemon" application.

[Yocto #7271]

(Bitbake rev: 1dd599ddfcdb547bee49bd7d86acddf64c675b42)

Signed-off-by: Randy Witt <randy.e.witt@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-19 11:58:45 +01:00
Randy Witt 37c0b9852b bitbake: localhostbecontroller: Fix server search debug message
When the output from the commands ran was redirected to
"toaster_server.log" instead of the console, the debug message stating
the port on which bitbake is running would no longer appear.

This change makes looks at "toaster_server.log" for the port rather than
the _shellcmd output. This makes the debug message useful again.

(Bitbake rev: 9097bae469cb1e005092c11610d92e908b8f19f5)

Signed-off-by: Randy Witt <randy.e.witt@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-19 11:58:45 +01:00
Robert Yang 3371a7bc96 bitbake: fetch2/hg.py: add clean function
Fixed when bitbake vim -ccleanall:

File: '/path/to/bitbake/lib/bb/fetch2/__init__.py', lineno: 1462, function: clean
     1458:    def clean(self, urldata, d):
     1459:        """
     1460:        Clean any existing full or partial download
     1461:        """
 *** 1462:        bb.utils.remove(urldata.localpath)
     1463:
     1464:    def try_premirror(self, urldata, d):
     1465:        """
     1466:        Should premirrors be used?
File: '/path/to/bitbake/lib/bb/utils.py', lineno: 633, function: remove
     0629:        subprocess.call(['rm', '-rf'] + glob.glob(path))
     0630:        return
     0631:    for name in glob.glob(path):
     0632:        try:
 *** 0633:            os.unlink(name)
     0634:        except OSError as exc:
     0635:            if exc.errno != errno.ENOENT:
     0636:                raise

Exception: OSError: [Errno 21] Is a directory: '/path/to/downloads/hg/vim.googlecode.com/hg/vim'

(Bitbake rev: 02763306662e15a4750395e5eab64ba98d1f9939)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-19 11:58:45 +01:00
Paul Eggleton aefc80c02e bitbake: lib/bb/utils: fix and extend edit_metadata_file()
Fix several bugs and add some useful enhancements to make this into a
more generic metadata editing function:
* Support modifying function values (name must be specified ending with
  "()")
* Support dropping values by returning None as the new value
* Split out edit_metadata() function to provide same functionality
  on a list/iterable
* Pass operation to callback and allow function to return them
* Pass current output lines to callback so they can be modified
* Fix handling of single-quoted values
* Handle :=, =+, .=, and =. operators
* Support arbitrary indent string
* Support indenting by length of assignment (by specifying -1)
* Fix typo in variablename - intentspc -> indentspc
* Expand function docstring to cover arguments / usage
* Add a parameter to enable matching names with overrides applied
* Add some bitbake-selftest tests

Note that this does change the expected signature of the callback
function. The only known caller is in lib/bb/utils.py itself; I doubt
anyone else has made extensive use of this function yet.

(Bitbake rev: 20059e4d5ab9bf0f32c781ccb208da3c95818018)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-19 11:58:45 +01:00
Paul Eggleton ba0546bfaf bitbake: lib/bb/utils: fix several bugs in edit_metadata_file()
* Fix unchanged assignments being dropped if other lines changed
* Fix not passing variable name from single-line assignments to the
  function
* Fix not trimming the trailing quote from values

(Bitbake rev: 0b0c82f49cf2de887967d305768cbd95314bb171)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-19 11:58:45 +01:00
Paul Eggleton 0d9c45289a bitbake: lib/bb/utils: add function to get layer containing a file
In certain contexts it can be useful to find the layer that a file (e.g.
a recipe) appears in.

Implements [YOCTO #7723].

(Bitbake rev: 3bf9c8830c5d5eea5502230d5af84ebd87ad5849)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-19 11:58:44 +01:00
Richard Purdie bc162d21dd bitbake: tests/parse: Add file missing from previous commit
(Bitbake rev: 76f095107a0eaf987a5a6a48eed7b98f87aea121)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-16 22:40:44 +01:00
Richard Purdie 71626c1b94 bitbake: tests/fetch: Add mirror of mirror tests
Add some tests of mirrors or mirrors to the fetcher unittests.

(Bitbake rev: e33d82bc10283d533f928836d56a6f0af80ea5c1)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-16 22:31:51 +01:00
Richard Purdie 2be2271906 bitbake: fetch2: Improve mirror building function error handling
First, when building mirror urls we don't do any fetching so we should never
be calling clean functions.

Currently, if a mirror url fails, we don't process it further to see
if there are any mirrors of the mirror.

We should do this even when the mirror url fails, else we may miss out
on valid/useful mappings, particularly in the case of file:// urls.

(Bitbake rev: b7fd3ec9994f664b17fc86423e6e7afac07e897b)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-16 22:31:51 +01:00
Liam R. Howlett b6140496e2 bitbake: fetch2: Allow GIT_SMART_HTTP to be passed through
Allow GIT_SMART_HTTP to be passed through to the fetch command so that
servers that cannot use GIT_SMART_HTTP can be used by the fetcher.

(Bitbake rev: e5c97a85bed0436d48eeaac2e32962cfb5371d2f)

Signed-off-by: Liam R. Howlett <Liam.Howlett@WindRiver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-16 22:31:51 +01:00
Mariano Lopez c41b5b6fc3 bitbake: bitbake: fetch2/git: Add URL when latest_revision generates an exception.
The URL is not sent when _latest_revision generates and exception.
When performing the sanity checks it is not possible to know the URI that failed.

This add the URL when latest_revision generates an exception.

[YOCTO: #7592]

(Bitbake rev: 9f2115b07a55cb14e4a74dc6fbd3707c28a234d0)

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-15 17:45:00 +01:00
Richard Purdie 2a56445b9b bitbake: tests/parse: Add very basic start of parse tests
This tests very basic usage of the parser and then adds a test
to ensure that incomplete functions raise an exception.

(Bitbake rev: b7bcef141b56fe8eb03724ea5251e3251fc63817)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-15 17:45:00 +01:00
Richard Purdie fd731142e9 bitbake: BBHandler: Error for incomplete function definitions
Add some sanity checks on the parsing state engine when returning data
so that incomplete functions raise parse errors.

This means a recipe doing:

do_somefunction {
     echo 1

VAR = "1"

will now raise a ParseError. To get the right file/line information,
__infunc__ was changed to a list.

[YOCTO #7633]

(Bitbake rev: 6b54a72638f57882d4fd5aab96b2752a09e065af)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-15 17:45:00 +01:00
Paul Eggleton c5316d4d57 bitbake: tests/fetch: ensure fetch tests preserve current dir
The fetcher calls os.chdir() in a number of places, which can affect
other tests (since the directory it changes into gets deleted) - let's
just put the current directory back to where it was when we're done.

(This fixes bb.tests.Path.test_unsafe_delete_path failing if it was run
as part of a full bitbake-selftest run, where the fetcher tests get to
run before it.)

(Bitbake rev: b1653855c74f86909c9f329ed6d2b10391c28395)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-15 17:45:00 +01:00
Paul Eggleton f5a87b4707 bitbake: fetch2/git: ensure the unpacked origin remote points upstream
If you're interested in using the checked out repository for development
(e.g. in OE with devtool) then you ideally want the origin remote to
point to the repository it was fetched from, so just set that after
cloning.

(As part of this I did a minor refactor so we have one function to
generate the repository URL, which was already in two places.)

Fixes [YOCTO #7756].

(Bitbake rev: 80ecd1c54d4c748cee3a7ce0d64013a346e7671e)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-15 17:45:00 +01:00
Richard Purdie 1dc2ee903b bitbake: runqueue: Handle cases where siginfo is now a parameter to the hashvadlidate function
In some cases we need to check specifically for siginfo files, in
some cases we need to check for the actual sstate objects themselves.

Therefore make this a parameter to the function. A fallback to the
previous function style is maintained for now.

(Bitbake rev: 18d3a03e1b07c98b2dce46eb94f30de1a2b4320b)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-15 17:45:00 +01:00
Alexandru DAMIAN e05be1fdd2 bitbake: toastergui: tables updates for HTML5 compliance
This is a patch to bring the generic table code in compliance
with HTML5 specification.

(Bitbake rev: 39e4ce20d1e0bf6c93ca7ef2dcc4019979de3e39)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-14 18:04:10 +01:00
Alexandru DAMIAN 80ca4f00f8 bitbake: toaster/contrib: adding TTS squashed patch
In order to move the Toaster Test System in Toaster itself,
we create a contrib directory.

The TTS is added as a squashed patch with no history.

It contains code contributed by Ke Zou <ke.zou@windriver.com>.

(Bitbake rev: 7d24fea2b5dcaac6add738b6fb4700d698824286)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-14 18:04:10 +01:00
Michael Wood 23f5b009e0 bitbake: toaster: Port All recipes, layers and machines to ToasterTables
Port of the main tables to the new ToasterTable widget.

(Bitbake rev: 6de539d5953b2dca2a9ed75556a59764337a194c)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-14 18:04:09 +01:00
Michael Wood 7f8c44771c bitbake: toaster: Add toaster table widget
This widget provides a common client and backend widget to support
presenting data tables in Toaster.

It provides; data loading, paging, page size, ordering, filtering,
column toggling, caching, column defaults, counts and search.

(Bitbake rev: b3a6fa4861bf4495fbd39e2abb18b3a46c6eac18)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-14 18:04:09 +01:00
Michael Wood 2ac26e4294 bitbake: toaster: managedcontextprocessor requires Project imported
Temporarily move this import into the main view so that the context
processor can return a value for projects (albeit an unused one) when in
non managed mode. This will be changed in the near future but this
will fix it until we finish the refactoring.

(Bitbake rev: db61587fdd78af071d8a2a50d494fe159bf169a6)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-14 18:04:09 +01:00
Michael Wood ec8bd70b27 bitbake: toaster: Remove old bldviewer application
The SimpleUI is obsolete and not maintained. It should be deleted as
there is no use to it.

[YOCTO #7709]

(Bitbake rev: 10b7c359613629bf6e3465234512990ba4742c48)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-14 18:04:09 +01:00
Alexandru DAMIAN c5a16235b8 bitbake: toaster: fix html5 compliance
This patch brings needed changes in all views so that
each view passes the HTML5 compliance test by the
W3C Markup Service.

(Bitbake rev: 6e60ed8ab71e4300ab571f42b7af5011086df697)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-14 18:04:09 +01:00
Belen Barros Pena 35d75c8352 bitbake: toastergui: change the import layer action help text
To match the form labels and the order in which the
information is requested. That should make
understanding the text easier.

(Bitbake rev: 3424110d1d95e3cc039e8775324f814e7fdd5470)

Signed-off-by: Belen Barros Pena <belen.barros.pena@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-14 18:04:08 +01:00
Alexandru DAMIAN 4df4098281 bitbake: toaster: adding art missing from Bootstrap
When Bootstrap 2.3 was imported, we missed a couple of art file.

This patch adds them to complete Bootstrap integrity.

License used is (as previously documented).

GLYPHICONS Halflings released with Bootstrap, CC BY 3.0

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
2015-05-13 13:30:49 +01:00
Liam R. Howlett 59ce7d02a5 bitbake: fetch2: Add BB_ALLOWED_NETWORKS support
BB_ALLOWED_NETWORKS is a list of hosts that the fetcher will be allowed
to use when BB_NO_NETWORK is not set.

If BB_NO_NETWORK is set, then networking is still disabled.

If BB_ALLOWED_NETWORKS is not set, the behaviour remains the same as
today.

If BB_NO_NETWORK is NOT set, and BB_ALLOWED_NETWORKS is configured, then
only the hosts in the list are usable by the fetcher.

eg:
BB_ALLOWED_NETWORKS="yoctoproject.org git.gnu.org"
The fetcher will be able to download from yoctoproject.org, git.gnu.org,
but not ftp.gnu.org or any other hostname that is not in the list.

There is also limited support for wildcards on the beginning of the
hosts, so BB_ALLOWED_NETWORKS="*.gnu.org" with match git.gnu.org and
ftp.gnu.org as well as foo.git.gnu.org

(Bitbake rev: c7263096ba31ba45daeeb9de90c1cb9ebef24a28)

Signed-off-by: Liam R. Howlett <Liam.Howlett@WindRiver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-12 12:17:13 +01:00
Daniel McGregor 9e24bde011 bitbake: fetch/hg: support submodules
Use hg clone and hg pull to copy the source into the build
directory rather than taring up the cloned repository and
untarring in the destination.

This allows submodules to be cloned. While here, make the default
behaviour keep the hg scm data to match the behaviour of the git
fetcher.

(Bitbake rev: f002b1ca80cb542a4ed0c06c53c914cd5e076565)

Signed-off-by: Daniel McGregor <daniel.mcgregor@vecima.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-12 12:14:14 +01:00
Jason Wessel d30aedcdf9 bitbake: fetch2: Allow git to use a different CA Bundle with private keys
Setting BB_ENV_EXTRAWHITE to include GIT_SSL_CAINFO is enough to
get "git ls-remote" to work for recipe parsing but it is not enough
to get the fetcher to clone properly.

This is because the fetcher has its own idea about what variables
should be exported in the environment for some operations.

It is desirable to use alternate CA Bundles for internal testing prior
to using public keys for https, so we should allow the GIT_SSL_CAINFO
to pass through.

(Bitbake rev: 40ff92282bbf32cf644b021bf7cbb1f393dbb856)

Signed-off-by: Jason Wessel <jason.wessel@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-12 12:14:14 +01:00
Scott Rifenbark cc63e1568b bitbake: bitbake-user-manual-metadata.xml: Alphabetized varflag list.
This list should have been in alphabetical order.  It is now.

(Bitbake rev: 9b7fae5c36c7b48237ce7903434602ebc1a07b20)

Signed-off-by: Scott Rifenbark <scott.m.rifenbark@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-12 12:14:14 +01:00
Scott Rifenbark 0e8f83fd92 bitbake: bitbake-user-manual-metadata.xml: Added two new varflag descriptions
Added brief descriptions for recideptask and lockfiles.

Reported-by: Robert P. J. Day <rpjday@crashcourse.ca>
(Bitbake rev: ee52aa304680a27529264cb186df6936b0375118)

Signed-off-by: Scott Rifenbark <scott.m.rifenbark@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-12 12:14:14 +01:00
Alexandru DAMIAN 3c4f3775f7 bitbake: toastergui: removing unused views
We remove url entries, views and templates that are not
used in the interface; these are leftovers from the development
process, and are not linked anywhere in the interface nor
mentioned in documentation.

(Bitbake rev: 96bdebaa1aa43a35dc0747eda3081e7cf1a3dbde)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-12 12:14:13 +01:00
Alexandru DAMIAN e7f4d9c8e1 bitbake: toastergui: fix whitespace after URL
This patch fixes whitespace after URLs in the tasks page,
fixing warnings from the linkchecker.

(Bitbake rev: 2f98e6e78845f699777856ee5dec7b26f4c88d08)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-12 12:14:13 +01:00
Alexandru DAMIAN 94d38a4e40 bitbake: toasterui: URL refactoring
This is a URL refactoring needed to remove the inadvertent
usage of server-side user session in a REST-style API.
We move the parameters that were stored in the user session
to the URL, making navigation more robust.

This refactoring allows a clean (no 500 HTTP return codes) crawl
of the website following inside links.

(Bitbake rev: cc251da0f211f3ee881ad07478733e2f4c1b7019)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-12 12:14:13 +01:00
Ed Bartosh a5193d3c7f bitbake: toastergui: Consider task name when restarting a build
'Run again' button now restarts the build using target:task if task
was specified for the build.

[YOCTO #7442]

(Bitbake rev: 420b197227394b341bcc1075bc298ecf2aabec46)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-12 12:14:13 +01:00
Mihail Stanciu 44b7b67154 bitbake: toaster: <Add class id to a few columns>
Added class ids to columns that were missing them in the Packages and Recipes pages. This is required for Toaster UI automation efforts.

[YOCTO #7377]

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-12 12:14:13 +01:00
Ed Bartosh 1e625fcc5f bitbake: toaster: fix typo in format string
Fixed TypeError: not all arguments converted during string formatting

[YOCTO #7618]

(Bitbake rev: 308b5529d4e183d550221864bfdbefd31cc23705)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-12 12:14:12 +01:00
Ed Bartosh b546be990d bitbake: toaster: Code cleanup: indent
Replaced tabs with 4 spaces.
Removed useless semicolons at the end of lines.

(Bitbake rev: 5bfe5eb0bf1e53ddb04c31e786e1ffc15542e6e8)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-12 12:14:12 +01:00
Ed Bartosh 13e88144bf bitbake: toaster: Make toaster script working in zsh
Fixed shell syntax not understood by zsh. Made toaster
script to work for both shells in both modes: sourced and
directly called.

[YOCTO #6964]

(Bitbake rev: e7b27843e2f37619dbe555a8883ac67f418f32dc)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-12 12:14:12 +01:00
Ed Bartosh 3b1bb2dfff bitbake: toastergui: decrease load time for configuration page
Used __in field lookup for QuerySet to get build variables faster.

[YOCTO #6691]

(Bitbake rev: 222e21d2fc3dc67449e582d983e11ea927e3eaef)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-12 12:14:12 +01:00
Alexander Kanevskiy 84a294edde bitbake: toaster: clear up toaster shutdown for mac osx
This patch properly shuts down toaster in interactive mode
under Mac OSx.

(Bitbake rev: a00cd5135a4bbd61d311fa17569894f974ab4420)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-12 12:14:12 +01:00
Maxin B. John 29c7111362 bitbake: prserv: serv.py: remove unused and duplicate imports
Remove unused xmlrpclib, atexit and duplicated threading module imports

(Bitbake rev: 3e12f4e8e3ec66d1df772a64be04b90ec72462ae)

Signed-off-by: Maxin B. John <maxin.john@enea.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-08 17:46:36 +01:00
Michael Wood 1fe445d0b5 bitbake: toaster: filtersnippet remove redefinition of filter type key
This redefinition is not needed and also used Mozilla specific API to
access the data thus breaking the date filter in all other browsers.

[YOCTO #7577]

(Bitbake rev: b64bbc2dd51369b64fae2b1a10502b9dd4ceebfd)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-08 17:42:06 +01:00
Michael Wood 286da989d9 bitbake: toaster: Add links to the layer add notification
Adds links to the layer details for the layer that has just been
removed or added

(Bitbake rev: 3b323af501f7ec81cce34adb4ea86a7d6599e60a)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-08 17:42:06 +01:00
Michael Wood bec5d16471 bitbake: toaster: Refactor and expand layer add remove mechanism
We have multiple pages which have buttons to add and remove layers this
patch adds functionality to libtoaster to abstract this and implements
it in the pages affected. We handle loading and showing the dependencies
dialog here too and generating the notification messages.
Also implemented is using the selectmachine api from the projectapp to
avoid having to handle this in each page that allows selecting machines.
A small number of jshint issues, help text and the machine page name
have also been fixed.

(Bitbake rev: ae7a656ba7fc6f4356b57aa309a9b6d035e51d2e)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-08 17:42:06 +01:00
Michael Wood a4cfca604b bitbake: toaster: projectapp Implement machine select command
Use the project page to select the machine rather than setting it and
then redirecting to the project page. This will also avoid having to have a
special handler in the machines page it's self.

(Bitbake rev: 9847e04d86063e4464afb402cb1352243b51f504)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-08 17:42:06 +01:00
Michael Wood ed26a06a07 bitbake: toaster: Move project context variables to common scope
We have a bunch of context data which are used in multiple pages so it
makes more sense to have this in a single place libtoaster.ctx that's
accessible from each page rather than request it from every page.

(Bitbake rev: 4ef2774a2f683929c700550a9acc7b8f6074195b)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-08 17:42:05 +01:00
Alexandru DAMIAN 4b59ba1bbb bitbake: toasterui: close build on BuildCompleted event
Moving the code to close the build and instantiate a fresh
buildinfohelper object to BuildCompleted event, as the
CommandCompleted/Failed/Exit events come in too early.

(Bitbake rev: af63abe88327fd5c1b3d7c00a84d9e408ef23285)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-08 17:42:05 +01:00
Alexandru DAMIAN 7759cd4931 bitbake: toasterui: proper exit code on toaster errors
This patch modifies the toasterui to properly return the exit
code based on the errors found in the toaster itself.

The upload event file API call will not delete event logs for which
toasterui showed an error. This will facilitate debugging.

Minor enhancement in the buildinfohelper to reduce the number
of lookups on unknown layer objects (prevented testing of the patch).

(Bitbake rev: 1ddd6a9e4280a4adf971132ff1fe7ec9b3252905)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-08 17:42:05 +01:00
Ed Bartosh db409697db bitbake: bitbake: reset build mtime cache before the build
Introduced build mtime cache structure. Reset it before the build
to prevent bitbake from crashing when build/tmp/stamps hierarchy
is removed.

[YOCTO: #7562]

(Bitbake rev: f8590547a198a78334debdf14bf40acb50c22ecc)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-04-29 10:59:54 +01:00
Ed Bartosh 4eeae70a45 bitbake: bitbake: use sys.exit to simplify the code
Used sys.exit instead of assigning exit code to
variable. This way it's more clear when bitbake
exists and which exit code is used.

(Bitbake rev: 5ecb8817bd49223652ede4fe513f1a42f2196798)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-04-29 10:59:53 +01:00
Ed Bartosh 37e46e49dd bitbake: bitbake: Check if bitbake versions match
Bitbake program and core versions must match.

Moved __version__ from main.py back to bin/bitbake.
Implemented check for version match in bin/bitbake.

(Bitbake rev: 2fe7d8c574ddf6a30278cff1a5a5c4089dc56d6d)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>

tbs

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-04-29 10:59:53 +01:00
Ed Bartosh 47e2dbd89a bitbake: bitbake: Handle BBMainException
Some error messages were lost because BBMainException was
inherited from bb.BBHandledException. When bb.BBHandledException
is processed error messages are not printed as they suppose to
be printed before raising this exception.

Stopped to inherit BBMainException from bb.BBHandledException.
Handled BBMainException in bin/bitbake and printed error message
to the stderr.

(Bitbake rev: c8e2a40c4e9865ebef9936d23644f2602a5c90f5)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-04-29 10:59:53 +01:00
Paul Eggleton 72d88f29da bitbake: lib/bb/utils: add safeguard against recursively deleting things we shouldn't
Add some very basic safeguard against recursively deleting paths such
as / and /home in the event of bugs or user mistakes.

Addresses [YOCTO #7620].

(Bitbake rev: 56cddeb9e1e4d249f84ccd6ef65db245636e38ea)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-04-19 13:37:31 +01:00
Ed Bartosh f9b160571f bitbake: cookerdata: Allow ~ in bblayers
Implemented processing of ~ in bblayer's paths if HOME
environment variable is approved.

(Bitbake rev: 3b8a656d3ccb543c32696229184ebf12237ad38e)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-04-19 13:37:31 +01:00
Scott Rifenbark 203aaaec08 bitbake: bitbake-user-manual: Cleaned up parallelism note and formatted user input
I updated the note in the second chapter that discusses the role for
BB_NUMBER_THREADS.  The updates make it a bit clearer.

Also scrubbed the manual for instances of user-supplied values to check
how they are being formatted.  I fixed the formatting to use the
<replaceable></replaceable> tags so they are in italics.

(Bitbake rev: e2879c60e905d7566091d40eab330372fa001313)

Signed-off-by: Scott Rifenbark <scott.m.rifenbark@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-04-17 13:28:47 +01:00
Chen Qi 547b189951 bitbake: bitbake-diffsigs: consider the situation where sigdata and siginfo files having the same hash values
For now, `bitbake-diffsigs -t <recipe> <task>' doesn't work. It always outputs
nothing.

The problem is that bitbake-diffsigs are comparing sigdata and siginfo files
that have the same hash value. This is not what we want. These two files are
actually duplicates considering the purpose of bitbake-diffsigs. So we need
to remove one of them so that bitbake-diffsigs could actually compare the
correct signature files.

(Bitbake rev: c34613eb69fd19770cbfc78ab8384221f10d5587)

Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-04-17 11:55:44 +01:00
David Reyna abc7f15960 bitbake: toaster: date range filter for project builds page
Port date range filter initialization code to the project builds page.

[YOCTO #7578]

(Bitbake rev: 294ea99263385525a1434bc6df690b653ee1f81b)

Signed-off-by: David Reyna <David.Reyna@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-04-17 11:53:57 +01:00
David Reyna db254cb513 bitbake: toaster: recipe path broken in recipe detail page
The recipe paths in managed mode are not processed correctly. They need
any prefix tags removed, and then any build path prefix removed only if
present.

[YOCTO #7523]

(Bitbake rev: ee5c569ebe21c065508cd816ae8cb8842baf0b7e)

Signed-off-by: David Reyna <David.Reyna@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-04-17 11:53:57 +01:00
Belen Barros Pena 77cad30d53 bitbake: toasterui: remove 'targets' from project notification
Remove the last remnant of the "targets" wording, which was
lurking in the notification we show you when you create
a project.

(Bitbake rev: ff475c1a6dec883f855dfce7cb0650135da33751)

Signed-off-by: Belen Barros Pena <belen.barros.pena@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-04-17 11:53:57 +01:00
David Reyna c6552273dd bitbake: toaster: searching all projects table resets page count
Add missing '_save_parameters_cookies' to preserve page count.

[YOCTO #7483]

(Bitbake rev: 62f486f81f59a3186a57f93c3b102404a3da8a6f)

Signed-off-by: David Reyna <David.Reyna@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-04-17 11:53:57 +01:00
David Reyna ac69236d1d bitbake: toaster: add layer source in admin throws an error
Preset uninitialized object members to bootstrap new layer source creation.

[YOCTO #7437]

(Bitbake rev: 4de721aa75a4aaad186312443726e6daa764934b)

Signed-off-by: David Reyna <David.Reyna@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-04-17 11:53:56 +01:00
Belen Barros Pena 86f982c3df bitbake: toasterui: correct terminology in landing page
Change the introductory description of Toaster
we provide on first use, to make sure it mentions
all the bits that make up the build system, and
not just BitBake.

(Bitbake rev: ed741d022e9bd13cbef5c71e33b1e68cb2e42d9b)

Signed-off-by: Belen Barros Pena <belen.barros.pena@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-04-17 11:53:56 +01:00
Michael Wood 0405ef3816 bitbake: toaster: BuildButton target input allow building targets with tasks
Allow the input for targets to contain tasks in the form target:task

[YOCTO #7501]

(Bitbake rev: cf7d0418ff2d4aec508c7942f96e9b52a491b25e)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-04-17 11:53:56 +01:00
Michael Wood fab29bd737 bitbake: toaster: base.js minor jshint fixes
Fixing errors identified by jshint, some missing semicolons and
preference for === operator. Also pass the urldata in libtoaster as an
object rather than constructing a query string.

(Bitbake rev: 8652fbaf5e8c56d9d28b7da57432f37313a19b4a)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-04-17 11:53:56 +01:00
Anders Darander 65e9859769 bitbake: fetch/git: Remove a possible trailing '/' in subpath
If the subpath parameter to the git fetcher ends with a trailing '/',
 bb.utils.prunedir() will be called on '/'...

Fixes [YOCTO #7620].

(Bitbake rev: 380a3fb372c8b0a53dd7528562e6e7a222dc76ef)

Signed-off-by: Anders Darander <anders@chargestorm.se>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-04-17 11:52:18 +01:00
Andre McCurdy 3c0664b687 bitbake: toastergui: minor spelling fixes
RRECOMENDS -> RRECOMMENDS
  TRECOMENDS -> TRECOMMENDS

(Bitbake rev: 8cb95090c4b35b3fa54b25110ed6be4ef0d18e15)

Signed-off-by: Andre McCurdy <armccurdy@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-04-17 11:52:17 +01:00
Richard Purdie df9069d958 bitbake: bitbake-layers: ensure non-extended recipe name is preferred
In show-recipes and show-overlayed, we only list one variant of each
recipe where multiple exist, therefore we should show the main one (e.g.
in OpenEmbedded, we now show openssl and not nativesdk-openssl which
would otherwise sort first.)

Fixes [YOCTO #7514].

(Bitbake rev: c4172fe33f0c63eda7f4af2bd36a5e9738e13595)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-04-13 22:33:24 +01:00
Christopher Larson a146e8ccfc bitbake: runqueue: pass finalized metadata to scenequeue callbacks
This ensures that _append, _prepend, overrides, etc are functional when used
on sstate variables (e.g. SSTATE_DIR).

[YOCTO #7564]

(Bitbake rev: 2e683c25b856b431198573f7f352d841587275e6)

Signed-off-by: Christopher Larson <kergoth@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-04-08 10:53:32 +01:00
Richard Purdie fe9c5b1ba9 bitbake: bin/image-writer: Add ext4 as a deployable image type
This is particularly problematic since qemu images switched to ext4 by
default and now cannot work properly with UIs like hob.

This patch adds in ext4 to the appropriate variable fixing this.

[YOCTO #7426]

(Bitbake rev: a1431b1b85bf90ed5f771552d0ce91de288047d4)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-03-31 22:39:21 +01:00
Richard Purdie 3cb87724a5 bitbake: cooker: Ensure bbappend files are processed in a determistic order
self.appendlist is a dict and as such unordered. This can lead to cases
where appends with different names (e.g. x_%.bbappend vs. x_123.bbappend)
can be reordered in application which in turn reorders the variables
that those bbappend files might touch. Reorderd variables changes the sstate
cache signatures causing real world issues.

To avoid this, use a list for the append files instead.

This patch is conservative and just adds a new data structure alongside
the existing one and uses it to resolve the core issue. Later patches
(post release) can handle some of the wider but less problematic ones
(e.g. issues in bitbake-layers flatten).

[YOCTO #7511]

(Bitbake rev: f980f060cd0d1e7fe5011f3c325c1b254f05eccf)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-03-31 22:39:21 +01:00
Alexandru DAMIAN ce2573b96a bitbake: Revert "toaster: machine not searchable in all builds/projects"
This reverts commit 1a86ed8f95649c5f5a3a66984ce36978d93b0e01.

Bug [YOCTO #7334] was incompletly fixed, and the proper fix is
not straightfoward. Consequently we revert the partial fix, as
a incomplete fix is worse than no fix.

(Bitbake rev: 3d838ddc2f1f92e2dd940e43c3467f6ba4ba043d)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-03-31 22:30:06 +01:00
Alexandru DAMIAN e5a441c3ff bitbake: toastergui: add ext4 option for IMAGE_FSTYPES
This patch adds "ext4" as a selectable option in IMAGE_FSTYPES,
to keep in sync with the OE-Core capabilities.

(Bitbake rev: e5a44e1096ba62b50bed4eee191f109c4f2c4105)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-03-31 22:30:05 +01:00
Belen Barros Pena ddee53e767 bitbake: toasterui: rename 'targets' to 'recipes'
A recent round of informal user feedback has revealed
that the term 'target' is quite problematic. For
all the users we spoke to the word refers to the
target arch. In Toaster, it refers to the software
you build.

This patch replaces the word 'target' with 'recipe'
across the Toaster interface. This is by no means
self-explanatory, but at least it cannot be confused
with target hardware, and it is also consistent with
the terminology we use in the analysis portion of
the interface.

(Bitbake rev: f4a64c2386a534aae9522629afde219d2aa00817)

Signed-off-by: Belen Barros Pena <belen.barros.pena@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-03-31 22:30:05 +01:00
Alexandru DAMIAN c6ea812964 bitbake: toastergui: prevent error on empty build list
This patch prevents errors being thrown on date limit
computations if the build list is empty.

[YOCTO #7513]

(Bitbake rev: d028fbe76962f3b86239633a0951626dfa66b8af)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-03-31 22:30:04 +01:00
Richard Purdie e029d27eaf bitbake: Update to version 1.27.0 post 1.26 release
(Bitbake rev: 0153e0b4e089f62a7d5a92ca6be2fa5a8f61a6e4)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-03-25 12:56:29 +00:00
Richard Purdie 5756684d67 bitbake: Update to version 1.26.0
(Bitbake rev: 14da7bc06bc6ea6fd051c0afd8d4839d96f415e7)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-03-25 12:56:29 +00:00
Alexandru DAMIAN 87c4e58dda bitbake: toasterui: enable sanity checker
Since toasterui acts as the user-facing UI, we need
to run sanity checks in order to let the GUI display proper
warnings and stop the build if something is wrong.

(Bitbake rev: 260dd77fa771ae3b777134f4178d344e96b6f3d6)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-03-25 12:39:58 +00:00
Dave Lerner eeef93ad07 bitbake: toaster: most-build-targets bug fixes
Fixes for the following for the "Most built targets" section of the
project page:
1) limit the number of most built targets to 5,
2) order the list of most built targets by build count,
3) select only those builds that had build status of 'fail', 'success',
or a BuildRequest of 'failed',
4) if there are no layers in the project then (on refresh) all of the
checkboxes in the 'Most built targets' list are disabled.

Note that per agreement with UI designer, the only sort on the most
target list is build count as described in (3) above; secondary sorting
is random.

[YOCTO #7177]

(Bitbake rev: dda2f07193e740860f4497c3761d340d8c5c2b41)

Signed-off-by: Dave Lerner <dave.lerner@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-03-25 12:39:58 +00:00
David Reyna 1a4c8a849a bitbake: toaster: PACKAGE_CLASSES variable does not show current value when editing
A typo in some element selectors caused the package_rpm setup to break.

[YOCTO #7448]

(Bitbake rev: 0479faa46f0364df6d8e151be34e1834076df49d)

Signed-off-by: David Reyna <David.Reyna@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-03-25 12:39:57 +00:00
Belen Barros Pena 3190ab6441 bitbake: toasterui: small changes to empty states
This commit:

* Makes some changes to the layout and styles of the
landing.html template

* Adds the empty state to layers.html

* Changes the message of the empty state of machines.html
to use the same wording used in all other pages

* Adds the project name and the right styles to
the empty state of the project.html template

* Makes small changes in style and wording to the
empty state of targets.html

* Updates all "configure a layer source" links to point
to the new Toaster manual.

(Bitbake rev: d886db3add1351a806ab58ec7519d2321687187b)

Signed-off-by: Belen Barros Pena <belen.barros.pena@linux.intel.com>
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-03-25 12:39:57 +00:00
Belen Barros Pena 38c3f2ec5e bitbake: toasterui: hide right column if empty
In the layer details page, if the background
information about the layer that shows in the
right hand column is completely empty, remove
it altogether.

(Bitbake rev: fe9fff960d0a6fc24e0e177e5194f93a7be9456b)

Signed-off-by: Belen Barros Pena <belen.barros.pena@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-03-25 12:39:56 +00:00
Belen Barros Pena 63510bf20d bitbake: toasterui: Improve messaging for empty layer details
Explain that, if information is found during the build
process, it will appear in the layer details page.

(Bitbake rev: 40b46ce28b0b93241f9ce581f1e295e95bbff284)

Signed-off-by: Belen Barros Pena <belen.barros.pena@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-03-25 12:39:56 +00:00
Belen Barros Pena 7c3bdebffe bitbake: toasterui: hide release when only one exists
If only one release has been set up in Toaster,
any project you create defaults to that release,
and we simply don't show the release selection
menu when creating projects.

However, we were still showing the release information
in the project page, which would leave users wondering
what that release thing is since they never were
exposed to it.

This patch hides the release information in the
project page when only one release has been set
up in Toaster.

(Bitbake rev: 0bd81be04c88104d7d460f913d0500e5fa9755fe)

Signed-off-by: Belen Barros Pena <belen.barros.pena@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-03-25 12:39:55 +00:00
Belen Barros Pena fd4085f380 bitbake: toasterui: update links to Toaster manual
Set the links to the Toaster manual in the base.html
and landing_not_managed.html templates to the URL of
the new Toaster manual.

(Bitbake rev: 88b1201a2f6dec53970b120cc8e13c0eb5ae21e1)

Signed-off-by: Belen Barros Pena <belen.barros.pena@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-03-25 12:39:55 +00:00
Belen Barros Pena 792716a385 bitbake: toastergui: changes to release change notifications
* Remove the branch name from the list of the layers
deleted (it does not apply).

* In the modal dialog that warns you before layer
deletion, use the release description instead of
the release name

(Bitbake rev: b4ffe490253108e7a009290474e206468255ce12)

Signed-off-by: Belen Barros Pena <belen.barros.pena@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-03-25 12:39:54 +00:00