Commit graph

3102 commits

Author SHA1 Message Date
Belen Barros Pena
a6867343b6 bitbake: toaster: Remove inline styles in h1
The templates bpackage.html, recipes.html and build.html included
an inline style declaration in the div containing the h1
tag to add a top margin of 40px. The extra top margin
is unnecessary in bpackage.html and recipes.html, but nicely
separates the Recent builds and All builds sections in build.html.

The changes remove the inline style declaration and create
a .top-air class in default.css to include the extra top margin
when needed, i.e. in the build.html template.

(Bitbake rev: 2841f0740024a8351606452a5f803b9b7f70c783)

Signed-off-by: Belen Barros Pena <belen.barros.pena@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-03-09 12:23:59 -07:00
David Reyna
438578e299 bitbake: toaster: extend Tasks to include Time, Disk IO, and CPU Usage
Update the All Tasks page to also cover the Time, Disk I/O, and
CPU Usage pages. Add filter count header support, and fix minor column
enablements.

[YOCTO #4387]

(Bitbake rev: 7e78836ebbddf0240094fd79a18cb057d6c4f322)

Signed-off-by: David Reyna <David.Reyna@windriver.com>
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-03-09 12:23:58 -07:00
Alexandru DAMIAN
93aa4aba74 bitbake: toasterui: add asserts on expected values
We add assert statements that validate expectations of
correct values and context when the data collection code is running.

These checks will help pinpointing unexpected data or
call flows, reducing debugging time.

Also contains a couple of very small fixes discovered through
these checks, including a virtual:native conflict with
regular tasks.

	[YOCTO #5553]

(Bitbake rev: e2fbd5c6fa6b53514e2cb23d42aa639020d8a475)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-03-09 12:23:58 -07:00
Alexandru DAMIAN
0daa189028 bitbake: toasterui: task data structure in toasterui
We update the structure used to hold interm task data,
before it is written to the database, to lower the changes
of key collision.

This will also lead to a cleaner data structure and easier
inspection.

(Bitbake rev: 49cb9f543526a161bc4c097f94422ea08b491ef9)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-03-09 12:23:58 -07:00
Marius Avram
1660519e5c bitbake: bb/ui: store_dependency_information optimization
This optimization is in support of the bug #5485. The function called
at the beginning of every build: store_dependency_information was taking
approximately 20sec and it was delaying the arrival of events from the
event queue. The change minimizes the calls to _save_a_task(),
reducing the time to half.

(Bitbake rev: b86fd2be40303d886fdb9ad3009355584d285acc)

Signed-off-by: Marius Avram <marius.avram@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-03-09 12:23:58 -07:00
Alexandru DAMIAN
06f9059abc bitbake: toaster: array-assignment fix from Chome
Apparently the JS engine in Chrome can't handle assignments to an array,
so I'm patching this up by using an intermediate array to hold the
values.

(Bitbake rev: 47f5fde1bd8cf2e6f7e5c4ec2534a2f9599c4ea2)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-03-09 12:23:58 -07:00
Alexandru DAMIAN
6fc246232a bitbake: toaster: mark dependency packages
We need to mark the package entries that are
created solely for dependency tracking purposes.

In order to avoid altering the database schema,
we mark the dependency targets with size = -1, since
this is not a valid size anyway and makes for easy
filtering.

	[YOCTO #5803]

(Bitbake rev: d11ed273dd6c520b16e9ccfe79476f340006a55d)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-03-09 12:23:57 -07:00
Alexandru DAMIAN
3d0bb418a0 bitbake: toasterui: adding new task outcome empty
In order to separate tasks with invalid states from the
no exec tasks, we add a new value OUTCOME_EMPTY for the tasks.

OUTCOME_EMPTY has the same value as OUTCOME_NA as to maintain
compatibility with already existing builds. New value for
OUTCOME_NA can be used to detect tasks with invalid states, i.e.
it should never appear after finishing a build.

Fixing noexec tasks outcomes.

	[YOCTO #5763]

(Bitbake rev: 475643ad78796835bf2e731b9d0fa5794ec80dd1)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-03-09 12:23:57 -07:00
Alexandru DAMIAN
7008a24792 bitbake: toastergui: fix task executed status display
The proper way to get a display-able value
in Django is to use a model method instead of clutter
the template with if/else.

	[YOCTO #5641]

(Bitbake rev: bb21b71dab70db163b804c7ebf27b85c59a39112)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-03-09 12:23:57 -07:00
Richard Purdie
4ae598195b bitbake: fetch2/wget: Fix downloadfilename functionality
Some of the previous cleanups broke the downloadfilename functionality.
This change fixes the code to ensure the commandline is correctly built.

Thanks Kristof Robot <krirobo@gmail.com> for reporting the issue.

(Bitbake rev: e008d9bb07e5d1a3584cc04ca2cd3dd906fd5759)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-03-09 12:07:01 -07:00
Richard Purdie
e9554464d4 bitbake: runqueue: Fix typo
slef.self is clearly meant to be self, fix typo.

Otavio spotted and reported, thanks.

(Bitbake rev: 316daad7928a58cdfc42e27b20e739f4dd74a02a)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-03-06 22:35:55 +00:00
Paul Eggleton
003d3170ed bitbake: data: add vardepvalueexclude varflag
On rare occasions it's useful to be able to exclude a part of a
variable's value from the variable's signature; for example if you want
to add an item to a list sometimes and not have the signature of the
variable change depending on whether the item is in the list or not. The
initial intended use case for this in OpenEmbedded is to allow adding a
function to SSTATEPOSTINSTFUNCS in buildhistory.bbclass and not have
that change any task signatures (so adding and removing
INHERIT += "buildhistory" won't lead to any rebuilds).

Part of the fix for [YOCTO #5897].

(Bitbake rev: f803bf8cfefafcbe212442e66b301ccd9c5aa2a5)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-03-06 17:31:11 +00:00
Richard Purdie
151eaca6e0 bitbake: fetch2: Drop svk fetcher
The svk fetcher never appears to be used by anyone and the development
on svk appears to have stopped in 2010. We might as well drop support
for it.

(Bitbake rev: 8239264753977bd06ad5b1b574245d3842af489b)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-03-02 17:25:29 +00:00
Richard Purdie
e955def4d3 bitbake: codeparser: Fix var_execs to append to execs, not references
When using the "execs" information in new code, it became clear that
the returned data was incorrect and there were missing exec'd functions.
This corrects the error and changes one of the test results to match
the correct behaviour.

(Bitbake rev: 8a24f2d3b735bbc59ca4a09670cabbadb1868c1a)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-03-02 17:25:29 +00:00
Richard Purdie
69b4614ff3 bitbake: fetch/wget: Separate out download and checkstatus functions
These two functions have little in common, separate them out.

(Bitbake rev: 7413e0fa4dca9571ea98f32dab87d4fd60bc8de9)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-03-02 17:25:29 +00:00
Richard Purdie
5f0e3a8800 bitbake: fetch/wget: Start to clean up command construction
Start to clean up wget fetcher command construction to allow clearer
and more extensible code structure. Drops support for ${URI} and
${FILE} directly in the commands.

(Bitbake rev: 4e59fe45be2088996abc21e9a631a32b9a9642c9)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-03-02 17:25:29 +00:00
Richard Purdie
6265744ec5 bitbake: wget: Drop usage of old style *COMMAND variables
These variables were dropped from OE-Core some time ago, drop their
usage from the fetcher as well.

(Bitbake rev: bd33e709ab65d6966b234010641861834d170e2b)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-03-02 17:25:29 +00:00
Richard Purdie
c0bdd18609 bitbake: fetch/svk: Drop usage of old style *COMMAND variable and MKTEMPDIRCMD
Clean up some horrible old code and drop usage of the old style *COMMAND
variable and MKTEMPDIRCMD whilst in here. This means we don't need to touch
OVERRIDES either.

(Bitbake rev: c127bb3a9b7b1d2ab2c833ff73186b6ead0dc29c)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-03-02 17:25:29 +00:00
Richard Purdie
c547149cae bitbake: fetch/perforce: Drop usage of old style *COMMAND variable and MKTEMPDIRCMD
Clean up some horrible old code and drop usage of the old style *COMMAND
variable and MKTEMPDIRCMD whilst in here. This means we don't need to touch
OVERRIDES either.

(Bitbake rev: 6b79789769da160d0e7fca0f9c6044dc1e11a107)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-03-02 17:25:29 +00:00
Richard Purdie
1aab9c797d bitbake: fetch/git: Separate out an ls-remote function
There is other code which can want to run ls-remote style commands with
different parameters so split out the function.

(Bitbake rev: 13f1138f5504feee0ee8e8f3a0675d0bea490351)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-03-02 17:25:29 +00:00
Paul Eggleton
c2895edcdd bitbake: tests: add test for gitsm fetcher
Use a newly created "git-submodule-test" repo on git.yoctoproject.org
which currently contains one submodule (the bitbake repository).

(Bitbake rev: a750c57242928c546a5aace632543e956ee908eb)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-02-28 14:50:43 +00:00
Paul Eggleton
0dd3a1bc1d bitbake: tests: add missing import
This was found when trying to run the data tests individually.

(Bitbake rev: e4bf4ad4b99978483541a719105c98ea124e8a34)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-02-28 14:50:43 +00:00
Paul Eggleton
2f8be92de6 bitbake: bitbake-selftest: enable specifying tests to run on command line
If you are just trying to fix one test at a time, it can be useful to be
able to specify an individual test(s) rather than running them all:

 bitbake-selftest bb.tests.codeparser bb.tests.cow

You can even specify the test class or function to run, e.g.:

 bitbake-selftest bb.tests.fetch.URITest
 bitbake-selftest bb.tests.fetch.FetcherNetworkTest.test_fetch

(Bitbake rev: 4df9c72663e972437131a848e6ddcf3769ae1d2b)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-02-28 14:50:43 +00:00
Paul Eggleton
957c9a80bc bitbake: fetch2: fix fetching git submodules with git 1.7.9.x or older
Git versions older than 1.7.10 put absolute paths in configuration files
for the submodule repositories, leading to errors when the repository
checkout is moved. We move the repository as a matter of course in the
gitsm fetcher; the failure occurs in do_unpack). Change the absolute
paths to be relative during processing to fix this.

(At the time of writing, Ubuntu 12.04.4 LTS ships Git version 1.7.9.5,
hence the desire to fix this rather than just mandating a newer Git
version.)

Fixes [YOCTO #5525].

(Bitbake rev: e700d5a41deed4ee837465af526ed30c8a579933)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-02-28 14:50:43 +00:00
Cristiana Voicu
81f4dd661c bitbake: hob: append bbfiles to bblayers.conf instead of local.conf
[YOCTO #5118]
(Bitbake rev: 12946da1353dc35b5c1c4ce56315408f4f5c1edf)

Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-02-28 14:48:40 +00:00
Cristiana Voicu
2557667aa0 bitbake: hob: remove the code that adds hob layer to bblayers
[YOCTO #5118]
(Bitbake rev: ef32d27b33477a5cbf127cc8b98012b48e89ad07)

Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-02-28 14:48:40 +00:00
Cristiana Voicu
ed19ed0db1 bitbake: hob: create a base hob image used to create custom images
In order to remove hob-image.bb from meta-hob, a hob-image.bb should
be created somewhere in the build directory. I've saved it in build/recipes/images
directory, and moved the templates to recipes/images/custom (here are those
templates saved by the user).
The image is created when hob starts. Also it appends to BBFILES the directory
where it is created.

Removed images directory from meta-hob.

[YOCTO #5118]
(Bitbake rev: 4587297b51b7ca71d314bdb2c06f2061e7d4aa7d)

Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-02-28 14:48:40 +00:00
Stefan Stanacar
510560338d bitbake: build: filter out expanded empty strings for lockfiles flag
If we have something similar to:

    LOCKFILES = "${TMPDIR}/my.lock"
    LOCKFILES_qemuall = ""

    do_task[lockfiles] += "${LOCKFILES}"

when expanded, lockfiles will be empty for qemu, resulting in

     File "/home/stefans/yocto/poky/bitbake/lib/bb/utils.py", line 630, in mkdirhier
	raise e
    OSError: [Errno 2] No such file or directory: ''

This should filter out the empty expansions.

(Bitbake rev: 7813e1bfd08cd48871f8c03cae2810265590105d)

Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-02-28 14:48:40 +00:00
Cristiana Voicu
72576e2d04 bitbake: hob: replace the use of hob-toolchain with populate_sdk task
Now bitbake has an API to run individual task for targets.
Hob can use this to build the sdk, instead of using the hob-
toolchain.

(Bitbake rev: 4a5009036e9cb38f6e0260a88278948931073bc6)

Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-02-25 08:06:11 +00:00
Richard Purdie
a1faa7df2a bitbake: fetch2: Fix mirror repo tarball creation
A typo was meaning that the mirror creation method wasn't being called
when it should have been. Fix the type to fix mirror tarball creation.

[YOCTO #5284]

(Bitbake rev: 348971d410bfd5d8b1757468d73e1d24ae78a594)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-02-24 13:00:36 +00:00
Martin Jansa
dd43700b5d bitbake: runqueue: Catch ValueError from pickle.loads
* exception like this keeps spinning quite quickly generating GBs of logs
  better to kill it asap and show invalid pickle

(Bitbake rev: a69eb4c12c71bba9d742c4e5578f25c388d9f825)

Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-02-24 12:42:40 +00:00
Jacob Kroon
f5a344441a bitbake: ast: Fix support for anonymous methods in wildcard .bbappend files
When using wildcard .bbappend files with anonymous methods in them,
bitbake/python fails to parse the generated code since the '%' is encoded
in the generated method name.

Fix this by including '%' in the convert-to-underscore list during
method name mangling.

While we're at it, move the method name mangling translation table
to a class variable, as suggested by Chris Larson.

[YOCTO #5864]

(Bitbake rev: 537f1f9bbe110acc9848ef95f43468c07d87af79)

Signed-off-by: Jacob Kroon <jacob.kroon@mikrodidakt.se>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-02-24 12:42:40 +00:00
David Reyna
deb3859820 bitbake: toaster: implement package summary page
Implement the updated design for the package summay page, with pop-up
boxes for the layer commit ids, column filtering, and column sorting
support.

[YOCTO #4318]

(Bitbake rev: c39b99792547b642570ea5152070e7396e812390)

Signed-off-by: David Reyna <David.Reyna@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-02-17 15:38:53 +00:00
David Reyna
77eeb4200e bitbake: toaster: implement recipe summary page
Implement the updated design for the recipe summay page, with pop-up
boxes for the dependecies and layer commit ids, column filtering, and
column sorting support.

[YOCTO #4294]

(Bitbake rev: 92b71c8e6cfcd656866fbf9bd00bf184b223c5fa)

Signed-off-by: David Reyna <David.Reyna@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-02-17 15:38:53 +00:00
Belen Barros Pena
994236e180 bitbake: toaster: Make popovers mutually exclusive
Table popovers should be mutually exclusive (only one
of them should be open at any given time), and should
close when you click outside their area. But this is not
the default popover behaviour in Bootstrap, so some
additional javascript is needed.

The code in main.js taking care of this in the design
prototype was quite ugly and didn't get on well with
certain browsers. I have replaced it with a better
solution (although still not ideal).

(Bitbake rev: d56633c00f6730c053f355570211eba1bdc41b62)

Signed-off-by: Belen Barros Pena <belen.barros.pena@intel.com>
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-02-17 15:38:53 +00:00
Belen Barros Pena
49a81f064a bitbake: toaster: Make popovers scrollable
Some recipes and packages have a lot of dependencies.
To stop their popovers from taking over the full
height of the screen, I have given them a maximum
height and made their content scrollable in
default.css

(Bitbake rev: b6416a98f441516100d2ce7baca30f09714a1d1d)

Signed-off-by: Belen Barros Pena <belen.barros.pena@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-02-17 15:38:53 +00:00
Belen Barros Pena
ec79df9c9e bitbake: toaster: Make all targets links to the build dashboard
Change the build.html template to remove the conditional check
that added links to the build dashboard only to those targets
that are image recipes.

All targets should be links to their corresponding build
dashboard.

(Bitbake rev: 05f52b9fad597173d813fa4305af65f1c9bb88a1)

Signed-off-by: Belen Barros Pena <belen.barros.pena@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-02-17 15:38:53 +00:00
Ravi Chintakunta
877f87baa0 bitbake: toaster: Select a radio button by default in Filter Dialog
If none of the filter options in the filter dialog are active,
then select the first radio button.

(Bitbake rev: 8aa63143cc446227c69f64688b314c65b74604d8)

Signed-off-by: Ravi Chintakunta <ravi.chintakunta@timesys.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-02-17 15:38:53 +00:00
Belen Barros Pena
f8f448e4e6 bitbake: toaster: Fix for task_color tag
This patch changes the task_color tag in
projecttags.py to make sure it adds the error
class to the rows corresponding to failed tasks
in the tasks table.

(Bitbake rev: 44c5b3a93a93a464429e8d998aace25f840ae724)

Signed-off-by: Belen Barros Pena <belen.barros.pena@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-02-17 15:38:53 +00:00
Belen Barros Pena
155302acb2 bitbake: toaster: Apply error style to cell links
This patch changes default.css so that any anchor
tag inside a table cell with the error class applied
inherits the class styles. This ensures that the
failed tasks listed in the builds table look red
like the failed tasks in the tasks table.

(Bitbake rev: e5cbb190317dba2eddd933678474569563ec840b)

Signed-off-by: Belen Barros Pena <belen.barros.pena@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-02-17 15:38:53 +00:00
Belen Barros
2e3fee7291 bitbake: toaster: Make "Edit columns" multiselect
Twitter Boostrap elements with the dropdown-menu class
close by default once a selection is performed. Such
behaviour is not appropriate for our "Edit columns" menu,
since users might want to check / uncheck several
columns.

This patch adds a call to the stopPropagation() jQuery
function to main.js to stop the "Edit columns" menu
from closing every time you change a checkbox.

(Bitbake rev: c2e43750bf0913523a1abcb2c8bf97b764da2524)

Signed-off-by: Belen Barros <belen.barros.pena@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-02-17 15:38:53 +00:00
Alexandru DAMIAN
97c81e089f bitbake: toaster: fix javascript for table filters
This patch adds javascript link for resetting page filters
in the tool tip "Show all" button.

Also fixes a JS bug for when the cookie is not set yet.

(Bitbake rev: e123922274ea875105e6ed855b3368b8b77fca64)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-02-17 15:38:52 +00:00
Belen Barros
4157c3849f bitbake: toaster: Add clear filter button to filter tooltips
This patch modifies the basetable_top.html template to add
the 'Show all' button to the applied filter tooltip. It also
adds a delay in the tooltip dismissal in main.js to allow
users to click the button comfortably.

The patch does not add the functionality to the button (when
you click on it nothing happens). Someone else will need
to add that in.

(Bitbake rev: e39857162b5d91cc2f92843a9a14599bab369849)

Signed-off-by: Belen Barros <belen.barros.pena@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-02-17 15:38:52 +00:00
Belen Barros
985017af3d bitbake: toaster: Clean up main.js
Clean up main.js to leave only what is being
used in the design prototype.

Initialise the Bootstrap tooltips for any anchor tag inside
a table heading with the btn-primary class applied. This
effectively styles the title attribute of the applied
filters to look like all other Toaster tooltips.

(Bitbake rev: 71a07268c3a2e969c088ef9f6bcf4c54254b3600)

Signed-off-by: Belen Barros <belen.barros.pena@intel.com>
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-02-17 15:38:52 +00:00
Belen Barros
b4416e324f bitbake: toaster: Clean up default.css
Getting rid of all unused css declarations, grouping
and annotating to make the file more readable.

(Bitbake rev: 90409869d49468d31daa60880f54774c4b24d5aa)

Signed-off-by: Belen Barros <belen.barros.pena@intel.com>
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-02-17 15:38:52 +00:00
Dave Lerner
620553df86 bitbake: toaster: Implementation of package detail views
Adds new package detail views.  The views are based on
specifications found in attachments to:
    https://bugzilla.yoctoproject.org/show_bug.cgi?id=4328
specifically:
    design-1.5.1-package-details.pdf, and
    design-1.1.1-included-package-details.

This patch includes a redefinition of constant numbers for
task dependency tasks. This is needed in order to achieve
sorting criteria from the design.

This change invalidates currently dependency information for
currently existing builds, as it breaks compatibility.

[YOCTO #4328]

(Bitbake rev: 6855925c06e7e5bb15ae9d0c08d77f3a9a2574bc)

Signed-off-by: Dave Lerner <dave.lerner@windriver.com>
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-02-17 15:38:52 +00:00
Ravi Chintakunta
47634378ac bitbake: toaster: Added sorting to other columns in All tasks table
Added sorting to recipe, time, cpu, disk i/o and log columns of
all tasks table

(Bitbake rev: 277d076d777f26f215f01cac59302bcf9cf44a9c)

Signed-off-by: Ravi Chintakunta <ravi.chintakunta@timesys.com>
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-02-17 15:38:52 +00:00
Ravi Chintakunta
6f38844000 bitbake: toaster: Fix builds view filter options to match changes in filter dialog
Filter dialog uses an array of tuples for filter options.Fixed the builds view
by changing the filter options from a dictionary to a list of tuples.

(Bitbake rev: bac1ca6e276b2e276b844d1009dba964810a2957)

Signed-off-by: Ravi Chintakunta <ravi.chintakunta@timesys.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-02-17 15:38:52 +00:00
Ravi Chintakunta
fd86e0e557 bitbake: toaster: Filter Dialog fix to display filter options as radio buttons
- Filter options are displayed as radio buttons in the filter dialog.
- To preserve the order of the filter options, the options are passed
as tuples inside a list, instead of key/value pairs of a dictionary. Changed
the filter dialog code to use the tuple

(Bitbake rev: 5e475b312348fa45312946b528648739c78da2ba)

Signed-off-by: Ravi Chintakunta <ravi.chintakunta@timesys.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-02-17 15:38:52 +00:00
Ravi Chintakunta
2b2d3d618f bitbake: toaster: Removed unused 'disabled' css class in Filter dialog
(Bitbake rev: 8cd978ec3cc56e3f30d58d17188e67296bdb120b)

Signed-off-by: Ravi Chintakunta <ravi.chintakunta@timesys.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-02-17 15:38:51 +00:00
Paul Eggleton
0008cd6444 bitbake: build: fix handling of task override for tasks with underscores in their names
Tasks whose names contain underscores (such as do_populate_sdk in OE)
when converted to a task override do not function properly. If we
replace underscores with hyphens we can still have a working override
for these tasks.

(Bitbake rev: cf90bd6b2a0ab7dce922bffb500d6a2ff2ff10e2)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-02-17 15:34:45 +00:00
Paul Eggleton
a56bd812a7 bitbake: parse: make vars_from_file return None for non-recipes
It doesn't really make sense to set PN from .conf files, for example.
More concretely, this avoids the config hash changing unnecessarily
within Hob due to PN effectively changing (since bblayers.conf is
parsed first and then .hob.conf).

(Bitbake rev: 22e03ef7ac9bb6b7245250347ae9c10c19f1d74e)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-02-17 15:34:45 +00:00
Richard Purdie
3221d97323 bitbake: runqueue: Fix silly variable overlap
A previous commit of mine used the target variable for two different uses
resulting in a lot more sstate being installed than is needed.

Fix the variable to use two different names and unbreak the setscene
behaviour.

(Bitbake rev: f975ca2cf728561bd6317ed8f76303598546113a)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-02-15 14:02:08 +00:00
Richard Purdie
b60ed2d0fd bitbake: runqueue: Ensure we do run 'masked' setscene tasks if specified as targets
If you specify multiple targets on bitbake's commandline and some of them are
setscene tasks which are "masked" by other tasks they may not get run.

For example <image>:do_rootfs <kernel>:do_populate_sysroot

the rootfs tasks "masks" the populate_sysroot task so bitbake would currently
decide not to run it. In this case, we do really want it to be run.

The fix is not to skip anything which has been given as an explict target.

(Bitbake rev: 0753899d1e855795cc18671357609a86f169b379)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-02-13 17:57:20 +00:00
Scott Rifenbark
9c61b6392c bitbake: user-manual-metadata.xml: Edits to the "Sharing Functionality" section.
Applied some review edits from Paul to the section.

(Bitbake rev: 56321b18808f5ed932543d907b9ebcfbf4420233)

Signed-off-by: Scott Rifenbark <scott.m.rifenbark@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-02-11 12:17:06 +00:00
Scott Rifenbark
d110f55a94 bitbake: user-manual-metadata.xml: Edits to "Conditional Syntax (Overrides)"
Re-wrote this section to use clearer more described examples.

(Bitbake rev: 6eea23c4783c591c2d2c7f0b2a98e7a0cc8aa3c3)

Signed-off-by: Scott Rifenbark <scott.m.rifenbark@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-02-11 12:17:06 +00:00
Scott Rifenbark
decc756e89 bitbake: user-manual-metadata.xml: Edits to "Defining Pure Python Functions"
Made a review edit here to the section per Paul Eggleton.

(Bitbake rev: 3b6ab095862367bc0101b995b002f62e7968ccf3)

Signed-off-by: Scott Rifenbark <scott.m.rifenbark@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-02-11 12:17:06 +00:00
Scott Rifenbark
2fc9281c49 bitbake: user-manual-metadata.xml: Edits through syntax section
I made some general improvements in the "Overview" and
"Basic Syntax" sections.  Additionally, I added a blank
section for "Variable Flags" that will eventually hold general
information on this concept.  Finally, come review edits to the
"Defining Pure Python Functions" section per Paul Eggleton.

(Bitbake rev: 665d655f436f1a353f5fe467c5d97588f7b121c5)

Signed-off-by: Scott Rifenbark <scott.m.rifenbark@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-02-11 12:17:06 +00:00
Scott Rifenbark
a2e5746f29 bitbake: user-manual-intro.xml: General edits to Introduction chapter.
I performed a general edit to this chapter.  Some significant changes
include changing the chapter's title to "Overview" when it was titled
"BitBake User Manual", doing some consolidation of text to eliminate
a couple sections that described methods to obtain a copy of BitBake,
and various improvements as needed.

(Bitbake rev: f635c4b36af79b8572095083a392fb58c11198c9)

Signed-off-by: Scott Rifenbark <scott.m.rifenbark@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-02-11 12:17:06 +00:00
Scott Rifenbark
3cdf800334 bitbake: user-manual-metadata.xml: Added "Checksums (Signatures)" section.
Added this section to the end of the Metadata chapter.

(Bitbake rev: b653c58284cafd0b79991520543ca6239705d36b)

Signed-off-by: Scott Rifenbark <scott.m.rifenbark@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-02-11 12:17:06 +00:00
Scott Rifenbark
89058e1ef7 bitbake: user-manual-ref-variables.xml: Review edits to several variables in glossary.
ASSUME_PROVIDED
BBCLASSEXTEND
SRC_URI
PACKAGES_DYNAMIC
BB_NUMBER_THREADS
BB_DANGLINGAPPENDS_WARNONLY

(Bitbake rev: 8e586ccee6d5e78070d28cda67058578e1fe91d7)

Signed-off-by: Scott Rifenbark <scott.m.rifenbark@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-02-11 12:17:06 +00:00
Scott Rifenbark
ddcd91c6d6 bitbake: user-manual-ref-variables.xml: Added 11 new variables to glossary.
SRCREV_FORMAT
BUILDNAME
BB_ENV_WHITELIST
BB_PRESERVE_ENV
BB_ENV_EXTRAWHITE
BB_ORIGENV
BB_TASKHASH
BITBAKE_UI
BBDUBUG
BB_DEFAULT_TASK
BBINCLUDED

(Bitbake rev: 0e3106bfdebfa255b2db08a8a4585516d8bebe20)

Signed-off-by: Scott Rifenbark <scott.m.rifenbark@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-02-11 12:17:05 +00:00
Scott Rifenbark
081d31c6bc bitbake: user-manual-ref-variables.xml: Edits to FILESPATH.
(Bitbake rev: 64bdd3ba6fe90ed6e0338ad053c8117db0acfe2e)

Signed-off-by: Scott Rifenbark <scott.m.rifenbark@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-02-11 12:17:05 +00:00
Scott Rifenbark
825366d7f0 bitbake: user-manual-ref-variables.xml: Added 6 new variables to glossary:
MULTI_PROVIDER_WHITELIST
CACHE
PERSISTENT_DIR
FETCHCMD
FILE
FILESDIR

(Bitbake rev: 4aa8c2f93cc126778f7e99229d99b398d603fcac)

Signed-off-by: Scott Rifenbark <scott.m.rifenbark@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-02-11 12:17:05 +00:00
Scott Rifenbark
5f59a85511 bitbake: user-manual-ref-variables.xml: Edits to PREFERRED_PROVIDER.
(Bitbake rev: 9232cca08ec3f099475ad8dfc10e4fa3c2b39366)

Signed-off-by: Scott Rifenbark <scott.m.rifenbark@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-02-11 12:17:05 +00:00
Scott Rifenbark
658d885c0c bitbake: user-manual-ref-variables.xml: Added PREFERRED_PROVIDERS to glossary.
(Bitbake rev: 86cacad8129502088011ebb622dd4d3d3663d39e)

Signed-off-by: Scott Rifenbark <scott.m.rifenbark@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-02-11 12:17:05 +00:00
Scott Rifenbark
dac581dbd2 bitbake: user-manual-ref-variables.xml: Added FAKEROOT* variables to glossary.
FAKEROOT
FAKEROOTCMD
FAKEROOTBASEENV
FAKEROOTDIRS
FAKEROOTENV
FAKEROOTNOENV

(Bitbake rev: 8480db0fa7d0363741e7075bb8169a9c1b6a71dd)

Signed-off-by: Scott Rifenbark <scott.m.rifenbark@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-02-11 12:17:05 +00:00
Scott Rifenbark
606e2cb517 bitbake: user-manual-ref-variables.xml: Added BB* variables to glossary.
BB_STRICT_CHECKSUM
BB_WORKERCONTEXT
BB_SIGNATURE_EXCLUDE_FLAGS
BB_HASHBASE_WHITELIST
BB_INVALIDCONF
BB_LOGFMT
BB_RUNFMT
BB_RUNTASK
BB_HASHBASE_WHITELIST
BB_STAMP_POLICY
BB_STAMP_WHITELIST
BB_SCHEDULER
BB_SCHEDULERS
BB_SETSCENE_DEPVALID
BB_SETSCENE_VERIFY_FUNCTION
BB_SIGNATURE_HANDLER
BB_SRCREV_POLICY
BB_VERBOSE_LOGS

(Bitbake rev: f02be6ca79fccc3c7bfbbadeef0c98242f661524)

Signed-off-by: Scott Rifenbark <scott.m.rifenbark@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-02-11 12:17:05 +00:00
Richard Purdie
410a3e1574 bitbake: runqueue: Fix setscene hard dependency problems
Commit c54e738e2b5dc0d8e6fd8e93b284ed96e7a83051 added in the idea of hard dependencies
such as the case a setscene has a hard dependency on pseudo-native and that
dependency wasn't available from sstate for some reason.

Unfortunately the implementation was a bit too enthusiastic, causing rebuilds
of things when it wasn't necessary. A test case was:

bitbake quilt-native
bitbake quilt-native -c clean
bitbake <some-image>

and then you'd watch quilt-native get rebuilt for no good reason.

The clue to the problem is in the for loop where it never depends on
the item being iterated over.

The fix is to include the exact list of hard dependencies rather than
guessing. With these changes, the use case above works, the one in
the original commit also works.

This patch also adds in or cleans up various pieces of logging to
allow issues like this to be more easily debugged in future.

(Bitbake rev: 81bd475585ff1b44b390036b1eca0feae7c149eb)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-02-11 11:53:45 +00:00
Laurentiu Palcu
095bb006c3 bitbake: bitbake: build.py: create separate function for shell trap creation code
Currently, the shell trap code was created in exec_func_shell(). Split
the function so that we can create the code separately.

Also, some whitespaces were automatically deleted by my editor. Since
this is not necessarily a bad thing, leave these changes too.

(Bitbake rev: c712e622d20c61a07c9c172b60e9dc6beae14197)

Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-02-09 10:01:38 +00:00
Richard Purdie
707cdcf905 bitbake: data: Account for pre/postfunc functions when calculating dependencies
pre/postfuncs were not being added to checksums. This meant that when reconfiguration
occurred, tasks were not always being rerun when they should. This include
sstate functions as well as systemd's do_install function in the OE metadata.

With the addition of postfuncs, its possible a shell task can have a python
pre/postfunc so we have to guard against this when generating shell output
in emit_func.

(Bitbake rev: b84d010144de687667cf855ddcb41c9b863c236e)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-02-04 22:03:48 +00:00
Richard Purdie
d93620baea bitbake: runqueue: Fix race against tasks sharing stamp files
Shared work directories work by assuming bitbake will not run
more than one task with a specific stamp name. Recent runqueue optimisations
accidentally broke this meaning there could be races. This fixes the code.

(Bitbake rev: b1628b1a260ddf43fc9985535b1ddcfcebbb1e5b)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-01-31 15:52:17 +00:00
Richard Purdie
bfa46b6872 bitbake: user-manual: Fix issue with previous revert
(Bitbake rev: 2d052690d46506ba3f4180ad5cdfa835ce85e59a)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-01-28 15:29:50 +00:00
Richard Purdie
14778a858e bitbake: Revert "usermanual: Updated CC License to version 3.0"
This reverts commit 0546bc19557de9263b448ceb1707884543d2de56.

This change was not discussed with the copyright holders so the license
needs to remain at version 2.5.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-01-28 14:29:01 +00:00
Robert Yang
d955d6a8f5 bitbake: siggen.py: fix the SignatureGenerator()
If we set:

BB_SIGNATURE_HANDLER = "noop"

Then we would get the following errors:

[snip]
File "runqueue.py", line 876, in RunQueue._start_worker(fakeroot=False, rqexec=None):
                "fakerootnoenv" : self.rqdata.dataCache.fakerootnoenv,
   >            "hashes" : bb.parse.siggen.taskhash,
                "hash_deps" : bb.parse.siggen.runtaskdeps,
AttributeError: 'SignatureGenerator' object has no attribute 'taskhash'
[snip]

This patch fixes the problem.

[YOCTO #5741]

(Bitbake rev: 2bfcb751891cf3b4050e996b3c8e28678c3a8bf4)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-01-28 00:52:57 +00:00
Richard Purdie
30bfec03e2 bitbake: runqueue: Simplify pointless len() usage
(Bitbake rev: 1f2bdd1b99075babe8dba91478cfc5d3501676cb)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-01-28 00:52:57 +00:00
Irina Patru
9a46332aa0 bitbake: hob: No need for enable_proxy and same_proxy in conf file
Hob doesn't read these variable from file and bitbake doesn't use them,
so they shouldn't be set in conf file.

(Bitbake rev: a8c9df86b96e27dc49028c2da42034d13988960c)

Signed-off-by: Irina Patru <irina.patru@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-01-28 00:52:57 +00:00
Irina Patru
914e50e5ea bitbake: hob: Don't always save proxy values in conf file
If enable_proxy is not activated, Hob should not save the proxy values
in the conf file when user hits save button.

[ HOB #5308 ]

(Bitbake rev: fbe0851221ecfcefea5bdd4b629a05ed4f5ac189)

Signed-off-by: Irina Patru <irina.patru@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-01-28 00:52:57 +00:00
Irina Patru
d24611228e bitbake: hob: don't display interruptions as fails
When Hob receives a bb.command.CommandFailed event, it should check if
it's a log kind of information.
"Forced shutdown" and "Stopped build" are messages that show when a build
is not complete, but Hob considered them error.

[HOB #5609]

(Bitbake rev: ea1939f7ec8d8a71ce16a60c251c2413d7d91eb3)

Signed-off-by: Irina Patru <irina.patru@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-01-28 00:52:57 +00:00
Irina Patru
1b6e11cdcb bitbake: hob: unsetting busy cursor after hitting Stop button
The busy cursor would never change after pressing Stop button.
It should be set after the possible return inside machine_combo_changed_cb()
method.

(Bitbake rev: d440d3ad4b2d99bc20e06d2d5f5e76d07864dff3)

Signed-off-by: Irina Patru <irina.patru@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-01-28 00:52:57 +00:00
Irina Patru
14e548dbb7 bitbake: bitbake: hob: check if parser has attribute 'shutdown'
It must be checked first if parser has the attribute 'shutdown' when
user hits Stop button and the forceshutdown state is given.

[HOB #5579]

(Bitbake rev: 46943b442ea4fa778f70590b6dcce483595efaf8)

Signed-off-by: Irina Patru <irina.patru@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-01-28 00:52:57 +00:00
Irina Patru
f43c2d8d79 bitbake: hob: change error_msg for CommandFailed event
When a bb.command.CommandFailed event is received by Hob, the error
message is stored inside event.error.
This information tells exactly why bitbake failed, so Hob should display
it instead of the current composed message.

(Bitbake rev: 24543ff6b45771712d624541ae35738d7d98f33c)

Signed-off-by: Irina Patru <irina.patru@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-01-28 00:52:56 +00:00
Richard Purdie
3c5b68356d bitbake: cooker: Add option of specifying indvidual tasks for targets
Currently its near enough impossible to tell bitbake to run target X,
task Y and target A, task B. We could hack various parts of the API
around but it would mean incompatibilities.

An alternative is to accept the syntax "<target>:do_<task>" as a target
name. The default task would be used where the task is unspecified.

This has the advantage that its neat/clean code and works from all
current APIs including the commandline.

(Bitbake rev: 55f6bee3114e582333a1784caeddb197b9163d02)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-01-28 00:52:56 +00:00
Scott Rifenbark
5280192555 bitbake: user-manual-ref-variables.xml: Added 10 new variables to the glossary.
Fixes [YOCTO #5472]

1. ASSUME_PROVIDED
2. BB_CURRENTTASK
3. BB_FETCH_PREMIRRORONLY
4. BB_FILENAME
5. BB_NICE_LEVEL
6. BB_HASHCONFIG_WHITELIST
7. BB_TASK_NICE_LEVEL
8. BB_NO_NETWORK
9. BB_NUMBER_PARSE_THREADS
10. STAMPCLEAN

(Bitbake rev: 72a01a54c4786a7a82ae8b86b8e2beedd07f7f21)

Signed-off-by: Scott Rifenbark <scott.m.rifenbark@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-01-27 21:03:26 +00:00
Scott Rifenbark
6201d18b79 bitbake: user-manual-ref-variables.xml: Fixed references to "Build Directory".
I made sure that these are lower-cased.  Also, removed the tags
for cross-referencing into the term section of the YP Development
Manual.

(Bitbake rev: f9e3de2bb3e73204ef35d102ff26ee7393056ede)

Signed-off-by: Scott Rifenbark <scott.m.rifenbark@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-01-27 21:03:26 +00:00
Scott Rifenbark
37ab0888a1 bitbake: user-manual-ref-variables.xml: Editing pass.
Corrected grammar and applied active voice where possible.
Also removed several cross-reference tags to the YP manual set.

(Bitbake rev: 17cbad436c97e904a04596237022e84853b10a21)

Signed-off-by: Scott Rifenbark <scott.m.rifenbark@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-01-27 21:03:26 +00:00
Scott Rifenbark
d51fb535a6 bitbake: template/titlepage.templates.xml: Suppress Title Text.
I commented out two blocks of code that are responsible
for getting the book's title to format onto the title page.
Commenting the code out suppresses the title from the
PDF version's title page yet retains the title text for the
HTML tab space of the browser.  The reason this is necessary
is because the BitBake User Manual uses an image file for
the title.  Thus, it is not necessary to print the title
again.

(Bitbake rev: d7f15880c5423d91fd786e291f3e062545342184)

Signed-off-by: Scott Rifenbark <scott.m.rifenbark@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-01-27 21:03:25 +00:00
Scott Rifenbark
4fb4928504 bitbake: user-manual-customization.xsl: Updated param list to pass
I added five parameters here to match the build process used
for the YP manual set.

(Bitbake rev: f95994be26986098c2603ef6d4cb10f06422b790)

Signed-off-by: Scott Rifenbark <scott.m.rifenbark@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-01-27 21:03:25 +00:00
Scott Rifenbark
9dcd60395e bitbake: user-manual-metadata.xml: Renamed section tag
There were two section tags that used the same id name "classes".
One was in the intro chapter and one was in the metadata chapter.
This was causing an exception in the PDF creation process and
breaking it.  I renamed the tag in the metadata chapter
"metadata-classes".

(Bitbake rev: d5f405586f7c50c602241519d32d02b1c7b1f345)

Signed-off-by: Scott Rifenbark <scott.m.rifenbark@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-01-27 21:03:25 +00:00
Scott Rifenbark
e501280162 bitbake: user-manual-ref-variavbles.xml: Added new glossary chapter.
Added a scrubbed copy of the YP ref-manual glossary.  The content
was scrubbed to contain BB variables only.  Removed broken
cross-references, made sure the PDF file built.

(Bitbake rev: aae6bcb7fb6e056eb7b1027a8054f6ea5f8ab2b2)

Signed-off-by: Scott Rifenbark <scott.m.rifenbark@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-01-27 21:03:25 +00:00
Richard Purdie
bd4140578e bitbake: user-manual-bitbakecommand: Various edits/cleanups
(Bitbake rev: f079e0886b1b191d84621057f9752cc32bfabb44)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-01-27 21:03:24 +00:00
Richard Purdie
5cce95c602 bitbake: user-manual-bitbakecommand: Standardise on section tags, not example
(Bitbake rev: c85a035608b7c862d964f45bb1b767fde072a1de)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-01-27 21:03:24 +00:00
Richard Purdie
2375f55045 bitbake: user-manual-bitbakecommand: Replace screen tags with literallayout
(Bitbake rev: 4fffe48ab607fd0d2f884186006ee168874b8414)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-01-27 21:03:24 +00:00
Richard Purdie
64f66d7821 bitbake: user-manual-bitbakecommand: Reform whitespace
(Bitbake rev: f1a6d7029e7adc147655809dca71782a143c5ac5)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-01-27 21:03:24 +00:00
Richard Purdie
32304baea2 bitbake: user-manual-bitbakecommand: Update help text output with that from bitbake master
(Bitbake rev: bd1da5001c03cba17a4c61cfb854ab394c50e732)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-01-27 21:03:23 +00:00
Richard Purdie
1669f39e47 bitbake: user-manual-fetching: Editing pass over the manual chapter
(Bitbake rev: b0fc79e6405b7070a42cfda035e87edc832f8334)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-01-27 21:03:23 +00:00
Richard Purdie
123b978c82 bitbake: user-manual-fetching: Change varname/emphasis tags to filename
(Bitbake rev: 3b04c18212f3f0c22a1167dc6f62210e54ed2002)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-01-27 21:03:23 +00:00
Richard Purdie
0de491b766 bitbake: user-manual-fetching: Change screen -> literallayout
(Bitbake rev: e0474be77c954cac3bb9b403d6c854ff447cc02d)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-01-27 21:03:23 +00:00
Richard Purdie
d0c16f9467 bitbake: user-manual-fetching: Change the file whitespace
(Bitbake rev: 7e32068d1ab4e1f342f70a8338bb8a51fc783da9)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-01-27 21:03:23 +00:00
Richard Purdie
552d54f2ec bitbake: user-manual-metadata: Clean up task documentation
(Bitbake rev: 55158ce6c5435544a62a60c0055724619bafde27)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-01-27 21:03:22 +00:00
Richard Purdie
ac576f40ac bitbake: user-manual-metadata: Clean up task flag section
(Bitbake rev: b2e8c56d0f554e3b118f23b42858319110501180)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-01-27 21:03:22 +00:00
Richard Purdie
7b99bc04ac bitbake: user-manual-metadata: Expand parsing process docuemtnation
(Bitbake rev: 2596dd2b42c06ef258032356294cc345a6e25fed)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-01-27 21:03:22 +00:00
Richard Purdie
d6b8f746d4 bitbake: user-manual-metadata: Add section about layers
(Bitbake rev: 0a99fe01dc273c548f7f83f9a7695c21184e7331)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-01-27 21:03:22 +00:00
Richard Purdie
76072eaa8e bitbake: user-manual-metadata: Add section about INHERIT
(Bitbake rev: f2ffe0365df7aa9c7d6c57b6bff8a5bbed198bfb)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-01-27 21:03:22 +00:00
Richard Purdie
ab18cca2dc bitbake: user-manual-metadata: Add section about running tasks and the environment
(Bitbake rev: b32524643c125c78848630a5ce18d1df36313bc7)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-01-27 21:03:21 +00:00
Richard Purdie
8705fe2383 bitbake: user-manual-metadata: Rework section about shell/python functions
(Bitbake rev: c2bcb5364ff7c702bc1ec2726169f608b445f979)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-01-27 21:03:21 +00:00
Richard Purdie
908fdb5cbc bitbake: user-manual-metadata: Add section about data store operations
(Bitbake rev: 85ffd05f292386de1b098debb6a3d3fe18c6b6b7)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-01-27 21:03:21 +00:00
Richard Purdie
c676bf8453 bitbake: user-manual-metadata: Add lists of common events
(Bitbake rev: 240d3f0439e7c1f12e06fa738acc20fbcac8ae65)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-01-27 21:03:21 +00:00
Richard Purdie
c8892fc919 bitbake: user-manual-metadata: Editing pass over the manual chapter
(Bitbake rev: 6f326f2f2785d6d48d7753abee6e8162852d8702)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-01-27 21:03:20 +00:00
Richard Purdie
98b41e41ff bitbake: user-manual-metadata: Use filename tags instead of literal and varname tags
(Bitbake rev: 56f8c4ad09c244522d68e203fe4cb76d593a9f2f)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-01-27 21:03:20 +00:00
Richard Purdie
a0f3328679 bitbake: user-manual-metadata: Replace screen tags with literallayout tags
(Bitbake rev: be3967f9f088e414c679e72a947817b60bab82d7)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-01-27 21:03:20 +00:00
Richard Purdie
0e90d0f1a4 bitbake: user-manual-metadata: Update whitespace (no content changes)
(Bitbake rev: 7eb1e340321ab4a5baa23e83eebf65ba13d23aef)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-01-27 21:03:20 +00:00
Richard Purdie
4dd8a0d6d6 bitbake: user-manual-metadata: Reorder sections to more logical order
(Bitbake rev: 6f2bed62bde5cd20f91c336b158f60f4a6bcb82f)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-01-27 21:03:20 +00:00
Richard Purdie
633db4f0ff bitbake: user-manual-metadata: Add section ids for most sections
(Bitbake rev: 4ad94fb161d5fe24869696187e01004bff61d569)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-01-27 21:03:19 +00:00
Scott Rifenbark
437791a23d bitbake: user-manual-hello.xml: Added new chapter for "Hello World Example"
This file was evidently a "working" file and not included in the
manual at the point Bill left off.  The wmat branch, however, had
a load of commits dedicated to this file.  Rather than attempt to
replay them all one-by-one, I simply copied the file from the
wmat branch and hand-inserted the changes to make it equal to what
was there.  Note also that I re-formatted the file to have the
same formatting standards I use in the YP manuals.

(Bitbake rev: 9ddbf31ba7d05a596ca53b8ed78d94221850894b)

Signed-off-by: Scott Rifenbark <scott.m.rifenbark@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-01-27 21:03:19 +00:00
Scott Rifenbark
1b08402519 bitbake: user-manual.xml: Cleanup on the title page.
1. Added commented out manual revision history.
2. Cleaned up the author stuff.

(Bitbake rev: df92da67ddbb2bcc672911626b9abd1a168cf436)

Signed-off-by: Scott Rifenbark <scott.m.rifenbark@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-01-27 21:03:19 +00:00
Bill Traynor
3a9593d409 bitbake: user-manual-intro.xml: Re-write of BitBake introduction.
(Bitbake rev: afc998a73f09f3d1a0d7e22425badfa5b7001aa1)

Signed-off-by: Scott Rifenbark <scott.m.rifenbark@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-01-27 21:03:19 +00:00
Richard Purdie
a713dade85 bitbake: user-manual-intro: Various minor cleanups/improvements
(Bitbake rev: 5e22c0c5d4bb9e63142a13618f5f87e53c83d9f2)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-01-27 21:03:19 +00:00
Richard Purdie
5b60a81c95 bitbake: user-manual-intro: Add obtaining bitbake and summary sections
Add new obtaining bitbake and summary sections from Bill Traynor

(Bitbake rev: 083e4e44f52ffdfca68ce6c56eae85ce3b719e1b)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-01-27 21:03:18 +00:00
Scott Rifenbark
e925730cd8 bitbake: user-manual-intro.xml: Added "Concepts" section.
Provided initial text for recipes, configuration files, and
classes.

(Bitbake rev: 55875bcf682979ce538845a8118452425ff96cfc)

Signed-off-by: Scott Rifenbark <scott.m.rifenbark@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-01-27 21:03:18 +00:00
Richard Purdie
2038b85a46 bitbake: user-manual-intro: Update/improve whitespace
(Bitbake rev: fb74a1926c3a603a4c1ebe0325b27fb2c3aec3ee)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-01-27 21:03:18 +00:00
Richard Purdie
2ee774782c bitbake: user-manual-intro: Expand introduction and fix spelling mistakes
Expand the introduction section, fill out the section ids and fix some spelling
mistakes.

Wording from Bill Traynor/Scott Rifenbark

(Bitbake rev: 780f61da6a59c52555de8574093a264d565b2a75)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-01-27 21:03:18 +00:00
Bill Traynor
7c98419b09 bitbake: documentation: Add README based off the YP docs one
Reusing the README file from the Yocto Project Documentation
directory but removed the YP specific content and changed to be
specific to BitBake.

(Bitbake rev: 67b6e9c54676e979c546c0d350b2d01443f13f36)

Signed-off-by: Bill Traynor <wmat@alphatroop.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-01-27 21:03:17 +00:00
Richard Purdie
bfa18b58d3 bitbake: user-manual: Separate out chapters into individual files
Splitting the manual into a chapter per file makes the content
a little more managable.

(Bitbake rev: 020178eba958d2d5142ee0909fc0fd133b97cd92)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-01-27 21:01:08 +00:00
Bill Traynor
034e49004f bitbake: user-manual: Import YP Docs templates for usermanual improvements
Import the necessary pieces to be able to build the BitBake User
Manual using make in a similar way that the Yocto Documentation is
built.  The Makefile has been edited to remove Yocto Project specific
content and adapt for bitbake's needs.

(Bitbake rev: d4199078692f86341ed5b42a7c2dd4b34819aead)

Signed-off-by: Bill Traynor <wmat@alphatroop.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-01-27 21:01:08 +00:00
Bill Traynor
eeb4c71647 bitbake: usermanual: Renamed manual dir to user-manual to be consistent.
Renaming the manual directory to user-manual in order to allow
for future additions of specific manual types, such as a
dev-manual.

(Bitbake rev: 4617aa5e8491067f7825062ed3baee27cf697e88)

Signed-off-by: Bill Traynor <wmat@alphatroop.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-01-27 21:01:08 +00:00
Bill Traynor
afac3d83bd bitbake: usermanual: Updated CC License to version 3.0
Updated the Creative Commons License to the most current version,
version 3.0.  Also updated the Creative Commons mailing address.

(Bitbake rev: 0546bc19557de9263b448ceb1707884543d2de56)

Signed-off-by: Bill Traynor <wmat@alphatroop.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-01-27 21:01:07 +00:00
Alexandru DAMIAN
e5b3918bb9 bitbake: toaster: fix package data gathering
Under OE-Core, the name under which a package would
be installed in a target may have been different than the
name under it has been built or recorded in the dependencies
listings.

This patch addresses the way that Toaster records package
names, and adds the field of "installed_name" to save the
name under which a package have been installed in an image.

(Bitbake rev: 24e0367429b248108b104ab5a2af05efcf7a8c39)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-01-27 21:01:07 +00:00
Alexandru DAMIAN
9677275d0d bitbake: toaster: fix builds page CSS and functionality
This patch fixes a set of CSS and functionality problems
with the build list page:

* Fix Recent Builds text styles
* Added proper links from Failed Tasks entries.
* Always Search returns to the first page of results.
* Clear search button appears only if search is active.
* Search shows the number of object, proper no objects found
* Various smaller fixes.

(Bitbake rev: 9164948e387a726f318f723f63e8d93435d7afe6)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-01-27 21:01:07 +00:00
Barros Pena, Belen
594eeeba71 bitbake: toaster: Set .error styles for tables
Override the default styles of Twitter Bootstrap for table rows
with the .error class applied, and ensure that table cells and
anchor tags inherit the .error styles when their table row has
that class applied.

(Bitbake rev: 8b44955bb836ccad384718247ceb08d713ebc152)

Signed-off-by: Belen Barros <belen.barros.pena@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-01-27 21:01:07 +00:00
Ravi Chintakunta
47d19237ef bitbake: toaster: All Tasks Feature with sorting and filtering
- Fixed the all tasks view and template  to match the UI design and use
the new basetable code.

- Added a method to views to add sort order icon to the view context.

- Default sort order when the page is loaded is displayed with the sort icon

- Filtering of columns

(Bitbake rev: b2f8de082c3ae41eb44e6ccdc283849b64d0b0f2)

Signed-off-by: Ravi Chintakunta <ravi.chintakunta@timesys.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-01-27 21:01:06 +00:00
David Reyna
84b239086d bitbake: toaster: Implementation of recipe detail views
Add the new recipe detail page and update the view context accordingly.
Rename the recipe summary page to 'recipes.html' and add the respective
links to the recipe details page.

The views are based on specifications found in attachments to
	https://bugzilla.yoctoproject.org/show_bug.cgi?id=4299

[YOCTO #4299]

(Bitbake rev: d561000b6c4927ef6ec269e9ab7c70249b3c344a)

Signed-off-by: David Reyna <david.reyna@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-01-27 21:01:06 +00:00
David Reyna
b45defd888 bitbake: toaster: Implementation of base build details page
This page is the standard base page for 'details' views. It is
based on 'basebuildpage.html' but does not include the build's
quick-link sidebar.

(Bitbake rev: 2d7e9a6f8464783e165804974a7d94b544c9a43f)

Signed-off-by: David Reyna <david.reyna@windriver.com>
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-01-27 21:01:06 +00:00
Ravi Chintakunta
05a684f6e5 bitbake: toaster: Added custom filter tags for use in templates.
- custom filter tag to return the css class based on
  the task execution status and execution outcome

- custom filters for active filter icon and tooltip text

- custom filter for displaying blank for None, zero, '0' and
  'Not Applicable'

(Bitbake rev: 1e9253984e6f107c6eed1c3b9df3a444076e2989)

Signed-off-by: Ravi Chintakunta <ravi.chintakunta@timesys.com>
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-01-27 21:01:06 +00:00
Ravi Chintakunta
87776ca32e bitbake: toaster: Base table fixes for header text, sort icon and filter icon
- Display the table column header as a link only if it is sortable.
   Non-sortable column headers are displayed as plain text.

- Display the sort order by an up / down arrow icon next to the
   column header

- Add style for header in default.css

- Set tooltip for the active filter icon

- Pass the view name to the filter dialog

(Bitbake rev: 53ede15926d45b555252d77919a0568a984c6d74)

Signed-off-by: Ravi Chintakunta <ravi.chintakunta@timesys.com>
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-01-27 21:01:05 +00:00
Ravi Chintakunta
4acdf8aaf1 bitbake: toaster: Base html fix to set the page title
Set title of the page if the context includes objectname.

(Bitbake rev: 8bfd140b93bcbfe8be4dd8a30268d3cc033e180f)

Signed-off-by: Ravi Chintakunta <ravi.chintakunta@timesys.com>
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-01-27 21:01:05 +00:00
Alexandru DAMIAN
dfd7a8a27a bitbake: toaster: settings remove duplicate entry
The 'toastergui' application had duplicate entries,
so this patch removes the duplicate ones.

Based on an original patch from Ravi Chintakunta <ravi.chintakunta@timesys.com>.

(Bitbake rev: 86d3a44e81aa3dbff947330dfaeff6040bbd625f)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-01-27 21:01:05 +00:00
Andreea Proca
baaab36f41 bitbake: toaster: Add initial tests
This adds the initial api tests for toaster, using Django's unittest modules.

(Bitbake rev: daf9a61fbf69a46b7afd781a6175b05b05fd452f)

Signed-off-by: Andreea Proca <andreea.b.proca@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-01-27 21:01:05 +00:00
Alexandru DAMIAN
b0b1acbe62 bitbake: toaster: Toaster GUI Build and Dashboard pages fixes
THis is a large set of fixes for the generic table, Build and
Dashboard pages.

Among the fixes:
* the table remembers which columns to show across refreshes,
based on saving the settings in a cookie
* added column timespent for a build which is  a denormalization
of the completed_on - started_on information due to limits in
computing datetime differences in the SQL engine
* fixed formatting of the time differences
* various sorting header links fixed
* correct error and warning CSS classes applied to the
respective rows
* fixes multiple divide-by-zero error in displaying duration
estimations

(Bitbake rev: 61e3dee55ac577fce1c0ae0fe7e0d3cf644e8ae6)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-01-27 21:01:04 +00:00
Alexandru DAMIAN
d27c7f26d3 bitbake: toaster: add jquery cookie plugin
We add the jquery cookie plugin to allow us to save
and read local client date in cookie.

jquery-cookie using version 1.4.0 under MIT licence.

(Bitbake rev: 8ce1668dd93269add878f8ea69deb8b53f3bca8b)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-01-27 21:01:04 +00:00
Alexandru DAMIAN
15e52903fa bitbake: toaster: add vim modelines for .py files
No-op patch that adds vim modelines for all .py files
intended to be user-edited.

(Bitbake rev: 73271a7c6f1913c68a4b39ab86414f44acc04776)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-01-27 21:01:04 +00:00
Alexandru DAMIAN
731a0ac2c8 bitbake: toaster: update Task classification fields
This patch updates the task classification fields (outcome, etc)
as to

* Changes outcome names from SSTATE to CACHED and
from EXISTING to PREBUILT

* NoExec tasks now recorded as Not Executed / script type NA instead
of Executed / script type NOEXEC. Script type NOEXEC is deleted.

* SetScene tasks do not get order numbers

* New task method that returns a QuerySet for setscene tasks related
to this task: Task.get_related_setscene()

* New custom TaskManager that allows searching for setscene tasks
related to a certain task: Task.objects.related_setscene(task)

(Bitbake rev: a4164821a142f8b625a5fdc209adc6dc80874241)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-01-27 21:01:03 +00:00
Alexandru DAMIAN
3e1dbed338 bitbake: toaster: update Simple UI references to avoid conflict
Since the Toaster GUI was cloned from the Simple UI, we need
to update the URL pattern names in Simple UI to
prevent conflict when determining the reverse URL path.

(Bitbake rev: 6ed1a28eb0e52a6de83a37664ff6f4418ce84ee4)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-01-27 21:01:03 +00:00
Richard Purdie
43a3f47189 bitbake: runqueue: Only attempt to print closest matching task if there is a match
(Bitbake rev: 1dbf400c662354b7826b2b97ee2e3e6d11af9fd2)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-01-27 11:14:44 +00:00
Richard Purdie
9a4855bc7f bitbake: fetch2/wget: Check downloaded file isn't zero size
I can't think of a reason we'd download zero sized files however there are
reasons zero length files can accidently make it onto source mirrors.

This check allows us to ignore the broken files and switch to another
mirror rather than fail with odd checksum failures.

(Bitbake rev: 300cba2e1a720dba4b83b0c76208ea93c608c1de)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-01-27 11:14:44 +00:00
Richard Purdie
c81d3ad1e3 bitbake: fetch2/git: Dereference unresolved tags with ls-remote
We need to deference tags when trying to map them to commit IDs with
ls-remote. If we don't do this, a given commit might not show up
later in a specific branch. There appears to be no good reason not
to do this.

(Bitbake rev: 8ef24f4c834298348172b96ec0b855bf09552b09)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-01-21 10:20:14 +00:00
Richard Purdie
d21c1537b7 bitbake: fetch2/git: Anchor names when using ls-remote
When specifying tags, they're searched for unanchored so foo/bar could
match:

refs/heads/abc/foo/bar
refs/heads/xyz/foo/bar
refs/heads/foo/bar

This change anchors the expressions so they are based against heads
or tags (or any other base level tree that has been created).

(Bitbake rev: df2e0972cd1db7abd5ec8b7cb295fb0c42e284a4)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-01-21 10:20:13 +00:00
Richard Purdie
594c2e01e7 bitbake: fetch2: Improve invalid SRCREV error message
The current message can be ambiguous, improve it (and also rename a
variable to clean up the rest of the function).

(Bitbake rev: 0c1bb7c0fce7b0f334311a2893ccb00385fa8d55)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-01-21 10:20:13 +00:00
Richard Purdie
453eb5432d bitbake: fetch2: Sanity check SRCREV matches rev/tag parameter
Add a sanity check so that if some SRCREV is set and a rev parameter is given
to the url, the revision given should match.

Any tag parameter behaves the same as rev. If both are specified, error to
tell the user we're confused rather than do something which may or may not
be what they intended.

Also add some unittests for this.

(Bitbake rev: e82a4ab48991035866da9914c8b75a9bfbc9a7fc)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-01-21 10:20:13 +00:00
Richard Purdie
721d3d6e0a bitbake: fetch2: Clean up srcrev_internal_helper
Currently INVALID and None are checked as incorrect values under different
circumstances. This code standardises those checks to be consistent. We
should phase out the use of "INVALID".

(Bitbake rev: 86ef4e65ce18b71dc69643586bd2aa8f48703171)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-01-21 10:20:13 +00:00
Robert Yang
6dcc4e7ded bitbake: fetch2/__init__.py: let try_mirror_url return correct value
The fetcher will try:

1) PREMIRROR
2) Upstream
3) MIRROR

If it fails to download from the Upstream, but succeeds from the MIRROR,
and ud.localpath != origud.localpath (for example, the git tarball),
then we will get the error (e.g.: xf86-video-omapfb):

ERROR: Function failed: Fetcher failure for URL: 'xxx'. Unable to fetch URL from any source.
ERROR: Logfile of failure stored in: /path/to/log.do_fetch.28024

It should not show the error and let the build go on since it succeeds.
(e.g.: xf86-video-omapfb)

[YOCTO #5686]

(Bitbake rev: c08ca1e4eeb04f78e1354780cf5a4c3855e49572)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-01-19 13:06:21 +00:00
Martin Jansa
5e5a7fc37c bitbake: SignatureGenerator: Add empty implementation for dump_sigs
* bitbake -S throws exception when 'noop' signature handler is used

[YOCTO #5738]

(Bitbake rev: 53352e8d388b7fc4da73f95b93dcc087e76d0426)

Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-01-18 23:29:07 +00:00
Richard Purdie
7b580cd25c bitbake: fetch2/gitsm: Use ud.basecmd instead of hardcoding git
This allows FETCHCMD_git to override the fetcher command as the git fetcher does.

[YOCTO #5717]

(Bitbake rev: 23ab943be3a33077d6ad8be68bba53cd1e2270b4)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-01-18 23:29:07 +00:00
Martin Jansa
a99fa6ded1 bitbake: fetch2: Don't allow '/' in user:pass, fix branch containing '@'
* currently decode_url regexp parses branch=@foo as username so it ends like this:
  - ('git', '', 'foo', 'git.openembedded.org/bitbake;branch=', '', {})
  + ('git', 'git.openembedded.org', '/bitbake', '', '', {'branch': '@foo'})
* http://hg.python.org/cpython/file/2.7/Lib/urlparse.py also assumes
  that there is at least one '/' as separator between netloc and path,
  params, so it looks reasonable to prevent including '/' in username

(Bitbake rev: 2c82742114091cb55055328b54223686816582f2)

Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-01-18 23:29:07 +00:00
Richard Purdie
19c75e92a1 bitbake: siggen: Remove fsync that is causing performance issues
This fsync was added for belt and braces protection for things like sstate
on NFS. To be honest, it probably doesn't buy much, if the rename isn't
atomic, all bets are off anyway and there are bigger issues to worry about.

The issue is that at the end of every task, the dump_sig() code is triggered
to save out information about the task and this was triggering an fsync(fd).

Whilst it may select the file descriptor, on file systems like ext4, it will
require large parts of the journal to be written out so it can have
significant impact. latencytop showed an average fsync() call overhead of about
2s and if that happens for 5000 tasks, the time mounts up. This blocks the next
task execution by that time.

We therefore drop the fsync since in reality its causing problems and is unlikely
to buy much.

(Bitbake rev: 46fd841319479f6079d850b3813e64bd8c2680a3)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-01-18 23:29:07 +00:00
Alexandru DAMIAN
1b636173ca bitbake: toaster: Toaster GUI, generic search, filter and order
This patch implements table searching, filtering and ordering, in a
generic mode reusable for all tables.

The search operates list of fields defined in the corresponding
class for each model, search_allowed_fields.

The search expression and filters are sent through GET requests
using a QuerySet-like input. The inputs are filtered and
validated before usage to prevent inadvertent or malicious use.

Filters and table headers are defined in the views for each table,
and rendered by generic code which is easily modified for various
tables.

The Build table and Configuration table are implemented using this
framework as an example of how it should be used.

    [YOCTO #4249]
    [YOCTO #4254]
    [YOCTO #4255]
    [YOCTO #4256]
    [YOCTO #4257]
    [YOCTO #4259]
    [YOCTO #4260]

(Bitbake rev: 2ca15117e4bbda38cda07511d0ff317273f91528)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-01-10 15:20:26 +00:00
Alexandru DAMIAN
5482409a37 bitbake: toaster: Build dashboard implementation
This patch adds the build dashboard page implementation,
which is the landing page for the Toaster GUI.

Also adds correct links from the main build page
to the various parts of the dashboard.

    [YOCTO #4258]

(Bitbake rev: bf7fbf5c0ee39564d813f82e194242f9d4f73c47)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-01-10 15:20:26 +00:00
Alexandru DAMIAN
2251426ae4 bitbake: toaster: Create the base page navigation structure
Updating the general container pages to use the graphical
design and features from the design phase.

In the process of adapting the Simple UI to the designed
interface, we create all the pages and the navigation
structure for the Toaster GUI.

Views for each page have been added, and the url mapping
has been updated to reflect newly added pages.

The table page has been refactored to be component-oriented
instead of class-oriented in order to facilitate reusage.

Changes are made in different layers of the template
(base, basetable) in order to maximize code reuse among
different pages in the build.

(Bitbake rev: d31f039ae31b77023722c06e66542751536a1362)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-01-10 15:20:25 +00:00
Alexandru DAMIAN
c2fc6ca2fa bitbake: toaster: update static media for Toaster GUI
In the process of making the GUI more usable, we
bring in the static assets used during the design
phase into the application.

Licenses used:

    Bootstrap 2.3.2 under Apache License 2.0
    Font Awesome under SIL OFL 1.1 (GPL compatible) and MIT licence
    GLYPHICONS Halflings released with Bootstrap, CC BY 3.0
    jQuery 2.0.3 under MIT licence
    prettify.js under Apache Licence 2.0

(Bitbake rev: 0b3d2000c0bcd2bb5a3af5d49a514a90f001a7d2)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-01-10 15:20:25 +00:00
Alexandru DAMIAN
47621ecb24 bitbake: toaster: clone Simple UI as base for Toaster GUI
This patch clones the Simple UI to provide the base code for
the development of the Toaster GUI. The clone takes the place
of the application that was reserved for Javascript MVC code.

The templates used for Simple UI are renamed to start with
an "simple_" to prevent name resolution conflict with the
Toaster GUI templates.

Minor changes are made to the settings.py and urls.py in the
toaster main section to account for the newly enabled application.

(Bitbake rev: e2fde84f16da017ba0d71aef6a1fa8e2b9255db4)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-01-10 15:20:24 +00:00
Richard Purdie
6ee5d95317 bitbake: fetch2/gitannex: Fix function arguments to match bitbake master
This code clearly uses an earlier fetcher API. Update it to match master.

(Bitbake rev: e13acb4113ce75226664c3006a9776cc885e860d)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-01-08 15:27:37 +00:00
Richard Purdie
854daab404 bitbake: gitannex: Add missing file from previous commit
(Bitbake rev: e14031fbe4924819ab4c9705eef2bedccae1506c)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-01-08 13:07:48 +00:00
Zhenhua Luo
c4bcaa3f7c bitbake: bitbake: fetch2/git: add description for nobranch
(Bitbake rev: b426740b2ae8245c8cf0f314bf4983b6fff7ecb7)

Signed-off-by: Zhenhua Luo <zhenhua.luo@freescale.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-01-08 12:49:08 +00:00
Otavio Salvador
2bd4477be7 bitbake: bitbake: gitannex.py: Add Git Annex support
This add a Git Annex backend which reuses the Git fetcher code; it
allows managing files with git, without checking the file contents
into git, being useful when dealing with files larger than git can
currently easily handle, whether due to limitations in memory, time,
or disk space.

(Bitbake rev: a61fc4db598e9d13c966712a6a0e4783e19448be)

Signed-off-by: Otavio Salvador <otavio@ossystems.com.br>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-01-08 12:49:08 +00:00
Zhenhua Luo
100c8831a5 bitbake: bitbake: fetch2/git: add nobranch option for SRC_URI to skip SHA validating for branch
For rebased git tree, some commits doesn't exist in any branch, and such commits are
valid in tag, the change is useful for such case.

(Bitbake rev: f594cb9f5a18dd0ab2342f96ffc6dba697b35f65)

Signed-off-by: Zhenhua Luo <zhenhua.luo@freescale.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-01-07 13:42:55 +00:00
Robert Yang
1d3a139016 bitbake: bitbake/lib/bb/build.py: fix the task flags cleandirs
The user manual said:

'cleandirs' - directories which should created before the task runs but should be empty

But it only removes the dir, doesn't create it

[YOCTO #5703]

(Bitbake rev: 0636797d75874ce4577f29011d69c56a4c6b9e89)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-01-07 13:42:55 +00:00
Robert Yang
4722965f82 bitbake: bitbake: usermanual.xml: fix a typo
Fixed:
"should created" -> "should be created"

[YOCTO #5703]

(Bitbake rev: 146318729dbeeee4136bcb4832e6f8f69f9a2489)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-01-07 13:42:55 +00:00
Alexandru DAMIAN
6c294f0a2e bitbake: toaster: add two-stage commit startup logic
Toaster start script lunches multiple process components
of the toaster system.

This patch adds logic into the startup script to
safely fail startup and do proper cleanup on any error
that may happen during system start.

Bitbake needs to return 0 if it will successfully lunches
the server-mode.

(Bitbake rev: f43d284e7ae752049711d8215a6020bee6966d45)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-01-07 13:03:05 +00:00
Robert Yang
f4e85cad1b bitbake: bitbake/lib/bb/cooker.py: remove a duplicated self.data
(Bitbake rev: 34afbdd0fc809b8fb20696aeef3e6a61d6812e16)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-01-03 15:16:42 +00:00
Robert Yang
2c9475edcc bitbake: bitbake: fetcher2: clean(): remove the .patch.done
There was a problem:

$ bitbake xf86-video-omapfb -cfetch && bitbake xf86-video-omapfb -ccleanall

Everything should be removed, but the
0006-omapfb-port-to-new-xserver-video-API.patch.done still exists in the
DL_DIR, this is because the clean() in the fetch2/__init__.py skips
removing the local file, so that it will skip removing the .done.

The local file (file://) isn't needed to be removed since it is not
downloaded into DL_DIR, but the .done should be removed, this patch will
remove the .done, and it doesn't remove anything else since the clean()
in local.py does nothing.

[YOCTO #5687]

(Bitbake rev: 2bc99b9dfa532430a13c39fca4e5ef3a2206b3b8)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-01-02 13:08:41 +00:00
Robert Yang
d8d4b0edb1 bitbake: bitbake: fetcher2: git.py: clean(): remove the .tar.gz.done
There was a problem:

$ bitbake xf86-video-omapfb -cfetch && bitbake xf86-video-omapfb -ccleanall

The git2_git.pingu.fi.xf86-video-omapfb.tar.gz has been removed from the
DL_DIR, but the git2_git.pingu.fi.xf86-video-omapfb.tar.gz.done still exists,
this is because the "open(ud.donestamp, 'w').close()" in try_mirror_url() will
create the git2_git.xxx.tar.gz.done, but no one removes it (the clean() in
fetch2/__init__.py removes the DL_DIR/git2/pkg.done)

This only happens on the git fetcher AFAIK.

[YOCTO #5688]

(Bitbake rev: fb2dc84875eb477661f421b21bc404d4805ce379)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-01-02 13:08:40 +00:00
Robert Yang
d305a05961 bitbake: bitbake: replace 3 spaces with 4
A part of fetch2/__init__.py uses 3 spaces as the indent, I
think that they should be typos.

(Bitbake rev: abafd85e2fcf23cee872e0e9e468898101430f1f)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-01-02 13:08:40 +00:00
Paul Eggleton
005af45191 bitbake: fetch2: avoid printing "no checksum" error message twice
Because of the way we were handling this error, it was printed twice -
once via logger.error() (to avoid the log being printed) and a second
time when the exception gets wrapped in a FuncFailed at a higher level.
Call logger.error() earlier and change the text we send in the
exception to be more brief, so it more closely resembles the behaviour
when there is an invalid checksum.

(Bitbake rev: 46765369d7f76ec7f67b90430131a79eb6a66235)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-12-22 14:29:13 +00:00
Paul Eggleton
1edb126af3 bitbake: fetch2: fail checksum validation if SRC_URI checksums set to ""
We were checking SRC_URI md5sum/sha256sum values against None here, so
if they were set to "" then no error was produced. Since the value is
still effectively unset in this case, this is not the right behaviour;
just check if the value doesn't evaluate to False instead.

(Bitbake rev: 040943a718795c64dc4e604abfcf08b26b7d00e6)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-12-22 14:29:12 +00:00
Richard Purdie
8bbaed40c1 bitbake: Update to version 1.21.1 for master
(Bitbake rev: 4cc6e61fe11eb233bdba7c1bdc110b8cdafa56f8)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-12-20 12:26:06 +00:00
Richard Purdie
0b5476a3fa bitbake: siggen: Fix reversed difference output
The output when comparing siginfo files for dict_diff is reversed and shows
additions when things were removed and vice versa. This patch reverses the operation
so the changes are shown correctly and makes the output less confusing.

(Bitbake rev: 9b4142df36619099670740a5d3bc94e404ab2b56)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-12-20 12:26:06 +00:00
Richard Purdie
eebe65c186 bitbake: runqueue: Further extend bitbake -S output to view signature differences
Based upon the list of difference starting points, we can use the siggen.find_siginfo()
function call and the difference printing code to provide a list of differences
between the current build target and whatever can be obtained from the sstate cache.

(Bitbake rev: 7a77861feb62750ef166d2d1e89ed1f444ca8dc7)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-12-20 12:26:05 +00:00
Richard Purdie
a4457cf939 bitbake: siggen: When printing signatures recursively, limit the output
Currently the code prints all differences. If the task dependencies have changed hash,
we recurse into those and print those differences as well. This leads to a lot
of output. The reality is if the parents changed signature, we might as well just
say that and recurse with no other output since we're much more interested in how
the parents changed in nearly all cases. The changes in the parent are probably
the same ones we'd have printed at each level anyway.

By doing this we focus the output more carefully on the thing the user wants/needs
to see.

(Bitbake rev: 7a17fd6e51a76d3582c357b79f5ef86e1969650c)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-12-20 12:26:05 +00:00
Richard Purdie
6d2f3440bb bitbake: runqueue: Fix data being written into siginfo/sigdata files
The way hash_deps was being generated was different to the way siggen generated
the data internally which lead to seemingly different sigdata/siginfo files
for the same checksum. The -S output correct but the files written during
builds contained superflous data which would look like a difference.

This patch removes the badly duplicated data and uses it from the source
which ensures its consistent.

(Bitbake rev: e6d5e925c402cd2cc7ee034e9de4cc6df8944a34)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-12-20 12:26:05 +00:00
Richard Purdie
848961e624 bitbake: runqueue: Add output for -S option for listing the changepoints compared with an sstate cache
Its useful to understand where the delta starts against an existing sstate cache
for a given target. Adding this to the output of the -S option seems like a
natural fit.

We use the hashvalidate function to figure this out and assume it can find siginfo
files for more than just the setscene tasks.

(Bitbake rev: c18b8450640ebfd55a2b35b112959f9ea3e0a700)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-12-18 17:08:08 +00:00
Olof Johansson
52ef6c210c bitbake: bb.fetch2.git: reuse basecmd attribute
The basecmd is initialized in urldata_init; there's no need redoing that
work.

(Bitbake rev: f8df6f746fb2e27f029a5449cee6c891b1f36f4f)

Signed-off-by: Olof Johansson <olof.johansson@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-12-18 17:08:08 +00:00
Richard Purdie
498de04737 bitbake: fetch2/git: Improve handling of unresolved names verses branches
Currently the fetcher doesn't distinguish between names that the fetcher
needs to resolve verses branch names that the user specified.

This meant that if you specify a tag and a branch, the fetcher broke. This
separates the two so that the branch name is preserved and can be used in
appropriate places.

(Bitbake rev: e85f39fe9d1b224414b5da0780da514f75c5df92)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-12-18 17:08:08 +00:00
Richard Purdie
ba82035412 bitbake: build/ast: Create strong task add/del API in bb.build
Currently its near impossible to control task addition/deletion from
metadata context. This adds stong add/deltask API to bb.build
which is traditionally where it resided. The rather broken
remove_tasks function was removed, it didn't appear to do anything
useful or have any users.

This allows us to clean up hacks currently in use in metadata and use
standard API for it instead.

(Bitbake rev: bf7138dd38fc1f8efca80891198e3422fef64093)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-12-18 17:08:07 +00:00
Richard Purdie
8a82a3835c bitbake: ast/BBHandler/build: Add support for removing tasks (deltask)
Back in the depths of time we did support task removal. In the pre
AST days it was nearly impossible to continue supporting it, it wasn't
used so it was dropped. With the modern codebase we can easily now support
deltask and it would be very useful within the metadata since it can
massively simplify dependency trees.

As an example, a core-image-sato had 47703 inter task dependencies before
this patch and a patch to native.bbclass, afterwards with the noexec tasks
deleted, we had 29883. Such a significant simplification is worthwhile
and justifies adding a deltask operation to the system.

(Bitbake rev: acecbde6fb70ff3c96deab3cdf819d8442e87ed4)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-12-18 17:08:07 +00:00
Richard Purdie
bf71844ce5 bitbake: data_smart: Fix hash corruption issue
We were accidentally using references to sets in the contains functionality
instead of creating a copy. This could cause data corruption and corruption
of the resulting sstate checksums.

This patch fixes this to make a copy of the set and resolved the corruption
issue.

(Bitbake rev: 8f4733257ad665aa7c7e7061c543379d5e4e3af2)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-12-13 12:07:48 +00:00
Richard Purdie
d754ddcec7 bitbake: process: Add timeout to select call
On some machines, a hang has been noticed where the system sits in
the select call despite the task having completed.

The exact reasons for this as unknown but adding a timeout unblocked
the builds and resolved the hangs in question.

(Bitbake rev: 5223ffb5b6a46d8b3f6ac3362bd2672e2edf2691)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-12-10 17:55:29 +00:00
Cristiana Voicu
e3b9923921 bitbake: hob: after adding a layer, hob do not parse the configuration
Moved the parseConfiguration method before obtaining the machines and
other configurations from bitbake. If not, Hob doesn't see the new machines
added by the new layer.

[YOCTO #5632]
(Bitbake rev: 8de14b2a481d61424eb32fd0234f7a38a961a75b)

Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-12-10 11:16:14 +00:00
Alexandru DAMIAN
3f611895c7 bitbake: toaster: Add more information for tasks in Simple UI
This patch adds more information about Tasks in the Simple UI:
* all local file system information is not listed in a single
column, showing the common relationship
* adding the display for the location of the task source
* we display the work directory for each task

(Bitbake rev: b102af0b7ebd2f06d6352b834ea083e959c21b0f)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-12-10 11:16:14 +00:00
Alexandru DAMIAN
192583a91e bitbake: toaster: add pagination to the Simple UI
In an effort to make the Simple UI more usable and
reponsive, this patch adds pagination support for the
pages with lots of entries: Builds, Configuration and
Tasks.

(Bitbake rev: d4f075c050ad9ecebe750420d49961a7f30d090b)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-12-10 11:16:14 +00:00
Alexandru DAMIAN
415ebb738d bitbake: toaster: migrate orm models to South
As Toaster advances, database schema alteration
will force users to delete old versions and lose
all data collected.

In order to prevent this, and to allow database
updates to happen without having to delete old data,
we use South to handle migrations for the ORM
application which stores the Toaster data.

    [YOCTO #5559]

(Bitbake rev: 6936faed8d94f3a2ab4055049cd27d02d8229003)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-12-10 11:16:14 +00:00
Alexandru DAMIAN
81ebce0778 bitbake: toaster: update to Django 1.5
In order to remain up to date with the relevant technologies,
Toaster is updated with this patch to Django 1.5. This also
makes headways to allow usage of emerging
Django-related technologies.

Changes include the startup script Django version check, usage
of TemplateView instead of deprecated simple function to do
redirects, and update to the new form of the _url_ template tag.

Support for Django 1.4.5 is now deprecated.

    [YOCTO #5558]

(Bitbake rev: 2d37a1731a2b681bc976f3f391d65abb7745b6f9)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-12-10 11:16:13 +00:00
Alexandru DAMIAN
5ab8545c4f bitbake: toasterui: do not link non-image targets
In the Simple UI, builds table, targets that are images
have link to the list of installed packages.

There is no point in having links enabled for the non-image
targets, so we don't link in this case.

    [YOCTO #5366]

(Bitbake rev: 5839e5b0af45d4c9e05145b16c4ed5817e152606)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-12-10 11:16:13 +00:00
Alexandru DAMIAN
bb70ade1c3 bitbake: toasterui: fix version numbers for no PE specified
PE is an optional field in recipes specifiying the epoch
for the recipe. The canonical form for the
full recipe version string is: PE:PV-PR

If no PE is specified, we shouldn't store the initial ":"
character, as it leads to inconsistency with how the
version string is used elsewhere. This patch drops the leading ":"

    [YOCTO #5459]

(Bitbake rev: f6031bd753917c459ab232d88d7dcfc3f10e8184)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-12-10 11:16:13 +00:00
Alexandru DAMIAN
54d0e30433 bitbake: toaster: change package storage model
Up until this patch, package information lived in two
places - one table for build packages and one table for
target installed packaged. This situation leads to
two problems: there is no direct link between a build
package and a installed package, and a lot of data is duplicated.

This change unifies all package types in a single table.
The SimpleUI remains the same for continuity sake,
but the REST API will be changed in a future patch.

The package dependencies and package files are now
kept in a single table.

Since we collect target installed package information at all times,
we need to expand it to supplement missing information if a
package is not actually built in the current build.

Small changes to the Simple UI reflect the updated database schema.

    [YOCTO #5565]
    [YOCTO #5269]

(Bitbake rev: f5d655bfaeb349c8680d74530617e34aa389d1f0)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-12-10 11:16:12 +00:00
Alexandru DAMIAN
f8120984f4 bitbake: toaster: update build stats reading
In the processes of removing local system accesses
from toaster UI, we remove the build stats
code that was moved to toaster.bbclass, and
adapt the database writing code to read the data
from BuildStatsList event sent by the toaster.bbclass

    [YOCTO #5604]

(Bitbake rev: 4930ff5b471761c2a8d16c1935cdab9cf141d2d8)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-12-10 11:16:12 +00:00
Alexandru DAMIAN
ea307bbcff bitbake: toaster: move layer information reading code
In the processes of removing local system accesses
from toaster UI, we remove the layer data reading
code that was moved to toaster.bbclass, and
adapt the database writing code to read the data
from event sent by the toaster.bbclass

    [YOCTO #5604]

(Bitbake rev: 33b60a940f58e8374a8c7baa9bf28a98f54cdf13)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-12-10 11:16:12 +00:00
Alexandru DAMIAN
6cbc83fe89 bitbake: toaster: remove Author field in SimpleUI
Removes a left-over Author field in the Simple UI
recipe page.

    [YOCTO #5449]

(Bitbake rev: a9c24343f13d33d159dab0ac2fd8f50262408980)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-12-10 11:16:11 +00:00
Alexandru DAMIAN
7d24b00d0e bitbake: toaster: Analysis API variablehistory update
This patch updates the Analysis REST API to expose
the variablehistory information. A UI tool can query
this API to get a picture on how a variable value got
to its final form.

The documentation for VariableHistory is updated on
Wiki https://wiki.yoctoproject.org/wiki/Analysis_REST_API_Contracts

(Bitbake rev: f304332da3a51b15e864a853989fe5fbaf5f6079)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-12-10 11:16:11 +00:00
Richard Purdie
4aac8d87d4 bitbake: data: Ensure we add the contains keys in a particular order
If we don't sort the data, the values can reorder changing the
signatures meaning we get confused builds and significant cache
misses.

(Bitbake rev: 8f453bb11d72afc90a986ac604b3477d97eaf9a8)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-12-09 10:25:47 +00:00
Richard Purdie
184038bedb bitbake: fetch2/git: Ensure check is made in the correct directory
(Bitbake rev: 73ddf1a3d1c516ad3bc785e0413483e616ce07a8)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-12-06 14:10:15 +00:00
Cristiana Voicu
5301c59503 bitbake: toaster: exclude "set in documentation.conf" from variable history
That has no impact on the builds themselves, so we should just
remove that line from the variable history.

[YOCTO #5561]
(Bitbake rev: ae0ed55e80b7bd30c775b128b4114b306a50ff69)

Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-12-05 16:27:13 +00:00
Cristiana Voicu
2bd4c2be9f bitbake: hob: changes to INHERIT var should use "append" operation
Setting INHERIT var means to remove other operations made on INHERIT.
This is too intrusive, so we decided to use append for this case.

[YOCTO #5448]
(Bitbake rev: a2d0122c198ee50325e6f0e2f5d1c2284475fc7b)

Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-12-05 16:27:13 +00:00
Richard Purdie
f19546e02d bitbake: fetch2/git: Add sanity check to ensure we really did fetch the correct revisions
The fetcher made the rather bold assumption that if it fetched from the upstream,
the revisions were present and correct. These checks are fast and ensure that
really is the case. The avoids accidental network accessed and missing
branch configuration problems.

(Bitbake rev: a9112a102a89049cda597dad449e922c9e957a5d)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-12-05 16:27:13 +00:00
Valentin Popa
c51730aaa9 bitbake: Hob: check if warnings are related to package build
If warnings come from recipes parsing
and not from package build, 'parent' object
will be None; so don't update the color for it.

[YOCTO #5621]

(Bitbake rev: f9d24f55a5ffa9257e7ba3257be1210687513733)

Signed-off-by: Valentin Popa <valentin.popa@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-12-03 17:45:54 +00:00
Paul Eggleton
0a5fdfbf36 bitbake: hob: fix package property dialog for changes to FILES_INFO
The FILES_INFO structure is now much simpler, so remove all of the
horrible mangling we had to do here in order to read it.

(Bitbake rev: 11a664292064dbf76850bf21ba386f78a43a56b6)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-12-03 17:45:54 +00:00
Paul Eggleton
27ae6871db bitbake: hob: name package files list variables appropriately
This treeview code was obviously copy-pasted from one of the config
dialogs and the variables were never renamed. Rename them now to improve
readability.

(Bitbake rev: 93b3ddad4396d757d1d0c199ac168f8b1a205fe3)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-12-03 17:45:54 +00:00
Paul Eggleton
8a02406f1c bitbake: lib/bb/ui: handle PKGSIZE change to bytes
PKGSIZE is now in bytes in pkgdata, so we need to treat it as such in
the UI code for Hob / Toaster.

(Bitbake rev: 3b5ff814cd4a3efa4b17c6b343ec39c9acca5c9e)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-12-03 17:45:54 +00:00
Richard Purdie
d1e66643ae bitbake: cooker/command/hob: Cleanup configuration init/reset functions and commands
initConfigurationData and loadConfigurationData are similar functions, the only
reason for them appears to be to be able to reset the pre/post configuration
files. The current code is confusing and unmaintainable.

Instead this patch creates a new Sync command which allows these to be explicitly
set. The init and load functions can then be merged into one. There is then no
need for a parseConfiguration command, we can simply reset the server to have the
settings take effect.

The reset fuction is not an instant value return and triggers an event so it should
be an Async command, not a sync one.

The number of calls for the set pre/post command is probably higher than it
need be but someone with more familiarity with the hob code base can probably
figure out the right places its needed (maybe just init_cooker?).

(Bitbake rev: bae5210d7e048022f083361964ebec7daf1608f7)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-12-03 17:45:54 +00:00
Richard Purdie
b8f0963592 bitbake: hob: Use BASEDATASTORE_TRACKING feature
Might as well use this functionality now we have it available.

(Bitbake rev: cd7f4d85e3f187140d1bb0aecf82f657a8f8701a)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-12-03 17:45:54 +00:00
Richard Purdie
eba506b301 bitbake: codeparser/data_smart: Optimise parsing speed
The previous "contains" changes caused a ~3% parsing speed impact.
Looking at the cause of those changes was interesting:

* Use of defaultdict was slower than just checking for missing entries
  and setting them when needed.
* Even the "import collections" adversely affects parsing speed
* There was a missing intern function for the contains cache data
* Setting up a log object for each variable has noticeable overhead
  due to the changes in the code paths uses, we can avoid this.
* We can call getVarFlag on "_content" directly within VariableParse
  for a noticeable speed gain since its a seriously hot code path.

This patch therefore tweaks the code based on the above observations to
get some of the speed back.

(Bitbake rev: fca802187a2a30686a8a07d2b6b16a3e5716e293)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-12-03 17:45:53 +00:00
Paul Eggleton
a17ffbdbb7 bitbake: hob: fix rotating progress icon animation
I've always found it somewhat annoying that this icon's rotation
animation wasn't quite correct; this was because it was looping around
to the second position instead of the first, which made it appear to
stutter.

(Bitbake rev: 78ad15b669b9c7cde41f7bd1ab884c1d2e0db91b)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-12-02 17:51:35 +00:00
Richard Purdie
35c25907a6 bitbake: imagedetailspage: Fix crash with more than 15 layers
If you had more than 15 layers the system would crash since one more
value is added to one array than the other. This fixes the code
so equal numbers of values are added to the arrays and hence
doesn't crash when many layers are enabled.

(Bitbake rev: 4e65463886a2ef245b2f8974e82e9cb942af224b)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-12-02 17:43:22 +00:00
Jason Wessel
a67fde0ab1 bitbake: bitbake.lock: Add host:port to bitbake.lock for memres server
The idea is to build on the --status-only option for bitbake and
expose a mechanism where the oe init scripts can easily switch between
memres server and the non-memres server.

In the case of the standard oe init script the following
can shut down the server:

if [ -z "$BBSERVER" ] && [ -f bitbake.lock ] ; then
    grep ":" bitbake.lock > /dev/null && BBSERVER=`cat bitbake.lock` bitbake --status-only
    if [ $? = 0 ] ; then
       echo "Shutting down bitbake memory resident server with bitbake -m"
       BBSERVER=`cat bitbake.lock` bitbake -m
    fi
fi

A similar function can be used to automatically detect if the server
is already running for the oe memres init script.  This new
functionality allows for the memres init script to be started in a new
shell and connect up to an alaready running server without seeing the
error of trying to start the server multiple times.

(Bitbake rev: b1803958de8d7c3c3279841e38604a08dc2316cc)

Signed-off-by: Jason Wessel <jason.wessel@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-12-02 17:33:14 +00:00
Jason Wessel
6ff9c9e39d bitbake: bitbake, xmlrpc.py: Implement memory resident auto port configuration/restart
This patch adds the ability to dynamically select a port for the
bitbake memory resident server when the BBSERVER port is set to -1.
This allows for running multiple instances of the bitbake memory
resident server on the same system in different build directories.

The client portion of the bitbake instance can also request that the
server automatically start when using the auto port feature.  This is
to deal with a bitbake instance that eventually times out and exits or
that has died for some unknown reason.

The new functionality allows for lazy startup of the server after
sourcing the init script for the memory resident functionality.

(Bitbake rev: d6abc07ff385357d312d8435b89e0a9c1f965433)

Signed-off-by: Jason Wessel <jason.wessel@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-12-02 11:28:27 +00:00
Jason Wessel
0bea372b0f bitbake: bitbake: Add --status-only option
The --status-only option is purely to check the health of the server.
The idea is to use it from the oe environment script so as to know if
the server needs to be started or not.

(Bitbake rev: 46b26ef5c27bfa9c91680f494f02750305b8098c)

Signed-off-by: Jason Wessel <jason.wessel@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-12-02 11:28:27 +00:00
Cristiana Voicu
fabdf2f57a bitbake: hob: disable layer drag and drop outside the containing widget
[YOCTO #5577]
(Bitbake rev: d39d957c77b9804670cad3bc9f7742084ead27e8)

Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-11-30 22:26:30 +00:00
Richard Purdie
f5f3ef7b09 bitbake: parse/ConfHander/BBHandler/utils: Fix cache dependency bugs
Currently bitbake only adds files to its dependency list if they exist.
If you add 'include foo.inc' to your recipe and the file doesn't exist,
then later you add the file, the cache will not be invalidated.

This leads to another bug which is that if files don't exist and then
you add them and they should be found first due to BBPATH, again the
cache won't invalidate.

This patch adds in tracking of files we check for the existence of so
that if they are added later, the cache correctly invalidates. This
necessitated a new version of bb.utils.which which returns a list of
files tested for.

The patch also adds in checks for duplicate file includes and for now
prints a warning about this. That will likely become a fatal error at
some point since its never usually desired to include a file twice.

The same issue is also fixed for class inheritance. Now when a class
is added which would be found in the usual search path, it will cause
the cache to be invalidated.

Unfortunately this is old code in bitbake and the patch isn't the
neatest since we have to work within that framework.

[YOCTO #5611]
[YOCTO #4425]

(Bitbake rev: 78d285871e4b8c54ccc4602d571e85f922e37ccd)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-11-30 22:26:30 +00:00
Nathan Rossi
f91a3f46a1 bitbake: cooker: Fix support for wildcards in bbappend filenames
The 'bbappend in f' incorrectly compares the current recipe with the avaliable
bbappends recipes. This comparsion causes unrequested bbappend files to be
appended, e.g. in the case of 'libgcc_4.8.bb', the bbappends for 'libgcc_4.8.bb'
and 'gcc_4.8.bb' are added to the filelist (because 'gcc_4.8.bb' is contained in
the 'libgcc_4.8.bb' string) which in turn causes the gcc_4.8.bbappend files to
be appended to the libgcc_4.8 recipe.

This should be a 'bbappend == f' to match the previous implementation of this
function, such that if no wildcard is present the recipe names must match
exactly.

This issue was introduced by commit 31bc9af9cd56e7b318924869970e850993fafc5f,
which it related to [YOCTO #5411].

(Bitbake rev: 991cbeedbde8bd25ce08c669b1bfac8b99e33149)

Signed-off-by: Nathan Rossi <nathan.rossi@xilinx.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-11-29 09:53:00 +00:00
Yi Zhao
463e56ff2b bitbake: crumbs/builder.py: Fix typo for indefinite article
(Bitbake rev: bc3c69e50d36ea5e43f55f7b3b1ff3b0ab3696ee)

Signed-off-by: Yi Zhao <yi.zhao@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-11-27 14:43:33 +00:00
Valentin Popa
82ff33fc39 bitbake: Hob: force notebook refresh
Redraw the entire notebook widget after
number-of-issues drawable is incremented
(because for this case, the notebook widget
doesn't refresh it's children automatically).

[YOCTO #5596]

(Bitbake rev: 347b2ead091f00ee60703f6f3d17cfdd9075ac07)

Signed-off-by: Valentin Popa <valentin.popa@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-11-26 23:01:33 +00:00
Richard Purdie
a10301f6a9 bitbake: data/codeparser: Improve handling of contains functions
One of the current frustrations with the sstate checksums is that
code like base_contains('X', 'y',...) adds a full dependency on X
and varies depend even on whitespace changes in X.

This patch adds special handling of the contains functions to expand
the first parameter and check for the flag specified by the second
parameter (assuming its a string).

The result is then appended to the value of the variable with a "Set"
or "Unset" status. If the flag is added/removed, the stored variable
value changes and hence the checksum changes. No dependency on X
is added so it is free to change with regard to other flags or
whitespace.

This code is far from ideal, ideally we'd have properly typed variables
however it fixes a major annoyance of the current checksums and
is of enough value its worth adding in a stopgap solution. It shouldn't
significantly restrict any propely typed variable implementation in
future.

(Bitbake rev: ed2d0a22a80299de0cfd377999950cf4b26c512e)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Christopher Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-11-26 23:01:33 +00:00
Richard Purdie
d061692372 bitbake: bitbake: Share BB_TASKDEPDATA with tasks
Currently tasks have no knowledge of which other tasks they depend
upon. This makes it impossible to do at least two things which would be
desirable/interesting:

a) Have the ability to create per recipe sysroots
b) Allow the aclocal files to be present only for the entries in
   DEPENDS (directly and indirectly)

By exporting task data through this new variable, tasks can inspect
their dependencies and then take actions based upon this.

(Bitbake rev: 84f1dde717dac22435005b79d03ee0b80a3e8e62)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-11-26 23:01:33 +00:00
Richard Purdie
4c69970bd3 bitbake: runqueue: Optimise next_buildable_task()
This unlikely looking function was found to be eating a lot of CPU time
since it gets called once per trip through the idle loop if we're not
running a maximum number of processes. This was particularly true in
world builds of 13,000 tasks.

Calling the computation code is pretty pointless because until some
other task finishes nothing is going to become available to build.
We can know when things become available so this patch teaches the
scheduler this knowledge.

It also:

* skips any coputation when nothing can be built
* if there is only one available item to build, ignore the priority map
* precomputes the stamp filenames, rather than doing it every time
* saves the length of the array rather than calculating it each time
  (the extra function overhead is significant)

Timing wise, initially, 5000 iterations through here was 20s, with
the patch 200000 calls takes the same time. The end result is that
builds get up and running faster.

(Bitbake rev: 4841c1d37c503a366f99e3a134dca7440e3a08ea)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-11-26 23:01:33 +00:00
Richard Purdie
3ca58c762f bitbake: codeparser: Drop unneeded variable separation
There is no good reason to separately track var_references and
references so merge them and remove the unneeded variable.

(Bitbake rev: 64d4cbd6360c96574cece70205ea3aecc3f8bae6)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-11-26 23:01:32 +00:00
Peter Kjellerstedt
0aef540dd8 bitbake: fetch2: Correct a few calls to latest_revision()
In 6a48474de9505a3700863f31839a7c53c5e18a8d the url parameter to a
number of functions was removed. However, not all calls to
latest_revision() were fixed...

(Bitbake rev: 7c94ca56b2fd85a989089f58b3dcce3172a778f2)

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-11-25 13:02:27 +00:00
Florin Sarbu
64372fa027 bitbake: bitbake: fetch2: Remove the unused urldata from the git fetcher.
(Bitbake rev: e0a2e9699e9f338dd7ade8c9eef0a12c7639ec05)

Signed-off-by: Florin Sarbu <florin.sarbu@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-11-25 13:00:18 +00:00
Richard Purdie
b1c15cb907 bitbake: data: Fix output inconsistencies for emit_var
VAL = ""     (not shown)
VAL = " "    (shown as "")
VAL = " x"   (shown as "x")

would all show up rather differently to what would be expected in the
bitbake -e output. This fixes things so they appear consistently.

The output for running some shell functions may also change slightly
but shouldn't change in a way that is likely to cause problems.

[YOCTO #5507]

(Bitbake rev: fcba5ef0053dc0ef5360e4912609e5d52f5046b0)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-11-25 13:00:18 +00:00
Richard Purdie
6ddb9bf097 bitbake: perforce: Fix path subdirectory issues
With a SRC_URI = " \
p4://depot/folder/...;module=localfolder/localsubfolder;changeslist=${P4CHANGELIST} \
"

the subfolders of //depot/folder/... get renamed when mapped to the
local folder structure. They lose the first 3 letters. This
patch fixes that.

Issue reported by and patch sent from katutxakurra@gmail.com

[YOCTO #5380]

(Bitbake rev: 40e06dc459d9c0b5d42d65b2d2c846196fd36b1f)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-11-24 12:25:37 +00:00
Paul Eggleton
4ca910d4cf bitbake: bitbake-layers: avoid loading configuration when not needed
In recent versions of bitbake, it is not possible to initialise a
BBCooker object without having it load the configuration first. Thus we
should avoid creating the Tinfoil object here in bitbake-layers which
does that internally until we actually need to, so you can run
"bitbake-layers help" and not have to wait several seconds for the
output.

(Bitbake rev: 8f1e280fbbb6432d7bcc1fb4241f402668c6c5ea)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-11-24 12:25:37 +00:00
Valentin Popa
300b6f7f49 bitbake: HOB: Show warnings in Issues tab
Proper update of warnings view during build.

[YOCTO #3496]

(Bitbake rev: 124428ba8a0b1cc85d4b96053bf71bce14a5774c)

Signed-off-by: Valentin Popa <valentin.popa@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-11-24 12:25:37 +00:00
Saul Wold
381d592018 bitbake: cooker: add support for using % as a wildcard in bbappend filename
There has been a continuing call for supporting wildcard in bbappend
filenames. The wildcard is actually allow matching of the name and
version up to the point of encountering the %.  This approach will
allow for matching of the major or major.minor.

Exampes:

busybox_1.21.1.bb
busybox_1.21.%.bbappend will match
busybox_1.2%.bbappend will also match

if we update to busybox_1.3.0.bb the above won't match, but a busybox_1.%.bb
will.

[YOCTO #5411]

(Bitbake rev: 31bc9af9cd56e7b318924869970e850993fafc5f)

Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-11-24 12:25:36 +00:00
Richard Purdie
bc267ef6fe bitbake: runqueue/bitbake-worker: Fix dry run fakeroot issues
When using the dry run option (-n), bitbake would still try and fire
a specific fakeroot worker. This is doomed to failure since it might
well not have been built.

Add in some checks to prevent the failures.

[YOCTO #5367]

(Bitbake rev: f34d0606f87ce9dacadeb78bac35879b74f10559)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-11-24 12:25:36 +00:00
Richard Purdie
9d7f8e2a20 bitbake: fetch2: Stop passing around the pointless url parameter
There is no good reason to keep passing around the url parameter when
its contained within urldata (ud). This is left around due to
legacy reasons, some functions take it, some don't and its time
to cleanup.

This is fetcher internal API, there are a tiny number of external users
of the internal API (buildhistory and distrodata) which can be fixed up
after this change.

(Bitbake rev: 6a48474de9505a3700863f31839a7c53c5e18a8d)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-11-22 12:03:02 +00:00
Richard Purdie
4acc7322a2 bitbake: fetch2: Clean up some url parameters to functions
urldata contains the url so we might as well stop passing around
pointless function parameters. This was done for legacy reasons but
its time to clean this mess up.

This is a first step in cleanup and is a standalone patch but there is
more to be done in a second patch.

(Bitbake rev: 06590cfebbcf6565a17b80cc298e3ecdfaba4656)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-11-22 12:03:02 +00:00
Alexandru DAMIAN
725fae6581 bitbake: toasterui: fix typo
A typo in buildinfo helper leads to a bug where
information about tasks is not correctly stored.

This patch fixes the typo.

(Bitbake rev: 67b752993a2c64cba9ccc4fa662f0bddf081e74a)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-11-20 14:06:49 +00:00
Alexandru DAMIAN
e19ee57c96 bitbake: bitbake: cooker, toaster: mark interrupted builds as failed
This patch changes bitbake to log an error to the
command line when the build is interrupted via Ctrl-C.

This is needed to inform the user that not all tasks
required for the build have been executed, and
the build is not complete.

Internally, the Bitbake server will return a CommandFailed
event that will be logged by Toaster as build failure.

(Bitbake rev: 9a658e8b1511f1b9f91663f546f748fdfbc8965f)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-11-20 14:05:52 +00:00
Peter Seebach
10717e5a0a bitbake: build.py: add single-quotes around already-expanded directory name
If the computed name of a directory contains an undefined variable
reference, bitbake dutifully creates a directory with a name that has
${...} in it. However, the actual task script created then tries to cd
to that directory, and the cd command fails, because no such directory
exists -- because the shell has helpfully removed the ${...} which did
not match any actual variables.

Since we want the name to be used exactly-as-is, add single quotes around
the name so this doesn't cause strange failures running tasks, which
allows us to progress past such failures and get to a point where they
can be diagnosed.

(Bitbake rev: 2809c2e6f2f35f9b08058950be896947ab5a0284)

Signed-off-by: Peter Seebach <peter.seebach@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-11-20 14:05:52 +00:00
Richard Purdie
bf7d135201 bitbake: runqueue: Fix hole in setsceneverify skipped task logic
We have do_bundle_initramfs which is a task inserted after compile and
before build. It is not covered by sstate.

If we run a build with a valid sstate cache present, the setsceneverify
function realises it will rerun the do_compile step (due to the
bundle_initramfs task) and hence marks do_populate_sysroot to rerun.
do_install, a dependency of do_populate_sysroot is left as marked as
covered by sstate.

What we need to do is traverse the dependency tree for any setsceneverify
invalided task and ensure any dependencies are also invalidated. We can
stop at any point we reach another setscene task though.

This means the do_populate_sysroot task has the data from do_install
available and doesn't crash.

(Bitbake rev: f21910157d873c030b149c4cdc5b57c5062ab5a6)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-11-20 13:48:01 +00:00
Richard Purdie
215dab864e bitbake: fetch2: Fix handling of SCM mirrors in MIRRORS
If an SCM mirror is in PREMIRRORS, the tarball is downloaded and then found
by the "upstream" check and handled correctly.

If an SCM mirror is in MIRRORS, the tarball is downloaded but not used
since there is no "upstream" run after MIRRORS completes. It therefore
sits there useless and unused. This code change forces the upstream to
run after a mirror tarball is found and fixes the usage of SCM mirrors
in MIRRORS.

(Bitbake rev: a66ee0994645aa5658b2f5ea134ed17d89f8751a)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-11-19 13:22:48 +00:00
Richard Purdie
9ccfe66074 bitbake: git: Use git branch for ancestor checks
Using git merge-base for checking for ancestors is nice but required git 1.8.0
which is not in many distrbutions yet. We therefore revert to a more ugly
check using git branch --contains until such times as we can upgrade.

(Bitbake rev: 31467c0afe0346502fcd18bd376f23ea76a27d61)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-11-19 13:22:47 +00:00
Konrad Scherer
e15893adf9 bitbake: serv.py: Give pr-server up to 5 seconds to commit data
The default value of 0.5 seconds before sending the pr-server a
SIGTERM is not enough to guarantee that sqlite has committed all
the pr data to the database. By polling the pid to see if it is
still running, this allows the pr-server process to shutdown
cleanly and finish the final pr data commit.

(Bitbake rev: 22eec978e70794923c85689928c6be0cfe71cdcd)

Signed-off-by: Konrad Scherer <Konrad.Scherer@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-11-18 17:19:11 +00:00
Richard Purdie
2354250a95 bitbake: git: Use merge-base instead of log for testing if a commit is present
The current use of git log to check if a given revision is present can be
a little fragile.

For example if revision X was on branch A, and then later added to branch
B, the update checks would not notice this since they just check for X
being in the repository.

We also had some autobuilder corruption where an older packed-refs file
was copied over a new repository containing newer pack files. There
was no update to the refs file since the revision was present but
not accessible in any branch.

The correct fix is to check that the required revisions are present
on the specific branches. This patch does this using merge-base.

(Bitbake rev: 89abfbc1953e3711d6c90aff793ee622c22609b1)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-11-18 17:19:11 +00:00
Alexandru DAMIAN
9fdd8adc17 bitbake: cooker, toaster: variable definition tracking
In order to track the file where a configuration
variable was defined, this patch bring these changes:

* a new feature is defined in CookerFeatures, named
BASEDATASTORE_TRACKING. When a UI requests BASEDATASTORE_TRACKING,
the base variable definition are tracked when configuration
is parsed.

* getAllKeysWithFlags now includes variable history in the
data dump

* toaster_ui.py will record the operation, file path
and line number where the variable was changes

* toaster Simple UI will display the file path
and line number for Configuration page

There is a change in the models to accomodate the recording
of variable change history.

[YOCTO #5227]

(Bitbake rev: 78e58fed82f2a71f052485de0052d7b9cca53ffd)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-11-15 11:48:53 +00:00
Cristiana Voicu
51084379d4 bitbake: toaster: task with outcome 2 (sstate), should have sstate_result!=0
0 (not applicable) is not a valid sstate_result for tasks with
outcome 2 (sstate), which should return 3 (restored), 2
(failed) or 1 (missed).
Sstate_result for tasks with outcome 2 is equal to the outcome
of _setscene corespondent task.

[YOCTO #5220]
(Bitbake rev: 8ff8d75318ea88ba80c744b471e486901ef6749a)

Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-11-15 11:48:53 +00:00
Cristiana Voicu
6f01250547 bitbake: hob/hoblistmodel: check if vals of packages/recipes names are not None
[YOCTO #5053]
(Bitbake rev: ba9fe77e37be31e8246431578902e871dd94515e)

Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-11-15 11:48:53 +00:00
Paul Eggleton
ad65846ba8 bitbake: cooker: warn if user specifies a target listed in ASSUME_PROVIDED
If the user explicitly asks to build a target that is listed in the
value of ASSUME_PROVIDED, show a warning mentioning that it will be
ignored.

(Bitbake rev: 65be09f5e2a4a41e65c9232e208d8154b822fc4e)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-11-15 11:48:52 +00:00
Cristiana Voicu
859b631b2f bitbake: hob: refresh tables when the user is redirected to Edit packages
When the build fails and I click "Edit packages", we should be returned
to the tab we launched the build from. Hob should remember the tab,
not the search state.

[YOCTO #5257]
(Bitbake rev: afdea16516084ef6c046d80e12bf244f25a9da50)

Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-11-15 11:48:52 +00:00
Alexandru DAMIAN
5f98364a62 bitbake: toaster: fix path to buildstats file
The buildstats file path changes based on the
optional PE variable that may be defined for a
recipe.

The toasterui simply ignored the PE value, and
as such it didn't correctly reach buildstats files
for some of the tasks.

This patch fixes the issue.

    [YOCTO #5073]

(Bitbake rev: 97b8ab88edc7c8dfb26b4cf305701ec96e52cc4f)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-11-15 11:48:52 +00:00
Alexandru DAMIAN
533cf81f8b bitbake: toasterui: mark failed sceneQueue tasks as failed
This patch addresses an issue where a failed sceneQueue task
entry was not updated on the Fail event. As a result, it
always showed the task as not-available.

    [YOCTO #5216]

(Bitbake rev: 9b99a417f58381bac4bda412bcfd11de50403318)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-11-15 11:48:52 +00:00
Alexandru DAMIAN
9f1e696aba bitbake: toaster: remove author field
The AUTHOR field in most recipes is not defined,
or it's not really consistently set in the metadata,
Also does it seem particularly useful.

This patch removes the AUTHOR variable from the
toaster system

    [YOCTO #5449]

(Bitbake rev: da3ac049300be84defab7b32b0b99ab07c7d0a27)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-11-15 11:48:52 +00:00
Alexandru DAMIAN
2445b313a2 bitbake: toaster: fix tasks showing as NoExec
Tasks without script type information showed by default
as NoExec; this happens for all Prebuild or Covered
tasks, as script type information comes only on TaskStarted
event. Such a default value may drive confusion, as NoExec value
should be reserved for the NoExec-flagged tasks.

This patch adds a new default value named Unknown that will be
used for all tasks that don't have script type information
available.

    [YOCTO #5327]

(Bitbake rev: ec6cac74290f0d4f5b60222019c23416b4b8e1ef)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-11-15 11:48:51 +00:00
Cristiana Voicu
ea92671d98 bitbake: cooker: replace "w" file opening mode with "a" mode
Also removed some redundant file manipulation code
Based on patch sent by Stefan Stanacar <stefanx.stanacar@intel.com>

(Bitbake rev: e054c1e7c8581f66082fcdfb89769401ca6e78a3)

Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-11-12 17:01:37 +00:00
Cristiana Voicu
8c534107d9 bitbake: toaster: convert build_package size to bytes to keep consistence
[YOCTO #5503]
(Bitbake rev: 19eb6e01b675c439ff0a817be6fa5e34ad42ba37)

Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-11-12 17:01:37 +00:00
Trevor Woerner
b9d81043df bitbake: newbb.vim: fix spelling
(Bitbake rev: 0bd1a3026a26c2c01abd31e265668541f6201bae)

Signed-off-by: Trevor Woerner <trevor.woerner@linaro.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-11-12 17:01:37 +00:00
Valentin Popa
67a7ac1ea7 bitbake: HOB: HIG compliant message dialogs
"More" HIG compliant dialogs for HOB.

(Bitbake rev: a1ccb5e70523be6af7f4bcaabd728af1ddaf6dc5)

Signed-off-by: Valentin Popa <valentin.popa@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-11-08 17:25:38 +00:00
Gary Thomas
34eb004435 bitbake: depexp: Improve wording on requirements
Since -g is required, tell the user exactly so.

(Bitbake rev: f05f74e98b0dd567a8b0cb85dc8183716619991b)

Signed-off-by: Gary Thomas <gary@mlbassoc.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-11-08 17:25:37 +00:00
Alexandru DAMIAN
57498cede4 bitbake: toaster: fix target package information
Toaster needs to record information about packages
installed on a built target image, and dependencies
between these packages.

This patch fixes a bug where the variable from the
server wasn't read correctly leading which caused
the buildhistory to not be processed correctly.

Additionally, two display issues in the package table
were fixed, issues that lead to package information
being displayed incorrectly.

    [YOCTO #5197]

(Bitbake rev: ab4bc18409d80de6d069e3dd76c3c54964fe5764)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-11-05 12:50:44 +00:00
Alexandru DAMIAN
9160f37606 bitbake: toaster: enable required classes in the toaster startup script
In order to use toaster, now you have to set INHERIT+="toaster buildhistory"
To keep it simple, I've done some changes in order to automate it. When toaster
is started, this line is added to a new file called toaster.conf.

This file is passed to the bitbake server with the --postread parameter.

Based on a patch by Cristiana Voicu <cristiana.voicu@intel.com>

(Bitbake rev: 029e868044989eda370340f8bf4200cfd2670fca)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-11-04 09:36:22 +00:00
Cristiana Voicu
8cbd1ce1b8 bitbake: toaster: add variable description for prefixed/suffixed variables
In the Configuration table, we need to link prefixed / suffixed
variables to the corresponding variable descriptions in documentation.conf.

[YOCTO #5198]

(Bitbake rev: 641d9c4fda5fe978154fdfab978c3c09e3906eab)

Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com>
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-11-04 09:36:22 +00:00
Alexandru DAMIAN
16067d5295 bitbake: toaster: server shutdown on terminal exit
If the terminal where the server was started is closed,
the bitbake server should shutdown. Currently the system
is left in hanging state.

This patch uses "trap" command to make sure the servers
are closed on terminal exit.

[YOCTO #5376]

(Bitbake rev: 5f8b97010f7b465753b6ff6275d18426006ee14b)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-11-04 09:36:22 +00:00
Alexandru DAMIAN
af32d95f3b bitbake: toaster: fix timezone settings
This patch fixes an issue where, if not defined,
the timezone defaults to 'America/Chicago'.

The solution is to set the timezone to current computer's
timezone.

[YOCTO #5186]

(Bitbake rev: a4102b549f04a9b52cdcd318bf511a18ab48067d)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-11-04 09:36:22 +00:00
Alexandru DAMIAN
0087db6257 bitbake: cooker: do not recreate recipecache in buildfile mode
When building a single file, the cooker will recreate
the recipecache from scratch.

I suspect this is a remnant of past code, since:
* the current recipecache works fine
* the new recipecache will not have all the fields as
requested by HOB_EXTRA_CACHES setting

This patch disables recreating the recipecache, leading
to shorter times when building single build files
(-b option) and better compatibility with Toaster.

(Bitbake rev: 618d69b00075981b8553513130d7deb1aed61578)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-11-04 09:36:22 +00:00
Alexandru DAMIAN
85b9194171 bitbake: build, toaster: record proper task type
Bitbake tasks may be of type 'python' or 'shell',
or they may not be executed at all, which is record
as task type 'noexec'.

In order to record proper task type, this patch:

* creates no exec task type as the default value in
the toaster model definition

* adds full task flags to the bb.build.TaskStarted event
in build.py

* if the task actually starts, the toaster ui will
record the type of the task as either 'python' or 'shell'
based on the task flags.

[YOCTO #5073]
[YOCTO #5075]
[YOCTO #5327]

(Bitbake rev: 6648c57e6d369fc009ea3a9fe939def5d2c67bf5)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-11-04 09:36:22 +00:00
Alexandru DAMIAN
a670859103 bitbake: cooker: add data to the dependency tree dump
Toaster needes to record extra data that needs to
be moved at the time of the dependency tree dump.

This data includes:
* layer priorities for recording in the layer section
* the inherit list for each PN which allows to determine
the type of the PN (regular package, image, etc).

This patch adds this data to the dependency tree dump.

(Bitbake rev: 7636aba37320aaf9b044d3832ddc21af51ccd69c)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-11-04 09:36:21 +00:00
Alexandru DAMIAN
f04a32162d bitbake: toaster: fixes for null values from events
Some of the data values may come of as None through the event system,
and the UI would encounter a problem saving the Configuration.
It would be trying to save these values as NULL in the
database, which is not allowed.

This patch adds more verification for data coming through
the event system.

Other minor updates:
* update for the event model from toaster.bbclass
* minor code flow fix in the event system

(Bitbake rev: 03fafd086381723c6486522873671515824e49f2)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-11-04 09:36:21 +00:00
Cristiana Voicu
f3541226b8 bitbake: hob: do not display the "Package list may be incomplete!" dialog
This dialog is not needed. Instead "Edit packages" button should
redirect to Packages list.

[YOCTO #5257]
(Bitbake rev: aa2e86a2ac28414f6d25fcd541a6ed2b50f15a2c)

Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-11-01 17:59:31 +00:00
Volker Vogelhuber
1c7bbf8dec bitbake: fetch/hg: Improve user/password handling
Trying to use a server with username and password authentication
within the URL of the SRC_URI variable doesn't appear to work.

This patch adds the missing parts to the hg fetcher to make this
work properly.

(Bitbake rev: dc3d6d73e44802c203b3f7247f6f212acc2f69bf)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-11-01 17:59:31 +00:00
Cristiana Voicu
fe3e26509d bitbake: hob: populate error_msg when hob receives a CommandFailed event
When Hob receives a bb.command.CommandFailed event, it doesn't show any error,
because the error_msg is empty. If so, it is hard to detect the issue, because
Hob continues to run till it blocks because of an information gap.

[YOCTO #5097]
(Bitbake rev: a5abd1826f34e6a7eefa837620b846e9b62ae758)

Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-11-01 17:59:31 +00:00
Nicolas Dechesne
de7e6efb80 bitbake: fetch2/svn.py: use log instead of info to retrieve revision
We have faced a corner case situation where the 'last changed
revision' returned from svn info is wrong. It happens when the last
revision is a directory move. e.g. if we assume that the svn
repository at revA has root/x/y/z/foo/bar and it is moved to
root/a/b/c/foo/bar in revB, then svn info 'last change revision' will
return revA. As such when using AUTOREV, we are going to attempt to
retrieve root/a/b/c/foo/bar (as per SRC_URI) but at revA when it did
not exist.

So this patch changes how we retrieve the latest revision and uses
'svn log --limit 1' which gives correct result in all tested cases.

(Bitbake rev: 17d8ef0b813a05c231e3dbe6e8bc82a4a9b1d2f8)

Signed-off-by: Nicolas Dechesne <nicolas.dechesne@linaro.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-11-01 17:59:30 +00:00
Olof Johansson
8707c39a10 bitbake: monitordisk: lower inode check warning to note
Filesystems like btrfs and reiserfs sets the inode count to 0, since
they don't have an inode concept. This is expected, and having a warning
show up every time you run bitbake can cause undue concern.

(Bitbake rev: f3ac2d3678f48c68a250a0a20c08cf8687322d38)

Signed-off-by: Olof Johansson <olof.johansson@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-10-18 16:02:16 +01:00
Alexandru DAMIAN
9a1dce10bd bitbake: toaster: add Toaster UI interface
Adding a new bitbake UI interface named 'toasterui'.

'toasterui' listens for events and data coming from a
bitbake server during a run, and records it
in a data store using the Toaster object model.

Adds a helper class named BuildInfoHelper that
reconstructs the state of the bitbake server and
saves relevant data to the data store.

Code portions contributed by Calin Dragomir <calindragomir@gmail.com>.

(Bitbake rev: 62200ff6694b21fbd5abf009a6f47ad93adf5309)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-10-18 11:13:49 +01:00
Alexandru DAMIAN
4e21d092f9 bitbake: toaster: adding frameworks for the Simple UI
This commit adds the 3rd party frameworks used for the web UI.

jQuery is licensed under MIT.

Bootstrap is licensed under APACHE-2.0

(Bitbake rev: 8395e257fed030f0d9b24feba17ed99664f26b2b)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-10-18 11:13:49 +01:00
Alexandru DAMIAN
164ab730cc bitbake: toaster: add toaster code to bitbake
This patch adds the Toaster component to Bitbake.

Toaster is a module designed to record the progress of a
Bitbake build, and data about the resultant artifacts.

It contains a web-based interface and a REST API allowing
post-facto inspection of the build process and artifacts.

Features present in this build:
* toaster start script
* relational data model
* Django boilerplate code
* the REST API
* the Simple UI web interface

This patch has all the development history squashed together.

Code portions contributed by Calin Dragomir <calindragomir@gmail.com>.

(Bitbake rev: d24334a5e83d09b3ab227af485971bb768bf5412)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-10-18 11:13:49 +01:00
Richard Purdie
d0072fc139 bitbake: utils: Don't show stack traces for BBHandledException
A BBHandledException means we already showed an error to the user so
we shouldn't show a stack trace as this just confuses things further.

(Bitbake rev: 8a8bafc8ded98364a31878b23c64503a53affcd1)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-10-18 11:13:49 +01:00
Richard Purdie
396d8f8d9a bitbake: bitbake: bitbake: bb: Drop deprecated functions
These functions in the main bb module have long been deprecated
and moved to other modules. Finally remove the compatibility links.

(Bitbake rev: ccd181c3ed4852e2b9169cf19aaf18aeacddcc18)

(Bitbake rev: d7f817518c5df2524a4bcf008ba63c71a8eb48bb)

(Bitbake rev: 89b31a4fb44f2c2ad0bb4210151652cd3730418d)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-10-14 16:59:12 +01:00
Paul Eggleton
e9662e8926 bitbake: siggen: handle recipe path changes in siginfo files
Avoid storing paths to files in SRC_URI when writing out the the
file checksums to siginfo files. This prevents a move of the source
directory being reported by bitbake-diffsigs as files being removed and
then added (the signature itself is not affected since the file paths
have never been included in the signature).

This has required the format of the file checksums in the siginfo file
to be changed from a dict to a list of tuples (in order to handle
multiple files with the same name under different paths, which is
uncommon but possible); the code remains backwards-compatible with older
siginfo files that use a dict however.

Fixes [YOCTO #5245].

(Bitbake rev: e4d3077c5b0cc57964640512f3646c2d73c1d855)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-10-14 16:59:12 +01:00
Cristiana Voicu
ce0a6f81ae bitbake: hob: Hob should ask for ports only when Manual proxy is selected
[YOCTO #5307]
(Bitbake rev: 09b3e4e1e3fac737ea4069457e8bbffe1a4fe09d)

Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-10-14 16:59:12 +01:00
Richard Purdie
3ca820de91 bitbake: cooker/command: Add error state for the server and use for pre_serve errors
Currently if errors occur when starting the PR service, there is a race that
occurs since the UI runs various commands including starting builds before
processing the CookerExit(). By adding the error state and refusing to run
async commands in this mode, builds are prevented from starting and the
UI reaches the exit code with the system shutting down cleanly.

(Bitbake rev: 42fa34142ea685f91115a551e74416ca28ef1c91)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-10-14 16:59:12 +01:00
Richard Purdie
f41922d0c7 bitbake: bitbake: Update post 1.20 to development version
(Bitbake rev: 03d051b6da52ec4ee26f97aa9038622f8d87e55d)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-10-14 16:59:12 +01:00
Cristiana Voicu
5445f71fc8 bitbake: hob: change tooltip for live images
This change is needed to inform the user that selecting
live type means that the system will build a hddimg and iso
image.

(Bitbake rev: 5051f59976de4e099bb434aeea414de5a67a069f)

Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-10-07 16:25:02 +01:00
Cristiana Voicu
81e678bc0a bitbake: bitbake: replace comment added in conf files
The patch that implements removeConfigurationVar method was made before
merging the patch that replaces "added by bitbake" with "added by hob".
This patch corrects this issue.

[YOCTO #5284]
(Bitbake rev: 42601a5edef0316767b952b162123534aa8fee18)

Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-10-07 12:52:05 +01:00
Paul Eggleton
3c5b6af991 bitbake: bitbake-dumpsig: introduce command line and error handling
This utility doesn't take any special arguments, but it's nice if it at
least knows how to deal with no arguments, --help and errors properly.

(Bitbake rev: 0cabdf1d0cde6687bc1372675a0d6242587c87a0)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-10-07 09:37:35 +01:00
Paul Eggleton
e57bd62e17 bitbake: bitbake-diffsigs: improve error handling
* Set up a logger independent of BitBake so we can log errors ourselves
* Handle common errors without printing a traceback

(Bitbake rev: 77b5f5b8dca4deebb06eeb06a8e7f2ccdbfff46f)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-10-07 09:37:35 +01:00
Paul Eggleton
e0e30c6239 bitbake: bitbake-diffsigs: refactor argument parsing slightly
* Use OptionParser to parse the two options to -t rather than trying to
  pick them out ourselves.
* Add a description shown with --help output

(Bitbake rev: daab42d19463b4108968fc88b207936e5ac84154)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-10-07 09:37:34 +01:00
Paul Eggleton
7c72144419 bitbake: bitbake-diffsigs: handle if task name is specified without do_ prefix
Adjust the task name automatically if the -t option is specified with
a task name that doesn't start with do_ (e.g. "configure" instead of
"do_configure").

(Bitbake rev: d182cbc63745303ef2dc9fa2cbbf5d87a68e0b52)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-10-07 09:37:34 +01:00
Richard Purdie
f18194c088 bitbake: bin/bitbake: Clarify server error message
Getting an error message about --remote-server being set when really
BBSERVER was is confusing, clarify the message.

(Bitbake rev: d7b5938a30a9b0ed83f899a06a88786e8392f8bd)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-10-07 09:37:34 +01:00
Richard Purdie
06afe1fafe bitbake: bin/bitbake: Clear ui-queue when starting a server
When starting a server we don't want to show the debug event queue which
is reserved for when when errors have occurred. This patch copies the UI
code to ensure the user doesn't see confusing output.

(Bitbake rev: a886cda58415085981646fb9a024fa7641f55865)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-10-07 09:37:34 +01:00
Richard Purdie
266a7d8c97 bitbake: bin/bitbake: No need to show tracebacks for BBHandledExceptions
For BBHandledExceptions, we've already displaced a sensible error to
the user so we don't need to do it again. Just exit with an error
value.

(Bitbake rev: 1ff5ec26eba70ab1c85674a60b7dac77317bf349)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-10-07 09:37:34 +01:00
Richard Purdie
22d5782ef5 bitbake: xmlrpc: Show a better error message for connection failures
Showing a traceback when unable to connect to a bitbake server is
rather ugly. This change allows us to show a sensible error message.

(Bitbake rev: 26913202f83fbbecdce95da59515af102bcde4a7)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-10-07 09:37:34 +01:00
Richard Purdie
a333693c7f bitbake: bin/bitbake: Catch establish connection log messages
If for example you try "bitbake -m" with an invalid BBSERVER, error
messages are not displayed. This change ensures logging is in place
to catch and display such errors.

(Bitbake rev: 719808f95adc7820fcc09743c592513414d03ce1)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-10-07 09:37:34 +01:00
Cristiana Voicu
ee7e64f116 bitbake: bitbake/hob: removing extra parameters from conf files using hob
In Hob settings, there is a tab to add/remove extra settings. This
patch implements a way to "remove" variables from conf files, through
bitbake. But, to keep the history assigment of the variables synchronized,
instead of removing, it replaces the lines with blank lines.

[YOCTO #5284]
(Bitbake rev: bd720fb63cef6b399619b8fbcaeb8d7710f2d6df)

Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-10-07 09:37:34 +01:00
Valentin Popa
45392cc67a bitbake: HOB: MACHINE should be saved in conf files using ?=
MACHINE var is saved using early assignment operator.
Calling  MACHINE=x bitbake core-image-... works properly.
Comment "#added by bitbake" is replaced with "#added by hob".

[YOCTO #5070]
(Bitbake rev: 2d0ec8ff083b636a6cf98de3278900eb95c3def6)

Signed-off-by: Valentin Popa <valentin.popa@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-10-07 09:37:33 +01:00
Ross Burton
95915910df bitbake: utils: use logger.warn instead of print in copyfile
print disappears into the ether, so use logger.warn and clean up the messages.

(Bitbake rev: 90f91f7402ff69f3fe9fba5f94a53d371303ce34)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-10-07 09:37:33 +01:00
Erik Botö
56f39bcf22 bitbake: lib/bb/monitordisk.py: fix spelling in error text
(Bitbake rev: 78efda1224a99ed3d2ca6befb9fd719d82f6b0ae)

Signed-off-by: Erik Botö <erik.boto@pelagicore.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-10-02 15:41:21 +01:00
Valentin Popa
b43c8126de bitbake: HOB: Busy cursor when saving configuration
Display a busy cursor while the configuration is saved.

[YOCTO #4846]
(Bitbake rev: fdb2e0120f189e97b8fdb6eb72d77a3100ba3018)

Signed-off-by: Valentin Popa <valentin.popa@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-09-30 22:11:58 +01:00
Richard Purdie
3059b09223 bitbake: bitbake: Update to version 1.20.0
(Bitbake rev: 2666865a27cedbffc82800b4cab1d6cff5a0222e)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-09-24 18:04:16 +01:00
Richard Purdie
d4b9713ec6 bitbake: tests/data: Whitespace in key names is a really bad idea
The parser never has supported it, the datastore API did happen to work
but whitespace in key names is a really bad idea and not something I think we
should encourage or support.

Fix test case failures after excplitly ignoring it for variable expansion
purposes.

(Bitbake rev: a2074ddaba6f53962d6caf34dbd27bdbc259935b)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-09-22 12:19:45 +01:00
Richard Purdie
d9955a14fa bitbake: data_smart: Variable references don't contain newlines, spaces or tabs
The code is happily trying to expand variable names containing newlines,
spaces and tabs which are illegal characters in variable names. This
patch stops it doing this. This will change dependency checksums
since some rather weird dependencies were being attempted to be expanded.

(Bitbake rev: 37e13b852b33d98fa40f49dc1e815b3bbe912ff0)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-09-22 12:19:45 +01:00
Cristiana Voicu
3c41bea907 bitbake: hob: populate "Save image" dialog when saving changes to a custom image recipe
If my build starts from a YP standard image recipe, the 'Save image
recipe' dialog should be empty.

If my build starts from a custom image recipe, the 'Save image recipe'
dialog should populate the 'Name' and 'Description' fields with the values
set for the custom image recipe.

[YOCTO #5004]
(Bitbake rev: 10757c529fe0b4b9a39740d269831347a3aab4a0)

Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-09-22 12:19:45 +01:00
Richard Purdie
d28b1816e9 bitbake: bin/bitbake: Improve --help text
The --help text was rather inconsistent in style and plain incorrect in places,
using confusing terminology in others. I guess most people know what the options
do and don't read this but its confusing to new users.

This updates it to use the terms recipe and task consistently, remove
the references to stage, bbread and generally try and make the output
more useful.

[YOCTO #4856]

(Bitbake rev: 516311946c7bd14c84947dc44c3bb0563e5a9667)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-09-22 12:19:45 +01:00
Richard Purdie
35a1d37ac8 bitbake: build: Add BB_TASK_NICE_LEVEL to task code
On Linux its not possible for processes to regain a previous nice level after
it has changed. Its therefore not possible to have a core low priority and
then raise the priorities of individual tasks.

This variable allows us to do something like:

BB_TASK_NICE_LEVEL = "5"
BB_TASK_NICE_LEVEL_task-testimage = "0"

to give priority to specific tasks which the BB_NICE_LEVEL functionality
doesn't give us the option of.

(Bitbake rev: 94d82997220c6cfc7028f76719df028ba8254a5c)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-09-22 12:19:44 +01:00
Alexandru DAMIAN
ace48c2866 bitbake: bitbake: runqueue: add task hash to Queue events
Adding the sstate-related hash for all runqueue and
scenequeue tasks, as it's needed in the WebHob data.

(Bitbake rev: b6e2ce1cf7a0ede890f08fabf536a556dc4263c5)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-09-22 12:19:44 +01:00
Alexandru DAMIAN
24ae636fde bitbake: bitbake: event: adding generic event for metadata usage
Adding the generic bb.event.MetadataEvent that is
targeted specifically at metadata usage. This is
needed in order to let the metadata code send and receive
events during asynchrous execution without having
to define each event specifically in Bitbake.

Metadata code should subscribe to and fire the MetadataEvent
in order to communicate asynchronously, and identify
the object using event.type field, and parse the
data in the event.data field.

Knotty UI will ignore these event by default.

This deprecates RequestPackageInfo/PackageInfo, and that
event pair will be removed in the future.

(Bitbake rev: ae1ea51aaab73e010d1c3db39df058bebebc11dd)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-09-22 12:19:44 +01:00
Alexandru DAMIAN
a05d474e5a bitbake: bitbake: cooker, command: add a command to return global data
Adding the 'getAllKeysWithFlags' read-only command that will
return a dump of the global data state, together with specified
flags for each key. The flag list is passed in as the first
parameter to the command.

This will be used by UI clients to get the build configuration.

(Bitbake rev: 3e094da513e1220319288806cb76ddf804772afd)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-09-22 12:19:44 +01:00
Alexandru DAMIAN
0c51d610e1 bitbake: bitbake: cooker: get extra information from recipe cache
The loaded cache modules may add extra attributes to
the recipecache, that will be populated by the cache
classes required by the UI. These attributes
will be used by the UI to display relevant information.

Adds cachefields cache class field to specify
for each cache class which attributes will be set
in the recipecache.

Adds code to automatically expand depends tree with the
fields exported by the extra cache class.

Fixes a cache field name in the HOB UI.

(Bitbake rev: 47c171005fb3803d936e65fcd4436c643883ae16)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-09-22 12:19:44 +01:00
Alexandru DAMIAN
5ee82d4048 bitbake: bitbake: build, runqueue: adds info to the *runQueue* events
This patch adds task identifying information for all
runQueue and sceneQueue events, and for bb.build.Task* events.
This will allow matching event to specific tasks in the UI
handlers processing these events.

Adds RunQueueData functions to get the task name and task
file for usage with the runQueue* events.

Adds taskfile and taskname properties to bb.build.TaskBase.

Adds taskfile and taskname properties to the *runQueue* events

(Bitbake rev: b4a5e4be50d871a80dbe0993117d73f5ad82e38f)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-09-22 12:19:44 +01:00
Alexandru DAMIAN
f7621f47d8 bitbake: bitbake: cooker,runqueue: send the task dependency tree
Adding a CookerFeature that allows UIs to enable
receving a dependency tree once the task data has been
computed and the runQueue is ready to start.

This will allow the clients to display dependency
data in an efficient manner, and not recompute the runqueue
specifically to get the dependency data.

(Bitbake rev: 75466a53b6eece5173a9bfe483414148e4c06517)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-09-22 12:19:43 +01:00
Alexandru DAMIAN
ba83eb315d bitbake: bitbake: cooker,xmlrpc,servers: implement CookerFeatures
Implementing feature set selection that allows a client
to enable specific features in the server at connection time.

Only enabling of features is supported, as there is
no way to safely remove data loaded into the cooker.
Once enabled, a feature will remain enabled for the
life of the cooker.

Client-server connection now supports specifying the feature
set required by the client. This is implemented in the Process
server using a managed proxy list, so the server cooker
will now load dynamically needed features based on what client
connects to it.

In the XMLRPC server the feature set is requested by
using a parameter for registerUIHandler function.
This allows observer-only clients to also specify features
for the server.

The server code configuration now is completly separated
from the client code. All hardcoding of client knowledge is
removed from the server.

The extra_caches is removed as the client can now specify
the caches it needs using the feature. The UI modules
now need to specify the desired featureSet. HOB is modified
to conform to the featureSet specification.

The only feature available is CookerFeatures.HOB_EXTRA_CACHES
which forces loading the bb.cache_extra:HobRecipeInfo class.

(Bitbake rev: 98e594837aab89ea042cfa9f3740d20a661b14e2)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-09-22 12:19:43 +01:00
Richard Purdie
f1ff3c2fdc bitbake: data_smart: Fix variable reference issues
The change to use the expansion cache in VariableParse was incorrect as
it was adding in references it shouldn't have been. This patch corrects
the codepaths and ensures the references are correct.

The cache version is bumped since the previous bug could have leave
to invalid checksum calculations and a clean cache is therefore desireable.

The impact of the bug was that sstate was not getting reused when it should
and some tasks were also being rerun when they should not have been.

(Bitbake rev: 8a42d082315bd6ce091d006bf83476db257fa48b)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-09-18 13:27:30 +01:00
Richard Purdie
eaf06bc284 bitbake: cooker: return a copy checkPackages
The syntax used for checkPackages implies a copy is returned but it
did not do so. Make it so.

This is fixes universe builds where error messaages were being shown
but there should have only been warnings.

[YOCTO #5222]

(Bitbake rev: 97db2a0792d605f27d434bc6a4acce52857deee1)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-09-18 08:41:55 +01:00
Scott Rifenbark
825cfeb2e6 bitbake: usermanual.xml: Two new sections added to BitBake "Description"
1. Added a new section "Appending and Prepending (override style
   syntax)".  This section shows how the append and prepend operators
   work using the override style syntax.

2. Added a new section "Removing (override style syntax)".  This
   section describes the new "_remove" operator.

(Bitbake rev: 6983afab0ce8d82d102142636d5a570f7d86a844)

Signed-off-by: Scott Rifenbark <scott.m.rifenbark@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-09-17 22:27:56 +01:00
Richard Purdie
bb939628ec bitbake: Revert "bb.fatal: Raise a BBHandledException instead of exiting"
Sanity test failures are no longer fatal with this change so whilst
its the right idea, the code paths need more work.

This reverts commit a50017ba71250e1710a6425b60ac7e3f03d88295.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-09-17 19:16:10 +01:00
Richard Purdie
d96c512db1 bitbake: bb.fatal: Raise a BBHandledException instead of exiting
With new bitbake UIs having the cooker exit at 'random' points
in the codebase is problematic. This patch raises an exception
which matches the siutation instead.

(Bitbake rev: a50017ba71250e1710a6425b60ac7e3f03d88295)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-09-17 14:11:05 +01:00
Richard Purdie
d15652d78c bitbake: data: Optimise flag exclusion list handling
Move the variable lookup to the outer loop for performance, replacing
a now unneeded parameter (after the previous changes).

(Bitbake rev: 8a59a8707cecbde257fca169775ce8ff7709928b)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-09-17 14:11:05 +01:00
Richard Purdie
9a32eca022 bitbake: data: Optimise flag lookup in build_dependencies
When looking up flag variable dependencies, large chunks of the function
aren't needed. Optimise the function flow accordingly for speed.

(Bitbake rev: 1bf3aee698ad35f6815ea2c75471a96511a29d55)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-09-17 14:11:05 +01:00
Richard Purdie
d7e7b991ce bitbake: data: Optimise build_dependencies a little
Instead of multiple calls to getVarFlag, make one call to getVarFlags, only expanding
the flags that need to be expanded. This improves performance.

(Bitbake rev: eba1e9545cc933820d40de96f023b2307b3c4d0b)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-09-17 14:11:04 +01:00
Richard Purdie
d6ac67f9af bitbake: data_smart: Cache the fact a variable accesses another even if its unset
If a variable references another but it isn't set at present, the
reference wasn't stored. It really should be marked as a reference
and the higher level dependency code can handle as appropriate.

(Bitbake rev: b05b748b2153c941b95cd36fb22aaafc4dbf3791)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-09-17 14:11:04 +01:00
Richard Purdie
385bd4410d bitbake: data_smart: Allow flags to use the expand cache
(Bitbake rev: a0122ab80df21597291ff32ff7fbaa4de0347a6f)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-09-17 14:11:04 +01:00
Richard Purdie
a61519f3fb bitbake: data_smart: Allow expansion of flags in getVarFlags
Allow a list of flags to expand to be passed into getVarFlags. This
is useful within bitbake itself to optimise performance of the
dependency generation code.

(Bitbake rev: a3ae7efdf750fc5bb9ff5a75defbcfdab1912dbe)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-09-17 14:11:04 +01:00
Alexandru DAMIAN
0902850e97 bitbake: data_smart: Add explict None checks
Simple if xxx checks end up calling len(xxx). We're interested in the specific case
of None which means we can break out the iterator much earlier after the first
item. This adds in the specific tests for None in what is a hot path in the
data store code which gives small performance gains.

(Bitbake rev: a4d81e44a7cd3dafb0bf12f7cac5ff511db18e60)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-09-17 14:11:04 +01:00
Richard Purdie
66c9c01b2b bitbake: data: Cache an list of export variables
Compute a cache of the list of potential export variables so
that we don't have to compute the list from scratch.

(Bitbake rev: f41f46f7eaa6889edeb3a4e4ddedc07084686c60)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-09-17 14:11:04 +01:00
Richard Purdie
36b4fcde7a bitbake: data: Use direct iteration, not keys()
Profiling shows the creation of keys() has overhead and we're better using
an iterator rather than the memory associated with the huge list of keys
when iterating the whoe datastore. We minimise the number of times
we do this to twice only per recipe.

(Bitbake rev: e63448d9ee331b0f45fb9a0197d0dbee49eb2fa0)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-09-17 14:11:04 +01:00
Richard Purdie
f78db82e1a bitbake: data_smart: use the expand_cache in VariableParse
When in VariableParse, use the expand_cache if possible rather than looking
up data. Ultimately it would come from the same place but this short cuts
a heavily used code block for speed improvements.

(Bitbake rev: f682b8b83d21d576160bac8dc57c4c989b4dc555)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-09-17 14:11:03 +01:00
Richard Purdie
43f1867e32 bitbake: data_smart: Improve variable expansion regexp
Debugging showed the variable expansion regexp was catching python
expressions (starting with @). Since these are caught by their own
dedicated regexp, stop matching these for the plain variable expansion
for small performance improvements.

(Bitbake rev: c630d564285f55f9db10c18269bd310df797430e)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-09-17 14:11:03 +01:00
Richard Purdie
c971868360 bitbake: siggen: Use lookup cache exclusively
All the values we need are already guaranteed to be in the lookupcache
so rather than fetch variables again, just use the cache. This gives a
small performance improvement and simplifies the code.

(Bitbake rev: 8ffaba61da7f195d7c3b64dce35b6a56272aecae)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-09-17 14:11:03 +01:00
Richard Purdie
43c670accc bitbake: data: Be explicit in data_db check
The if statement current causes the size of parent to be calcuated which
is like a len() operation on a datastore. Since we're only interested
whether the value is none, checking explictly for this gives a
small performance gain.

(Bitbake rev: 43a245bde318545ea75ca4ce7894395c1cf9b32a)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-09-17 14:11:03 +01:00
Richard Purdie
602bb695cf bitbake: cooker: Allow profiling of the parser in profile mode
(Bitbake rev: f8a6e4caed4dc3dcf207aecc4ea5f438027da8be)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-09-17 14:11:03 +01:00
Alexandru DAMIAN
eb4854f903 bitbake: cooker: Avoid duplication for taskdata creation
Clean-up to avoid duplication and promote code reuse to factor
taskdata creation into a common function.

[RP: minor tweaks]

(Bitbake rev: 468c221449290c4f196e87f7d8e23fcd7db86135)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-09-17 14:11:03 +01:00
Alexandru DAMIAN
a828c89822 bitbake: runqueue: add runQueueTaskSkipped event
Adding a runQueueTaskSkipped to notify that the tasks that are not
run either because they are set-scened or they don't need an update
(timestamp was ok).

(Bitbake rev: cf4a0c7aa82090876ae652b611acfab3ce2688f7)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-09-17 14:11:02 +01:00
Richard Purdie
1181e69119 bitbake: build: Add logfile to add TaskBase events
We add the path to the logfile for all Task events except TaskInvalid
so that we can trace back the logfile locations at some future point.

TaskInvalid doesn't ever have a logfile.

(Bitbake rev: 8344d84c609446f59f9619cc7ca0d693b7e2bbd6)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-09-16 13:12:34 +01:00
Richard Purdie
ca1b5ddb86 bitbake: cooker: updateCache should rause exceptions, not sys.exit
Exiting from the server is antisocial, instead we should raise an exception. This
will correctly fail the current command and reset the server state. We use
the handled exception since for these conditions to occur, something was
already displayed to the user.

(Bitbake rev: dacc94bcace85a2e95aee2dccd8e680c59e4545f)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-09-14 08:20:38 +01:00
Richard Purdie
926b60f6e4 bitbake: knotty: Cleanup error/interruption handling
Only display a CommandFailed ERROR: message if there is an error to display.

Only display an errors summary if we actually displayed errors.

(Bitbake rev: 568ea00acd226d48e725bb01d4f8c410ed1eaa61)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-09-14 08:20:38 +01:00
Richard Purdie
ef7e3882a9 bitbake: command: Treat empty messages as failures, not CommandCompleted
Empty messages should trigger CommandFailed, not CommandCompleted as
otherwise the exit code will be incorrect.

(Bitbake rev: 70a8ead31f9ffc987d9c6db61a926f7a9af8f8b1)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-09-14 08:20:38 +01:00
Richard Purdie
0519d1ae13 bitbake: cooker/command: Add finishcommand to reset cooker state
After running a command on the server, it needs to reset to the initial
state. This ensures that subsequent clients start from a known state
and notice any configuration changes.

Ultimately we may want to do more than this buts a good start and better
than nothing.

(Bitbake rev: dd15648fc2654b8d7c3e00ea7ab3dbf04f24f24b)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-09-14 08:20:38 +01:00
Richard Purdie
7663a52061 bitbake: cooker: Rename confusing 'stop' state to 'forceshutdown'
The shutdown state causes the server to finish what its doing, stop was
them meant to completely stop it. It doesn't mean the server is stopped
though. Renaming the current stop event for forceshutdown gives more
meaning to what it actually does. The stopped namespace then becomes
available to indicate a completely stopped server.

(Bitbake rev: 12e9d33bfae5294e3870dfd1202f63383ad05e92)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-09-14 08:20:38 +01:00
Richard Purdie
2174a51ee8 bitbake: cooker: Clean up init/reset configuration code
Currently the cooker event data isn't rebuilt upon reset and the cache
configuration cannot be changed after init. These are both bad things
and this patch refactors the init/reset code so that it is possible
to reconfigure the server.

(Bitbake rev: 1193b8d76fcb6cb87e9ec135a2514370d7dd90ac)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-09-14 08:20:38 +01:00
Richard Purdie
fe1258d478 bitbake: cooker: Drop obsolete worker test
This call only ever happens in cooker context now so we can drop the
nasty worker check from here.

(Bitbake rev: bc0b30199a8e3624c5b9914430adbcc7c6bd4497)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-09-14 08:20:37 +01:00
Alexandru DAMIAN
a392877e57 bitbake: bitbake: xmlrpc: delete unused code BitBakeUIEventServer
BitBakeUIEventServer is an unused class that pushes UI
events over a separate thread.

The current version of XMLRPC server works just fine with
the classic UI event handlers, so this class is not needed.

(Bitbake rev: 8e8e17631d790271b1be747c4b45059ec38ab606)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-09-13 17:45:22 +01:00
Richard Purdie
dd36930f3f bitbake: data_smart: Account for changes in append/prepend/remove in the config hash
bitbake wasn't reparsing when _remove items were added to its configuration
and equally, appends/prepends were also being badly tracked. This
change enrures these variables are accounted for in the configuration
hash.

[YOCTO #5172]

(Bitbake rev: 62914f9208ef2427a34daa523af857f4027900eb)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-09-12 17:03:17 +01:00
Richard Purdie
19c3c6dff3 bitbake: utils: Fix BB_PRESERVE_ENV
BB_PRESERVE_ENV wasn't working since data.inheritFromOS wasn't getting a
correct list of keys to import into the data store. This fixes
things so it does add all environment variables into the data store
when BB_PRESERVE_ENV is used.

(Bitbake rev: 843e9339c5ee3c99657a40a0e2c7dbd777b6ef06)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-09-11 11:05:05 +01:00
Alexandru DAMIAN
2044668a41 bitbake: bitbake: runqueue: add sceneQueueTaskCompleted event
Adding an event to be fired when a scene task is completed.
It is analogous to the run task completed event, and has
been missing for some reason.

(Bitbake rev: 73b8f4d3fbeaf1b330a66d76012d0a5cef8dbe2d)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-09-10 15:57:20 +01:00
Robert Yang
87771db34c bitbake: runqueue.py: check whether multiple versions of the same PN are due to be built
There would be an race issue if we:

$ bitbake make-3.81 make-3.82

This because they are being built at the same time which would cause
unexpected problems, for example:

[snip]
ERROR: Package already staged (/path/to/tmp/sstate-control/manifest-qemux86-make.populate-sysroot)?!
ERROR: Function failed: sstate_task_postfunc
[snip]

Or there would be python's strack trace such as:

[snip]
 *** 0004:    mfile = open(manifest)
     0005:    entries = mfile.readlines()
     0006:    mfile.close()
     0007:
     0008:    for entry in entries:
Exception: IOError: [Errno 2] No such file or directory: xxx
[snip]

[YOCTO #5094]

We can quit earlier to avoid this kind of issue when two versions of the same PN
are going to be built since this isn't supported.

(Bitbake rev: ab377c00c33a2d296bfda1b0b6c2a62b29d1004f)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-09-09 16:28:46 +01:00
Robert Yang
0583714a57 bitbake: providers.py: enhance the runtime debug degbug messgae
The runtime provider debug message is the same as the build time debug
message, make them different would be better.

[YOCTO #5067]

(Bitbake rev: 92b624cbc2711d3d859994099fb63918dfd0031a)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-09-09 16:28:46 +01:00
Richard Purdie
2ab1bf27f7 bitbake: prserv: Ensure data is committed
In exclusive mode, we need to complete the transaction for writes to make
it to the database. Therefore add sync calls to ensure this happens.

Autocommit mode is significantly (100 times) slower so caching the
data is of significant benefit.

(Bitbake rev: 4e55f7821786a59c2cd7dbd8bfa2a22f5f196e99)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-09-08 10:51:56 +01:00
Christopher Larson
3dccd57360 bitbake: contrib/vim: merge changes from vim-bitbake repo
commit 00ca441614695b4261d8d4f31b7ef0e3e3784282
Merge: 8cc367d bb88c0f
Author: Christopher Larson <kergoth@gmail.com>
Date:   Thu Aug 22 16:42:42 2013 -0700

    Merge pull request #6 from staticshock/multi-line-strings

    Remove "keepend" and "excludenl" directives

commit bb88c0fd4ad2b7b9c8c4c73def2b3cb20c473ac3
Author: Anton Backer <olegov@gmail.com>
Date:   Sat Jul 13 01:24:15 2013 -0400

    Remove "keepend" and "excludenl" directives

    It looks like these were never actually used correctly, and were doing
    more harm than good. "keepend" on bbString, for instance, prevented
    proper nesting of ${@python} in strings. Similarly, a balanced pair of
    { } braces inside a shell function would force the function to terminate
    early if the closing brace was on its own line.

    So far I've seen absolutely no negative consequences from removing
    these, but a bunch of positive consequences.

    Fixes #1

commit 8cc367d01f4c699be5fcc072de59e6f2f14a138b
Merge: c58628c eec6b7f
Author: Christopher Larson <kergoth@gmail.com>
Date:   Thu Aug 22 09:46:46 2013 -0700

    Merge pull request #4 from staticshock/function-names

    Parse function names with nested vars

commit c58628ca517cd25985361fc0d27863521cc28a5d
Merge: dfb0f7c a890982
Author: Christopher Larson <kergoth@gmail.com>
Date:   Thu Aug 22 09:43:40 2013 -0700

    Merge pull request #5 from yoyko/master

    syntax: python expansion (${@...}) inside shell functions

commit a890982b7c33a6e363b12d6cb69e22b4bbc0f317
Author: Jozef Šiška <yoyo@ksp.sk>
Date:   Thu Aug 22 13:20:45 2013 +0200

    syntax: python expansion (${@...}) inside shell functions

    Signed-off-by: Jozef Šiška <jsiska@nuvotechnologies.com>

commit eec6b7f6f0472787929f424968f9a0d78ac4af08
Author: Anton Backer <olegov@gmail.com>
Date:   Fri Jul 12 22:16:01 2013 -0400

    Parse function names with nested vars

    For instance, pkg_postinst_${PN}

    Fixes #3

commit dfb0f7c0d51556448cba79b474b8c19b9cded9af
Author: Christopher Larson <chris_larson@mentor.com>
Date:   Fri Jun 1 18:57:13 2012 -0400

    syntax: add ?= flag def

    Signed-off-by: Christopher Larson <chris_larson@mentor.com>

commit 589a62a00709ca822a42327e7086008aba2d9933
Author: Christopher Larson <kergoth@gmail.com>
Date:   Fri Dec 9 22:25:47 2011 -0700

    ftplugin: set commentstring

    Signed-off-by: Christopher Larson <kergoth@gmail.com>

commit 7ffc80b3fb4ddf68cc5a69bdc63ab03d70c44f87
Author: Chris Larson <chris_larson@mentor.com>
Date:   Thu Jun 2 15:27:48 2011 -0700

    Handle +=/=+ for flags

    Signed-off-by: Chris Larson <chris_larson@mentor.com>

(Bitbake rev: f5f479bbe9b74622cd54c8d6ba8786661a3ae3e6)

Signed-off-by: Christopher Larson <kergoth@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-09-06 22:31:33 +01:00
Richard Purdie
b093b6c524 bitbake: knotty: Reduce refresh of footer
When displaying larger number of events the client can get caught up in displaying
the footer, then immediately overwriting it. To avoid this, wait for pauses
in the event stream before displaying the footer to give a slightly more
friendly feel to the UI.

(Bitbake rev: 5d706c7cd6ee8d83b67ff18312d4c8119bea8878)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-09-06 22:31:33 +01:00
Peter Kjellerstedt
373e209605 bitbake: bitbake: Ensure ${DATE} and ${TIME} are consistent
Due to the worker split the ${DATE} and ${TIME} variables could end up
with different values for different workers.

E.g., a task like do_rootfs that is run within a fakeroot environment
had a slightly different view of the time than another task that was not
fakerooted which made it impossible to correctly refer to the image
generated by do_rootfs from the other task.

(Bitbake rev: 756cc69ebf8bfe8455d0c90f288dd51be2499773)

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-09-06 22:31:33 +01:00
Cristiana Voicu
c8957a4fbe bitbake: hob: remove custom image from the images list when not needed
Selected custom image recipes should be cleared from the combo the moment
you change your selection. The idea is to always perform the selection of
those images in the same way (i.e through the "Select from my image
recipes" option).

[YOCTO #5001]
(Bitbake rev: 94483ee5ae9f4051bccd660c4718c36564e17161)

Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-09-06 22:31:33 +01:00
Cristiana Voicu
b501b832eb bitbake: hob: limit the description size when a custom image is saved
[YOCTO #5003]
(Bitbake rev: 9aec9ee41d4d893325d9bf92b8a53f2e68e4973d)

Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-09-06 22:31:32 +01:00
Khem Raj
da29dcce83 bitbake: newbb.vim: Use 'git config' instead of git-config
Newer versions of git do not have the '-' concatenated command

(Bitbake rev: 7adb05978b917e624016bae1700db23bd280b41a)

Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-09-06 22:31:32 +01:00
Richard Purdie
e92b2dbfcd bitbake: data_smart: Move getVar expand cache handing to fix _remove operations
DISTRO_FEATURES_remove = "opengl" wasn't working as expected. The reason
turned out the be the indirect reference to opengl and the fact _remove was
operating on unexpanded data.

This patch rearranges some code to ensure we operate on expanded data
by moving the expand cache handing into getVarFlags instead of getVar.

(Bitbake rev: 181899bd9665f74f8d1b22d2453616ad30d26d9e)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-09-06 22:31:32 +01:00
Cristiana Voicu
f41b7a7d4d bitbake: hob: remove PACKAGE_INSTALL variable setting from hob
Before saving the packages for a custom images in a .bb file,
the packages were saved in bitbake memory. Now all the variables
are saved in conf file, so saving PACKAGE_INSTALL is not needed anymore.
Moved were LINGUAS_INSTALL is set, because both conditions are for testing
if a custom image is saved.

[YOCTO #5101]
(Bitbake rev: 8757f962b92e7668f40d2d8bd9e762b152f91f7b)

Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-09-04 14:18:49 +01:00
Paul Eggleton
e63c577573 bitbake: bitbake-worker: ensure BUILDNAME is available during execution
BUILDNAME is set from cooker by default, so since the worker split it
will not be set when executing functions. In OpenEmbedded this results
in /etc/version (which is populated from BUILDNAME) not having any
content. Pass this variable value through to the worker explicitly to
fix the issue.

Fixes [YOCTO #4818].

(Bitbake rev: 92940b0427d9b2b3f95e27c230ec1e36638a34bc)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-09-04 14:18:49 +01:00
Cristiana Voicu
1b814498b6 bitbake: bitbake/cooker: fix some calls of cookerdata.findConfigFile method
Cookerdata.findconfigFile method has a new parameter. Changed some calls.

(Bitbake rev: dce0f9d4afe0986e2dd0146944fc4ac9dde275e4)

Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-09-02 12:26:20 +01:00
Richard Purdie
c7994f83ba bitbake: runqueue: Fix scenequeue to pass file descriptors, not a float
This was missed off in a previous patch.

(Bitbake rev: ad7664edd40fa46e6f6fec2144403e3b6fc3a639)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-09-02 09:16:24 +01:00
Alexandru DAMIAN
6d9e3a1938 bitbake: server/xmlrpc: stop server on client exit
If the server only expects a single client to connect (i.e.
no bind parameter set, so there is no way for the
clients to get the server port), stop the server after
the first client exits.

(Bitbake rev: eb6bae56f62082bf147045311154cbae4bca0f4c)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-09-01 15:51:11 +01:00
Richard Purdie
c7b3429032 bitbake: prserv/serv: Settle on two threads for optimal performance
Using the threading mixin class resulted in large amounts of memory
being used by the PR server for no good reason. Using a receiver thread
and a thread to do the actual database operations on a single connection
gives the same performance with a much saner memory overhead so
switch to this.

(Bitbake rev: e08455d5f3b8e96765942b9c3b9767c30650557d)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-09-01 15:51:11 +01:00
Richard Purdie
6e15fee9ee bitbake: build: Fix profile file names
Using the basename of the .bb file is not unique, for example xxx-native
and xxx can overwrite each other. If this happens whilst running, you can
get odd backtraces as one file is parsed as another tries to write out
new data.

Avoid issues by using PN for the output filename instead.

(Bitbake rev: c9534f8e59d44b885334607ed90a3be2e492ec69)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-09-01 15:51:11 +01:00
Richard Purdie
bd023ef9c0 bitbake: serv/db: Don't use BEGIN/COMMIT
Since we don't support using multiple servers on the same database file,
don't use the BEGIN/COMMIT syntax and allow writes to the database
to work ~100 times faster with no transaction locking.

(Bitbake rev: 42144a54979658f93fbbb43f7e271c1fff4d88ff)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-09-01 15:51:11 +01:00
Richard Purdie
883813deae bitbake: serv/db: Take an excluside lock on the database
We only support one server using the database at a time so take an exclusive
lock and avoid later lock overhead.

(Bitbake rev: e3e39be6f2d063858c92971ce8ccd89c95d4f26d)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-09-01 15:51:11 +01:00
Richard Purdie
3e5abff7da bitbake: serv/db: Fix looping upon database locked issues
If the database is locked we will get an immediate error indicating so,
there is no retry timeout. The looping code is therefore useless, the loop
count is near instantly exceeded.

Using a time based retry means we can wait a sensible time, then gracefully
exit.

(Bitbake rev: 9f9e6d87007ea87e62495705464f4232c996a165)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-09-01 15:51:11 +01:00
Richard Purdie
b306d7d9a4 bitbake: server/process, server/xmlrpc, runqueue: Use select.select() on fds, not time.sleep()
The existing backend server implementations were inefficient since they
were sleeping for the full length of the timeouts rather than being woken when
there was data ready for them. It was assumed they would wake and perhaps did
when we forked processes directory but that is no longer the case.

This updates both the process and xmlrpc backends to wait using select(). This
does mean we need to pass the file descriptors to wait on from the internals
who know which these file descriptors are but this is a logical improvement.

Tests of a pathaolgical load on the process server of ~420 rapid tasks
executed on a server with BB_NUMBER_THREAD=48  went from a wall clock
measurement of the overall command execution time of 75s to a much more
reasonable 24s.

(Bitbake rev: 9bee497960889d9baa0a4284d79a384b18a8e826)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-09-01 15:51:10 +01:00
Richard Purdie
5fd04b0258 bitbake: prserv: Allow 'table is locked' matching for retry loop
Try and avoid errors like "ERROR: database table is locked: PRMAIN_nohist"
by retrying if we see the string "is locked".

(Bitbake rev: 1a175b51f80d13f747b653d29e9c0d2201b5109c)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-08-30 17:53:26 +01:00
Richard Purdie
cb939416b7 bitbake: server/xmlrpc: Increase timeout to 60s
This is a better value that the earlier infinite timeout yet still
allows for servers with high loads. It does mean the bitbake process
can hang at exit for the timeout period but that should never happen
and only happened for me in some test cases which wouldn't happen
in normal use.

(Bitbake rev: ab8d926b9bc27c58011e7db9327e031ac76ba34b)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-08-30 17:42:39 +01:00
Richard Purdie
4322948564 bitbake: cookerdata: Allow bblayers.conf to be found using BBPATH
It should be possible to run a build anywhere on the filesystem and have
bitbake find the correct build directory if its set somehow. The BBPATH
variable makes perfect sense for this usage. Therefore use any available
value of BBPATH to search for conf/bblayers.conf before walking the parent
directory structure.

This restores the option of being able to run bitbake from anywhere if
the user has set things up to operate in that environment.

(Bitbake rev: e86336b3fe245bc97fe74c9b9d6a21d38a536fb7)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-08-30 17:42:39 +01:00
Richard Purdie
c7a389729a bitbake: cookerdata: Set TOPDIR when using bblayers.conf
By definition, bblayers.conf is at the top of the build tree. We'd like
to support running bitbake anywhere within that build tree but TOPDIR
gets set to wherever cwd is. Change the code to reset TOPDIR
to the top of the build directory.

This shouldn't break anything but does make the system more usable.

(Bitbake rev: b266db27de0bba19a418e4d42e870649136b116b)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-08-30 17:42:39 +01:00
Richard Purdie
d5e860be4c bitbake: prserv/serv: Multithread the server
This makes the PR server multithreaded and able to handle multiple connections
at once which means its no longer a build bottle neck when serving one connection
at a time. I've experimented and database connection for each thread seems to
cause the least issues, pushing the contention for sqllite to handle itself.

This means moving the db/table connection code into the actual function methods.
It doesn't abstract well as a function since we need the db object around for
the lifetime of the function as well as the table else we lose the connection.

(Bitbake rev: bf9be2029b2bded5f532bdda4c38ae3dff5d1cf6)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-08-29 00:13:22 +01:00
Richard Purdie
06d7664590 bitbake: prserv/db: Threading fixes
Enabling threading for the PRServer causes a number of issues. Firstly is
the obtuse error:

sqlite3.InterfaceError: Error binding parameter 0 - probably unsupported type

which is due to the class not being derived from object. See:
http://docs.python.org/2/library/sqlite3.html#registering-an-adapter-callable

Secondly, we want to enable multithreadded access to the database so we do this
when we open it. This opens the way up to multithreading the PR server.

(Bitbake rev: 5709efc2ff1e36529bd28f49cd093ccfa7abff7f)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-08-29 00:13:22 +01:00
Jason Wessel
84708a4524 bitbake: serv.py: Fix regression from 972bc43e6d5b
commit 972bc43e6d5b1207b944b3baa8f9805adb35dda7 (serv.py: Fix hang
when spawned dynamically with bitbake) introduced a regression,
because the wrong patch was submitted.  The syntax was incorrect in
the original patch.  The logger iterator must be used with a call to
getLogger().

[YOCTO #5059]

(Bitbake rev: 85fed8acc3af3e15bf119db2f51c486a9de3646b)

Signed-off-by: Jason Wessel <jason.wessel@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-08-28 09:29:46 +01:00
Christopher Larson
87f413f617 bitbake: data_smart: allow removal of multiple words at once with _remove
FOO = "foo bar baz"
    FOO_remove = "foo baz"

(Bitbake rev: 04127dec207d6dfc0ada56c5cc67ec9ad30517a8)

Signed-off-by: Christopher Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-08-28 00:34:13 +01:00
Christopher Larson
092190e20b bitbake: data_smart: use a split/filter/rejoin for _remove
This is more idiomatic, and from the limited performance testing I did, is
faster as well. See https://gist.github.com/kergoth/6360248 for the naive
benchmark.

(Bitbake rev: 1aa49226d5a2bac911feeb90e3d9f19529bc1a3e)

Signed-off-by: Christopher Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-08-28 00:34:13 +01:00
Jason Wessel
da6260f95f bitbake: serv.py: Fix hang when spawned dynamically with bitbake
The PRServer has the possibility to hang indefinitely blocking on a
semaphore processing a xmlrpc request to send an event back to the
main bitbake instance.  This was observed during a "bitbake -e" on a
heavily loaded machine and the main bitbake instance and cooker exited
before the PRServer emitted its first log.

The stack trace is provided below as to show what happens every time a
logger.info() is executed in the PRServer.  Not only does it write to
the stream handler but it also tries to send the event to the main
event processor.

    self._notempty.acquire()
    self.queue.put(event)
    _ui_handlers[h].event.send(event)
    fire_ui_handlers(event, d)
    fire(record, None)
    self.emit(record)
    hdlr.handle(record)
    self.callHandlers(record)
    self.handle(record)
    self._log(INFO, msg, args, **kwargs)
    (self.dbfile, self.host, self.port, str(os.getpid())))
    self.work_forever()
    pid = self.daemonize()
    self.prserv.start()
    singleton.start()
    self.prhost = prserv.serv.auto_start(self.data)
    cooker.pre_serve()
    bb.cooker.server_main(self.cooker, self.main)
    self.run()
    code = process_obj._bootstrap()
    self._popen = Popen(self)
    self.serverImpl.start()
    server.detach()
    server = start_server(servermodule, configParams, configuration)
    ret = main()

It was never intended for the PRServer to send its logs anywhere but
its own log file.  The event processing is an artifact of how the
PRServer was forked and it inherits the event log handlers.  The
simple fix is to clean up and purge all the log handlers after the
fork() but before doing any of the typical PRServer work or logging.

(Bitbake rev: 972bc43e6d5b1207b944b3baa8f9805adb35dda7)

Signed-off-by: Jason Wessel <jason.wessel@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-08-28 00:25:56 +01:00
Cristiana Voicu
1472b9ebff bitbake: hob: fixes for image combo box
When an image from scratch is selected, and recipes parsing
is canceled, the image shown by the combo box isn't correct.

[YOCTO #5000]
(Bitbake rev: f8166ace0bd9155199166990ce15da24eb2e793b)

Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-08-27 11:47:35 +01:00
Cristiana Voicu
39052cd703 bitbake: hob: add event handlers filtering in Hob
Create the _evt_list for hob; it is longer than the knotty
uses because it handles more events.

(Bitbake rev: 715aed74f972bb6e9b6a5130ca9ede48d4f79f0a)

Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-08-27 11:16:58 +01:00
Cristiana Voicu
8d134e999e bitbake: bitbake/event.py: UIhandler filter should work without a mask
The default for the mask will be * (all the handlers)

(Bitbake rev: 4c95e5f46cf2a656100bbf5a0e5a09d506abf9b9)

Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-08-27 11:16:58 +01:00
Richard Purdie
c34bb765fa bitbake: command.py: Call updateCache for all states != running
updateCache handles the logic for shutting down the parsing so we need
to call it for all cases when we're not running.

This fixes hangs if Ctrl+C is pressed during parsing.

(Bitbake rev: 552b8935dd2f9f11e8d5c08a597a7e966b891480)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-08-26 13:43:47 +01:00
Richard Purdie
d7571c0390 bitbake: data_smart: Add _remove operator
There are long standing complaints about the fact its very difficult
to remove a portion of a variable. The immediate request is for a -=
and =- operator. The trouble is that += and =+ are "immediate"
operators and are applied straight away. Most people would expect
-= and =- to be deferred to have the effect most people desire and
therefore implementing -= and =- would just make the situation more
confusing.

This deferred operation is much more similar to the override syntax
which happens at data store finalisation. The _remove operator is
therefore in keeping with the _append and _prepend operations.

This code is loosely based on a patch from Peter Seebach although it
has been rewritten to be simpler, more efficient and avoid some
potential bugs.

The code currently only works on space delimited variables, which
are by far the most commom type. If bitbake is ehanced to support
types natively in future, we can adjust this code to adapt to that.

(Bitbake rev: 9c91948e10df278dad4832487fa56888cd58d187)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-08-26 11:29:46 +01:00
Richard Purdie
69aab78dd8 bitbake: bitbake: Add ui event handlers filtering
Add functionality to allow UIs to update and change the types of events they
recieve. To do this we need to add a new command and also need to be able
to obtain the current event hander ID. In the case of xmlrpc, this is
straightforward, in the case of the process server we need to save the result
in a multiprocessing.Value() so we can retrive it. An excplit command
was added to the server API to facilitate this.

The same function can also be used to mask or unmask specific log messages,
allowing the UI to optionally differ from the standard set of message
filtering.

Based upon work by Cristiana Voicu <cristiana.voicu@intel.com>

(Bitbake rev: ba5a6c88785d9889d4172ec79937ac2a5555327e)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-08-26 11:29:46 +01:00
Richard Purdie
bfab986ccd bitbake: server/xmlrpc/prserv: Add sane timeout to default xmlrpc server
The standard python socket connect has long timouts which make sense for remote
connections but not local things like the PR Service. This adds a timeout
parameter to the common xmlrpc server creation function and sets it to a more
reasonable 5 seconds.

Making the PR server instantly exit is a good way to test the effect of this
on bitbake.

We can remove the bodged timeout in the PRServer terminate function which
has the side effect of affecting global scope.

(Bitbake rev: 8c01cff94787abbb64fbdf0c16cd63f8f97a7e03)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-08-26 11:29:46 +01:00
Richard Purdie
a03a423c60 bitbake: process: Improve exit handling and hangs
It turns out we have a number of different ways the process server termination can
hang. If we call cancel_join_thread() on the event queue, it means that it can be left
containing partial data. This means the reading of the event queue in the terminate()
function can hang, the timeout and block parameters to Queue.get() don't make any
difference.

Equally, if we don't call cancel_join_thread(), the join_thread in terminate()
will hang giving a different deadlock.

The best solution I could find is to loop over the process is_alive() after requesting
it stops,  trying to join the thread and if that fails, try and flush the event
queue again.

It wasn't clear what difference a force option should make in this case, we're
gracefully trying to empty queues and shut down regardless of whether its a SIGTERM
so I've simply removed the force option.

(Bitbake rev: c5c8f33ca4b81877a0115887849881001b745bf0)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-08-26 11:29:46 +01:00
Martin Jansa
aa0b237144 bitbake: parse: Return IOError when including file with absolute path
* resolve_file was behaving different when relative and absolute
  paths were passed to it

* include relative-path/non-existent-file.inc
  works correctly resolve_file throws IOError, BBHandler.py:handle()
  doesn't catch it, ConfHandler.py:include() catches IOError and shows:
  DEBUG: CONF file 'relative-path/non-existent-file.inc' not found
* include /absolute-path/non-existent-file.inc
  was failing, because resolve_file just returns fn,
  BBHandler.py:handle() calls bb.parse.mark_dependency(d, abs_fn)
  which throws:
  OSError: [Errno 2] No such file or directory: '/absolute-path/non-existent-file.inc'
  and parsing fails.
  Ad isfile() test for absolute fn and throw IOError to make
  resolve_file behavior consistent for both paths.

* I know we had some issues with -b relative-path-to-recipe.bb and
  absolute path, so consider this patch only as RFC and documentation of
  this problem

* Catch OSError too in ConfHandler.py:include() e.g. in case the file exists, but user
  cannot read it or something like that.

(Bitbake rev: b0bbd89a4f0b98fa1ab28b8e0526cd9ddb76fa57)

Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-08-26 11:29:46 +01:00
Richard Purdie
501e1a321d bitbake: prserv/serv: Fix pid file removal
Mark Hatle spotted there were pid files being left around. This patch
fixes things so the removal function is called correctly, the code
contained a typo.

(Bitbake rev: c696a16c8200c31c52750037eeafe07e065b6517)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-08-23 17:43:40 +01:00
Richard Purdie
25e410b743 bitbake: event/msg: Add primitive server side UI log record filtering
Currently one of the bigger bottlenecks in bitbake is passing all the
log messages over IPC to the UI. This is worthwhile if the UI is going
to use them, pointless otherwise. The memory resident bitbake suffers
from this performance issue particularly badly.

This patch filters the log events on the server side with the global
log levels and hence reduces the traffic. This speeds up parsing
(18.5s down to 17s) and bitbake general command overhead is reduced
(7.3s for a NOP to 6.2s).

What isn't added here is general event filtering or the ability to
change the log levels once set. Provision is made for adding this
in a follow up patch though.

(Bitbake rev: 1bf0e88f57ba0bca62532e81d0d62cf88e2abcbb)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-08-23 17:29:54 +01:00
Nicolas Dechesne
9f0f799c46 bitbake: build.py: create symlink for run.do_xxx scripts
The 'courtesy' symlink for log.do_xxx are quite useful when debugging, so
with this commit, we now get similar 'courtesy' symlink for run.do_xxx
scripts.

We only create symlink for tasks, not individual functions.

The symlink is create right before the actual runfile is created, indeed
we cannot create the symlink right after running the task since a failure
or execption can happen, in which case the symlink wouldn't be created,
and symlink are particularely useful when the task failed!

Another option would be create the symlink after the runfile is created,
and before the script is executed, but that means we need to duplicate the
code in case of Shell vs Python task.

(Bitbake rev: a672b39c5d529ba85d72eee8fef4c4273eaa5397)

Signed-off-by: Nicolas Dechesne <nicolas.dechesne@linaro.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-08-23 17:29:54 +01:00
Cristiana Voicu
136cb5e6b8 bitbake: hob: changes to image combo box from image configuration page
In some cases when a new machine is selected, the image combo
shows the same option. Fixed this issue, because the image
combo box should be reseted.

Fix the counter for the options in the combobox. This bug
was introduced by the templates functionality. The combo box had some
last changes, and I forgot about this counter.

[YOCTO #4858 & #5000]
(Bitbake rev: 457fd80ee6b1b2bcef463e3a83e048da2f8bf805)

Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-08-22 12:51:59 +01:00
Andrei Dinu
8d1e77ec31 bitbake: propertydialog.py : Scrollable windows for long tooltips
In some cases, the length of the description and the brought
in by field was too big. That led to the size of the property
dialog exceeding Hob's size. For long tooltips we use
scrollable windows now.

[HOB #4321]

(Bitbake rev: 78ecabf19bf01e5a662b6e2b865cd93bf47d962b)

Signed-off-by: Andrei Dinu <andrei.adrianx.dinu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-08-22 12:38:12 +01:00
Peter Seebach
f6ab9274bc bitbake: build.py: Provide useful diagnostics when exiting.
Running scripts with 'set -e' produces silent failures with no
diagnostic. Add an exit handler which produces diagnostics, including
details of what was running if the shell seems to be bash.

(Bitbake rev: e213e6a4c297a4f1c22eed15bd7b4cbc0e9eab4f)

Signed-off-by: Peter Seebach <peter.seebach@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-08-20 22:43:42 +01:00
Cristiana Voicu
1b3f8ac9cf bitbake: hob/retrieveimagedialog: change labels
Some labels needed to be changed in order to respect the convention.

[YOCTO #4999]
(Bitbake rev: 110b485eac6adea242297b7942da8710403c465a)

Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-08-16 11:21:03 +01:00
Paul Eggleton
840a427ed2 bitbake: taskdata: report close matches with NoProvider errors
Assuming there is no known reason why an item is not provided, show
close matches on the assumption that it might have been a typo or
other mistake.

(Bitbake rev: ed81b0856b4a3892b53d39871eaaa6273390ea75)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-08-16 11:21:02 +01:00
Paul Eggleton
baaa7adc16 bitbake: runqueue: report close matches for an invalid task name
Help to pick up mistakes such as "bitbake -c cleanstate xyz" (instead
of "bitbake -c cleansstate xyz".)

(Bitbake rev: 15c3db1cffdffd85641c6b12e77f19ce7a553472)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-08-16 11:21:02 +01:00
Richard Purdie
712622e2db bitbake: hob: Add missing file from previous commit
This should have been part of 'hob: implement the "retrieve image dialog" +
changes to image combo box' but got lost in the merge process. This adds
the missing file.

(Bitbake rev: 0eadcc073f270c0b323955cf7719b77195cf4890)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-08-12 10:23:32 +01:00
Ross Burton
89f42c9dcb bitbake: depexp: fix typo in variable name
(Bitbake rev: e3b8585738abea96a9fd1d1204731004a35e0bc9)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-08-06 13:02:27 +01:00
Ross Burton
44db452d8f bitbake: depexp: insert data instead of append/set, for speed
(Bitbake rev: 9f50a841f4929e68333ccf2ace32f5d3ca1e7d0d)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-08-06 13:02:27 +01:00
Ross Burton
3412a89dfd bitbake: depexp: make parse() a member function
(Bitbake rev: e5709f2b5592eb6448c7d10b9aeb7cf3bef4864e)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-08-06 13:02:27 +01:00
Cristiana Voicu
e76fdac6f1 bitbake: hob/imageconfigurationpage: changes to image combobox
Place the "Create your own image recipe" field to the
end of the image recipes list.

[YOCTO #4193]
(Bitbake rev: 288bbda31164efffd07a370a728a7682db775c08)

Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-08-06 13:02:27 +01:00
Cristiana Voicu
4182c53e56 bitbake: hob/imagedetailspage: add tooltip for save image recipe button
[YOCTO #4193]
(Bitbake rev: de3cb77ec7ceb8e0671cf3edfd3713f907d1d0bc)

Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-08-06 13:02:27 +01:00
Cristiana Voicu
8db39d2f66 bitbake: hob: set focus on entry name when the name is not correct
[YOCTO #4193]
(Bitbake rev: bbae370bc57296723cc87e48d64da79882f1e5b6)

Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-08-06 13:02:26 +01:00
Cristiana Voicu
ecc56f204c bitbake: cooker: save packages in IMAGE_INSTALL instead of PACKAGE_INSTALL
Hob retrieves the list of recipes and packages using the IMAGE_INSTALL
variable, so a custom image should be saved using this variable.
Changed how the image is saved in a bb file

[YOCTO #4193]
(Bitbake rev: edf3f52c05d86d49b71770cdafde583213e2034d)

Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-08-06 13:02:26 +01:00
Cristiana Voicu
c83f8185a8 bitbake: hob: implement the "retrieve image dialog" + changes to image combo box
Tha changes related to the image combo box are related to the
action done in the retrieveImageDialog. When the user wants to select
a customize image, but then he cancels the action, the combo box is set to
--select a base image--.
If the user selects an image using the new dialog, a new item with its name
is added to the combo box list and then it is activated.

[YOCTO #4193]
(Bitbake rev: f25322de7e47719b31808397174e5c4f6d8649f2)

Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-08-06 13:02:26 +01:00
Cristiana Voicu
42a4f33fc0 bitbake: hob: retrieve file name of an image
The entire file name (with the path) is needed to know
if the image is located in the "build" directory or it comes
from layers. According to this information, the image is placed
differently in the combobox.

[YOCTO #4193]
(Bitbake rev: 7d15eccc25b6c96851e4d01401f9f9b7821730b1)

Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-08-06 13:02:26 +01:00
Cristiana Voicu
4640fd053b bitbake: hob: changes to image combo box
Added an item for the custom images.
Added a separator in the combo box.

[YOCTO #4193]
(Bitbake rev: 1eed84c11269c25c13bb444871d84c5dfeabcb73)

Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-08-06 13:02:25 +01:00
Cristiana Voicu
63157e2c87 bitbake: hob/bitbake: when an image is saved, it should require an image from layers
Saving an image that requires another image will cause issues when
the second on is removed. So, we have agreed to "require" only the images
from layers.
The functionality is implemented in bitbake, in order to be more abstract,
and it is used by Hob when an image recipe is saved.

[YOCTO #4193]
(Bitbake rev: 28296ca78507ba2e414eb136c81afee65a8e25e5)

Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-08-06 13:02:25 +01:00
Cristiana Voicu
08ccfacbc1 bitbake: hob: the saved image should be editable
Also, the image can be saved when the a name is filled.

[YOCTO #4193]
(Bitbake rev: 0edad0ab3ccd165125726d2aa3dfeb19dad246c9)

Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-08-06 13:02:25 +01:00
Cristiana Voicu
3bef5a75d6 bitbake: hob: change the name and description for the saveimagedialog
The image name and description should be saved for a future
save.

[YOCTO #4193]
(Bitbake rev: 6dc0fc243ac6046714523d08df4d8f88c48698cc)

Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-08-06 13:02:25 +01:00
Cristiana Voicu
bb8a265b0f bitbake: hob & bitbake: append a value to a variable from hob throught bitbake
It was necessary to append ${TOPDIR}/recipes/images to BBFILES.
Implemented the mechanism to append a value to a variable: a command and
the method in cooker.

[YOCTO #4193]
(Bitbake rev: 4aedbee90bd92395c2460a68702e6ede00e256c9)

Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-08-06 13:02:24 +01:00
Cristiana Voicu
116219ce09 bitbake: hob: make changes in order to permit from UI to edit an image after saving it
Added the image name to the list model, in order to show the image name as
the user named it.

[YOCTO #4193]
(Bitbake rev: 0aba493103d1fe50026a47db16529febbbbd77a2)

Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-08-06 13:02:24 +01:00
Cristiana Voicu
564c83be5e bitbake: hob/bitbake: save the description of a custom image
When an new image is saved, the dialog for this action has
a field for the description. Changed how an image is saved, by
appending the DESCRIPTION variable at the end of the .bb file.

[YOCTO #4193]
(Bitbake rev: 5629007f2b984005e3a8ac5d9b71422cbc2f1409)

Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-08-06 13:02:24 +01:00
Cristiana Voicu
140744c470 bitbake: hob/bitbake: create a template (a .bb file) from hob through bitbake
Modified generateNewImage function from cooker, in order to be used to
save a template in Hob.
Created a command to ensure that some dirs are created. The templates
(recipes) will be saved in {TOPDIR}/recipes/images folder.
Called these methods from Hob.

[YOCTO #4193]
(Bitbake rev: 96ffa00945c7eb09a0132fa47159aef3ef20fb3e)

Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-08-06 13:02:24 +01:00
Cristiana Voicu
ee4fe5a229 bitbake: hob: create save image dialog used to save a template
Implemented a new dialog used by Hob. This dialog was desinged
in order to permit to save only in a particular directory.
Also, it has a field where the user can type a description
for the image.
Implemented in the handler a method to retrieve the topdir variable,
because the changes will be saved in {topdir}/recipes/images directory.

[YOCTO #4193]
(Bitbake rev: 117d4809a62e28ffe7e9dcda5433993d76f7d934)

Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-08-06 13:02:23 +01:00
Cristiana Voicu
843ef10832 bitbake: hob: labelling changes in Hob
When the design document for templates in Hob was created, we've noticed
that some labels need to change.

[YOCTO #4193]
(Bitbake rev: fcbadbb73a8a94a3d5e330e1a5fa9550130d2c62)

Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-08-06 13:02:23 +01:00
Cristiana Voicu
adcce841b4 bitbake: hob/packageslist: change image size label to estimated image size
On the "Edit packages list" page, the image size may be computed before
building. We didn't find another way to give a more accurate size, so
we have agreed to inform the user that this an estimated size.

[YOCTO #4388]
(Bitbake rev: 136eda2dcbc32aba4f59783049352dc1375cc945)

Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-08-06 13:02:23 +01:00
Cristiana Voicu
4c96451fa5 bitbake: hob/advancedsettings: change toolchain labels to sdk labels
In order to be easier for the user to understand, we have changed the
toolchain labels to sdk.

[YOCTO #3808]
(Bitbake rev: 008afbcf6b1b315eb5463ecfb39fc50e6303687e)

Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-08-06 13:02:23 +01:00
Valentin Popa
888d1f35ee bitbake: bitbake-worker: import needed signal module
bitbake-worker makes use of the signal module
but it doesn't import it. This patch fixes the issue.

[YOCTO #4750]

(Bitbake rev: c2ed639690f135994199eb24d964e37f57259e3a)

Signed-off-by: Valentin Popa <valentin.popa@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-07-31 06:59:59 +01:00
André Draszik
d45486eb1c bitbake: fetch2: fix unpack of .xz files
If a file ends with .xz, it currently gets overwritten during unpack:
The decompress command for .xz files is:
  'xz -dc %s > %s' % (file, efile)
and as efile == file, we end up overwriting file (the source).

Fix this by adding .xz to the list of suffixes that that need to
be removed from a file name for an extract command, leaving the
bare file name. Now, for a given file foo.xz,
file == foo.xz and efile == foo, similar to how .gz .bz2 and .Z
files are treated.

(Bitbake rev: 2cd2d0a48e12ab4358fb967eaf7a56c17993f48d)

Signed-off-by: André Draszik <andre.draszik@linaro.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-07-31 06:59:59 +01:00
Petter Mabäcker
bd1c441a21 bitbake: bitbake: runqueue: add warning if invalidating invalid task
Add a warning if 'bitbake -C' is executed with a task that does not
exist.

Fixes [YOCTO #4877]

(Bitbake rev: 6459c1d0eb8f1007246df36149e2496ee531e25f)

Signed-off-by: Petter Mabäcker <petter@technux.se>
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-07-29 15:25:09 +01:00
Cristiana Voicu
0e9049a011 bitbake: hob: sstatemirror is standard also for file://.* regex
(Bitbake rev: 2e3c87569a8c40f7a2ebda22b01eee3bb7c96d1d)

Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-07-29 15:25:08 +01:00
Cristiana Voicu
e28904e1de bitbake: hob: changed the split of SSTATE_MIRRORS variable
Until now the split made searching the string "file://", but
this is not ok when SSTATE_VARIABLE has the following form:
SSTATE_MIRRORS ?= "\
     file://.* http://someserver.tld/share/sstate/PATH \n \
     file://.* file:///some/local/dir/sstate/PATH"

In the documentation I've found that \n is the list separator.

[YOCTO #4857]
(Bitbake rev: 73bcd96928cb2df390e1fc6d3a8b7ce3e9d546a4)

Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-07-29 15:25:08 +01:00
Robert Yang
e605ecfd95 bitbake: providers.py: print a debug message for dynamic pacakge
We can't know the dynamic pacakge's name exactly, there might be a
problem, for example, when we use:

IMAGE_INSTALL_append += "ncurses-lib12344"

The ncurses-lib12344 matches ncurses' dynamic packages pattern:

PACKAGES_DYNAMIC = "^${PN}-lib.*"

so there is no errors before the rootfs creation though there is no
ncurses-lib12344.

We can warn this, but I think that we'd better not since there are many
dynamic packages, or there would be too many warnings, for example, the
perl and kernel modules, maybe we can print a debug message for it.

[YOCTO #4798]

(Bitbake rev: df372ca057f0c8c2152223b3e26ad9a30958bab6)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-07-29 15:25:08 +01:00
Valentin Popa
dc86293f04 bitbake: HOB:Proper handle of SIGINT
Modal dialogs doesn't run on the main loop so they cannot
catch any signal from the terminal. This patch makes sure
the dialogs are destroyed when a SIGINT is sent to HOB.

[YOCTO #3329]

(Bitbake rev: 6eee0cc37438cc3f91531b7df524330fba27161b)

Signed-off-by: Valentin Popa <valentin.popa@intel.com>
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-07-05 15:52:48 +01:00
Cristiana Voicu
8a186a6b38 bitbake: hob: save button from settings called a nonexisting method
The method was removed when the process for saving configuration
in Hob was changed. Replace the call with the right function.

[YOCTO #4793]
(Bitbake rev: b6aa2b63d71cbe82850a375381b2dbc750cf1905)

Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-07-03 08:13:35 +01:00
Richard Purdie
624160af6e bitbake: cookerdata: Improve message if BBPATH is unset and bblayers.conf not found
If BBPATH isn't set and bblayers.conf isn't found, improve the message
shown to the user to help their understanding of what the problem might
be.

[YOCTO #3271]

(Bitbake rev: 0e639f5cbc813c8d4719019cfdd4287e9a429610)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
A
2013-06-28 10:34:16 +01:00
Li Wang
10e44f162c bitbake: bitbake: python funcname can not include special character @
[YOCTO #4772]

When path:file change to python function, it maybe include '@' character.
So, add the special character to change to '_' for avoid error.

(Bitbake rev: 684bc6dcb11ecb1fd7a4d25c08909ad9879e8342)

Signed-off-by: Li Wang <li.wang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-06-27 10:06:09 +01:00
Cristiana Voicu
4852c4f6a1 bitbake: bitbake: adding a new comment should be placed on a new line
In this case, the comment is appended to the end of the file.
Some text editors, do not place  a '\n' to the end of the file
after saving it.

[YOCTO #4636]
(Bitbake rev: 2beb9589b1bd9773f587b4dc08afdfe50f4ea913)

Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-06-27 10:06:09 +01:00
Richard Purdie
011608a7b0 bitbake: data_smart: Fix bug with overrides and weak default values
(aka pay the cookie monster for weak defaults)

If you have code like:

MYVAR = "a"
MYVAR_override ??= "b"

then MYVAR will get the value "a" even when override is in OVERRIDES. The
reason is that the value of ??= is set as a flag not a value and the cookie
monster isn't paid.

The fix is to ensure appropriate payment is made for a defaultval varflag
matching the usual setVar case.

(Bitbake rev: 3d8044bc79c482c5ea008ddf12a8128dcd1527ee)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-06-21 13:00:57 +01:00
Richard Purdie
353212a533 bitbake: data_smart: Ensure variable flags are accounted for in config data hash
Currently if the flags set against a variable in the base data store
change, it doesn't automatically trigger a reparse when it really
should. For example with the blacklist class setting:

PNBLACKLIST[qemu] = "bar"
PNBLACKLIST[bash] = "foo"

will not trigger a reparse if only one entry is changed and a
blacklisted recipe can still be built.

I did consider using BB_SIGNATURE_EXCLUDE_FLAGS in here however it
doesn't make sense, we want to trigger a reparse when any of the
flags change too (which is different to the sstate signatures which
we wouldn't want to change in those cases).

[YOCTO #4627]

(Bitbake rev: ed74ea50043f6feb698c891e571feda2b9f8513d)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-06-21 09:19:28 +01:00
Cristiana Voicu
78bcf871c6 bitbake: hob: save variables in conf/ directory
Until now the variables were saved in bitbake configuration,
now they are saved in configuration files, in order to be read by
bitbake-worker. This helps to assure the consistency for the rest
of the variables.

(Bitbake rev: ea65ebf43525f173205183aa2fd5d8db303ffd4a)

Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-06-20 16:26:22 +01:00
Cristiana Voicu
f3246953c2 bitbake: hob: replace the init cooker with reset in order to keep the history
When the cooker initialization is called, the variables history is lost.
It need a reset, in order to load the configuration files.
Also, concatenated the value for INHERIT variable.

(Bitbake rev: 2b8c0c09e09ab69edc53501b84050c797f0686e4)

Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-06-20 16:26:22 +01:00
Cristiana Voicu
71520c5045 bitbake: bitbake: call the initConfiguration when reparsing
As it is implemented now, when parsing the configuration files is
needed, it also needs an initialization.
Also, implemented a function to create an extra configuration files
which can be used as postfile/prefiles when parsing.

(Bitbake rev: 5fe23c1b51da14f58f8c483f43f30b48766a0913)

Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-06-20 16:26:22 +01:00
Richard Purdie
c5a5d23ec4 bitbake: cooker: Ensure configuration matches variable tracking state
When we reset, the current tracking value may change unexpectedly. This
patch ensures that the default value is preserved over a datastore reset.

(Bitbake rev: 0aa27598d9fdc8c3d18839b3ef93d0ba9698f761)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-06-20 15:10:32 +01:00
Richard Purdie
db6bd113d0 bitbake: build.py: Cleanup data store references
Clean up a number of old style accesses to the datastore.

(Bitbake rev: d872fef2c38749c3c6f5d84344db3ec2f9f134ce)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-06-20 15:10:32 +01:00
Cristiana Voicu
7d02de844c bitbake: bitbake: remove the function for reparse because it does nothing
The command for reparse does nothing, so I have managed to remove it,
because it can be misleading. When a reparse is needed, it can be used
the parse function.

(Bitbake rev: 188eaba121789112ffeb1188f0984d23dfe8df4f)

Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-06-20 13:29:53 +01:00
Peter Kjellerstedt
3a74f25cf1 bitbake: Correct a typo
(Bitbake rev: 087ec3a1b2dde04a8e5ce2943802fccd424ab3f3)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-06-20 13:26:57 +01:00
Peter Kjellerstedt
d1ad98b64b bitbake: Allow checksums to be used for files retrieved using SFTP
(Bitbake rev: fd438a334d1f90ff07dded61c9648987da42c34f)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-06-20 13:26:57 +01:00
Cristiana Voicu
77264ca2ea bitbake: hob: check if imageconfiguration page has the attribute 'warnings_bar'
[YOCTO #4571]
(Bitbake rev: 53c6b65bbcd5a6d3caed8581533f31ac373163a7)

Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-06-20 13:26:57 +01:00
Cristiana Voicu
01f9d1b338 bitbake: bitbake & hob: modify some commands and call them from hob
This patch adds a new command to create an extra configuration
file, in order to create it from Hob.
Also, it calls the enable/disable data tracking when needed.
Modified the prefiles and postfiles variables because the cooker
expects some lists.

(Bitbake rev: 92393701adcfa181a1090e3632bc6df7ee7852e6)

Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-06-20 13:26:57 +01:00
Cristiana Voicu
64eda3e058 bitbake: hob: remove the rest of the old template functionality
Until now, some configuration settings was saved in a hob specific
directory. From now on, it will be saved in conf directory through bitbake.

(Bitbake rev: c53e902b010d1c3f1550f7e60e744f40120f73c2)

Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-06-20 13:26:56 +01:00
Richard Purdie
801fbe5a0c bitbake: fetch2: Enhance error message for BB_NO_NETWORK
BB_NO_NETWORK can be set by bitbake internally by the use of
BB_FETCH_PREMIRRORONLY so update the error message to give users a
hint about this.

[YOCTO #3222]

(Bitbake rev: cac3060d0bf8c7deeacda18d06d92787911380d0)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-06-20 13:10:47 +01:00
Richard Purdie
5af68d42ba bitbake: runqueue/build: Add recideptask flag
Currently, tasks like fetchall are slightly broken since if a recipe
has specific [depends] which occur after do_fetch and add items not listed
in DEPENDS and RDEPENDS, they are not caught by recrdeptask. We've gone
around in circles on this issue (e.g
http://git.yoctoproject.org/cgit.cgi/poky/commit/bitbake/lib/bb/runqueue.py?id=5fa6036d49ed7befe6ad50ec95c61a50aec48195
) and in many cases the behaviour of recrdepends is correct but tasks like
fetchall need the other behaviour.

To address this we add a recideptask flag which can be used in conjuction
with the recrdeptask flag to specify which task to to the inspection upon.
This means entries like do_rootfs[depends] which have do_fetch tasks are
caught and run.

I'm not 100% happy with needing another flag but I don't see any rational
way to get the correct behaviour in all cases without it.

[YOCTO #4597]

(Bitbake rev: f8c9b292b02ce2c28741b74901205f5e5807ca87)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-06-20 13:10:47 +01:00
Alexandru DAMIAN
baff2630b3 bitbake: bitbake: read BBSERVER environement for remote server
This change allows bitbake to connect to a remote server
by reading the BBSERVER environment variable and effecting
the config. Basically a shortcut to the command line parameters.

(Bitbake rev: 81929f86c57ed0a4ad0cda7aaa820fceabaa61e9)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-06-17 16:09:10 +01:00
Alexandru DAMIAN
f5cadbe005 bitbake: knotty, bitbake: add option to terminate a remote server
I add an option to terminate a remote server gracefully
as not to need a kill command.

(Bitbake rev: 7495f835666a9561c2c7d84da7aaa74e4df55b9a)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-06-17 16:09:10 +01:00
Alexandru DAMIAN
3ea9d647ec bitbake: knotty, xmlrpc: add observer-only mode
I add an observer only mode for the knotty UI and
the XMLRPC server that will allow the UI to register
a callback with a server in order to receive events.

The observer-UI is able to send read-only commands to the
server, and also is able to register as an event handler.

Read-only commands are the commands that do not change
the state of the server and have been marked as such in
the command module.

The observer can switch to a full client if it calls addClient
at any time, and the server has no other client running.

(Bitbake rev: 4de9ee21f1fa4d04937cc7430fb1fc8b7a8f61e2)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-06-17 16:09:10 +01:00
Richard Purdie
e023620773 bitbake: utils: Fix multiprocessing pool reference
Interestingly the previous version comparision was broken and we were always using
the compatibility code, masking this bug. Oops.

(Bitbake rev: d48e8bcb24e8fa5d4fd60fd2c9927a95976d8d8b)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-06-14 22:01:09 +01:00
Richard Purdie
441c699acb bitbake: compat/server/utils: Jettison pre python 2.7.3 workarounds
Now we've moved to require python 2.7.3, we can jettison the compatibility
workarounds/hacks for older python versions.

(Bitbake rev: a51c402304f2080a76720f9b31d6dfdbed393bba)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-06-14 17:26:30 +01:00
Richard Purdie
6c058341f9 bitbake: runqueue: Improve handling of failing setscene tasks with hard dependencies
If a setscene task has a hard dependency on a task like pseudo-native, its
expected that the setscene task will not run unless the dependency is met.

This adds code to ensure that is the case, otherwise a bug would show up
with a usecase like:

bitbake gnome-common
bitbake pseudo-native -c cleansstate
bitbake gnome-common -c clean
bitbake gnome-common

With the double wrapper script environment, we'd not see issues like
this as it would be masked. The problem theoretically affects code like
useradd too as well as anything using a sstate postinstall.

(Bitbake rev: c54e738e2b5dc0d8e6fd8e93b284ed96e7a83051)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-06-14 17:08:40 +01:00
Alexandru DAMIAN
4aebdecdd7 bitbake: xmlrpc: removes the event servers controller
XMLRPC API ran a XMLRPC server at port+2 in order
to provide endpoints to register an event server
in which to dump the events.

This is no longer used, so we remove it.

(Bitbake rev: e171a363913a86e56266f4c9d107110c7f5221e3)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-06-14 14:34:59 +01:00
Bogdan Marinescu
5aa5fc470e bitbake: bitbake: Add event mask flag support
Add a flag to event handlers which lists the events a given handler wishes to
process. By default event handlers recieve all events but this means
we can stop running code in many cases if we know it doesn't want the event.

This is part of the fix for YOCTO #3812, but implements filtering only
for class event handlers; the other part (events filter for UIs) will be
the subject of a different patch.

(Bitbake rev: 074003a4e7530a72863b9c685fc5c31b0f08c039)

Signed-off-by: Bogdan Marinescu <bogdan.a.marinescu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-06-14 12:52:58 +01:00
Richard Purdie
542d6a87db bitbake: bitbake: Update version to 1.19.1
(Bitbake rev: 6acd444a557bba977ae1772371fdadf5f510e3b6)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-06-14 12:52:58 +01:00
Richard Purdie
8408f3dfca bitbake: bitbake: Bump minimum python version to 2.7.3
(Bitbake rev: a01d153a8baac7136c2797c95357e74dd8872026)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-06-14 12:52:57 +01:00
Richard Purdie
5ebd9bfff1 bitbake: prserv: Adapt autostart to bitbake-worker
With the change to bitbake-worker we need to ensure the workers know
how to contact the PR service, the magic 0 port and singleton is
no longer enough.

(Bitbake rev: c761751e259bb8e940552a28794b45887b5a72d9)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-06-14 12:52:57 +01:00
Richard Purdie
efb877bcdb bitbake: runqueue: Spawn a separate worker for fakeroot tasks
(Bitbake rev: 860ec42b220b7ed3f3bbe52c3546bba66644eac8)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-06-14 12:52:57 +01:00