Commit Graph

3258 Commits

Author SHA1 Message Date
Richard Purdie c501492132 bitbake: bitbake-worker: Improve sigterm handler
When processes terminate, we really want all of the child processes to
terminate too. This was not happening for worker processes which spawned their
own multiprocessing pools, leading to build hangs. This change ensures any
sigterm gets passed to the whole process group. In local tests, this resolved
some hanging process workloads I could generate. It does rely on signals
being delivered in a timely fashion and there is a multiprocessing bug we have
to work around there.

(Bitbake rev: 96f8ea07ace1379380fab2d78eb592fa40c867d4)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-08-23 09:31:42 +01:00
Richard Purdie a4f077b1fe bitbake: knotty: Improve exception handling
Currently, IOErrors are just passed over due to the broken Exception
clause. A command like "bitbake X | <invalid command>" would break stdout
triggering a traceback. With these changes we print the exceptions, shut down
the server gracefully and exit which is a much nicer behaviour and is less
confusion to the user.

(Bitbake rev: 9544108f7b413038d871ce6ca88232de2f2434d9)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-08-23 09:31:42 +01:00
Richard Purdie db50630948 bitbake: process: Deal with infinite looping of the server
Currently if an exception occurs, we just run the idle handler again and
again, usually looping indefintely. Chances are the exception that occurred
will keep occuring and this is not a good place to be.

This was breaking the autobuilders with gigabytes of logs.

At least improve things so the cooker shuts down gracefully when this happens.
Some trace of the original problem may still be present on the console too!

(Bitbake rev: 1f28d8d3311262427938180435b68f0a35c2b330)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-08-23 09:31:42 +01:00
Robert P. J. Day 34436672f3 bitbake: runqueue.py: Fix typoes/grammar in comments.
(Bitbake rev: 000fa81013205dd9bc907ff7a61f06f57637212d)

Signed-off-by: Robert P. J. Day <rpjday@crashcourse.ca>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-08-19 20:41:26 +01:00
Andre McCurdy 02455ebfbb bitbake: fetch2/local.py: fix first line indent of search path debug output
Make OE do_unpack logs slightly prettier.

(Bitbake rev: cec5859dec56529d76a0a685cc67b8da29453c69)

Signed-off-by: Andre McCurdy <armccurdy@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-08-19 20:41:26 +01:00
Robert P. J. Day f1dc26ebf4 bitbake: daemonize.py: Non-functional comment/aesthetic fixes.
(Bitbake rev: fd293c5fcc89bc7a541d09e88a6aec5e0b2a1db2)

Signed-off-by: Robert P. J. Day <rpjday@crashcourse.ca>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-08-19 20:41:26 +01:00
Robert P. J. Day 8a91a24ce9 bitbake: runqueue.py: Correct several misspellings of "notifing".
(Bitbake rev: 4e9aef14d747c37444a4fc683f9641906906afe9)

Signed-off-by: Robert P. J. Day <rpjday@crashcourse.ca>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-08-19 20:41:26 +01:00
Robert Yang c3f7f4e015 bitbake: newbb.vim: remove PR
We don't need the "PR = r0" in the bb, so remove it to avoid misleading
the user.

(Bitbake rev: 17fc0174f177b444815487ba67a5d623e47ee8b1)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-08-19 20:41:26 +01:00
Richard Purdie 25129a6afe bitbake: runqueue: Add sceneQueueComplete event
Its useful to have an event emitted when all of the sceneQueue tasks
have completed since the metadata can hook this for processing.
Therefore add such an event.

(Bitbake rev: 38d4f65bf1cbcdd5a2d60dff0e1d2859c34ed62e)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-08-03 12:40:41 +01:00
Namhyung Kim f401e8f88c bitbake: cooker: Get rid of duplicate 'import re'
The re module was imported at the top, no need to do it again.

(Bitbake rev: c29c2370635075949c75937c237602600f4f10dc)

Signed-off-by: Namhyung Kim <namhyung@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-08-02 09:57:11 +01:00
Namhyung Kim 70d0316d57 bitbake: knotty: Move second event check into a proper block
so that it doesn't check it twice for non-empty events

(Bitbake rev: e304ace6d8f7d0808ee401b7c01146b2798a81b4)

Signed-off-by: Namhyung Kim <namhyung@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-08-02 09:57:11 +01:00
Scott Rifenbark 68eb17da93 bitbake: bitbake-user-manual-metadata.xml: Fixed variable flags example.
(Bitbake rev: 78b6c89931a78b2c722d2fcf61cf553b45566a5a)

Signed-off-by: Scott Rifenbark <scott.m.rifenbark@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-08-02 09:30:01 +01:00
Scott Rifenbark 4643e6b76c bitbake: bitbake-user-manual-fetching.xml: Minor edits to clearcase fetcher
(Bitbake rev: 33cd736d84eb8d4719e430450f26acaee5b31782)

Signed-off-by: Scott Rifenbark <scott.m.rifenbark@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-08-02 09:30:01 +01:00
Scott Rifenbark d0dec8496f bitbake: bitbake-user-manual/bitbake-user-manual-intro.xml: Fixed link
I inadvertantly used a &YOCTO_* variable in the book to get
a link to the landing YP page.  Of course this does not work.
I replaced it with the actual URL.  Unfortunately, a previous
commit slipped through that would have prevented the manual
from "making" correctly.

(Bitbake rev: 63f5a4b06a575e1fb18805b45a01549bffdc4a03)

Signed-off-by: Scott Rifenbark <scott.m.rifenbark@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-08-02 09:30:01 +01:00
Scott Rifenbark 53436d97e0 bitbake: bitbake-user-manual-fetching.xml: Added new clearcase fetcher module.
Reported-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(Bitbake rev: dc669c4beda04ee88a50c9b48facfbd8cf17a331)

Signed-off-by: Scott Rifenbark <scott.m.rifenbark@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-08-02 09:30:01 +01:00
Richard Purdie 89d1788412 bitbake: codeparser cache improvements
It turns out the codeparser cache is the bottleneck I've been observing when running
bitbake commands, particularly as it grows. There are some things we can do about
this:

* We were processing the cache with "intern()" at save time. Its actually much
  more memory efficient to do this at creation time.

* Use hashable objects such as frozenset rather than set so that we can
  compare objects

* De-duplicate the cache objects, link duplicates to the same object saving
  memory and disk usage and improving speed

* Using custom setstate/getstate to avoid the overhead of object attribute names
  in the cache file

To make this work, a global cache was needed for the list of set objects as
this was the only way I could find to get the data in at setstate object creation
time :(.

Parsing shows a modest improvement with these changes, cache load time is
significantly better, cache save time is reduced since there is now no need
to reprocess the data and cache is much smaller.

We can drop the compress_keys() code and internSet code from the shared cache
core since its no longer used and replaced by codeparser specific pieces.

(Bitbake rev: 4aaf56bfbad4aa626be8a2f7a5f70834c3311dd3)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-07-26 08:50:14 +01:00
Richard Purdie a05435fc59 bitbake: cache: Optimise invalid cache file handling
If there is a corrupt/invalid cache file, we'd keep trying to reopen
it. This is pointless, simplify the code paths and delete the dead
file.

(Bitbake rev: c22441f7025be012ad2e62a51ccb993c3a0e16c9)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-07-26 08:50:14 +01:00
Richard Purdie 97f4d8fa78 bitbake: cache: Don't reload the cache file since we already have this data in memory
If we're writing out merged data to disk, its safe to assume that either
we loaded the data or couldn't. Loading it again is relatively pointless
and time consuming.

(Bitbake rev: 14c31e18e4a4a52ee54056aeefb09ff8c295b475)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-07-26 08:50:13 +01:00
Robert Yang 8f52c69183 bitbake: bitbake: remove choices for dump-signatures
The SIGNATURE_HANDLER can be defined by the user, so we can't use choices.

(Bitbake rev: 0036b36e333a68c6d30a6bf5c21bfd4fd3e8441b)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-07-24 12:31:39 +01:00
Alexandru DAMIAN f4456df240 bitbake: toasterui: fixing event.data clash
This patch fixes a name collision on the event.data
in the MetadataEvent class. event.data is used in the
event system as a copy of "d" in certain situations,
and this collision triggered a bug leading to data loss.

[YOCTO #6332]

(Bitbake rev: 3f191b7cfe95aea4d4e96babf001d62d45dd3aaa)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-07-23 20:07:48 +01:00
Alexandru DAMIAN 37d2066c83 bitbake: toaster-requirements.txt: document requirements for the python environment
Since Toaster adds a number of specific requirements managed
by pip, we document these requirements in order to be able
to run bitbake and toaster under virtualenv. The target here
is to break the dependency on distro packages for specific
python libraries.

In order to start bitbake/Toaster in a distro-independent manner,
we use virtualenv and pip. We add venv to the .gitignore list
to make room for the virtualenv deployment in this directory.

Use this command sequence to setup the virtualenv:

$ virtualenv venv
$ . venv/bin/activate
$ cat toaster-requirements.txt | xargs pip install

bitbake and Toaster will then run normally under virtualenv.

We also fix bugs related to toaster starting in managed mode
through this script:
  * toaster will not use bldcontrol application in interactive mode
  * a proper message will be displayed if toaster detects a managed
start request in a build environment

(Bitbake rev: b5789c7ea7e6d1a4b67ec60fc0e4233ec543ba3a)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-07-23 20:07:43 +01:00
Alexandru DAMIAN 1b9175af3f bitbake: toaster: properly set layers when running a build
This patch enables the localhost build controller to
properly set the layers before the build runs.

It creates the checkout directories under BuildEnvironment
sourcedir directory, and runs the build in the buildir
directory.

Build launch errors are tracked in the newly added BRError table.
These are different from build errors, in the sense that the
build can't start due to these errors.

(Bitbake rev: 1868d5635b517e0fe1b874674ea7a78910b26e2e)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-07-23 20:06:58 +01:00
Alexandru DAMIAN 6e71c276b5 bitbake: toaster: add project main edit page
This is the first commit on the project main edit page.

At this point we have:
* the default settings for a newly created project
* the ability to add targets
* the ability to trigger a build command, and have
the build executed

Project layers now have an optional field, allowing for
removal. Default meta, meta-yocto and meta-yocto-bsp
layers cannot be optional.

We add XHR calls for interactivity in the main page.

(Bitbake rev: 4e438854120cbd10319df1b571ec93e334002325)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-07-23 20:06:58 +01:00
Alexandru DAMIAN 8a3789a7b1 bitbake: toaster: move bldcontrol initial_data fixture to code
We move the initial_data fixture to a code migration,
in order to make sure that we don't overwrite modified data.

The data is inserted only if no data is present, instead
of always overwriting the entry.

The check settings returns 0 if the check was ok.

(Bitbake rev: 525c316ac95c06c52fbcfc836ca5599ee9dbfe8b)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-07-23 20:06:58 +01:00
Alexandru DAMIAN 5aba3d7fcc bitbake: toaster: improved Project models
A layer may live in a subdirectory of a git repository,
so we add a field to track this setting in the Project layers.

We add the Project schedule_build function, which creates
a build request from the current project configuration.

We also fix an import problem with Projects in views.

(Bitbake rev: 1b5835e5d48cbfb7d38e38437c45d161052dfb37)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-07-23 20:06:57 +01:00
Alexandru DAMIAN 6cfb76fa8b bitbake: toaster: add fields for sourcedir and builddir paths
We add explicit absolute paths for a directory where
the layer sources will be checked out (sourcedir) and
where the build activities will take place.

Adding minimal checking when starting the application in
order to make sure that BuildEnvironment (BE) settings are
usable. This check is ran by the toaster script at startup.

Modify the localhost bbcontroller to use the BE settings
instead of trying to self-configure on checked out sources.

(Bitbake rev: d17500d3f73fdeeef5f11fb3773a65e927be3f02)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-07-23 20:06:57 +01:00
Robert P. J. Day 653b557770 bitbake: "per-package" should say "per-recipe"
(Bitbake rev: 1cd369883469747a8158826bb8d67dcca2a8577f)

Signed-off-by: Robert P. J. Day <rpjday@crashcourse.ca>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-07-23 10:02:13 +01:00
Scott Rifenbark 8b7116d25e bitbake: Updated the the example 'bitbake -h' output to match the actual output, which has been recently patched to fix the '-S SIGNATURE_HANDLER, --dump-signatures=SIGNATURE_HANDLER' option.
Note also, that I have added a new option that has since appeared
in the output but was not in the manual's example.  That option
added is '--token=XMLRPCTOKEN'.

Reported-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(Bitbake rev: 8fbffd25d42f1f2d9a85ae1d9cc4852e835730d4)

Signed-off-by: Scott Rifenbark <scott.m.rifenbark@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-07-22 08:33:25 +01:00
Dennis Meier 5bcda42a94 bitbake: fetch2: Add module for ClearCase (ccrc://)
The clearcase fetcher is used to retrieve files from a ClearCase
(http://en.wikipedia.org/wiki/Rational_ClearCase) repository.

Usage in the recipe:

    SRC_URI = "ccrc://cc.example.org/ccrc;vob=/example_vob;module=/example_module"
    SRCREV = "EXAMPLE_CLEARCASE_TAG"
    PV = "${@d.getVar("SRCREV").replace("/", "+")}"

The fetcher uses the rcleartool or cleartool remote client, depending on which one is available.

Supported SRC_URI options are:

- vob
    (required) The name of the clearcase VOB (with prepending "/")

- module
    The module in the selected VOB (with prepending "/")

    The module and vob parameters are combined to create
    the following load rule in the view config spec:
                load <vob><module>

- proto
    http or https

Related variables:

    CCASE_CUSTOM_CONFIG_SPEC
            Write a config spec to this variable in your recipe to use it instead
            of the default config spec generated by this fetcher.
            Please note that the SRCREV loses its functionality if you specify
            this variable. SRCREV is still used to label the archive after a fetch,
            but it doesn't define what's fetched.

User credentials:

    When using cleartool:
            The login of cleartool is handled by the system. No special steps needed.

    When using rcleartool:
            In order to use rcleartool with authenticated users an `rcleartool login` is
            necessary before using the fetcher.

(Bitbake rev: 76cff0aaad4cf10b325bceadd8ea90b3d75edcc2)

Signed-off-by: Dennis Meier <meier.dennis@siemens.com>
Reviewed-by: Roger Meier <r.meier@siemens.com>
Reviewed-by: Christian Liechti <christian.liechti@siemens.com>
Reviewed-by: Henrique Mendonca <henrique.mendonca@siemens.com>
Reviewed-by: Pascal Bach <pascal.bach@siemens.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-07-21 19:24:29 +01:00
Richard Purdie 13a03f2e9c bitbake: command/runqueue: Fix shutdown logic
If you hit Ctrl+C at the right point, the system processes the request
but merrily continues building. It turns out finish_runqueue() is called
but this doesn't stop the later generation and execution of the
runqueue.

This patch adjusts some of the conditionals to ensure the build really
does stop.

(Bitbake rev: 39b08c604ba713100e174c136f81f18eca6ef33d)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-07-21 19:24:29 +01:00
Robert Yang 231c7373fb bitbake: bitbake: update help message for dump-signatures
The bitbake -S had been updated which always reuires an argument, so
update the help info:
- Add the two args in the help message: none and printdiff

- Use type="choice" so that we can get more friendly error messages,
  for example:
  bitbake: error: option -S: invalid choice: 'printdiffX' (choose from 'none', 'printdiff')

- dump-signatures=DUMP_SIGNATURES -> dump-signatures=SIGNATURE_HANDLER

(Bitbake rev: 021b778fa4685bdde39e1a0f6c7c57632dcf792a)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-07-21 11:00:58 +01:00
Scott Rifenbark d74b8824ae bitbake: bitbake-user-manual-metadata.xml: fixed some example whitespace issues
I added some whitespace to bring the example to use 5 spaces
rather than for so that it follows the manual's convention.

(Bitbake rev: 4c6e7d88d715db18e61bb263dfd2672ebb6c6b09)

Signed-off-by: Scott Rifenbark <scott.m.rifenbark@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-07-21 11:00:58 +01:00
Scott Rifenbark 335defe010 bitbake: bitbake-user-manual-metadata.xml: KERNEL_FEATURES example updated.
I added text to separate this example code from the general
BitBake User Manual's spirit of standing alone and away from
YP or OE stuff.  Also, added another space to move the examples
code so that 5 spaces are used for literal indentation.

(Bitbake rev: 17fcac8a7b21afa5c7d201e94cbf603ab07ba70c)

Signed-off-by: Scott Rifenbark <scott.m.rifenbark@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-07-21 11:00:58 +01:00
Scott Rifenbark c723fa02f6 bitbake: bitbake-user-manual-metadata.xml: Fixed whitespace
I noticed that the examples from some patches were not indenting
by 5 spaces.  I have fixed these to be consistent.

(Bitbake rev: c487341bd20b70aae6b430ee1c74a7f20345f81a)

Signed-off-by: Scott Rifenbark <scott.m.rifenbark@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-07-21 11:00:57 +01:00
Robert P. J. Day c08d72475c bitbake: bitbake-user-manual-metadata.xml: Removed fakeroot from list
"fakeroot" is now specified as a function attribute, not as a task
flag.

(Bitbake rev: 749b39a8e5e8060222c748a349efc31f541da94e)

Signed-off-by: Scott Rifenbark <scott.m.rifenbark@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-07-21 11:00:57 +01:00
Robert P. J. Day eb169fe8b3 bitbake: bitbake-user-manual-metadata.xml: Added second conditional metadata example.
I think having a couple real-life examples here would make a huge
difference.

Comment from Scott: I modified the example description to indicate
that it is from an OpenEmbedded metadata-based recipe (i.e from
YP).  We have been trying very hard to keep the BitBake User
Manual as separate as possible from the YP docs.

(Bitbake rev: 834780772a08ecce7ed863e96d0674b47d0e589d)

Signed-off-by: Scott Rifenbark <scott.m.rifenbark@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-07-21 11:00:57 +01:00
Robert P. J. Day 49627bbad3 bitbake: bitbake-user-manual-metadata.xml: Adding some flag examples.
As long as i've understood everything correctly, this is just
throwing in some physical examples of a couple concepts.

(Bitbake rev: a18cc69c2fef6484a6acd78ea008d1da71198e68)

Signed-off-by: Scott Rifenbark <scott.m.rifenbark@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-07-21 11:00:57 +01:00
Scott Rifenbark e83032ce0a bitbake: bitbake-user-manual-metadata.xml: Edits to some operators
Added wording to clearly state immediate or delayed application
for various operators.

Reported-by: Robert P. J. Day <rpjday@crashcourse.ca>
(Bitbake rev: 877c4c9a466e26953abd6fe416c2cba092607dac)

Signed-off-by: Scott Rifenbark <scott.m.rifenbark@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-07-21 11:00:57 +01:00
Robert P. J. Day af4f1f4a98 bitbake: bitbake-user-manual-execution.xml: Minor edits to execution chapter.
(Bitbake rev: 8ff1b363df74f7e48da67fce9cf6b7e868c5e28e)

Signed-off-by: Scott Rifenbark <scott.m.rifenbark@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-07-21 11:00:57 +01:00
Robert P. J. Day 696973af24 bitbake: bitbake-user-manual-intro.xml: More edits to the intro chapter.
Some applied as is, some not applied, some modified.

(Bitbake rev: af98b2ff4f44f892b63a6598e7fdfc144972ed3d)

Signed-off-by: Scott Rifenbark <scott.m.rifenbark@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-07-21 11:00:57 +01:00
Robert P. J. Day 4df50d3827 bitbake: bitbake-user-manual-intro.xml: Minor edits to start of intro chapter
(Bitbake rev: 4e5db4bc460c12194a233dd283c273009606bb64)

Signed-off-by: Scott Rifenbark <scott.m.rifenbark@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-07-21 11:00:57 +01:00
Robert Yang b62b682aef bitbake: bitbake-user-manual-metadata.xml: Replaced "populate_staging"
There is no "populate_staging" any longer, should be
"populate_sysroot".

(Bitbake rev: dbe5902e64259e2143f09ab1aa9afa03aa02f4c4)

Signed-off-by: Scott Rifenbark <scott.m.rifenbark@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-07-21 11:00:57 +01:00
Scott Rifenbark 435e425976 bitbake: bitmake-user-manual-fetching.xml: Added transportuser parameter for SVN
Fixes [YOCTO #6475]

A new parameter has been added to make sure the user is not
passed as the username for the subversion commands with the
SVN fetcher.

(Bitbake rev: 366c70dbb0e9916f50ce1a4ff3ad4bd2757832a3)

Signed-off-by: Scott Rifenbark <scott.m.rifenbark@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-07-21 11:00:56 +01:00
Alexandru DAMIAN 1dcdd877c7 bitbake: toasterui: fix build - project identification
This patches fixes the build - project identification when
running under managed mode. The build is assigned to the
project from which it was triggered, and to the
build request, as to simplify relationships queries
 in the database.

(Bitbake rev: af1d3373706d365f9138caec110fcb20a5966b7b)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-07-14 14:10:03 +01:00
Alexandru DAMIAN e7fa0c325d bitbake: toaster: new project page implementation
We add the logic to create a new project. This page
also serves as user registration and silent login
for users.

Once the project is added, the main project page is displayed.

(Bitbake rev: 8855daebe55917c4f5855413d02ae1f3f7f76571)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-07-14 14:10:03 +01:00
Alexandru DAMIAN d19b784c1a bitbake: toaster: whitespace fix
This patch is just a whitespace (end-of-line) fix.

(Bitbake rev: 5f33d1a240faa49bd8259a48eb15ef57a6b4ce55)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-07-14 14:10:03 +01:00
Alexandru DAMIAN 9cfa66bd13 bitbake: toaster: add automated login in new project page
Toaster uses the Django authentication system to assign
user accounts to the projects that are being created.

In the current implementation, the user accounts are
created/authenticated automatically, on the fly, based
on the fields specified in the create new project page.

(Bitbake rev: a9062d9692525e24e59b5b2bb4dfdef90b41bf2a)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-07-14 14:10:03 +01:00
Alexandru DAMIAN cff19351a8 bitbake: toaster: add project pages
We add the new project and project page skeletons.

In the process, we add an identifier in the settings.py
to detect whenever Toaster is running in managed mode,
and a context processor to make this value available
to the template processor.

(Bitbake rev: 927a27c68e24cfe13f62ca5f0e60878b04fa4e24)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-07-14 14:10:03 +01:00
Dave Lerner 6b62a0fd6e bitbake: toasterui: Show in the log that ToasterUi is ready to receive events.
Issue: TA53702

It was observed that a sequence in a script such as
   bitbake --server-only ..
   bitbake --observe-only ..
   bitbake <some target>
could generate events from the server to the observer before
the observer was ready to read the events, and the early events
of builds were consistently dropped.  Adding a "ready" note in the
log allows the script to scan for that message before proceeding
to building a target.

(Bitbake rev: 9949948f92bbe2717a05a380d3df63a332d39c9a)

Signed-off-by: Dave Lerner <dave.lerner@windriver.com>
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-07-14 14:10:02 +01:00
Alexandru DAMIAN c35b678053 bitbake: toaster: automatically enable applications
We automatically enable local applications in INSTALLED_APPS
based on detecting a models.py or views.py file.
This allows Toaster extensibility by adding applications,
without having to edit configuration files.

(Bitbake rev: 05b246095d161143ca2ebc126d9e70d198ab8bb8)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-07-14 14:10:02 +01:00
Alexandru DAMIAN 5790b60d00 bitbake: toaster: automatically discover urls.py files
Added code to automatically discover and add url dispatchers
to the urlpattern list. This allows extension of Toaster
through adding applications that will be automatically
registered with the URL dispatcher.

(Bitbake rev: 4000d8224206f404d174a5bc4aa49af55cb44d27)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-07-14 14:10:02 +01:00
Alexandru DAMIAN 1712c715f3 bitbake: toaster: remove strftime calls in filters
We remove the unneeded strftime calls in filters, as the
filters can deal directly with datetime types.

[YOCTO #6379]

(Bitbake rev: 149693a342fe3a56027943d3ecb8bf2e1d0d652b)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-07-14 14:10:02 +01:00
Richard Purdie a5531a2b89 bitbake: fetch2/svn: Add transportuser parameter
There may be a need to set the user for the transport rather than the subversion
command itself. Add a parameter to the url to allow this.

[YOCTO #6475]

(Bitbake rev: 56c294dc30b6c2575b1cf904e26b8b8bef7677c2)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-07-04 09:28:36 +01:00
Robert Yang 053857479c bitbake: lib: fix no newline at end of file
Add a '\n' to the last line of the file to fix:

No newline at end of file

(Bitbake rev: 54f1359ed2e9d47980cd221b7b43ef56543fe06d)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-07-04 09:28:36 +01:00
Robert Yang edf8cb1713 bitbake: toaster: fix no newline at end of file
Add a '\n' to the last line of the file to fix:

No newline at end of file

(Bitbake rev: 5eb65d92c35264087e5d82c35638f3b8805b1b3e)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-07-04 09:28:36 +01:00
Scott Rifenbark 49b416dde5 bitbake: templates: Added newline character to end of script.
Prevents errors when editing in vi and writing and quiting.

(Bitbake rev: 01692654cd792eaf6362bbac9c34314db4a7ffea)

Signed-off-by: Scott Rifenbark <scott.m.rifenbark@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-07-03 13:58:57 +01:00
Scott Rifenbark 70439569c8 bitbake: bitbake-user-manual-ref-variables.xml: Edits to OVERRIDES variable.
More detail.

(Bitbake rev: 0ac8a504de1798dae02eb2dade2776e7e25d034d)

Signed-off-by: Scott Rifenbark <scott.m.rifenbark@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-07-03 13:58:57 +01:00
Robert P. J. Day 0b96a4ba62 bitbake: bitbake-user-manual-execution.xml: Minor fixes.
Collection of typo and grammar fixes from "Execution"
chapter.

(Bitbake rev: ebca1b7847ff34213e6da71c79f00cb4b6b0757b)

Signed-off-by: Scott Rifenbark <scott.m.rifenbark@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-07-03 13:58:57 +01:00
Robert P. J. Day 538cbf4932 bitbake: bitbake-user-manual-intro.xml: Minor fixes mostly grammar adjustments.
(Bitbake rev: fedc5470e9f4f5438d3de957f47985f76a484533)

Signed-off-by: Scott Rifenbark <scott.m.rifenbark@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-07-03 13:58:56 +01:00
Scott Rifenbark ac8ac1724d bitbake: bitbake-user-manual-fetching.xml: Small fix in Git Fetcher
Fixed how we refer to the "rev" paramater.

(Bitbake rev: 90b1aec42cd29716f1cbd058757941cbfda235b0)

Signed-off-by: Scott Rifenbark <scott.m.rifenbark@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-07-03 13:58:56 +01:00
Robert P. J. Day 3a7258afd5 bitbake: bitbake-user-manual-execution.xml: Typo and grammar fixes.
(Bitbake rev: c469ceb86c1f76e9fa867ffb8a5c9b120ed7942b)

Signed-off-by: Scott Rifenbark <scott.m.rifenbark@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-07-03 13:58:56 +01:00
Scott Rifenbark 00ade7ecd0 bitbake: bitbake-user-manual-execution.xml: Edits to "Providers and Preferences"
I split this section into two sections and tried to simplify what
was being said about PROVIDES.

(Bitbake rev: 394f64123d5ef1e9b4628d083488b9432eccbded)

Signed-off-by: Scott Rifenbark <scott.m.rifenbark@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-07-03 13:58:56 +01:00
Scott Rifenbark 0736649c57 bitbake: bitbake-user-manual-fetching.xml: Small wording change.
(Bitbake rev: 66529a17d50bbce211361d98ace012b9424459c9)

Signed-off-by: Scott Rifenbark <scott.m.rifenbark@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-07-03 13:58:56 +01:00
Scott Rifenbark 157c80f488 bitbake: bitbake-user-manual-ref-variables.xml: Edits to PROVIDES variable.
Added some more detail to help describe the variable.

(Bitbake rev: 4b2019e67324f0f86e1a49d256eddb4f9ca597f4)

Signed-off-by: Scott Rifenbark <scott.m.rifenbark@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-07-03 13:58:56 +01:00
Scott Rifenbark 41b46d7826 bitbake: bitbake-user-manual-style.css: Added permalink style for display
Fixes [YOCTO #5772]

Added a couple styles to the CSS style sheet so that the displayed
permalink symbol is barely visible unless you hover over it with
the mouse.

(Bitbake rev: ee8db0cda1c8716ab0151a4859bfff84b2f3bd58)

Signed-off-by: Scott Rifenbark <scott.m.rifenbark@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-07-03 13:58:56 +01:00
Scott Rifenbark d3e1ce60b1 bitbake: bitbake-user-manual-customization.xsl: Added support for variable permalinks.
Fixes [YOCTO #5772]

Including the gloss-permalink.xsl file in the customization layer
so that making the manual now generates permalinks for each variable
definition in the glossary.

(Bitbake rev: 04467594a599472c367365dffe205bf727c4d592)

Signed-off-by: Scott Rifenbark <scott.m.rifenbark@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-07-03 13:58:56 +01:00
Scott Rifenbark dd09b077c0 bitbake: template: Added XSL template to support variable glossary permalinks.
Fixes [YOCTO #5772]

This file enables creation of permalinks for variables defined
in the glossary.

(Bitbake rev: 772f2bdc10fa242646e4d09f6bdbaf8558f06b22)

Signed-off-by: Scott Rifenbark <scott.m.rifenbark@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-07-03 13:58:56 +01:00
Scott Rifenbark ca77f37bf8 bitbake: bitbake-user-manual-intro.xml: Edits to "Obtaining BitBake"
Updated the last bullet item that describes a final way to
obtain a copy of BitBake.  The example was too specific regarding
Poky and OE.  So, I made it quite generic and kept with the
convention of not having any specific links from the BitBake
manual into YP docs or such.

(Bitbake rev: a2a3b534871aa23a5d442b93dd9660d5f9e27536)

Signed-off-by: Scott Rifenbark <scott.m.rifenbark@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-07-03 13:58:55 +01:00
Robert P. J. Day 296bb748f6 bitbake: bitbake-user-manual-intro.xml: Basic proofreading, grammar fixes.
(Bitbake rev: 266a0b5ef41dcc4048e2a4d1c43567568d7449d7)

Signed-off-by: Scott Rifenbark <scott.m.rifenbark@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-07-03 13:58:55 +01:00
Belen Barros 7c1a975a1c bitbake: toaster: Fix spacing and layout in no image files notification
Close the definition list before adding the notification, and
restrict the notification width to avoid uncomfortable line
lenghts.

(Bitbake rev: b82f724c271c43218dc28440219adcea968e2a26)

Signed-off-by: Belen Barros <belen.barros.pena@intel.com>
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-06-20 14:03:58 +01:00
Farrell Wymore fee3a88695 bitbake: toaster: display message if no images are generated
Display message if no images are generated for a target. This
commit was amended to have more conditions for displaying the
message.

[YOCTO #6094]

(Bitbake rev: ac1911eb857f4759ed085c8a4fc4b47cefafcd7a)

Signed-off-by: Farrell Wymore <farrell.wymore@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-06-20 14:03:58 +01:00
Ionut Chisanovici 457091dc56 bitbake: toaster: Add performance testing script
This is implemented as a django management command.
For the moment the 'manage.py perf' command will track the toaster
'gui' urls http response code and load time.

To use it:
1. do your toaster builds
2. ensure toaster is started
1. cd bitbake/lib/toaster
2. ln -s ../../../build/toaster.sqlite
3. ./manage.py perf

(Bitbake rev: 4a1fc6851d21500150715f0e8fa03c0b228ec5f2)

Signed-off-by: Ionut Chisanovici <ionutx.chisanovici@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-06-20 14:03:58 +01:00
Farrell Wymore 7ccbea9ea5 bitbake: toaster: refactor the target page
the target (packages) page had to refactored to
allow the displays to differ depending on the caller
namely the license manifest link. Amended to make the
link from the tag rather than the pathname itself.

[YOCTO #6291]
[YOCTO #6079]

(Bitbake rev: b21a2af9411da17d49521820fa512292e89c856e)

Signed-off-by: Farrell Wymore <farrell.wymore@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-06-20 14:03:58 +01:00
Richard Purdie f3d08464ef bitbake: build.py: Ensure shared work siginfo files are written to the correct location
Right now shared work signature data is saved to the non-shared directory
which is confusing to everyone including bitbake. Whilst its messy,
extra the stampbase data instead, which ensures the sig data is written
to the correct location alongside its corresponding stamp file.

(Bitbake rev: 7ae1d4844d9d3a76f86ef32c5a794e51e334e588)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-06-19 17:55:11 +01:00
Robert P. J. Day 5b1c329804 bitbake: bitbake-user-manual-fetching.xml: Grammar, typo, and tweaks.
Various grammatical and typo tweaks all in fetching chapter, including
moving CVS section after wget section for more logical content flow.

(Bitbake rev: 39bbc8f82894ca521d35645cd618dd131fde38ef)

Signed-off-by: Scott Rifenbark <scott.m.rifenbark@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-06-18 13:48:39 +01:00
Scott Rifenbark 8be77bb03f bitbake: bitbake-user-manual-style.css: Removed dead .PNG file
Fixes [YOCTO #6430]

I removed an old .PNG file that is not used.

(Bitbake rev: dc5adbb8ce140b6898f18bb99bac02aac16f2ac8)

Signed-off-by: Scott Rifenbark <scott.m.rifenbark@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-06-18 13:48:38 +01:00
Scott Rifenbark 74551392b3 bitbake: bitbake-user-manual: Added permalink support.
Fixes [YOCTO #5772]

I added the five XSL templates to the template folder.  I updated
the bitbake-user-manual-customization.xsl layer file to include
the new templates.

(Bitbake rev: e1c24a79ededb2dd4ac5ce09fcfdf93218261907)

Signed-off-by: Scott Rifenbark <scott.m.rifenbark@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-06-18 13:48:38 +01:00
Scott Rifenbark c65de86d81 bitbake: bitbake-user-manual-customization.xsl: Removed stale TOC variable.
I deleted the commented out xsl:param line that attempts to
set labeling for Appendices.  It had been replaced by a working
line.

(Bitbake rev: bf79174e02cc113e0512bcadc3ab0fdffd138215)

Signed-off-by: Scott Rifenbark <scott.m.rifenbark@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-06-18 13:48:38 +01:00
Scott Rifenbark d87cb56aa8 bitbake: bitbake-user-manual-fetching.xml: Edits to the Git Submodule fetcher section.
Exchanged "SRC_URI" for "URI" for clarity.

(Bitbake rev: 90cc450671eaabf917c10a8d01b729b6bc0f544c)

Signed-off-by: Scott Rifenbark <scott.m.rifenbark@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-06-18 13:48:38 +01:00
Scott Rifenbark 77166a2a88 bitbake: bitbake-user-manual-fetching.xml: Added new Git Submodule Fetcher section.
Added the "GIT Submodule Fetcher (gitsm://)" section to the
fetchers chapter.  This information was provided by Chris
Morgan.

Reported-by: Chris Morgan <chmorgan@gmail.com>
(Bitbake rev: a3a3331e7118fd3139fe5a80b157a5a5b38631de)

Signed-off-by: Scott Rifenbark <scott.m.rifenbark@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-06-18 13:48:38 +01:00
Alexandru DAMIAN e163522205 bitbake: toaster: build control functionality
We add the build control functionality to toaster.

* The bldcontrol application gains bbcontroller classes
that know how to manage a localhost build environment.

* The toaster UI now detects it is running under build
environment controller, and update the build controller
database and will shut down the bitbake server once
the build is complete.

* The toaster script can now run in standalone mode,
launching the build controller and the web interface instead
of just monitoring the build, as in the interactive mode.

* A fixture with the default build controller entry for
localhost is provided.

[YOCTO #5490]
[YOCTO #5491]
[YOCTO #5492]
[YOCTO #5493]
[YOCTO #5494]
[YOCTO #5537]

(Bitbake rev: 10988bd77c8c7cefad3b88744bc5d8a7e3c1f4cf)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-06-13 11:55:34 +01:00
Alexandru DAMIAN 87b99274e9 bitbake: toastergui: fix built ETA calculation
We fix the calculation showing the estimated time
to build completion.

(Bitbake rev: dc1daae51ef5201475b5c1a69d966ae57b66dcb6)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-06-13 11:55:34 +01:00
Alexandru DAMIAN 74cd8c38aa bitbake: xmlrpc: add support for token reusing
We add support to specify a connection token in the command line
and in the environment variable BBTOKEN.

When a client registers to a bitbake server, that client
will have exclusive access to the server. The client is identified
by a server-supplied token. If a client terminates, we cannot
reconnect to the server as the token is lost.

This patch adds the capability to specify the connection token
in the command line for xmlrpc clients. This allows us
to have bitbake work as an already-authenticated
client with the server and resume sending commands to a server.

(Bitbake rev: db5390940c0afbcdc9fbcf1225761968ae51d4a7)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-06-13 11:55:33 +01:00
Alexandru DAMIAN c7ae6bd307 bitbake: toaster: add function to get the database url
We add a function that returns the DATABASE_URL
for the current 'default' database settings. This
is useful to be able to start other toaster instances
with the same database settigns as the currently running
instance.

(Bitbake rev: 272a4bba0804bb6b5e0d498d3453321b5ed1dc76)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-06-13 11:55:33 +01:00
Alexandru DAMIAN 3bd8e9adc8 bitbake: toaster: create models for bldcontrol and enable it
We create the model classes that store information
about triggering builds, and the available build
environments.

We add a fixture with a default build environment
for build control, using a "build/" directory under
the poky checkout directory.

We enable the bldcontrol in toaster starting script
and in the toaster settings as to allow the actual database
to be kept in sync with the source code.

(Bitbake rev: d4bfe9059f765f11244b97e324c0131f32f8e400)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-06-13 11:55:33 +01:00
Alexandru DAMIAN e09cb4017a bitbake: toaster: initial bldcontrol application
The build features of Toaster will be contained into
a separate application, as to modularize the Toaster
project and provide multiple options for deployment.

This patch adds the application as a barebone Django
application in the Toaster project.

(Bitbake rev: 08556b79b7b2af08aaeedf8733b1b8996f387c4e)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-06-13 11:55:33 +01:00
Ross Burton 8e0c54cd0e bitbake: bb/utils: fix contains_any()
"set1 in set2" doesn't do what you'd expect, and if it did do a subset test
that's not the logic required by contains_any().

Instead get the intersection of checkvalues and val and check if the resulting
set is empty or not (by evaluating it in a boolean context), which tells us if
there are any elements in common.

Based on a patch by Valentin Popa <valentin.popa@intel.com>.

(Bitbake rev: 2e742c03e8dfdfa67899e7f5d579ed14bd87e139)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-06-12 17:47:59 +01:00
Ross Burton 2261e9f4c8 bitbake: bb/tests/data.py: add tests for contains() and contains_any()
contains_any is buggy so write a test case to demonstrate this.

(Bitbake rev: 25d10c349aab77bf99745c0a90dd8f9b90abccac)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-06-12 17:47:59 +01:00
Robert Yang 065fb5dd56 bitbake: cache.py: print debug info when EXCLUDE_FROM_WORLD
This gives us an easy way to find out which recipes have been excluded
from world when there are many layers.

(Bitbake rev: cf33d4bfc53ae8dc6353fa295e1acd789d4fc2d2)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-06-12 17:47:59 +01:00
Alexandru DAMIAN b5a928af72 bitbake: toaster: add Project class models to orm
This patch adds the Project class models to the
toaster GUI data model. There is no specified use,
and the link in the Build class is optional.

(Bitbake rev: 38f77a43d16e202d1f40632b27cb70dd20d02a5e)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-06-11 10:35:47 +01:00
Alexandru DAMIAN 9c3b53f12e bitbake: toaster: comment out bldcontrol migration command
This fixes toaster startup failing after the preparatory patches
for bldcontrol application have been merged.

I slipped a bldcontrol-specific line into the toaster startup
script and this prevents toaster from starting. We comment
out this line until the bldcontrol application have been merged.

(Bitbake rev: d095eec6fc958d0aeb514cdc206734617fd7c930)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-06-11 10:35:47 +01:00
Ash Charles 66d5be204f bitbake: bitbake: show wildcard appends for bitbake-layers
The 'bitbake-layers show-appends' command can use the built-in
get_file_appends() method which correctly identifies bbappends that
include a '%' wildcard in their filename.

(Bitbake rev: 2732dbae67c1945b668c38cc4cc5678c4aafe3d6)

Signed-off-by: Ash Charles <ashcharles@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-06-06 10:33:19 +01:00
Alexandru DAMIAN 36a5f66096 bitbake: toasterui: fix django settings environment value
Previously, the buildinfohelper only set a django settings module
environment variable if none were set.

This may lead to problems when the starting the toasterui
from an already existing Django environment.

As such, we always override the variable to provide the
correct name for the local Django settings module.

(Bitbake rev: 8271e61a2fbddd3fc49556829675478d7505d58f)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-06-06 10:32:55 +01:00
Alexandru DAMIAN 07cd9a3b34 bitbake: toaster: add project related models
We introduce the notion of a project in Toaster as the item
that holds the specification for triggering a build: the set
of layers used, the set of configuration variable values, and
the set of targets to be build.

Builds triggered through Toaster will be associated with a
Project, and they will be configured based on the project
settings at the moment when the build is ordered.

(Bitbake rev: 0bf0251ae05442ae260c6099b389bf765c4fef26)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-06-06 10:32:55 +01:00
Alexandru DAMIAN b68b74ddd4 bitbake: toaster: update toaster to run in managed mode
We disable bitbake self start to prevent race condition.

ToasterUI will shutdown the server when the build is done
if running in managed mode.

We fix usage of kill server flag in the bitbake binary.

(Bitbake rev: 30159dbda3a40fa596302f91c705cb5f148c97a9)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-06-06 10:32:54 +01:00
Alexandru DAMIAN b610aaee9d bitbake: toaster: startup script standalone mode
The toaster starter script was designed to be sourced in
a build environment and set up the build recording environment
to be used in an interactive mode.

For the standalone web server mode, we modify the toaster
script to be run directly from the checked-out sources,
without a build environment set up, and run the web server
alone. In the standalone mode, the build environemnts and
all build activities are controled through the web interface.

(Bitbake rev: c1db4ccf27bedcbab2f03e7539fdb11b042c4fb9)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-06-06 10:32:54 +01:00
Alexandru DAMIAN 05cf6fe0ff bitbake: toaster: startup script noweb mode
We add an option to the startup script, named "noweb" that
will start toaster without the embedded web server.

This is useful to start the system for build-only environments,
where the web server code is running on a different machine.

(Bitbake rev: c39838201301b9732581288a93783400bebe6591)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-06-06 10:32:54 +01:00
Alexandru DAMIAN 89fe052949 bitbake: toaster: read database settings from the environment
We add the capability to read the database settings for
Toaster from the environment. The DATABASE_URL is intepreted
and used to override the default settings.

This capability is essential for easy deployment of Toaster
in a managed hosted environment, and for creating
build environments with custom database settings.

(Bitbake rev: d16d19dafb83448fc214fce4fbdc2bcbf4bf9ce3)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-06-06 10:32:54 +01:00
Alexandru DAMIAN 506b5bd729 bitbake: uievent: retry on handler registration failure
The registration of a remote UI event handler may fail
if the server cooker is currently in some certain states.
This may happen, for example, when a remote UI is started
very fast after the bitbake server is started, and the
server hadn't time to finish initial configuration parsing.

Rather than fail outright, we have the remote UI event retry
registration for five time at one-second intervals,
in the hope it will succeed.

(Bitbake rev: c3d520c92ae4ae80d31926a416456df510654b6a)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-06-06 10:32:54 +01:00
Alexandru DAMIAN e89db137f0 bitbake: xmlrpc: client - remove fatal errors
When we use the XMLRPC client API to connect to a bitbake server,
we want to receive errors from the API instead of having the
API exiting without warning.

Thus the "bb.fatal" calls have been replaced with "bb.warn" calls,
and we re-raise the original exception for handling by the
original caller.

The bitbake starting script has been modified to properly test
for failures in calling the client API and handle them.

Additional error handling added in the client, as to prevent
fatal crashes.

(Bitbake rev: eb63f08c33644f64752aaae2146a000956ce894a)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-06-06 10:32:54 +01:00
Alexandru DAMIAN a5d01e9ec7 bitbake: bitbake: move configuration reading code
The configuration reading code should live in the
main bitbake entry point, and the server modules should
be supplied with correct configuration instead of attempting
to parse from configuration files.

This patch moves the endpoint address reading from XMLRPC
to the bitbake main script.

(Bitbake rev: ac5753274ff932e1d6f073ab4dab7bd6fe5355a1)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-06-06 10:32:54 +01:00
Richard Purdie 60c40e5db5 bitbake: siggen: Print warning about tainted tasks
The big warning printed when people use -f is easily ignored/forgotten.
To raise user awareness, print a warning any time we include a tainted
stamp file into a build instead.

(Bitbake rev: 18f9bcbad059608e22fca20309314e1c399acec7)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-06-01 14:29:32 +01:00
Richard Purdie 8a43a6a32b bitbake: build/utils: Fix broken exception handling
Checking for explicit exception names is bad, we also want to be able top
rely on inheritance. Fix these checks to be part of the real except clauses
so SkipPackage is recognised as being inherited from SkipRecipe.

(Bitbake rev: b131229145e1f2c372d6230a7b554e436c13c3f9)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-06-01 14:29:32 +01:00
Richard Purdie 4a78f52d87 bitbake: bb.fatal: Raise a BBHandledException instead of exiting
With new bitbake UIs having the cooker exit at 'random' points
in the codebase is problematic. This patch raises an exception
which matches the situation instead.

(Bitbake rev: 181a9735d02ebd517378558e909efc8b1b118973)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-06-01 14:29:32 +01:00
Richard Purdie 91a164cb87 bitbake: event: Add SkipRecipe event to replace SkipPackage
In the depths of time we were rather confused about naming. bb files
are recipes, the event to skip parsing them should be SkipRecipe,
not SkipPackage. This changes bitbake to use the better name but
leaves the other around for now. We can therefore start removing
references to it from the metadata.

(Bitbake rev: 98d9e6e0f514a7cb7da1d99bf4bd5602b89426d6)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-06-01 14:29:31 +01:00
Richard Purdie 044296ce4c bitbake: event.py: Allow passthrough of BBHandledException events
We need BBHandledException events to be passed through to the higher
layers, they don't need addition of any traceback since they've already
been reported to the user.

(Bitbake rev: f63b61f8997862439519f474cc09f3e05e95288c)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-06-01 14:29:31 +01:00
Richard Purdie ab29933898 bitbake: cookerdata: Improve error handling
If we see errors during parsing, firstly its bad to show a traceback
for an expansion error so lets suppress this.

Secondly, raise a BBHandledException instead of a SystemExit to show
we've informed the user about the condition (printing a traceback in the
default unknown case).

(Bitbake rev: e01988d9a1b7c40e31161c6ce7b85c4405671068)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-06-01 14:29:31 +01:00
Richard Purdie a26667054b bitbake: event: Fix event handlers to raise SkipPackage
If an event handler triggers a SkipPackage event, we really want that
event to be received and processed by the higher code levels. Currently
it was getting caught and ignored which was leading to recipes
being present when they clearly shouldn't have been.

In general this exception catching looks to be doing the wrong thing. It
was introduced in
http://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/bitbake/lib/bb/event.py?id=37cb4cc02b2e2b6c338c5943747e0a1ef15176b3
but we likely want exceptions to pass through to the higher layers.

This patch therefore removes that code.

(Bitbake rev: 79211b3434855942f9fad4a1db69ce7be911327c)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-06-01 14:29:31 +01:00
Paul Eggleton c2852ea835 bitbake: utils: avoid printing traceback on ExpansionError during parsing
If an ExpansionError occurs during better_exec() we should just raise it
instead of printing the traceback, so that recipe errors (such as broken
URLs in SRC_URI) are more easily comprehensible.

(Bitbake rev: 5b0da8932c318813138c113d2bb20498145dbd42)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-05-30 10:20:25 +01:00
Paul Eggleton 429bb2ae80 bitbake: fetch2: improve handling of two classes of URL parameter mistakes
Handle the following situations in a URL (e.g. in SRC_URI):

* Trailing semicolon in a URL - this is now ignored.
* Parameter specified with no value (no equals sign). This still
  produces an error, but at least it is MalformedUrl with a proper
  message rather than "ValueError: need more than 1 value to unpack".

(Bitbake rev: bfd13dfbc4c9f1dd8315002271791b1d9e274989)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-05-30 10:20:25 +01:00
Richard Purdie 91b6edd092 bitbake: Revert "toaster: toaster oe-selftest support"
This reverts commit bb5b1d6b139b886e54bfdc0c17f2b556db6a7fde.

Applied to incorrect repo.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-05-30 10:20:25 +01:00
Volker Vogelhuber ce5aadc677 bitbake: fetch2/hg: Fix missing proto param for hg checkout with user and pw
A fix for the former patch when checking out a repository with
username and password using HG

(Bitbake rev: 0e7b594ccbceb3149f38776cea204807031ef69f)

Signed-off-by: Volker Vogelhuber <v.vogelhuber@digitalendoscopy.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-05-28 08:29:18 +01:00
Paul Eggleton 8cdc7926ec bitbake: bitbake-layers: show-cross-depends: add support for RRECOMMENDS
RRECOMMENDS must be satisfied at build time, and these could cross layer
boundaries, so report these if they exist.

(Bitbake rev: 5569b3dca61e6d962494ca65c7aad09b2eb2ae63)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-05-28 08:29:18 +01:00
Paul Eggleton 8ae64ff2e4 bitbake: bitbake-layers: show-cross-depends: ignore self-satisfied RDEPENDS
Overlayed recipes caused this to show false positives because the
overlaying version appeared to be satisfying the overlayed version's
RDEPENDS; but you'd never be building both at the same time.

(Bitbake rev: b94318174fe7f92b9a20eabb0bc4055066cb3d51)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-05-28 08:29:18 +01:00
Paul Eggleton 603f3fbe98 bitbake: bitbake-layers: show-cross-depends: ignore global inherits
It's not particularly useful to show globally inherited classes here
since they do not normally represent a dependency.

(Bitbake rev: d16948bb88fcf44d861985838030be7c08697963)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-05-28 08:29:18 +01:00
Paul Eggleton e124c0f9a8 bitbake: bitbake-layers: show-cross-depends: add option to ignore a layer
By default, show-cross-depends shows dependencies on OE-Core (i.e.
"meta") which is not particularly useful. Add an option to allow you to
hide those. For example, to hide all dependencies on OE-Core:

  bitbake-layers show-cross-depends -i meta

Multiple layers can be specified by using commas as separators (no
spaces).

(Bitbake rev: 0e9062e65acbb05c1d9b3a9145eb866c3d562309)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-05-28 08:29:18 +01:00
Richard Purdie 122079e9b0 bitbake: data_smart: Fix an unusual variable reference bug
If you try:

Y = ""
Y_remove = "X"

in OE-Core, bitbake will crash with a KeyError during expansion. The reason
is that no expansion of the empty value is attempted but removal from is it
and hence no varparse data is present for it in the expand_cache.

If the value is empty, there is nothing to remove so the best fix is simply
not to check for None but check it has any value.

Also add a test for this error so it doesn't get reintroduced.

(Bitbake rev: af3ce0fc0280e6642fa35de400f75fdbabf329b1)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-05-21 16:47:01 +01:00
David Reyna bf9ff3de5f bitbake: toaster: reduce redundant foreign key lookups
Replace redundant foreign key lookups with "with" to improve all
recipes page load time. Do depends pre-lookup in the view class,
and use python itertation instead of filter() all to achieve x16
processing speedup.

[YOCTO #6137]

(Bitbake rev: a68a6dc50c11cc59e7c873414e3e22ac2644dea7)

Signed-off-by: David Reyna <David.Reyna@windriver.com>
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>

Conflicts:
	bitbake/lib/toaster/toastergui/views.py
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-05-21 16:47:01 +01:00
Farrell Wymore defe227135 bitbake: toaster: sort columns properly after edit columns
If a sorted column is made invisible through the edit columns function,
resort the table the its default order.

[YOCTO 5919]

(Bitbake rev: 64618f7489eb9eb13a97d03cd2d353384f5faa70)

Signed-off-by: Farrell Wymore <farrell.wymore@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-05-21 16:47:01 +01:00
Ionut Chisanovici 8a8d7176fc bitbake: toaster: toaster oe-selftest support
This patch adds toaster tests using the oe-selftest infrastructure.
You need to have builds done - the tests will verify data integrity
after the toaster collection phase.

Once you have your toaster builds done, to run the automated backend
tests via oe-selftest do the followings:

1. Update builddir/conf/bblayers.conf to contain the meta-selftest
layer
2. From the builddir run:

'oe-selftest toaster'

or if you just want to run a single test:

'oe-selftest toaster.Toaster_DB_Tests.testname'

This first part adds the meta/lib/oeqa toaster file.

(Bitbake rev: bb5b1d6b139b886e54bfdc0c17f2b556db6a7fde)

Signed-off-by: Ionut Chisanovici <ionutx.chisanovici@intel.com>
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-05-21 16:47:01 +01:00
Chris Morgan 09edf20070 bitbake: fetch2/gitsm: Documentation improvements for gitsm:// fetcher
(Bitbake rev: ef2bf63e56b87e19d1a9059dd2d81a9a1a537254)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-05-19 12:04:34 +01:00
Richard Purdie 50067e069c bitbake: fetch2/perforce: Ensure command has a default
Ensure there is default command available if FETCHCMD_p4 isn't set.

Also clean up the getVar references whilst here.

(Bitbake rev: d150226d11d5f041f78c8c3ce4abc5465dbc81d8)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-05-11 15:26:25 +01:00
Richard Purdie 56cd778eb0 bitbake: parse: Improve file inheritance logging
The file inheritance logging has been here since the dawn of time. It
duplicates output many times over and logs to debug level 2. When running
with the debug option, its understandable the user may want to see the
paths of files included in the build.

These changes remove pointless/duplicate output and print the
include paths clearly at debug level one in a form which
users should be able to more easily parse.

(Bitbake rev: 633a56987df639295fe9128418da5634050063e5)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-05-11 13:15:12 +01:00
Richard Purdie c0bdb8100f bitbake: knotty: Fix output printed to the user at debug log levels
If a user runs with the -v or -D options, its understandable they'd
expect to see log output from the workers yet right now a bug in the
log handling does not show this.

Fix the conditional to ensure such log output is shown on the terminal
when it has been requested. Ideally this data should always flow to
the logfiles but that is for another patch.

This also fixes the code to do what was always intended in the comments,
i.e. if the user specifies -D or -v, output is shown from the tasks,
otherwise notes are suppressed.

(Bitbake rev: 20a3c93d8572969e76563f29bff89400b93ffae7)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-05-11 13:15:12 +01:00
Richard Purdie bc391938c4 bitbake: parse/ast: Show append logging at lower log level
It was reported that bitbake -D made no mention of which append files it
was using. bitbake -DD does but it makes sense to increase the log level
of this piece of debug information.

[YOCTO #6262]

(Bitbake rev: 5824bf9c6feea05567d155911f4ab2e371911d34)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-05-11 12:27:56 +01:00
Volker Vogelhuber 60436dc604 bitbake: fix cloning of mercurial repository with username and password specified in url
(Bitbake rev: 9a328262e047562a6edf7b2ec7b8a1949e287cbe)

Signed-off-by: Volker Vogelhuber <v.vogelhuber@digitalendoscopy.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-05-11 12:27:56 +01:00
Alexandru DAMIAN 646b089c38 bitbake: toaster: modify SQL data type constraints
This patch modifies the data type constraints for the SQL
fields.

The original constraints have been based on guess worked, and
they weren't strictly enforced for sqlite. On MySQL, the
constraints are strictly enforced, and need to match the real
data that we have.

After measuring data on sample builds, the new constraints
have been validated to work with MySQL, and this patch updates
the database schema with the new field types / constraints.

(Bitbake rev: aaefbc0b4e7484d6f4854ce49602821a6e4adc4b)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-05-08 16:18:12 +01:00
Alexandru DAMIAN 09a1ff9743 bitbake: toaster: fix migration code for MySQL
This is a patch that fixes the 0004, 0005 migrations of the toaster
model to get them to properly work with MySQL.

These migrations had a conflict around Build.timespent field.
The sqlite3 constraits were not enough to detect the conflict, and
the migrations worked as expected.

MySQL objected to adding the field twice, so I did regenerate
the migrations with the correct model listing. The net effect
is the same, so the migrations work the same, but now we
can use these two migration on the MySQL and other more advanced
SQL engines.

(Bitbake rev: 29afc29154b948d270ce52978a1eed4cd8887f92)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-05-08 16:18:12 +01:00
Dave Lerner 1a10bd3036 bitbake: toaster: fix span2 headings for packages
[YOCTO 6147]

Fix bug in views.py that put 'span2' class id in a 'clclass' key
instead of the 'dclass' key.  This fix exposed another problem in
target.html and the target view function that used 'package_size'
instead of 'size' as the 'clclass' which is inconsistent with the
model field name, causing table data columns to be shown when
the header is hidden.

(Bitbake rev: 436865ac8520c1d75401dde7df373dba9a386c9d)

Signed-off-by: Dave Lerner <dave.lerner@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-05-08 16:18:12 +01:00
Belen Barros Pena 1278267807 bitbake: toaster: Make sure layer branch only shows when not empty
In the recipe details and package details pages, the layer branch
definition list elements should only show when the branch field in
the database is populated. If the branch field is empty, we don't
show them.

The patch also removes all unnecessary data-toggle and
data-original-title attributes.

[YP #6152]

(Bitbake rev: 72adcadb931519f803dad7488544767241561fb7)

Signed-off-by: Belen Barros Pena <belen.barros.pena@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-05-08 16:18:12 +01:00
David Reyna 961f6226fa bitbake: toaster: sort outcome and sstate columns by string value
Sort the 'outcome' and 'sstate_result' columns by their
description string values and not their numerical values.

[YOCTO #5921]

(Bitbake rev: eee0abe70e4d85a0753f0b122518d7a9f06181bc)

Signed-off-by: David Reyna <David.Reyna@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-05-08 16:18:12 +01:00
Nick Lewis 138ed54992 bitbake: Fix refetch of local files when they are read only
(Bitbake rev: 42b7be1a6c5ff559f4a213eed04b370ca955460d)

Signed-off-by: Nick Lewis <nick.lewis@usa.g4s.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-05-08 16:17:05 +01:00
Richard Purdie a671ced0ea bitbake: data_smart: Ensure all possible overrides are cached including those with '_' in the name
Unfortunately we've been neglecting to pay the correct tributes to the
cookie monster and hence the datastore is malfunctioning.

Currently tributes are only paid on the last part of a variable after
the last "_" character. We need to split by *all* "_" characters since
an override may contain the character.

This fixes the code so the correct number of tributes are made. Paradoxically
parsing appears to be faster after this change.

(Bitbake rev: d1c712fd3a59fa804e6fd451612c30487671f3a2)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-05-07 10:29:58 +01:00
Richard Purdie 0304d92a97 bitbake: prserv/db: Avoid fsync() calls
If the power were to fail, it doesn't matter to us much if the data
makes it to disk or not, we'd have other problems. However an fsync()
call on a multi build autobuilder is painful so lets avoid them.

(Bitbake rev: 4eb2dc8048e2722d64d589f453df1ce6262c71b8)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-05-03 11:24:48 +01:00
Richard Purdie 99c9b337ac bitbake: persist_data: Avoid fsync() calls
If the power were to fail, it doesn't matter to us much if the data
makes it to disk or not, we'd have other problems. However an fsync()
call on a multi build autobuilder is painful so lets avoid them.

This is particularly true in this case if a timeout causes a reconnect
during a build.

(Bitbake rev: ec28256ac2a30f047585e8f61200d764bc295ded)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-05-03 11:24:48 +01:00
Richard Purdie c1f4df4e88 bitbake: codeparser: Fix to better catch all getVar references
Currently if you do localdata.getVar, the code parser simply ignores
the references. Change the code to use endswith() to catch more of the
references. These names are probably unique enough to get away with this.

Bump the cache version to ensure things get updated.

(Bitbake rev: cf763cddc3faa2361b4c4dbd08419e4ebabf208f)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-05-03 11:24:48 +01:00
Richard Purdie afcf8e87aa bitbake: bitbake: Bump version to 1.23.1
(Bitbake rev: c40a4100ddb841d231360344616e59ab98e61fb5)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-05-01 22:47:58 +01:00
Otavio Salvador 17daa2ba62 bitbake: bb.utils, bb.codeparser: Add bb.utils.contains_any
This includes contains_any in the special handling code for sstate. It
does not take into account the equivalence of the values. In current
code, considering 'bb.utils.contains_any("A", "foo bar", ...)':

A = "foo"
A = "bar"
A = "foo bar"

All those will get different signatures.

(Bitbake rev: d1e3345d715e488ec3f5515fb0e1fb39366346bc)

Signed-off-by: Otavio Salvador <otavio@ossystems.com.br>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-04-29 23:38:54 +01:00
Christopher Larson 107269d9d0 bitbake: codeparser: don't interact with the cache for subshells
Doing so was causing leakage between the execs of the main value and that of
the subshell value, and was causing the cached subshell value to be used for
the overall variable. At the least this could cause execs contamination
between two variables that while differing, run the same subshell. Beyond
that, it's possible we could have been using an incomplete cached value of
a subshell for that of the main value.

Before this, bb_codeparser.dat would change between parses with differing
bbfile parse order. After, it does not change.

The codeparser cache version is bumped, to ensure we don't use potentially
incorrect cached values from previous runs.

This should hopefully resolve the difficult-to-reproduce issues we've seen at
Mentor Graphics where bitbake emits a script to run a task and misses
dependent functions, resulting in 'command not found' failures. This issue has
also been mentioned on the oe devel list, where someone hit a case where
oe_runmake was missing from a do_install task (IIRC). Adding debug information
showed that bitbake's information about the variable dependencies for this
task is inaccurate in the failure cases.

(Bitbake rev: 97537e4786a1e3a329249497498b59b8f5174fc3)

Signed-off-by: Christopher Larson <kergoth@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-04-29 17:34:24 +01:00
Jacob Kroon 3d34b49f4a bitbake: fetch2: Cleanup file checksum generation, v2
The initial version of this patch was reverted in

0a94e568152de550dedc8135a766beb18bf064ab

However, it was later agreed upon that it was reverted due to
a misunderstanding during code review; the patch is ok.

This version of the patch also removes an unneeded initialization of
the 'checksum' variable outside the scopes it is being used in.

(Bitbake rev: 9ee19a3ca2f8e11a91f0289ea3486310c61d40f2)

Signed-off-by: Jacob Kroon <jacob.kroon@mikrodidakt.se>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-04-29 17:33:58 +01:00
Volker Vogelhuber 0f717ea4b5 bitbake: HG Fetch with username and password in url
Following up a previous patch for mercurial fetcher, I just fixed a
problem when calling update on a repository with subrepositories enabled.

Attached a patch that fixes this problem

>From caae519a2bd64bf7c729bb26aff344827def47fb Mon Sep 17 00:00:00 2001
From: Volker Vogelhuber <v.vogelhuber@digitalendoscopy.de>
Date: Tue, 29 Apr 2014 15:29:47 +0200
Subject: [PATCH] bitbake: - fixed authentication issues in case of using sub
 repositories

(Bitbake rev: dfa041c940caad12da50126a559afc8de089eeda)

Signed-off-by: Volker Vogelhuber <v.vogelhuber@digitalendoscopy.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-04-29 17:33:57 +01:00
Richard Purdie 183dac4036 bitbake: Revert "fetch2: Cleanup file checksum generation"
This reverts commit 6dfb33fd7204e09815bde7ba5bf21941a73fe965.

It causes an unindeded change in behaviour.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-04-28 12:54:06 +01:00
Jacob Kroon a688c9cbab bitbake: bitbake-diffsigs: Fix runtime error when no arguments are given
If bitbake-diffsigs is run without arguments it will error:

    Traceback (most recent call last):
      File "./bin/bitbake-diffsigs", line 121, in <module>
        if output:
    NameError: name 'output' is not defined

Fix this by moving the check for output into the inner else-clause.

(Bitbake rev: 7d3545a66863ad7183a7650b2af57eee091c45ae)

Signed-off-by: Jacob Kroon <jacob.kroon@mikrodidakt.se>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-04-27 01:27:55 +01:00
Jacob Kroon d10353039e bitbake: fetch2: Cleanup file checksum generation
Cleanup the fix done in f9416e76e272ba3249abff099f6f3a47fe82e03e.

Instead of adding continue statements we can just move the last
statements into the final else-clause.

(Bitbake rev: 6dfb33fd7204e09815bde7ba5bf21941a73fe965)

Signed-off-by: Jacob Kroon <jacob.kroon@mikrodidakt.se>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-04-27 01:27:55 +01:00
Richard Purdie c7c58dbe42 bitbake: fetch2/git: Stop git from triggering fsync() calls
We only ever clone other repositories, if there were a problem such as power
failure, we'd blow away data and rebuild. As such we don't need fsync(). With
filesystems like ext*, the fsync pushes nearly all the data out to disk
which impacts all running processes.

We therefore set a configuration parameter to disable the fsync() calls.

Also fixup a case where basecmd wasn't being used for no good reason.

(Bitbake rev: 0a26abaf3a1e34d556c9375068dd17c879568d0f)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-04-27 01:27:55 +01:00
Richard Purdie 81a1b83954 bitbake: bitbake-worker: Drop BBHASH variables
Iterating through and calling setVar on this number of variables has significant
overhead in the profiling data. By not setting this, we save 3,000 calls
to setVar which gives a noticeable improvement to the speed of task execution.

The BBHASH variables have since been replaced by accessing that data through
the siggen code and going forward, that is the preferred way work with it.

(Bitbake rev: 92526eadd09d19938762290e0492076174367583)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-04-23 11:43:29 +01:00
Richard Purdie 249a24eee0 bitbake: parse/ast: Optimise data finalisation
The optimisation where only the data we're interested in was finalised
was good but it turns out we can do better. In the case where a
class-extension is to be targeted, we can skip the other targets.

This change does that and speeds up parsing at the bitbake-worker
execution time. Specifically, you can see an improvement in the speed
of bitbake X -n.

(Bitbake rev: b56918c7ef7913e84356c69ee9b269844a446728)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-04-23 11:43:29 +01:00
Richard Purdie 39c5cfbef7 bitbake: runqueue: Do not write out stamp files in dry_run mode
In dry run mode, stamps for noexec tasks are being written out which
is incorrect. Avoid this.

(Bitbake rev: aa6448a0552ba2947ac262b8b5314a593d1058d3)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-04-23 11:43:29 +01:00
Richard Purdie 75c6b781fc bitbake: runqueue: Fix task weighting algorithm
When looking at a list of tasks, do_patch and do_unpack were being
given equal priority when one clearly depends on another. The
reason for this was the default task weights of 0 being to tasks.
This is therefore changed to 1 to allow correct weighting of dependencies
which means the scheduler has better information available to it about
tasks.

Weight endpoints differently (10) for clearer debugging of priorities.

(Bitbake rev: 12dc1d17fac3e8ec420fcafb06186d32fd847d89)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-04-23 11:43:29 +01:00
Richard Purdie d8ef08c96b bitbake: runqueue: Fix handling of zero priority task
The zero priority task should be run first but was being confused with
the None value the priority field defaulted to. Check for None
explicitly to avoid this error.

In the real world this doesn't change much but it confused the debug
output from the schedulers.

(Bitbake rev: 49c9d8c9400f74c804c2f36462639236e0841ff0)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-04-23 11:43:29 +01:00
Belen Barros Pena 5cbf2318b1 bitbake: bitbake: toaster: Fix Empty tasks filter
Somehow the counter was set to count tasks with outcome
'not available', instead of outcome 'empty'. This patch
fixes the problem.

[YOCTO #6146]

(Bitbake rev: 1cc2a299c842185ba14074817d5cd2925275610b)

Signed-off-by: Belen Barros Pena <belen.barros.pena@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-04-23 11:43:29 +01:00
Belen Barros Pena 5345985b47 bitbake: bitbake: toaster: Fix total number of tasks in build dashboard
The total number of tasks in the build dashboard was counting
_setscene tasks, which are not exposed by Toaster as separate
tasks. This patch makes sure that _setscene tasks are not
counted when calculating that number.

[YOCTO #6145]

(Bitbake rev: f8817c060042147a173d3451121304b25ffc4b6c)

Signed-off-by: Belen Barros Pena <belen.barros.pena@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-04-23 11:43:29 +01:00
Scott Rifenbark 57e8d56a6b bitbake: bitbake-user-manual-hello.xml: Edits to the "Hello World" Appendix.
Fixes [YOCTO #1973]

These are review changes applied from Paul Eggleton.

(Bitbake rev: 431181705f1bc3fc51439faa04a193debc1d03fd)

Signed-off-by: Scott Rifenbark <scott.m.rifenbark@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-04-21 23:03:19 +01:00
Scott Rifenbark 61a680fb5d bitbake: bitbake-user-manual-hello.xml: First draft of "Hello World" appendix
Fixes [YOCTO #1973]

Added the complete "Hello World" appendix first draft beyond
what Bill Traynor had supplied.

(Bitbake rev: 176baaf3d0f033091f5ab9d9cfb2cec3d25a5442)

Signed-off-by: Scott Rifenbark <scott.m.rifenbark@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-04-21 23:03:19 +01:00
Scott Rifenbark 4687351d16 bitbake: bitbake-user-manual-hello.xml: Edits to environment for hello world.
Some general edits to the sections that get you set up to run
the "Hello World" example.

(Bitbake rev: 43286d31c22e6e2fa96ba1b82445e85db0a0f18c)

Signed-off-by: Scott Rifenbark <scott.m.rifenbark@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-04-21 23:03:19 +01:00
Scott Rifenbark f1b9c7ea32 bitbake: bitbake-user-manual-metadata.xml: Added section on pathname syntax
Added a brief subsection to the "Basic Syntax" section that
describes the issue using the tilde character (~) when providing
a pathname as part of a statement.  BitBake does not expand
this character like the shell does.  We recommend to not use
it in pathnames.

(Bitbake rev: 8e4c5cee932c3f38d4147c59612d76ca0b002727)

Signed-off-by: Scott Rifenbark <scott.m.rifenbark@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-04-21 23:03:19 +01:00
Scott Rifenbark ef7fac5bba bitbake: bitbake-user-manual-ref-variables.xml: Edits to the BBPATH variable.
I took the stuff about running bitbake outside the build
directory and removed the note syntax.

(Bitbake rev: 783f29a593ce16a37b275b02f71b3ec918ef8c91)

Signed-off-by: Scott Rifenbark <scott.m.rifenbark@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-04-21 23:03:19 +01:00
Scott Rifenbark 45880c7235 bitbake: bitbake-user-manual-ref-variables.xml: Fixed BBPATH example
The example used in the note in the BBPATH variable description
had incorrect syntax.  Spaces existed in the BBPATH = statement.
I have removed them.

(Bitbake rev: 896a1c102a4afda0e5d07d009bd7918044f88269)

Signed-off-by: Scott Rifenbark <scott.m.rifenbark@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-04-21 23:03:19 +01:00
Scott Rifenbark 8f4b4e6f31 bitbake: user-manual-hello.xml: General edits
I worked through the chapter and made some minor edits based
on my user experience.  Part of what I did was to update
the console output for the examples.

(Bitbake rev: 13bf6cf71ff49572c3413435a33d87e5b6a8df0d)

Signed-off-by: Scott Rifenbark <scott.m.rifenbark@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-04-21 23:03:19 +01:00
Scott Rifenbark 1313bacfbe bitbake: bitbake-user-manual-hello.xml: Fixed two broken links.
(Bitbake rev: 7a0948beba2c882de31ec9b98c7740bba230aed9)

Signed-off-by: Scott Rifenbark <scott.m.rifenbark@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-04-21 23:03:19 +01:00
Scott Rifenbark ea4e822b8d bitbake: bitbake-user-manual-metadata.xml: New section on anonymous Python functions
Per Paul Eggleton's suggestion, I added a new section on
anonymous Python functions into the "Functions" section.
I also updated the intro text to account for the added
type of functions.

(Bitbake rev: 983d03c1a082e2b83187f0788e61a7941670b242)

Signed-off-by: Scott Rifenbark <scott.m.rifenbark@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-04-21 23:03:19 +01:00
Scott Rifenbark 9889a91a5b bitbake: bitbake-user-manual-metadata.xml: Edits to flexible inheritance section.
Fixes [YOCTO #5472]

Applied review edits from Paul Eggleton to this section.
Minor edits and some re-writing.

(Bitbake rev: 7259d9d40aad8254751f7674653cd362a9023054)

Signed-off-by: Scott Rifenbark <scott.m.rifenbark@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-04-21 23:03:18 +01:00
Richard Purdie 392de9d568 bitbake: doc: Rename user-manual -> bitbake-user-manual
This manual gets combined with other manuals and in that context, it helps
a lot if its seen as the Bitbake User Manual. Renames are a pain but
this is worthwhile so that other docs can correctly be combined with this
one. This also clarifies things like google search results which is helpful.

(Bitbake rev: 452a62ae0c2793e281d6769fd3e45500a74898d6)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-04-21 23:03:18 +01:00
Richard Purdie 4ba4ffeb05 bitbake: bitbake: Update to version 1.23.0 for master
(Bitbake rev: bb4980c63db386ce7d30d9a6b86e9f3861b3bc3a)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-04-10 17:57:56 +01:00
Richard Purdie c3a216c5f5 bitbake: bitbake: Update to version 1.22.0
(Bitbake rev: a8a32de0fb74ff63e5490e3b5e4419146a8f7367)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-04-10 17:57:56 +01:00
Paul Eggleton 915ec3d02e bitbake: cache: don't trigger reparse on recipes with wildcards in SRC_URI
Since we now get wildcards in the file checksum list in the cache, we
need to ignore them when checking to see if they still exist. This
fixes connman-gnome reparsing on every bitbake execution in OE-Core.

(Bitbake rev: 340c250fc664414ab2715a454bedbd19e8efe103)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-04-10 17:57:55 +01:00
Alexandru DAMIAN 046a9ea303 bitbake: toaster: Allow toaster to start without pytz
This patch allows toaster to start without pytz.

Django can work with or without pytz, but in the
time zone fix I mistakenly added a hard dependency
on this module.

This patch eliminates the hard dependency.

(Bitbake rev: 40027a6e093c3b7480bfaccbd57e0e613d9a7b71)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-04-09 18:21:42 +01:00
David Reyna efa4a98fe4 bitbake: toaster: add URI search path list
Display the URI paths from SSTATE_DIR and SSTATE_MIRRORS
in the Task page.

[YOCTO #5996]

(Bitbake rev: 6d07b531736c2b304da2dfe661239fd3612b0541)

Signed-off-by: David Reyna <David.Reyna@windriver.com>
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-04-09 17:00:02 +01:00
Dave Lerner d3e5632efb bitbake: toaster: sort on size in detail pages
[YOCTO 5778]

Implements the features described in the attachment to bugzilla 5778
   - new global changes to the format of size data, and
   - adding sorts by selected columns to specific detail pages.

Although new pagination and row search capabilities are shown on the
screen shots for the 5778 attachment, those features are specified in
a different bugzilla entry 5777 and are not implemented in this commit.

Also, the 5778 spec includes table sorting for the recipe package
detail page, but sorting for that page was not implemented in this
commit due to complications with sorting then returning to a page that
is only one URL fragment in a template.

The scope of file changes are described below.
Changes to support new 'size' field column formats...
    default.css - added sizecol class style (right justified)
    projecttags.py - changed filtered_filesizeformat to allow
        ".0" suffixes

Changes that add class 'sizecol, span2(as spec'd) ' to <th> and/or
<td> size columns were made to...
    dirinfo.py,
    package_built_dependencies.html,
    package_included_dependencies.html,
    recipe.html,
    bpackage.html, and
    target.html

More significant changes to support detail page table sorting
are:
    - tablesort.html: New created to implement the sort icons,
        directions, and table headings, and
        suppress sort handling if 'disable_sort' in context,
        without search or pagination elements ingrained
        in basetable_top. Confining the changes to this small file
        reduces the impact (testing and risk) on the larger set of
        files that arleady include basetable_top/bottom files.
    - view.py: Modified the following view functions with
        - trivial changes for size formatting to the views: target,
        - changes to package_built_detail, package_included_detail,
            package_included_reverse_dependencies to handle the sorting
            implementation as well as moving headings and size
            formatting for size columns from templates to the views.
    - Implementation of the detail sorting using above in:
            package_built_detail.html,
            package_included_detail.html, and
            package_included_reverse_dependencies.html
        to include the tablesort heading setup, format the size column,
        and iterate over the new sorted objects, suppressing sorts if
        table row count less than 2.

(Bitbake rev: d16126e9abfffde66ab70865a81997322847d44e)

Signed-off-by: Dave Lerner <dave.lerner@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-04-09 17:00:02 +01:00
David Reyna dc7d3d2910 bitbake: toaster: hide tasks without order or outcome
Hide the incomplete tasks, those without order numbers and/or outcomes.

[YOCTO #6060]

(Bitbake rev: bdbba81715765681a0404fa82f28e471b241051a)

Signed-off-by: David Reyna <David.Reyna@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-04-09 17:00:01 +01:00
David Reyna 74e2f85019 bitbake: toaster: regex alternation filter caused django error
The combination of a regex filter specification that uses alternate,
plus a search string, plus multiple search_allowed_fields, leads to
a Django fatal error. Replace this regex filter for variables in
local files with a simpler 'contains' against the project's directory
plus a '/conf/' string.

Alex rebased this on top of fix for #6048.

[YOCTO #5962]

(Bitbake rev: fd57128dc3a35ca87031f3df1a531a085e89baf0)

Signed-off-by: David Reyna <David.Reyna@windriver.com>
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-04-09 17:00:01 +01:00
Alexandru DAMIAN f88a343773 bitbake: toaster: fix filtering query for multiple filters
This is a fix for using multiple expressions in filters.
Three different issues are touched:
* added an explicit error message for incorrect filter usage
* changed the value separator to something that will pass
through from the browser to the actual code
* changed the "and" operator for combining Q from a dubios
lambda function to the standard operator.and_

(Bitbake rev: 845b081fc108c656f04d4a70afa4695defc13c9f)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-04-09 17:00:01 +01:00
Alexandru DAMIAN b14482ef61 bitbake: toaster: keep number of rows across searches
This patch will make Toaster remember the selected
number of rows to be displayed in tables across different
searches.

The number of rows setting will not be remembered if the
the user leaves the page and subsequently returns to it.

(Bitbake rev: a84f296591be26972b808e98816e9d92cadf3eb5)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-04-09 17:00:01 +01:00
David Reyna 5a293287af bitbake: toaster: show unique set-in files in configvar
When listing the set-in files in the configure varaible table, only show
each file once in that summary view.

[YOCTO #6048]

(Bitbake rev: e69a57fc3302ed8fac21631ba95ea66e93b243e5)

Signed-off-by: David Reyna <David.Reyna@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-04-09 17:00:00 +01:00
Dave Lerner 4b17947f24 bitbake: toaster: disable 'size over total' sort
[YOCTO 6061]

Disable size-over-total table column sorts in the image installed
packages view because that field is a computed field and only model
fields are sortable in the current table toaster implementation.

(Bitbake rev: ba6937c9bffcf81f71ef6fa9f0d29fbbd6e17b2d)

Signed-off-by: Dave Lerner <dave.lerner@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-04-09 17:00:00 +01:00
Alexandru DAMIAN 1d631fba8a bitbake: toaster: do not load all available timezones
This patch makes sure that we only load pytz-recognized
timezones. Pytz is used to transform the timezone information
for the database queries, and needs to be able to deal with
the TIME_ZONE value that we set up.

[YOCTO #6093]

(Bitbake rev: bfe67472e3ee778b78ef004b2153fa88b3807b92)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-04-09 17:00:00 +01:00
Paul Eggleton 70288db2e9 bitbake: bitbake-selftest: add tests for local fetching
Add some explicit tests for unpacking local files to the appropriate
location. Some of these tests are actually testing for broken behaviour;
these have been called out in the comments, and associated bugs have
been filed.

(Bitbake rev: ca921c773c52392a5a338b2f493ad38c8132f708)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-04-09 14:31:14 +01:00
Paul Eggleton 68ec4c1d1a bitbake: bitbake-selftest: fix help message to include command line
(Bitbake rev: 3016af554d9cc67aec2429b96d8d085b88315e77)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-04-09 14:31:14 +01:00
Peter Kjellerstedt 8f683bf782 bitbake: bitbake: knotty: Clear footer before outputting to stderr
With the recent change to split the log output to stdout and stderr,
error messages that appeared while the footer was printed got all
messed up. This was because the messages to stderr was output _after_
the footer, then clearFooter() tried to remove the footer but removed
the error message and parts of the footer.

(Bitbake rev: 4fafea4fa69542b491e84463f6eae0d5bf645673)

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-04-08 18:05:26 +01:00
Paul Eggleton 327ed0bfce bitbake: fetch2: fix traceback when a wildcard matches a directory
If there is a directory matching a wildcard in SRC_URI when getting file
checksums, we should recurse into that instead of producing an error.

(Bitbake rev: ae87b7eb414e3d5eefd2effec7b30c22d2186b02)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-04-08 18:05:26 +01:00
Paul Eggleton 991af87183 bitbake: fetch2: handle wildcards correctly when recording file checksums
The Local fetcher's localpath is returning the parent directory for a
wildcard match; we need to handle this and add the wildcard
specification so that we checksum the correct files.

Fixes [YOCTO #6127].

(Bitbake rev: 4a90edd62c16cbf41b5b93280e155077564c774a)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-04-08 18:05:26 +01:00
Irina Patru bbc9aafbbf bitbake: hob: add "recipes/images/" to BBFILES when Hob is launched
The path for "recipes/images/" was not added in BBFILES when Hob had
to search for an image recipe. Therefore, it could not find it and an
error occurred.
This path needs to be added when Hob is launched.

[HOB #6086]

(Bitbake rev: 35c67281775b08925957c32663d587d486944e0e)

Signed-off-by: Irina Patru <irina.patru@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-04-08 13:07:25 +01:00
Paul Eggleton 1094be01e3 bitbake: fetch2: avoid cache ignoring missing files
Previously, if a file listed in SRC_URI was missing at the time of
parsing, and then was added and bitbake run again, it would not be
picked up because the recipe was cached without it. If we allow the file
to be added to the list of files to checksum, then it will be checked
for and found on the second run.

Fixes [YOCTO #4790].

(Bitbake rev: 71da822762cb298261cccdfa54b9c0fea02c3c5d)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-04-08 13:07:25 +01:00
Richard Purdie bb66113bde bitbake: fetch2: Fix bug in file checksum generation
For a while its been puzzling me why connman-gnome rebuilds as often as it
does. It turns out you can trigger this with a new checkout of the metadata.

The SRC_URI that is causing the problems is:

SRC_URI = "file://images/*"

and rather oddly the results in checksums for a file "." being added to
the tree, e.g.:

('.', 'ab48a68186f0e0f277c21ef4cb390b4b')

The problem is that when iterating files lists, the checksum variable can
become set yet we don't break the out from the for loop, which leads to
odd (and non-deterministic) entries being added into the file checksum list.
The exact item added probably depends on the order of items on the disk.

Before this change, bitbake-diffsigs on connman-gnome:do_fetch would report:

This task depends on the checksums of files: [
('connman-signal-03.png', 'f6c16aee57b37b73793a2f1dea433ffa'),
('connman-signal-02.png', 'ad0cd22710c097d8174121fc1023c3be'),
('connman-signal-01.png', '8842bd83d2fa9ba56480df34c727c629'),
('null_check_for_ipv4_config.patch', 'a23271e41c9fe81551244d875106af10'),
('connman-signal-05.png', '808589e7e8d502b44c7b007e9e68d48c'),
('connman-signal-04.png', 'ab48a68186f0e0f277c21ef4cb390b4b'),
('null_check_for_ipv4_config.patch', 'a23271e41c9fe81551244d875106af10'),
('0001-Removed-icon-from-connman-gnome-about-applet.patch', 'e2d8269357c1e8c84291606da24eea85'),
('0001-Removed-icon-from-connman-gnome-about-applet.patch', 'e2d8269357c1e8c84291606da24eea85'),
('.', 'ab48a68186f0e0f277c21ef4cb390b4b')]

Afterwards:

This task depends on the checksums of files: [
('connman-signal-03.png', 'f6c16aee57b37b73793a2f1dea433ffa'),
('connman-signal-02.png', 'ad0cd22710c097d8174121fc1023c3be'),
('connman-signal-01.png', '8842bd83d2fa9ba56480df34c727c629'),
('null_check_for_ipv4_config.patch', 'a23271e41c9fe81551244d875106af10'),
('connman-signal-05.png', '808589e7e8d502b44c7b007e9e68d48c'),
('connman-signal-04.png', 'ab48a68186f0e0f277c21ef4cb390b4b'),
('null_check_for_ipv4_config.patch', 'a23271e41c9fe81551244d875106af10'),
('0001-Removed-icon-from-connman-gnome-about-applet.patch', 'e2d8269357c1e8c84291606da24eea85'),
('0001-Removed-icon-from-connman-gnome-about-applet.patch', 'e2d8269357c1e8c84291606da24eea85')]

which is correct and deterministic without the "." entry.

(Bitbake rev: f9416e76e272ba3249abff099f6f3a47fe82e03e)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-04-06 11:31:26 +01:00
Farrell Wymore 396e2153a9 bitbake: toaster: added covered task list
if a task has a 'covered' indication, the list of tasks that
covered the task are computed and displayed. amended to add tooltip.

[YOCTO #5925]

(Bitbake rev: bb05ee13f53f10988579b6238802327732041d0c)

Signed-off-by: Farrell Wymore <farrell.wymore@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-04-05 14:55:29 +01:00
Farrell Wymore 1029726121 bitbake: toaster: correct package count
The package count was incorrect because it was counting anonymous
packages. the full path of the image files was shortened to just
the filename.

[YOCTO 6087]
[YOCTO 6091]

(Bitbake rev: 06b190b2c23799bd2c9749be28e11bf5d59ed4fc)

Signed-off-by: Farrell Wymore <farrell.wymore@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-04-05 14:55:29 +01:00
Belen Barros Pena 9db433246f bitbake: toaster: Fixes to the build dashboard
* Add "on" before the date in the build status

* Get rid of the extra space before the "and" in the
build status

* Make the errors and warnings links in the build
status show the message content

* Make sure that the "Images" section in the left navigation
appears only for successful builds

* Link the number of packages installed to the "Packages
included" tab in the image information page

* Remove unnecessary inline styles in the dashboard
template

* Make sure target names in the h1 are listed in
alphabetical order

(Bitbake rev: 644a38e9a90728af52ebda9846d2037dd6831b41)

Signed-off-by: Belen Barros Pena <belen.barros.pena@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-04-05 14:55:29 +01:00
Belen Barros Pena 37e2670951 bitbake: toaster: Make tables happy in Chrome
There is a known layout issue with Bootstrap tables in
Chrome. This change applies a css fix for it, and changes
the span classes in the variables table to improve
formatting.

(Bitbake rev: a7fee015b1cc34351d5ad8b854fc6bf368416400)

Signed-off-by: Belen Barros Pena <belen.barros.pena@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-04-05 14:55:29 +01:00
Belen Barros Pena 7322dddeed bitbake: toaster: Fix the empty packages alert
* Remove a tag in all capitals
* Remove an unnecessary div
* Fix a typo

(Bitbake rev: 1ca139ca157d62cee9c48b22063e77614fa34325)

Signed-off-by: Belen Barros Pena <belen.barros.pena@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-04-05 14:55:29 +01:00
Belen Barros Pena 9e81244b8a bitbake: toaster: Fix "Recent builds" links
* Link the number of errors to the #errors section in the
build dashboard

* Link the number of warnigns to the #warnings section in
the build dashboard

* Link the build time to the build time page

* Move some javascript that applies only to builddashboard.html
to that template from main.js

(Bitbake rev: 94c61d0d9ff24e0c781ea0a8e4bfea4c4ac94d4e)

Signed-off-by: Belen Barros Pena <belen.barros.pena@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-04-05 14:55:29 +01:00
David Reyna 4707c7c3ee bitbake: toaster: catch IndexError error in dictionary lookups
In the get_dict_value filter, there is the edge case where halted
builds can lead to IndexError errors in dictionary lookups, so we
need to catch those.

[YOCTO #6067]

(Bitbake rev: f6fcce974ce3b145bc472cd4e9721d56191828a4)

Signed-off-by: David Reyna <David.Reyna@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-04-05 14:55:28 +01:00
Belen Barros Pena 52ed413c3c bitbake: toaster: Changes to help text
All help text strings have been reviewed by the
technical writer, and some of them by Paul Eggleton.
This patch implements their suggested changes.

(Bitbake rev: ce89530b178be2f3202d45523ef1340e00df05be)

Signed-off-by: Belen Barros Pena <belen.barros.pena@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-04-05 14:55:28 +01:00
Belen Barros Pena 6831b7ad16 bitbake: toaster: Remove html validation errors in tasks.html
Remove the duplicated class attribute and the name
attribute.

Also, make sure that the row blue highlight works
without the name attribute when you land on the
tasks table after clicking the order link in a
task details page.

The commit also fixes a typo in default.css.

[YOCTO #6033]

(Bitbake rev: 471234f1ab1dbcd736a892720e99a305363db5ff)

Signed-off-by: Belen Barros Pena <belen.barros.pena@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-04-05 14:55:28 +01:00
Alexandru DAMIAN 53b781f41e bitbake: toasterui: save warnings from non-build context
Some warnings and errors may come from a non-build context,
and they were not saved, even if they were counted for the current
build.

This patch saves these messages in memory until we have the
entire build context available.

	[YOCTO #5642]

(Bitbake rev: fc7a74e7961775b5d7ff25298abed10138d24dc9)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-04-05 14:55:28 +01:00
Alexandru DAMIAN 1afb2a2620 bitbake: toasterui: avoid variable name conflict
We rename a local variable in as to prevent a conflict with
a similary named function parameter.

(Bitbake rev: f4e57f794651c4894600445e843ca9d5e104cd84)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-04-05 14:55:28 +01:00
Belen Barros Pena 86aefcdd4d bitbake: toaster: Update API
Make sure the REST API includes the latest changes
to the database schema.

(Bitbake rev: fb3d1d189f010488a8726872b01313857697751b)

Signed-off-by: Belen Barros Pena <belen.barros.pena@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-04-05 14:55:28 +01:00
David Reyna 0d39bee035 bitbake: toaster: link task order to right tasks page
When linking from a task order number to the All Tasks
page, automatically display the correct page for that link
anchor.

[YOCTO #5933]

(Bitbake rev: e5b6681432b627eb73bfa766d3154162206f374e)

Signed-off-by: David Reyna <David.Reyna@windriver.com>
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-04-05 14:55:28 +01:00
Marius Avram 56c776de3b bitbake: cooker: Overwrite IMAGE_BASENAME to default in custom image
This solves a problem of custom images which inherit a base
image with IMAGE_BASENAME overwritten in their recipe by a
different value than its default one: ${PN}.

The value of IMAGE_BASE causes a crash when hob will try to
create symbolic links to the resulting images from the deploy
directory, because it will look for names similar to
<original_recipe_name>-edited-timestamp-machine.rootfs.*
which might be different from the actual resulting image.

The solution is to simply overwrite IMAGE_BASENAME in the
custom recipe to the default value in the case IMAGE_BASENAME
is found in the base recipe.

Some recipes which were affected by this problem are those
from meta-fsl-demos (e.g.: fsl-image-test).

[YOCTO #6017]

(Bitbake rev: e42ee93519000f827be49659b6b5fb7717b3d592)

Signed-off-by: Marius Avram <marius.avram@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-04-04 15:08:26 +01:00
Marius Avram 07231022e2 bitbake: hob: fix customization of empty image recipe
This fixes a problem which happened when you tried to build an
image by selecting 'Start with an empty image recipe' from the
Image configuration page of hob.

The reason on of the bug was that the name of the resulting image
was threated the same way as ordinary custom images, when in fact
they should use the default name: "hob-image", because they do not
derive from any other recipe.

[YOCTO #6102]

(Bitbake rev: fa4ea3b4b40e7e9e6767e0cd51c6701e0af07135)

Signed-off-by: Marius Avram <marius.avram@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-04-04 15:07:01 +01:00
Cristiana Voicu f41241cecb bitbake: hob: adding INHERIT += " testimage " affects image recipes list
To find if a recipe is for an image, Hob checks if it inherits
image.bbclass. But when you add testimage in local.conf, this will
be added for each recipe, and it pass the test. Adding a "/" before
"image.bbclass", will check only for image.bbclass.

[YOCTO #6117]
(Bitbake rev: be8511c9d474c570f6ca7078e28919c8a5175a42)

Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-04-04 15:07:01 +01:00