Commit Graph

2871 Commits

Author SHA1 Message Date
Michael Wood e6d967dba2 bitbake: toaster: buildinfohelper Create a copy of the built layer and recipe
Create a copy of the built layer and the recipes associated with it.
This is so that the user can view the historical information about a
build. i.e. a snapshot of the layer version and artifacts produced at
that build.

(Bitbake rev: 0683d9a2b15e3234a94437aaebac84bfcca1420b)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-29 14:11:38 +01:00
Michael Wood 922503f4c1 bitbake: toaster: Create a relationship between build information and toaster layers
Previously this layer relationship was done by trying to match path
information that came back to the buildinfohelper with trying to query for
data in toaster's layers table. This rarely matched due to the lose coupling.

(Bitbake rev: 838e77c7c3c4006abd1327343a004590ab652de9)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-29 14:11:38 +01:00
Markus Lehtonen c6eaef01c5 bitbake: tinfoil: remove logging handler at shutdown
Otherwise the logger gets multiple handers (and the user get duplicate
logging output) if another tinfoil instance is initialized after one is
shut down().

(Bitbake rev: 74d67be7a4b591fab2278f7c184f282d11620c62)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-23 22:44:56 +01:00
Elliot Smith 15b482b16b bitbake: toaster: Add fake entry to Target_File for filesystem root
The files-in-image.txt file is produced by bitbake after an
image is created, listing all the files in the image.
However, this list doesn't include the root directory ('/').

buildinfohelper.py then tries to construct the filesystem
tree from this file, assuming that every directory apart from
the root directory (which is special-cased) can be assigned
a parent. But because the root directory isn't listed in
files-in-image.txt, an object for the root directory is never
created.

The direct subdirectories of the root ('./bin', './usr' etc.)
then can't be assigned a parent directory, as the object
representing the root directory doesn't exist. This
results in a Target_File lookup error and causes the
directory listing page to fail.

Fix this by creating a fake entry for the root directory
in the Target_File table, so that the direct subdirectories
of / can be assigned a parent. Note that it doesn't matter
that the root is faked, as its properties are never shown
in the directory structure tree.

[YOCTO #8280]

(Bitbake rev: a4015768183e5a3fa39a6c2b4dea0088ca182d80)

Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-23 22:44:55 +01:00
Ed Bartosh 08000eb27e bitbake: toaster: don't re-create Target objects
Due to re-creating Target objects from bitbake events task information
stored in original objects is lost.

There is no valid reason to remove existing objects. It's safer to query
them instead of re-creating as original object contain more information
than events coming from bitbake.

(Bitbake rev: aab4aff75eefb31aa53885d7735feee5daa294aa)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-23 22:44:53 +01:00
Romain Perier 7b86c771c8 bitbake: bitbake: bb.fetch2.git: Import errno module
Currently this module is dereferencing errno.ENOENT but the python module "errno"
is not imported, which causes a crash when fetching from a git repository.

(Bitbake rev: 93e4c9bb2393b1074f5a01e7eaaac742a59d8086)

Signed-off-by: Romain Perier <romain.perier@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-19 22:38:44 +01:00
Richard Purdie 5b4e380b9b bitbake: data_smart: Ensure OVERRIDES dependencies account for contains()
The dependencies of OVERRIDES were not including DEFAULTTUNE in OE-Core.
This is pulled in by a bb.utils.contains() reference which the override
dependency tracking code wasn't accounting for.

This patch ensures we do track contains references too.

(Bitbake rev: f3ee534cb0560dbb5f88a0ffe01e9305bae102e1)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-18 15:14:04 +01:00
Richard Purdie 60d019dde0 bitbake: data_smart: Correctly handle OVERRIDE values set using ??=
Right now, OVERRIDES dependency variables set using ??=, e.g. TARGET_ARCH
in OE-Core don't have their dependencies tracked. This is a bug, fix it.

(Bitbake rev: 944734503768f9e9223ef041f2d7873455418a54)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-18 15:14:03 +01:00
Richard Purdie 701ad76270 bitbake: data_smart: When considering OVERRIDE dependencies, do so recursively
Sadly its not enough to consider the dependencies of OVERRIDES, we
need to resolve their dependencies and so on recursively. If we don't
do this, some variable can be changed and the resulting data store is
incorrect.

(Bitbake rev: 82143ac064d391300e762ba7520ef1f8df18b574)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-18 15:14:03 +01:00
Richard Purdie 4325f6f03d bitbake: data_smart: Expand overrides cache recursively
If the values that make up OVERRIDES are themselves overridden,
we end up into some horrible circular logic. Unfortunately some
metadata does depend on this functionality.

e.g:
DEFAULTTUNE_virtclass-multilib-xxx = Y
which changes TUNE_ARCH
which changes TARGET_ARCH
which changes OVERRIDES

As a solution, we iterate override expansion until the values don't
change. If we iterate more than 5 times we abort and tell the user to
report the issue.

(Bitbake rev: 10279697c701e01bf6fdd5e9f92792ef5134807b)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-18 15:14:02 +01:00
Christopher Larson 3a5e46b6ed bitbake: bb.fetch2.{git, hg}: remove tarball if it needs updating
We were maintaining state in the form of ud.repochanged to determine whether
we need to write the tarball in the case where the tarball already exists, but
this state is maintained only in memory. If we need to update the git repo,
set ud.repochanged to True, and then are interrupted or killed, the tarball
will then be out of date.

Rather than maintaining this state, simply remove the out of date tarball when
we update the git repo, and it will be re-created with updated content in
build_mirror_data.

[YOCTO #6366]

(Bitbake rev: eaaa81393f181432c8586b17ade623f42c9fed2e)

Signed-off-by: Christopher Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-18 09:05:36 +01:00
Mark Hatle 2bb600a579 bitbake: tests/fetch.py: Fix recursion failure in url mapping
Instead of reproducessing the same line over and over and over, we remove the
current line from the mirror list.  This permits us to re-evaluate the list
while excluding all matches that have previousily occured.

Without this fix, adding this test results in a failure:

RuntimeError: maximum recursion depth exceeded in cmp

(Bitbake rev: 24a8e9a5b0ba145ae589178d74365c986ebca325)

Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-18 09:05:34 +01:00
Mark Hatle 4c3d4eceb1 bitbake: fetch2/__init__.py: uri_replace regex handling
We should only substitute one time.  If we do it without a max count, we can
end up matching over and over.

Before this change:

https://downloads.yoctoproject.org/releases/bitbake/bitbake-1.0.tar.gz

with a mirror of

https://.*/[^/]*    http://AAAA/A/A/A/

would end up either recursing indefinitely or result in:

http://AAAA/A/A/A/A/A/A/A/A/A/bitbake-1.0.tar.gz

(Bitbake rev: 4d254e02e2867dd9a6663508c8ca9f2733af71a8)

Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-18 09:05:34 +01:00
Elliot Smith 8c63d6049d bitbake: toaster: Read correct cooker log path from toasterui
The BB_CONSOLELOG variable changes by the time we read it in
BuildInfoHelper. This means that the log file location we
are using is incorrect, so the links to the cooker logs don't
work.

Instead, read it at the point when the BuildStarted event occurs
in toasterui. The BB_CONSOLELOG variable has the correct value
here, so pass that to BuildInfoHelper.

[YOCTO #8209]

(Bitbake rev: 20609eebee0d2318806cf81913e7ce6dc1005507)

Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-18 09:05:33 +01:00
Richard Purdie 3165af39bf bitbake: data_smart: Separate out update_overridevars into separate function
No code changes, just refactoring to allow for functionality
changes by moving things to a separate function.

(Bitbake rev: 2eb934814179ccf42e3d424dabe26b17d013a7ed)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-18 09:05:32 +01:00
Richard Purdie 07aef86c3a bitbake: cache: Handle spaces and colons in directory names for file-checksums
If there is a space in a directory name containing a file in file-checksums
(e.g. from a file:// url), you currently get tracebacks from bitbake. This
improves the code to handle colons and spaces in the file-checksums names
since it possible to figure out the correct names.

[YOCTO #8267]

(Bitbake rev: 87282b283921a58426f24fb21151db457c5bca66)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-18 09:05:31 +01:00
Richard Purdie 289007e5eb bitbake: utils: Add signal_on_parent_exit() function
Add a new bb.utils.signal_on_parent_exit() function so that a process
can register to recieve a signal when the parent dies. There is no
POSIX standard for this and the implementation is Linux specific.
Alternatives would be having an open pipe or polling os.getppid()
for changes but this seems more effective and less invasive to most
of bitbake's code structure.

We need to be able to determine when parents die to ensure child
processes stop running in a variety of circumstances to avoid
locks being held and ensure clean shutdown.

Roughly based on https://gist.github.com/evansd/2346614

(Bitbake rev: 34974f5e30e9b09c016481e4c81c156a5f379784)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-09 14:27:56 +01:00
Richard Purdie 384533bdb4 bitbake: server/process: Handle SIGTERM more gracefully
Currently if you send a SIGTERM to the bitbake UI process, the system basically
hangs if tasks are executing. This is because the server process doesn't
actually try any kind of shutdown before exiting.

This patch trys executing a stateForceShutdown command first, which is
enough to stop any active tasks before the system exits.

I also noticed that terminate can execute multiple times, once at SIGTERM
from the handler and once from the real exit. Double execution leads to
stack traces and potential hangs (writes to dead pipes), so ensure
the code only can run once.

With these fixes, bitbake much more correctly deals with SIGTERM to the
UI process.

(Bitbake rev: 1032ddddbe3241da02ebb3608a1c40f9123b9e80)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-09 14:27:55 +01:00
Christopher Larson a0246316c6 bitbake: bb.fetch: don't remove the clone when an update fails
When our clone exists, but is out of date, and the attempt to update it fails,
we don't necessarily want to remove the entire clone, particularly if it's
a large repository.

(Bitbake rev: 19af272ba5256653edeff6acbceeb09e3e478d61)

Signed-off-by: Christopher Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-04 16:24:00 +01:00
Christopher Larson ba50c2f4b9 bitbake: bb.fetch: handle checksums consistently for mirrors
If the main fetch method doesn't support checksums, the user will not be
defining them in the recipe, so we don't want to check them for
premirrors/mirrors either. This ensures that we never error due to missing
checksums on a git mirror tarball.

(Bitbake rev: 24c79bbed361b37f12d3351af13602e3d4386f4c)

Signed-off-by: Christopher Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-04 16:24:00 +01:00
Christopher Larson abba11daf9 bitbake: bb.fetch.git: remove leading '.' from gitsrcname
When using an absolute file URI, there's no host, and the path starts with
'/', the dir under ${DL_DIR}/git2/ ends up starting with '.', so is hidden.
Remove any leading '.' to fix this.

(Bitbake rev: 8dce6964d56b36a77fb113f2ad496cc992a5ff36)

Signed-off-by: Christopher Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-04 16:23:59 +01:00
Christopher Larson 7c4fdb8c17 bitbake: bb.cookerdata: include useful traceback for ExpansionError/ParseError
Show the user only the portion of the traceback which was from the metadata,
nothing from bitbake's internal calls.

(Bitbake rev: c45054aef03393fa0bf70e853ddcfc55988493cf)

Signed-off-by: Christopher Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-04 16:23:59 +01:00
Christopher Larson 5fe590cd6e bitbake: bb.data_smart: retain traceback from ExpansionError
This gives us the needed context of the original ExpansionError, which is
invaluable when we have a chain of function calls in the expansion.

(Bitbake rev: c514b6fbea77ede1b7871b89592a33ed39b1d71c)

Signed-off-by: Christopher Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-04 16:23:58 +01:00
Alex Franco 7ef2f951d6 bitbake: cooker/runqueue: Allow bitbake commands starting with do_
The output of "bitbake, -c listtasks pkg" lists tasks with their real names
(starting with "do_"), but then "bitbake -c do_task" fails, as "do_" always
gets unconditionally prepended to task names. This patch handles this error
by checking whether a task starts with "do_" prior to prepending it with it
when the task runlist is being constructed (and a few other corner cases).

[YOCTO #7818]

(Bitbake rev: dd3050ceef37ac556546e940aa596ce96ef6c8df)

Signed-off-by: Alex Franco <alejandro.franco@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-04 16:23:58 +01:00
Richard Purdie 4b9de767f0 bitbake: event/server: Add _uiready flag to handle missing error messages
If you start and suspend a bitbake execution so the bitbake lock is held,
then try and run "bitbake -w '' X", you will see bitbake return an error exit
code but print no message about what happened at all.

The reason is that the -w option creates a "UI" which swallows the messages. The
code which handles this exit failure mode thinks a UI has printed the messages
and therefore doesn't do so.

This adds in an extra parameter to the UI registration code so that we
can figure out whether its a primary UI or not and base decisions on whether
to display information on that instead. This fixes the error shown above and
some bizarre failures on the Yocto Project Autobuilder.

[YOCTO #8239]

(Bitbake rev: d1d60a68c2de40c2984d5040d14251c1be121b0b)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-04 16:23:56 +01:00
Benjamin Esquivel 7c079463c9 bitbake: utils: Fix a potential error in movefile
bitbake utils' movefile is now prone to malform the destination
file with duplicated file name strings. Fixing it to force a file
name append iff the dest argument is a dir not a file name

(Bitbake rev: 38dd27f7191da002a16c561be3790ce487045b01)

Signed-off-by: Benjamin Esquivel <benjamin.esquivel@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-04 16:23:56 +01:00
Elliot Smith 214ba674d0 bitbake: toaster: Improve how default project is identified and fetched
Command line builds are associated with a "default project"
(as we currently require a build to have a project). This
acts as a container for builds initiated outside Toaster.
Currently, this project is marked as the default by its ID
being 0. However, this doesn't work with MySQL, as MySQL
won't allow 0 in a foreign key which references an
autoincrement field.

Instead, use an is_default field to track the default Project
for builds initiated outside Toaster.

Add a method to fetch this default project, rather than fetching
a project with a magic ID.

Add this default project in a migration, rather than as a side
effect of a get_or_create() style method.

Also ensure that builds always have a project explicitly assigned
to avoid any magic with a build's project foreign key defaulting to
0 (as it no longer does).

[YOCTO #7932]

(Bitbake rev: 71b709a1bbc26d89d61873763b467d21e625b274)

Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-03 16:52:48 +01:00
Benjamin Esquivel f321655ad8 bitbake: utils: Specify dest file name in movefile()
When moving a file via the os.rename function, it was missing the
 destination file name which caused an OSError

 [YOCTO#8180]

(Bitbake rev: b147ba0341d87e077bd2b09ef4355976ecd2d26b)

Signed-off-by: Benjamin Esquivel <benjamin.esquivel@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-03 12:33:08 +01:00
Christopher Larson 383450c78a bitbake: build: delete tasks thoroughly
We want addtask to be able to bring back a deleted task, but we don't want its
previous dependencies to come back with it, so rather than marking a task as
deleted and then skipping tasks marked as such, actually delete the task and
its dependency information in deltask.

While we're in that part of the code, also fix a couple 'not foo in bar'
instances to 'foo not in bar', which is preferred in python.

(Bitbake rev: 94b3f3d6bdfbfa47f7eb3c3de64940a145b2ddd1)

Signed-off-by: Christopher Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-03 12:33:08 +01:00
Paul Eggleton 294948c732 bitbake: lib/bb/cooker: add interrupted flag to BuildCompleted event
Allow any listeners for this event (such as buildhistory.bbclass in
OpenEmbedded) to find out if the build was interrupted rather than
completing normally. The value will be 0 if not interrupted, 1 if
interrupted waiting for remaining tasks to complete, or 2 if force
interrupted (stopping any running tasks immediately).

(Bitbake rev: df2b778efd2ecc48f6c5a3ed446f6459f2250035)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-29 14:03:49 +01:00
Paul Eggleton bb34181b42 bitbake: lib/bb/main: avoid importing all server/UI modules on every execution
We're importing the server and UI modules in order to check they are
valid, but it turns out that that has some nasty side-effects. We don't
actually need to do this except when --help is passed or the module
doesn't exist, so rearrange the code so that we only do the module
listing in those two cases.

Additionally, let's just go ahead and catch all errors on import; we
really don't care to have them cause a failure now.

(Bitbake rev: c9dc6d9c86e8b887821a6d00346bd0b09e1da97c)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-24 23:47:07 +01:00
Randy Witt a3d228a484 bitbake: main: Handle RuntimeError exception in list_extension_modules
This exception was triggered in toaster from recent changes and is completely
breaking the whole of bitbake. Add the exception to the list so at least only
toaster is affected.

(Bitbake rev: f64def7cb6069dc1134fcd546bb59e4030c7376f)

Signed-off-by: Randy Witt <randy.e.witt@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-20 22:04:43 +01:00
Alexandru DAMIAN 25d4a35b03 bitbake: toaster logger: fix pylint issues
This patch fixes pylint issues in the toaster build data logger,
toasterui. The following types of warnings are touched here:

* fixing imports
* unused variables are set to _
* logger calls now use lazy evaluation instead of formatting
the string
* correct whitespace identation
* removes unneeded "pass" statements, and extra parantheses
* disable specific pylint warnings when decideing to override
them

(Bitbake rev: 947d47f15048baa967f88e03d80014e88ce152aa)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-19 18:05:55 +01:00
Alexandru Damian c9455a7484 bitbake: toaster: fix pylint errors
Prompted by issues discovered during running pylint on
the toaster sources, this patch fixes:

* missing import in toaster ui
* improper call of function in toaster ui (logger.debug)
* improper function definitions in bbcontroller
* invalid references to objects in bldcontrol.models
* proper initialization of object fields in ToasterTables

Also inhibiting specific pylint errors in files where
the problems are mis-identified.

(Bitbake rev: 1c71955c416fb68455f7f70669aba4202c411807)

Signed-off-by: Alexandru Damian <alexandru.damian@intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-19 18:05:55 +01:00
Paul Eggleton a3967e2ba5 bitbake: bitbake-layers: fix mapping files to layers
bitbake-layers needs to map recipe and class files to the layer they
came from within the show-recipes and show-overlayed commands. However,
it turns out that mapping a file to the layer it came from is not as
trivial as it might seem. To do it properly we need to match the path to
an entry in BBFILES then map that to the collection name using
BBFILE_PATTERN, then map that to the actual layer using variable
history. If it doesn't match any entry in BBFILES, then we can fall back
to BBFILE_PATTERN (to handle classes and conf files).

This fixes the layer name not showing up properly in the output of the
show-recipes and show-overlayed commands for recipes in layers such as
meta-intel that have subdirectories in BBFILE_PATTERN. It also fixes the
priority not showing up in show-layers for such layers.

As part of this I've added a function to VariableHistory which for a
space-separated list variable gives you a dict mapping the items added
to the files in which they were added. I've also fixed
bb.utils.get_file_layer() and reduced some of the duplication by using
this function in bitbake-layers. Also fixes the priority not showing up
for layers such as meta-intel

Fixes [YOCTO #8160].

(Bitbake rev: e852f6cabd7489585477ab567a1afeb2252377ac)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-19 18:05:49 +01:00
Paul Eggleton 5807ab3266 bitbake: lib/bb/main: consolidate UI/server extension listing and loading
Provide us with a means of showing the list of UIs / server choices for
the command line help, and do the processing in one place for both.

(Bitbake rev: 24035c1daad5a904c3372d21d44191ee8182338f)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-19 18:05:48 +01:00
Paul Eggleton 9b05ef581c bitbake: lib/bb/parse: properly handle OSError when updating mtime cache
If a file no longer exists, drop it from the cache silently instead of
generating a traceback. This was visible in some cases when a recipe was
deleted when bitbake was resident in memory.

(Bitbake rev: fe105b9042bdac4afd9f38fcf92bfdc2c04ec23f)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-19 18:05:48 +01:00
Paul Eggleton 3c3ff92882 bitbake: cooker: drop appendlist
Now we have the bbappends list and all users have been converted over to
use it, we don't need this anymore.

(Bitbake rev: 279770c42d4c63aa2cebce331b55a92a564b50ac)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-19 18:05:46 +01:00
Paul Eggleton b84bf58c19 bitbake: cooker: drop appliedappendlist
Whilst collecting this list on the fly may be quicker, doing so within a
function that's meant to *query* the list of bbappends is poor practice.

(Bitbake rev: 5c12aa3b0010d7d1733e54a0ca7d0af465454210)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-19 18:05:46 +01:00
Paul Eggleton e2ddc18874 bitbake: Fix -m handling if cannot connect to server
If we can't connect to the server we should error out, because it might
not be that the server is actually dead - it might just be unable to
execute commands.

(Bitbake rev: d4b921676859d6ba4e1922fa4682ee941652f483)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-19 18:05:46 +01:00
Paul Eggleton 7ce27b9fce bitbake: cooker: ensure prefile/postfile can work in memory resident mode
The prefile/postfile options weren't working in memory resident mode
because they weren't being passed through to the server, so ensure that
they do get passed through and that the server is reset when the values
come through.

(Bitbake rev: a3f7dc042fc7b1c308bfd248431930eb8ba50326)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-19 18:05:45 +01:00
Richard Purdie 37a2c062a5 bitbake: command: ensure sync commands that read configuration see updates
Add a means of ensuring that synchronous commands that read the results
of the configuration trigger a reparse of the configuration if any
underlying files have changed.

(Bitbake rev: aaf3cc024315450c1674819edf2a4e5cdf293f35)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-19 18:05:45 +01:00
Paul Eggleton 6e5ac6ba8e bitbake: cooker: further limit inotify watches
Unfortunately we were acting on inotify notifications about any files
changing within the watched directories, not just the ones we actually
care about. In OE the build directory is in BBPATH and hence it gets
watched, and we write things like bitbake.lock and
bitbake-cookerdaemon.log to that directory, hence effectively
notifications were being tripped on every bitbake invocation. To avoid
this, record which file/subdirectory we're interested in against each
watched directory so we can ignore any events for files/subdirectories
we don't care about.

Additionally, if we move up to the parent dir, ensure we haven't already
seen it before adding a watch on it (we were previously calling
watcher.add_watch() on the same directory multiple times before in a
typical OE configuration).

(Bitbake rev: bc39b8da34c046b629c43fd0a8cac2efbf1c060f)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-19 18:05:44 +01:00
Paul Eggleton e0daf78408 bitbake: command: intercept SystemExit to avoid trashing the server
If sys.exit() is called within a command run over XMLRPC, the XMLRPC
server is effectively trashed (apparently listening but no longer able
to respond to commands). We need to intercept the SystemExit exception
and deal with it as we would any other exception.

(Bitbake rev: 95e391acbc3b4efd6c77637a1ce815012ae0f09b)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-19 18:05:44 +01:00
Paul Eggleton 715d857174 bitbake: Fix default function parameter assignment to a list
With python you should not assign a list as the default value of a
function parameter - because a list is mutable, the result will be that
the first time a value is passed it will actually modify the default.
Reference:

http://docs.python-guide.org/en/latest/writing/gotchas/#mutable-default-arguments

(Bitbake rev: 7859f7388f2e3f675d0e1527cfde18625f36f637)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-19 18:05:44 +01:00
Mariano Lopez 0b80ef5f8a bitbake: runqueue.py: Add provides to taskdepdata
Currently bitbake is the only one that knows the relation
between PN and PROVIDES. In some cases it is needed to have
this relation in the data store (the bootloader it's a good
case).

This adds the PROVIDES to the taskdata, so it would be in
the data store as one field of BB_TASKDEPDATA.

(Bitbake rev: a660787311d2961c66c0443bf0e2e094c9baef1b)

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-10 13:57:59 -07:00
Richard Purdie a8b723498c bitbake: cooker: Resolve file monitoring race issues when using memres bitbake
If you load memory resident bitbake, then change bblayers.conf or other
configuration files, then execute your first command, bitbake will not
notice the changes. This is because it adds the file watches during
inital parsing, which happens at the time the first command is run.

To fix this, we move the addition of the file watches to earlier in
the process, so bitbake then does track them correctly. This also
avoids some issues Paul was seeing with tinfoil2.

(Bitbake rev: b9375c73e736003e66575969c9ea244403e47aeb)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-03 07:36:25 +01:00
Christopher Larson 22078d5e53 bitbake: bb.parse: properly error out on filesystem errors
We've had a long-standing bug where a legitimate error reading a file (IOError
or OSError) is always suppressed as though it was a 'file not found' case. As
a concrete example, if you do a `chmod 000 conf/local.conf`, it'll silently
not parse local.conf, rather than erroring to let the user know about the
problem.

Fix this by handling the ENOENT case specifically.

(Bitbake rev: e691312a3add222b04e7b2f52f8df6abcb9068bf)

Signed-off-by: Christopher Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-01 22:24:20 +01:00
Christopher Larson 75fc6bf0a0 bitbake: bb.cookerdata: don't show traceback for ParseError/ExpansionError
Tracebacks are of extremely limited usefulness in this context. The exceptions
carry the necessary context already, and the user doesn't care about the calls
in bitbake internals that led to an expansion or parse failure.

(Bitbake rev: 9b95fa94eaae452ac7814f1e67c2f7a6314c52f1)

Signed-off-by: Christopher Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-01 22:24:19 +01:00
Ed Bartosh a403ddbbba bitbake: bitbake: toaster: Wait for toaster gui to come
Set logging level for the Toaster logger to get messages
to toaster_ui.log

Reverted previous workaround.

[YOCTO: #7965]

(Bitbake rev: f1d8e0ff7233f17cff3a56db10d2443b392b407d)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-01 11:29:35 +01:00