Commit Graph

783 Commits

Author SHA1 Message Date
Andreas Oberritter 2f0c23d28d manual: document scmdata=keep for CVS, SVN and Git
(Bitbake rev: f42e6d33b986183eee2c74aa7b1bbc9b339276a9)

Signed-off-by: Andreas Oberritter <obi@opendreambox.org>
Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:51 +00:00
Andreas Oberritter bd34e7d6df fetchers: Add parameter scmdata=keep to include .git/ and others in generated tarballs.
* Allows generating version information from SCMs during build.
* Note that tar doesn't need to use --exclude '.git', because
  git checkout-index doesn't clone the repository.

(Bitbake rev: 05cbc1d1a01c667c77688f36fbc5b61c5f452a3a)

Signed-off-by: Andreas Oberritter <obi@opendreambox.org>
Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:51 +00:00
Chris Larson 7bbde5b149 build: fix FuncFailed for the no logfile case
(Bitbake rev: 3a4b5eb20e41bd3cecd9d7e8dff7d076d8a7bb9a)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:50 +00:00
Chris Larson ddf2c687d8 process: handle OSErrors other than file not found
(Bitbake rev: 7d80a5355cb540aae8d3082c1efebb72da4f93c6)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:50 +00:00
Chris Larson 3d51fd2b7d build: ensure LogTee has a valid name property
(Bitbake rev: 0ebb46e25261cfc85aaef2790cba7c1ec057c306)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:50 +00:00
Chris Larson ea91b1dd87 Fix PWD issue with new exec_func_shell
The previous attempt was incorrect.  The issue isn't that subprocess fails to
set PWD, it's that PWD is in the metadata, inherited from the environment, and
is re-exported, overwriting the actual accurate one in the shell environment
with the old one from the metadata.  So, ensure that PWD in the metadata is
not exported.

We can ditch this when the environment handling is reworked (e.g. poky's
commit to do so).

(Bitbake rev: 2c8683234acf514706b2b69f5b29405485e664dd)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:50 +00:00
Chris Larson 9260c08e1d Ensure that the invalid UI error goes to stderr
(Bitbake rev: 76aeea2c23a518046d0b26f1a6fc43287a180a9d)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:50 +00:00
Chris Larson e05d6fa927 build: set PWD in the subprocess preexec hook
(Bitbake rev: efa5485c8f9d488ba058c40734cc55296d3de6eb)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:50 +00:00
Richard Purdie 2b7c2a8420 bitbake: Transfer noexec runqueue messages to the UI
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:49 +00:00
Chris Larson fcba92f354 Move the runq task start/failed messages to the UI
(Bitbake rev: ab831e867f09b47001cb8da2f8f060e04febf237)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:49 +00:00
Chris Larson 23735bb765 Silence python 2.7 nested context manager warning
(Bitbake rev: 1c38f19fb8da2e6c55394162d0142e7146387e54)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:49 +00:00
Chris Larson 098f63d672 Rename command events, adjust compareRevisions
- Moved the logic for comparing revisions from cooker into command
- Removed 'Cooker' from the event names
- Renamed the 'ExitCode' event into CommandExit, and changed CommandFailed to
  be a subclass of CommandExit

(Bitbake rev: c51ed5d7a9971fad6019dac6c35a71b8a54ab16a)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:49 +00:00
Chris Larson 5a92e67b86 event: use cPickle for events
(Bitbake rev: b16c0c1dc3b7e03b02555e9803767a0d695e67f1)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:49 +00:00
Chris Larson d2123cc395 utils: fix 'lock' variable reference
(Bitbake rev: f57f8f3cc980e3ae1693c2e065227e951ed0b1c0)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:49 +00:00
Chris Larson 212414b13f bb.event: fix MsgBase ref in fire_class_handlers
(Bitbake rev: db7f960e5f103a424a4319f1867fb540e643c1ec)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:48 +00:00
Chris Larson 3d3af810fa Kill the uncaught exception handler
We now wrap the main() in a try/except, ensuring that both the main portion of
bin/bitbake and the UI raising an exception will be shown to the user.  For
the server and workers, we can ensure in the server itself that exceptions are
handled correctly.

(Bitbake rev: 240d4a7ae80a6636c302ae84266ddfed7a7fcedd)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:48 +00:00
Chris Larson 6f80455777 build: fix -D with shell functions
(Bitbake rev: 1c8be64732fdf4f3a608c090b3dc92065d6058d6)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:48 +00:00
Chris Larson aaa55880ac build: send logging messages to the log file for python functions
(Bitbake rev: ee1cce6ab21ddda60a7a070d03e98ff8485a5e71)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:48 +00:00
Chris Larson 25e519a80f utils: fix calls to close() in the lock/unlock functions
(Bitbake rev: 4262c26d36d1c1b6801ac422716a227c1f6b9589)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:48 +00:00
Chris Larson e2363f3cdd build: use a contextmanager for locks
Also don't bother passing logfile to exec_func_python, at least until we start
adding the logfile as a file handler to the bitbake logger.

(Bitbake rev: f99ee4680c9f67b7ed13fc06044ba2382f9a782c)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:48 +00:00
Chris Larson b4eff9fcef build: use bb.process instead of os.system
(Bitbake rev: 53740977521bc81ffa37adfa7bbeb8f2a80ea165)

build: write logfiles per task, not per function
Based on d14f9bf6 from poky, reworked for master and other cleanup.

(Bitbake rev: beadff2eca1eb95f0411115dd72ddb4c3c44c604)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:48 +00:00
Chris Larson 87b6cdf547 process: add subprocess-based bits
(Bitbake rev: c63e55564a8840083dbd8634b10fe6f76d1f1354)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:47 +00:00
Chris Larson c8d2dad049 Rename the ui 'init' method to 'main'
As these may run the UI in a blocking fashion and then return the exit code,
'init' was an inappropriate name, and 'main' is more appropriate.

(Bitbake rev: 4d081a0ed759bd526ab01849d650bd9e8d80ddd1)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:47 +00:00
Chris Larson 72c6953488 cooker: use re match, not search in re_match_strings
We want to match the requested pattern at the beginning of the string,
otherwise things behave in an unintuitive manner wrt ASSUME_PROVIDED (e.g.
ASSUME_PROVIDED += "gtk+" will also assume foo-gtk+ is provided), and the user
can always use '.*gtk+' to get the old behavior.

(Bitbake rev: 5670134ab2eb573d39df3c3231677cdb1a1dfc72)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:47 +00:00
Chris Larson 431160caf1 taskdata: use 'any' in re_match_strings
(Bitbake rev: e48e9a2150ee76aaf151f6d5bc9e86e6ae4de514)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:47 +00:00
Chris Larson 246bd18613 cooker: add shutdown/stop methods
(Bitbake rev: fc64eff03fe1f41e59128cb3df0ef2161e24a8cb)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:47 +00:00
Chris Larson 825e5045f4 cooker: merge cookerState and cookerAction
(Bitbake rev: c7c8945ef7ca9465312e630b7fa5f0a87ac8b6c7)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:47 +00:00
Chris Larson ac4d926f41 cooker: use a pool, abort on first parse error
(Bitbake rev: 9caf65e79f95fe0045e727391e974c4c1e7411ff)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:46 +00:00
Chris Larson e5624a4ed3 knotty: use enumerate for task waiting
(Bitbake rev: 89ce8df075ac8c9a5478c86405e6e6b60346a51c)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:46 +00:00
Chris Larson 7ffd6f88b8 knotty: shift non-interactive progress into a class
(Bitbake rev: c3d005cbbae3d56da9926666cfb1501c2bf96ea7)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:46 +00:00
Chris Larson 7ea3c96938 Fix the <100 recipe progress fix
(Bitbake rev: 424428a764651183218f9cc93bc05496867aa5de)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:46 +00:00
Chris Larson 1e3b83f96b Error more pleasantly when trying to use python2.4
(Bitbake rev: 57402ee727c8c1d8f5017534fb0f5e0b20aaaa5b)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:46 +00:00
Chris Larson 7481c11e34 Don't show a traceback for a python version error
(Bitbake rev: 44c325306e55336b7b416fd88b39874012a3f603)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:46 +00:00
Chris Larson c4e07bc033 cooker: don't choke on <100 recipes to parse
(Bitbake rev: 2be5169a01e57bcfcb1f75cbd6bac3bf0f230edd)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:45 +00:00
Chris Larson b8aedaa6b5 cooker: no cached in progressbar and add ETA
Rather than updating the progress bar based on the recipe being processed
(whether cached or parsed), consider only parsed recipes.  This reduces the
instability in progress rate introduced by the cached entries, and allows the
ETA to be resurrected and be a bit more useful.

(Bitbake rev: 618480f7739f6ae846f67a57bee5a78efb37839d)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:45 +00:00
Chris Larson b890c19a33 cache: change to more incremental format
(Bitbake rev: 4fe4ffbef3885887c97eebe021edc3f23feab9ea)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:45 +00:00
Chris Larson 5bff22988c cooker: pass back child exceptions to the server
(Bitbake rev: 0f68f8bcd0e0aa944f76f88a4a85c9bcc1e42bee)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:45 +00:00
Chris Larson 8faec0b672 vercmp: don't choke on empty first version components
(Bitbake rev: e75aa94e9477933c5a40021b2a8e844db54f29da)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:45 +00:00
Chris Larson 6810622484 Use __file__, not sys.argv[0]
(Bitbake rev: 97e92abe49663eee189c89c1dc91fe69891faf73)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:45 +00:00
Chris Larson 8a938d567f cache: ensure 'pn' is included in the pkgvars
(Bitbake rev: cefc28a2aa1e5703119dacfd885c8c159a1e47a3)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:44 +00:00
Chris Larson a92caf45c8 knotty: drop the ETA from the progressbar for now
Currently, the progress bar is an indication of the processing of our recipes,
which includes loading the cache file, then for each recipe, either adding the
existing cached information to the CacheData or parsing the recipe from disk.
These tasks clearly take different amounts of time, so the ETA is unreliable
today.  We'll resurrect this functionality after we revamp the progress
handling, fully incorporating the load of the cache file.

(Bitbake rev: 80867372dcbef91ebaf7d77a77ca871741dd3f74)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:44 +00:00
Chris Larson 7a7e2f4e59 cooker: ensure that the cache sync completes
Without explicitly joining the thread, it's possible for the process to end
(e.g. after a bitbake -p) and kill off the thread without waiting for it to
exit cleanly.  So, register the thread join with atexit.

(Bitbake rev: 97ce57e6f860d3e6f34cc7a603ed1eeac4f423d3)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:44 +00:00
Chris Larson f4a06aac98 cooker: get number of threads in constructor
(Bitbake rev: e7fd259d1528f48b0812fb7b0a54dd6d5d22f4a9)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:44 +00:00
Chris Larson a519031f23 cooker: don't add info for skipped recipes
(Bitbake rev: ac596d55de4ad86e1cdc84dabfea1350dd54efd0)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:44 +00:00
Chris Larson 60226ff35c cooker: show progress bar before initializing the cache
This ensures that the time spent loading the cache from disk occurs with the
progress bar up.  Though the progress bar stays at 0% during this period, I
think this is an improvement over the multi-second stall which occurred
previously before the progress bar came up.  Ideally, we'd integrate cache
loading from disk into the progress display, but this is a first step.

(Bitbake rev: f6d0a5c219f9deb84f702450d30d868ba6271f77)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:44 +00:00
Chris Larson 6cd15a1ea0 cooker: save progress chunk value (total/100)
(Bitbake rev: 09333737cbeeb9875d938521ddcd519fc808bcc3)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:43 +00:00
Chris Larson 739bb5a2d1 cooker: stop loading the cache for -b
Previously, the cache was actually being loaded from disk twice whenever using
-b or -e -b.  This also moves the bb_cache instance into the CookerParser, as
it's not needed by the cooker itself at all.

(Bitbake rev: dd0ec2f7b18e2a9ab06c499b775670516bd06ac8)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:43 +00:00
Bob Foerster 065da895d2 cooker: don't fire unnecessary parse progress events
(Bitbake rev: 7cf7d5f14405681496fced3640a50a20ef1acac1)

Signed-off-by: Bob Foerster <robert@erafx.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:43 +00:00
Chris Larson 7846d8db11 cache: don't add info to cache if cache is disabled
(Bitbake rev: f12bb303f3d86a68d0b3dda1112dd654b9251704)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:43 +00:00
Chris Larson 7c9444e9a5 cache: sync the cache file to disk in the background
This version uses a thread rather than a process, to avoid problems with
waitpid handling.  This gives slightly less overall build time reduction than
the separate process for it did (this reduces a -c compile coreutils-native by
about 3 seconds, while the process reduced it by 7 seconds), however this time
is quite insignificant relative to a typical build.

The biggest issue with non-backgrounded syncing is the perceived delay before
work begins, and this resolves that without breaking anything, or so it seems.

(Bitbake rev: 5ab6c5c7b007b8c77c751582141afc07c183d672)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:43 +00:00
Chris Larson 95d2f56126 Simplify cache syncing
Rather than adding nocache items to the cache, then copying the cache and
removing them to sync it, don't add them in the first place.  Also use 'with'
for the cachefile.

(Bitbake rev: 343b6f6255ad020c39e30742175a241f0859a5a6)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:43 +00:00
Chris Larson 9ffbd9fe27 Experimental usage of the 'progressbar' module
(Bitbake rev: 64feb03bc2accecb49033df65e0a939ef5ab5986)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:42 +00:00
Chris Larson 32ea766871 Implement parallel parsing support
This utilizes python's multiprocessing module.  The default number of threads
to be used is the same as the number of available processor cores, however,
you can manually set this with the BB_NUMBER_PARSE_THREADS variable.

(Bitbake rev: c7b3ec819549e51e438d293969e205883fee725f)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:42 +00:00
Chris Larson 570bec37a8 cache: don't write out the cache unnecessarily
If the only recipes's we reparsed this run were those flagged as not to be
cached, there's no point in re-saving the cache, as those items won't be
included anyway.

(Bitbake rev: 1e0c4dbcbec886a30b89f8b4bb365c3c927ef609)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:42 +00:00
Chris Larson acca344057 cache: create and use a RecipeInfo class
This class holds the particular pieces of information about a recipe which are
needed for runqueue to do its job.

By using it, I think we improve code clarity, reduce method sizes, reduce
overuse of primitive types, and prepare for parallel parsing.  In addition,
this ditches the leaky abstraction whereby bb.cache attempted to hide the
difference between cached data and a full recipe parse.  This was a remnant
from the way things used to be done, and the code using it had to know the
difference anyway.  If we choose to reimplement caching of the full recipes,
we can do it in bb.parse, in a completely transparent way.

(Bitbake rev: 992cc252452221f5f23575e50eb67528b2838fdb)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:42 +00:00
Bob Foerster c6328564de Prefer xrange over range for small performance gain.
range() allocates an actual list when called.  xrange() is just an iterator
and creates the next range item on demand.  This provides a slight
performance increase.

In python 3, range will do what xrange does currently, but the upgrade will
be handled by the 2to3 tool.

(Bitbake rev: 73b40f06444cb877a5960b2aa66abf7dacbd88f0)

Signed-off-by: Bob Foerster <robert@erafx.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:42 +00:00
Chris Larson e81fc749f3 Revert "svn fetcher: warn people to switch to SRCREV"
1) too spammy
2) can be implemented in the metadata instead

This reverts commit 8da9744fcdf856abebcfbe9e3bc1b8cf07bc317b.

Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:42 +00:00
Chris Larson 66b364cf8e cache: make loadDataFull a classmethod
(Bitbake rev: 16a9d689e61c35cfca94bbecd9772eab3c5072ba)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:41 +00:00
Bernhard Reutner-Fischer 81a81a163c svn fetcher: warn people to switch to SRCREV
as noted by rp in ac00ca89a4e43cd4f38ba86455079d31be78e644

(Bitbake rev: 8da9744fcdf856abebcfbe9e3bc1b8cf07bc317b)

Signed-off-by: Bernhard Reutner-Fischer <rep.dot.nop@gmail.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:41 +00:00
Bernhard Reutner-Fischer 4df0d6adca fetch: be more pythonic
no functional changes

(Bitbake rev: e88834fb7c6821cc29c12d296f2edd51f6eb3746)

Signed-off-by: Bernhard Reutner-Fischer <rep.dot.nop@gmail.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:41 +00:00
Bernhard Reutner-Fischer b7d667f252 fetch: use os.path.join
(Bitbake rev: c360b01df18d90a513a3d61d395f905102e7568e)

Signed-off-by: Bernhard Reutner-Fischer <rep.dot.nop@gmail.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:41 +00:00
Bernhard Reutner-Fischer 87b6f7d27a fetch: add common helper _strip_leading_slashes()
Several fetcher need a way to strip leading slashes off a local path.
This helper-function consolidates all such occurances.

(Bitbake rev: 823a02185ed109054c6c1ae366221aaed0353f24)

Signed-off-by: Bernhard Reutner-Fischer <rep.dot.nop@gmail.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:41 +00:00
Bernhard Reutner-Fischer d0d67a9fe2 fetch: rename suppports_srcrev to supports_srcrev
osc had it already spelled correctly?!

(Bitbake rev: b8bb4433de7a981c6826173e926ca34705c4ac70)

Signed-off-by: Bernhard Reutner-Fischer <rep.dot.nop@gmail.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:41 +00:00
Chris Larson 814c80f53c cache: Add debug msg for a nonexistant dep file
If a recipe depends on a file, and that file is out of date, we show a
message, but if that file was removed, we do not, until now.

(Bitbake rev: 67984ba0ac2db79874541bc031f2e3e9ff7a6c32)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:41 +00:00
Chris Larson 84f02831a7 cache: use new style classes
(Bitbake rev: e084430446be2544dd1a6b627088f888c37cc7f0)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:40 +00:00
Chris Larson c4d939079e cache: pyflakes/pep8/pylint cleanup
(Bitbake rev: 06420ff839ffc37de3e42474b8b0e47c4608a985)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:40 +00:00
Chris Larson 77d52cb1b8 cache: use set for clean, checked
(Bitbake rev: 3a776ef947e23b1a04bdd9e78a3a0c4616249b0e)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:40 +00:00
Chris Larson f207bb7c93 cache: make use of defaultdict
(Bitbake rev: 7171019b11ad656d0edb979564941fcf92ab0a02)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:40 +00:00
Chris Larson 9c7c18d06f Pre-explode rundeps/runrecs in CacheData
(Bitbake rev: 0c8e0f3191252ccedb83c431c48e5c7d905e2717)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:40 +00:00
Chris Larson 3a2176fe73 persist_data: handle locked db for SELECT
Parallel processes interacting with the persist_data db can quite easily
explode without this.

(Bitbake rev: b3d5432cff0ff28f4c8a5bcf10efa3e383b4fd4d)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:39 +00:00
Bernhard Reutner-Fischer f565258368 BBHandler: use os.path in inherit()
(Bitbake rev: 5b85de2c71973ba490b95a5d9ab634635f395142)

Signed-off-by: Bernhard Reutner-Fischer <rep.dot.nop@gmail.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:39 +00:00
Bernhard Reutner-Fischer b44100eafc BBHandler: simplify supports()
(Bitbake rev: 858d704d713d15bf97053eb1374758c74b0d8874)

Signed-off-by: Bernhard Reutner-Fischer <rep.dot.nop@gmail.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:39 +00:00
Bernhard Reutner-Fischer 1d8a9b8856 bb.cache: only log if the respective action was taken
This avoids alot of misleading log-messages like "Removing FOO from cache"
if FOO was not in the cache and as such is not a removal candidate.

(Bitbake rev: de34a403e206867e09410ad4925c7b9cff04fee6)

Signed-off-by: Bernhard Reutner-Fischer <rep.dot.nop@gmail.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:39 +00:00
Chris Larson 7e1b709231 bb.pysh: add Case support to format_commands
(Bitbake rev: 6e24f573a0e95068eb9237c1d264ad1148b2f690)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:39 +00:00
Chris Larson 32b4bd9224 bb.pysh: fix writing pyshtables all over the place
(Bitbake rev: ff4753e362714a3c4c759c2fad8a9e5b8fe5bef5)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:39 +00:00
Chris Larson 4addbd191d Move the pysh package into the bb package
The pysh we're using is modified, and we don't want to risk it conflicting
with one from elsewhere.

(Bitbake rev: 1cbf8a9403b4b60d59bfd90a51c3e4246ab834d6)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:39 +00:00
Peter Chubb 489d17596d Allow mercurial fetcher to follow tip
There are occasions when developing when I want a package always to
grab the latest copy of a package.  Witht eh CVS fetcher you can do
this by setting the `date' tag to `now'.  This patch adds similar
functionality to the mercurial fetcher: if the revision to fetch is
`tip' then always grab from the server, and don't use the cached
tarball.

Oh, and I fixed a typo in the Class comment.

(Bitbake rev: 01b85608d8a37f8af66dfd80133e950120679079)

Signed-off-by: Peter Chubb <peter.chubb@nicta.com.au>
Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:38 +00:00
Chris Larson 17ea152990 Fix GraphViz .dot output for rdepends and rrecs
Use bb.utils.explode_deps to break up the rdepends and rrecommends strings.
This fixes the same issue which was fixed by a number of patches floating
around, but uses explode_deps rather than regular expressions.

(Bitbake rev: 83cdb23f8b89453a3527a276bd0b4deb85d63deb)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:38 +00:00
Chris Larson 14889f9cca Make 'cache is clean' message debug
(Bitbake rev: 92637a355d55cb66de91b4314bc0e7cf1ac64ade)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:38 +00:00
Eric BENARD 3f7000cd3a bitbake: lib/bb/fetch/hg: fix fetching from a mercurial repository
* without this fix, we get :
updating working directory
74 files updated, 0 files merged, 0 files removed, 0 files unresolved
abort: There is no Mercurial repository here (.hg not found)!

(Bitbake rev: 75ea005ac8fc05b2b3afca803d77a6b5f558efee)

Signed-off-by: Eric Bénard <eric@eukrea.com>
Tested-by: Frans Meulenbroeks <fransmeulenbroeks@gmail.com>
Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:38 +00:00
Chris Larson 493f0cff71 Resurrect display of failed files
This was inadvertantly removed when trying to reduce the amount of duplicated
information the user sees when a failure occurs.

(Bitbake rev: 850d6158ea9daa58e896fd6b258d586df797dcf4)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:38 +00:00
Chris Larson 283b0a20c2 Access metadata vars as locals in python snippets
Example:
FOO = "bar"
BAR = "${@FOO + '/baz'}"

${BAR} == "bar/baz"

(Bitbake rev: 606fa1fd97cbd47a6a7ebdc7a2e6aa93a8f65cf5)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:38 +00:00
Chris Larson f3406dd288 Fix __getitem__ for DataSmart
Ensure it raises KeyError for a missing key, this is required to use this as a
mapping in various places, e.g. as locals in an eval.

(Bitbake rev: 8d661ce0c303e8d69f17c1d095545d5ed086d1d5)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:37 +00:00
Khem Raj 6491ed9e2e fetchers: Use tar --exclude pattern to remove SCM files
This option will exclude the SCM metadata from tar files.

Tested with gcc where svn tar which used to be 156M for gcc 4.5
is now 77M

(Bitbake rev: f264cb6d43472525ad787b0887764ea696ec52ba)

Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:37 +00:00
Chris Larson c81cd1be64 Fix syntax issue and don't include 'd' in ui_queue
(Bitbake rev: bed8e09971dc577f5443ad3d89aa14634c54eb16)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:37 +00:00
Chris Larson 26eda93337 Queue up events before the UI is spawned
- Queue up any events fired to the UI before the UI exists
- At exit, check if UIs exist, and if not, flush the queue of LogRecords to
  the console directly.
- When establishing a connection from the UI to the server, flush the queue of
  events to the queue in the server connection, so the UI will receive them
  when it begins its event loop.

(Bitbake rev: 73488aeb317ed306f2ecf99cc9d3708526a5933c)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:37 +00:00
Chris Larson 30cef6bade Fix another little bug with undefined var from cache
(Bitbake rev: f29ceb22b6a79ff62287a1eb398811fd81ec5c18)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:37 +00:00
Chris Larson a964761283 bb.cache: fix bug from last commit with loading from cache
(Bitbake rev: 99c324fe7395a44da78403c615797104413503a5)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:36 +00:00
Chris Larson 3f40a2efe8 Reduce bb.cache memory usage a bit
- Don't store key/value pairs when the value is None
- Delete the depends_cache when we're done with it

This reduces the memory usage after sync on initial parse by roughly 11.5% on
this machine.

(Bitbake rev: c7eb4c989459d182fdf9c81a627d32b7ef11626b)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:36 +00:00
Chris Larson 666513982f Fix FuncFailed construction
(Bitbake rev: 085e66f9c14123ea2c0f1e34f7737cf77071f86d)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:36 +00:00
Chris Larson 136a631b2e Restore FuncFailed argument compatibility
(Bitbake rev: c07cc08f7fd503ac3013ccc43c79198c4c3b7b29)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:36 +00:00
Chris Larson 311c9f5042 Only reference logfiles that exist
(Bitbake rev: 760f647ba044009150ee219869fc9dea171a7535)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:36 +00:00
Chris Larson c90bfa57f5 Fix bitbake -k issue introduced by build exception cleanup
A SystemExit from a python function wasn't being raised as a FuncFailed, which
resulted in it not being caught by the exception handlers in the runqueue for
the worker process, which resulted in a SystemExit exit, rather than os._exit,
which causes all manner of problems when used in a forked process.  This fixes
it by ensuring we raise a FuncFailed when seeing exceptions which aren't
instances of Exception.

(Bitbake rev: dafe92fe9f387450d9f9e9ff41c99388998b7495)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:36 +00:00
Chris Larson bfe4bec8ad Don't show tracebacks for SystemExit or KeyboardInterrupt
(Bitbake rev: d71984b3934c3dd9791c3bc00f332b79a1985a05)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:35 +00:00
Chris Larson 7afe34e2c2 Fix exit code display for task failure
Per the python documentation, os.waitpid returns the exitcode shifted up by 8
bits, and we weren't compensating, resulting in a display of 'failed with 256'
when a worker process exits with a code of 1.

(Bitbake rev: 90c2b6cb24dc9c82f0a9aa9d23f2d1ed2e6ff301)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:35 +00:00
Richard Purdie 7011ae3f78 FIXUP: runqueue bb.debug refs
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:35 +00:00
Chris Larson 9897d56861 Simplify build exception handling
- Drop EventException
- Use FuncFailed as the primary function failure exception, using TaskFailed
  for the event (leaving it up to the process running exec_{func,task} to
  display the more detailed information available in the exception).
- Switch InvalidTask to an exception rather than an event, as that's a
  critical issue.
- Reduce the number of messages shown to the user when a task fails -- they
  don't need to be told it fails 12 times.  Work remains in this area though.

(Bitbake rev: 06b742aae2b8013cbb269cc30554cff89e3a5667)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:35 +00:00
Chris Larson 1e7204a7b5 Turn Event into a new style class
(Bitbake rev: b42221cabeb1193ade134d1d3c0318203ab8eb93)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:35 +00:00
Chris Larson 39d61dc498 Work around issue with pickling of traceback objects
(Bitbake rev: 82928613256bad92fde9f4071244a53e20fc89ee)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:34 +00:00
Chris Larson 0533a8465b Don't show uncaught exception message for KeyboardInterrupt
(Bitbake rev: 29634acd262b06fd14f6ef1e134346f274cf448f)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:34 +00:00
Chris Larson a4b195f180 Apply fix for issue where the loggers were constructed with the wrong class
(Bitbake rev: 1b21daf052c49f3126dac001712ec01ad63c5f60)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:34 +00:00
Chris Larson 9c2bcae35e Fix providerlog NameError
(Bitbake rev: 992e460f24d4da707c76d6e6d74d3684c9646279)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:34 +00:00
Chris Larson fc27334d1a Drop 'fn' arguments to bb.msg functions
(Bitbake rev: 8341458e3d21b45db84e46bd32f8ad270000ce3c)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:34 +00:00
Chris Larson 55c0b36e36 Add pending deprecation warnings to the bb.msg functions
(Bitbake rev: 60293a42b5500b6139bcd912bf294f862ef9936b)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:34 +00:00
Chris Larson ecc68fa4fb Switch bitbake internals to use logging directly rather than bb.msg
We use a custom Logger subclass for our loggers

This logger provides:
- 'debug' method which accepts a debug level
- 'plain' method which bypasses log formatting
- 'verbose' method which is more detail than info, but less than debug

(Bitbake rev: 3b2c1fe5ca56daebb24073a9dd45723d3efd2a8d)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:33 +00:00
Chris Larson d3a45c7d41 Use logging in the knotty ui, and pass the log record across directly
This kills firing of Msg* events in favor of just passing along LogRecord
objects.  These objects hold more than just level and message, but can also
have exception information, so the UI can decide what to do with that.

As an aside, when using the 'none' server, this results in the log messages in
the server being displayed directly via the logging module and the UI's
handler, rather than going through the server's event queue.  As a result of
doing it this way, we have to override the event handlers of the base logger
when spawning a worker process, to ensure they log via events rather than
directly.

(Bitbake rev: c23c015cf8af1868faf293b19b80a5faf7e736a5)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:33 +00:00
Chris Larson 4855548ffb Use the python logging module under the hood for bb.msg
(Bitbake rev: 47ca82397bc395b598c6b68b24cdee9e0d8a76d8)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:33 +00:00
C Michael Sundius 3e57e63b2d Allow %20 in a file name in the SRC_URI
(Bitbake rev: f7c181a0f6ab0b4d33bf80a0e24a788de441f82b)

Signed-off-by: C Michael Sundius <msundius@sundius.com>
Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:33 +00:00
Chris Larson 5c50d43cfa Ignore python warnings that come from places we don't care about
Only shows warnings that come from bb, oe, or <string>

(Bitbake rev: 57018687f60b222ab220dd904c4bf870780171e9)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:33 +00:00
Chris Larson 2dd8c01513 Make DataSmart inherit the MutableMapping ABC
Provide __len__, __iter__, and the getitem/setitem/delitem methods, and its
mixed in versions of keys(), values(), items(), etc will automatically behave,
making the DataSmart act more like a real mapping.

(Bitbake rev: 89b5351c656d263b0ce513cee043bc046d20a01e)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:32 +00:00
Chris Larson ebe3850bee Split out 'find next buildable task' into a separate generator function
It needs to be a generator, so scheduler subclasses have the option to skip
buildable tasks and return a later one.

(Bitbake rev: a8c61e41bc6277222e4cde667ad0b24bd1597aa0)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:32 +00:00
Chris Larson 0d1034d2ea Let the RunQueueScheduler control the number of active tasks
(Bitbake rev: 1387423e747f59866fd1cb99a7d90605e668823f)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:32 +00:00
Chris Larson 384c5cc8ac Add the ability to use runqueue schedulers from the metadata
If you create a runqueue scheduler class in a python module, available in the
usual python search path, you can now make it available to bitbake via the
BB_SCHEDULERS variable, and the user can then select it as they select any
other scheduler.

Example usage:

    In a test.py I placed appropriately:

        import bb.runqueue

        class TestScheduler(bb.runqueue.RunQueueScheduler):
            name = "myscheduler"

    In local.conf, to make it available and select it:

        BB_SCHEDULERS = "test.TestScheduler"
        BB_SCHEDULER = "myscheduler"

(Bitbake rev: 4dd38d5cfb80f9bb72bc41a629c3320b38f7314d)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:32 +00:00
Holger Hans Peter Freyther 902b5da3f6 misc: Update the email address to a working one.
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:36:54 +00:00
Chris Larson 36318110da Fix IndexError encountered when a 'depends' flag references a nonexistant task
(Bitbake rev: 4b0fd70539e73d99282fa89d47ad2d5f642ca4f4)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:36:54 +00:00
Chris Larson 151c4a7a33 Drop the 'ui failed to start' message, as the ui_init does more in knotty
(Bitbake rev: 39087138ffd5d427f07ecaa580a40885c5ffaff3)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:36:54 +00:00
Chris Larson bac0c60b61 Make realfile optional for better_exec, using the existing fn in the code object
(Bitbake rev: 499a2d28d578cdd6df7cd30ccb79cc2b2796fb65)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:36:54 +00:00
Chris Larson 96e7518472 When shutting down, send SIGTERM, not SIGINT
SIGINT should be from the user, not a script.  It also doesn't work as
reliably to shut down processes, as it's not always interpreted as a
termination request.  In addition, it causes KeyboardInterrupt exceptions in
the worker processes, which can interfere with our exception handling.

(Bitbake rev: e5f6e0e9de4c6d1dfdd269d2bf7f83c00c415a27)

Signed-off-by: Chris Larson <clarson@kergoth.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-12-31 10:34:08 +00:00
Yu Ke 6ee0c26e21 bb.utils: check if lock file is writable, to fix bug 606
Bug 606 report that if $DL_DIR is read-only, do_fetch will
simply hang without any error message.

The root cause is that: bb.fetch.go()->bb.utils.lockfile()
will try to lock file ${DL_DIR}/xxxxx.lock. Since ${DL_DIR}
is read-only, it will cause IOError exception. Although
lockfile() can catch the exception, currently code simply
ignore all the exception and continue the loop. it make
sense if the exception is caused by locking contention,
but in the read-only $DL_DIR case, it cause endless waiting
unfortunately.

So this patch add read-only check for lockfile to avoid the
silent hang.

Fix [BUGID #606]

Signed-off-by: Yu Ke <ke.yu@intel.com>
2010-12-30 09:28:45 +00:00
Joshua Lock ab83f5f266 bitbake/depexp: fix early exit
Stupdi typo/thinko from me had depexp exiting once recipes had parsed
as I'd used a return the while loop where I'd meant a continue...

Signed-off-by: Joshua Lock <josh@linux.intel.com>
2010-12-22 18:36:37 +00:00
Richard Purdie 3bda5b1301 bitbake/fetch: Fix fetch error
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-12-21 15:15:42 +00:00
Kevin Tian 05963b0a1b runqueue.py: set BB_TASKHASH approriately after moving to fork()
In previous exec() model, cooker is re-initialized from scratch with environmental
variable exported accordingly. Now in fork() model, environmental variables are
not exported again, and thus original method to export BB_TASKHASH doesn't apply
now which breaks all sstate packages. Now we can set data variable directly instead.

Signed-off-by: Kevin Tian <kevin.tian@intel.com>
2010-12-21 08:48:17 +00:00
Richard Purdie 2d1007b33f bitbake/fetch: Only checksum downloads when they're first downloaded and a checksum is present
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-12-21 00:52:10 +00:00
Richard Purdie 52aada2eaf bitbake/runqueue.py: Somehow the python environment mapping is failing so do it manually
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-12-21 00:51:24 +00:00
Richard Purdie d4ba59c87e bitbake/fetch: Add missing return so if a checksum isn't present, it isn't checked
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-12-20 17:38:03 +00:00
Richard Purdie 7abdf3e5c3 bitbake/fetch: Checksum validity fixes
If the checksum check failed, the .md5 stamp file would still have been created
meaning subsequent builds would proceed with the corrupt file. Reorder the calls
to avoid this. Also raise a specific error for the checksum not specified error
case.

Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-12-20 16:04:51 +00:00
Yu Ke b8d69d6f6e FetchData: add SRC_URI checksum
This patch add the per-recipe SRC_URI checksum verification.

- SRC_URI format
The format of SRC_URI checksum follow OE definition:

1. SRC_URI has single src
SRC_URI = "http://some.domain/file.tar.gz"
SRC_URI[md5sum] = "xxxxxxxxxxxxxxx"
SRC_URI[sha256sum] = "xxxxxxxxxxxxxxxxxxxxxx"

2. SRC_URI has multiple src, every src need specify name
SRC_URI = "http://some.domain/file1.tar.gz;name=name1 \
           http://some.domain/file2.tar.gz;name=name2 "
SRC_URI[name1.md5sum] = "xxxxxxxxxxxxxxx"
SRC_URI[name1.sha256sum] = "xxxxxxxxxxxxxxxxxxxxxx"
SRC_URI[name2.md5sum] = "xxxxxxxxxxxxxxx"
SRC_URI[name2.sha256sum] = "xxxxxxxxxxxxxxxxxxxxxx"

- SRC_URI checking invocation:
the checksum checking is invoked in do_fetch phase,
so it can be invoked manually by

# bitbake -f -c fetch <recipe_name>

if recipes has no SRC_URI checksum item, bitbake will show warning:
"
WARNING: Missing SRC_URI checksum for xxxx.tar.gz, consider to add
SRC_URI[md5sum] = "5c69f16d452b0bb3d44bc3c10556c072"
SRC_URI[sha256sum] = "f4e0ada8d4d516bbb8600a3ee7d9046c9c79e38cd781df9ffc46d8f16acd1768"
"
thus recipe author can add it to recpie file after SRC_URI

- control variable BB_STRICT_CHECKSUM
when SRC_URI checksum is missing, this variable decide pass or not
if BB_STRICT_CHECKSUM = "1", bitbake should fatal in this case, otherwise bitbake just pass

Signed-off-by: Yu Ke <ke.yu@intel.com>
2010-12-20 15:24:52 +00:00
Kevin Tian 834f0c5a8d siggen.py: print taskname when seeing mismatched hash in cache
Signed-off-by: Kevin Tian <kevin.tian@intel.com>
2010-12-20 14:50:22 +00:00
Richard Purdie 1f3e313fd5 bitbake Revert bitbake exec() and go back to fork() for performace wins (first draft)
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-12-17 18:25:06 +00:00
Richard Purdie a86f960f65 bitbake/runqueue: Revert previous setscene noexec changes and ensure setscene stamp exists for setscene noexec tasks
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-12-17 10:47:02 +00:00
Richard Purdie 56ef2a01be bitbake/runqueue: Clean up message log levels
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-12-17 10:47:02 +00:00
Kevin Tian aa1a780d84 siggen.py: make sure stamp directory existing
or else "bitbake -S" from scratch may report "No such file or directory" error.

Signed-off-by: Kevin Tian <kevin.tian@intel.com>
2010-12-16 15:33:44 +00:00
Richard Purdie ace52b0da3 bitbake/runqueue.py: We should just failoutright in scenequeue for noexec tasks, not skip them so stamps are created
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-12-16 15:14:13 +00:00
Kevin Tian bb783afd4d runqueue.py: use correct task ID when checking validity of setscene tasks
sstate hash validation is done at initialization of RunQueueExecuteScenequeue.
However the index of 'valid' list returned from the validation doesn't
correspond to setscene task ID. It's just an intermediate namespace between
runqueue and sstate hash func. Use it as setscene task ID fully mess the flow.
Previously this doesn't cause trouble because all setscene tasks are passed. Commit
58396a5d24 add 'noexec' concept to setscene
tasks which grabs some tasks out of the list and thus trigger this problem

Without this fix there're ~50 recipes (gzip-native, glib, ...) rebuilt weirdly
with a minimal build, even though existing sstate packages could accelerate them.

there's another typo using wrong task ID in a debug message which further hide
this issue

Signed-off-by: Kevin Tian <kevin.tian@intel.com>
2010-12-14 22:40:16 +00:00
Paul Eggleton 80e6408b9f bitbake: add optional expansion to getVarFlag()
Add a parameter to getVarFlag() to auto-expand the value of the flag. This
makes getVarFlag() more consistent with getVar(), and allows expansion of
vardeps and vardepsexclude (which has been done in this commit).

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2010-12-14 22:36:19 +00:00
Paul Eggleton a52e4063f4 bitbake: allow excluding dependencies of variables
Adds a vardepsexclude flag that can be used to exclude a dependency of
a variable (the opposite of vardeps). This will allow the exclusion of
variables from the hash generation much more selectively than blanket
whitelisting using BB_HASHBASE_WHITELIST.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2010-12-14 22:35:50 +00:00
Richard Purdie c47cfc56c7 bitbake/bitbake-runtask: Since the parent has set the environment up cleanly we don't need to filter this ourselves anymore
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-12-10 12:40:23 +00:00
Richard Purdie 0ba9a9fffe bitbake: Overhaul environment handling
Currently, anything whitelisted in the environment makes it into the worker
processes. This is undesireable and the worker environment should be as
clean as possible. This patch adapts bitbake sosme variables are loaded into
bitbake's datastore but not exported by default. Any variable can be exported
by setting its export flag.

Currently, this code only finalises the environment in he worker as doing so
in the server means variables are unavailable in the worker. If we switch
back to fork() calls instead of exec() this code will need revisting.

Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-12-09 13:16:14 +00:00
Joshua Lock c538ef33aa bitbake/cooker: fix idle command processing in servers
idle command processing in each of the servers does not handle an explicit
None return value, which means the goggle UI ends up repeatedly adding
"Tasks Summary:" rows to the list.

This patch modifies BBCooker.buildTargets.buildTargetsIdle to return False
when BuildCompleted is fired, as is done in BBCooker.buildFile.buildFileIdle.
It may be that the correct way to fix this is to change the idle command
processing in the servers.

Signed-off-by: Joshua Lock <josh@linux.intel.com>
2010-12-09 12:41:44 +00:00
Joshua Lock 5526a64f4e bitbake/goggle: add a ProgressBar for parse progress
Construct a ProgressBar and pass it to the RunningBuild.handle_event() so
that goggle users are notified of metadata parsing progress.

UI's with status make users less nervous

Signed-off-by: Joshua Lock <josh@linux.intel.com>
2010-12-07 13:53:06 +00:00
Joshua Lock 48e27575eb bitbake/goggle: automatically show most recently added message
It seems likely that the user would want to view the most recently emitted
messages so this patch sets the message dislay treeview to scroll to any
newly added rows.

Signed-off-by: Joshua Lock <josh@linux.intel.com>
2010-12-07 12:53:05 +00:00
Joshua Lock b9617bf6e1 bitbake/xmlrpc: Modify xmlrpc server to work with Python 2.7
Python 2.7's library changes some of xmlrpclib's internal implementation such
that interacting with a proxy to BitBakes SimpleXMLRPCServer would cause
BitBake to crash.

The issue was traced to changes in the xmlrpclib.Transport implementation and
Python bug #8194 (http://bugs.python.org/issue8194).

This patch introduces a workaround by create a subclass of
xmlrpclib.Transport, which overrides the offending methods with the Python
2.6.6 implementation copy and pasted from the Python 2.6.6 xmlrpclib, and
using this BBTransport implementation for both xmlrpclib.Server objects we
create.

Signed-off-by: Joshua Lock <josh@linux.intel.com>
2010-12-07 12:51:59 +00:00
Joshua Lock 84384aa8a9 bitbake/crumbs: add optional pbar parameter to RunningBuild.handle_event()
Defaults to None, but if set will pass the ParseProgress sofar and total to
pbar's update() method.

Signed-off-by: Joshua Lock <josh@linux.intel.com>
2010-12-07 12:51:16 +00:00
Joshua Lock 1307d64286 bitbake/depexp: Factor ProgressBar into a separate class in crumbs/
ProgressBar will be useful in other UI elements so make it it's own class.
Make ProgressBar a subclass of gtk.Dialog, rather than gtk.Window, so that we
can suggest the window manager parent the ProgressBar to the widget passed at
as parent.

Signed-off-by: Joshua Lock <josh@linux.intel.com>
2010-12-07 12:50:55 +00:00
Joshua Lock 8cbaa9f0d9 bitbake/goggle: interaction tweaks
Set the goggle window to a more sane default size (640x480) and hook up the
close button.

Signed-off-by: Joshua Lock <josh@linux.intel.com>
2010-12-07 12:50:38 +00:00
Joshua Lock cb6eff7c80 bitbake/crumbs: fix the event name determination
Due to some recent change *somewhere* we need to explicitly look at the
name attribute on the instances class, rather than the name attribute of
the instance.

Signed-off-by: Joshua Lock <josh@linux.intel.com>
2010-12-07 12:50:09 +00:00
Joshua Lock bd8ff30da0 bitbake/crumbs: do the test for ignored messages sooner
Move the test for ignored messages to the start of the message handling loop to
avoid doing work for messages which are only going to be ignored.

Signed-off-by: Joshua Lock <josh@linux.intel.com>
2010-12-07 12:49:38 +00:00
Joshua Lock 96da2ea1f9 bitbake/crumbs: Fix crumbs UI for bitbake event class name changes
Some of the events we where trying to look for have had their class names
changed, fix these references.

Signed-off-by: Joshua Lock <josh@linux.intel.com>
2010-12-07 12:49:09 +00:00
Kevin Tian 4336d676d4 siggen.py: fix the wrong usage on BB_TASKHASH_WHITELIST
BB_TASKHASH_WHITELIST is expected to filter out native tasks from the
dependency list for target recipe's checksum. However current code
actually implements the opposite. All native sstate packages end up
to have empty task dependency while target sstate packages still have
native tasks counted into the checksum.

Signed-off-by: Kevin Tian <kevin.tian@intel.com>
2010-12-07 12:45:08 +00:00
Richard Purdie 7aacc310b5 bitbake/data_smart: Refactor _append/_prepend code to remove duplication
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-12-07 12:25:24 +00:00