Commit Graph

547 Commits

Author SHA1 Message Date
Richard Purdie c47cfc56c7 bitbake/bitbake-runtask: Since the parent has set the environment up cleanly we don't need to filter this ourselves anymore
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-12-10 12:40:23 +00:00
Richard Purdie 0ba9a9fffe bitbake: Overhaul environment handling
Currently, anything whitelisted in the environment makes it into the worker
processes. This is undesireable and the worker environment should be as
clean as possible. This patch adapts bitbake sosme variables are loaded into
bitbake's datastore but not exported by default. Any variable can be exported
by setting its export flag.

Currently, this code only finalises the environment in he worker as doing so
in the server means variables are unavailable in the worker. If we switch
back to fork() calls instead of exec() this code will need revisting.

Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-12-09 13:16:14 +00:00
Joshua Lock c538ef33aa bitbake/cooker: fix idle command processing in servers
idle command processing in each of the servers does not handle an explicit
None return value, which means the goggle UI ends up repeatedly adding
"Tasks Summary:" rows to the list.

This patch modifies BBCooker.buildTargets.buildTargetsIdle to return False
when BuildCompleted is fired, as is done in BBCooker.buildFile.buildFileIdle.
It may be that the correct way to fix this is to change the idle command
processing in the servers.

Signed-off-by: Joshua Lock <josh@linux.intel.com>
2010-12-09 12:41:44 +00:00
Joshua Lock 5526a64f4e bitbake/goggle: add a ProgressBar for parse progress
Construct a ProgressBar and pass it to the RunningBuild.handle_event() so
that goggle users are notified of metadata parsing progress.

UI's with status make users less nervous

Signed-off-by: Joshua Lock <josh@linux.intel.com>
2010-12-07 13:53:06 +00:00
Joshua Lock 48e27575eb bitbake/goggle: automatically show most recently added message
It seems likely that the user would want to view the most recently emitted
messages so this patch sets the message dislay treeview to scroll to any
newly added rows.

Signed-off-by: Joshua Lock <josh@linux.intel.com>
2010-12-07 12:53:05 +00:00
Joshua Lock b9617bf6e1 bitbake/xmlrpc: Modify xmlrpc server to work with Python 2.7
Python 2.7's library changes some of xmlrpclib's internal implementation such
that interacting with a proxy to BitBakes SimpleXMLRPCServer would cause
BitBake to crash.

The issue was traced to changes in the xmlrpclib.Transport implementation and
Python bug #8194 (http://bugs.python.org/issue8194).

This patch introduces a workaround by create a subclass of
xmlrpclib.Transport, which overrides the offending methods with the Python
2.6.6 implementation copy and pasted from the Python 2.6.6 xmlrpclib, and
using this BBTransport implementation for both xmlrpclib.Server objects we
create.

Signed-off-by: Joshua Lock <josh@linux.intel.com>
2010-12-07 12:51:59 +00:00
Joshua Lock 84384aa8a9 bitbake/crumbs: add optional pbar parameter to RunningBuild.handle_event()
Defaults to None, but if set will pass the ParseProgress sofar and total to
pbar's update() method.

Signed-off-by: Joshua Lock <josh@linux.intel.com>
2010-12-07 12:51:16 +00:00
Joshua Lock 1307d64286 bitbake/depexp: Factor ProgressBar into a separate class in crumbs/
ProgressBar will be useful in other UI elements so make it it's own class.
Make ProgressBar a subclass of gtk.Dialog, rather than gtk.Window, so that we
can suggest the window manager parent the ProgressBar to the widget passed at
as parent.

Signed-off-by: Joshua Lock <josh@linux.intel.com>
2010-12-07 12:50:55 +00:00
Joshua Lock 8cbaa9f0d9 bitbake/goggle: interaction tweaks
Set the goggle window to a more sane default size (640x480) and hook up the
close button.

Signed-off-by: Joshua Lock <josh@linux.intel.com>
2010-12-07 12:50:38 +00:00
Joshua Lock cb6eff7c80 bitbake/crumbs: fix the event name determination
Due to some recent change *somewhere* we need to explicitly look at the
name attribute on the instances class, rather than the name attribute of
the instance.

Signed-off-by: Joshua Lock <josh@linux.intel.com>
2010-12-07 12:50:09 +00:00
Joshua Lock bd8ff30da0 bitbake/crumbs: do the test for ignored messages sooner
Move the test for ignored messages to the start of the message handling loop to
avoid doing work for messages which are only going to be ignored.

Signed-off-by: Joshua Lock <josh@linux.intel.com>
2010-12-07 12:49:38 +00:00
Joshua Lock 96da2ea1f9 bitbake/crumbs: Fix crumbs UI for bitbake event class name changes
Some of the events we where trying to look for have had their class names
changed, fix these references.

Signed-off-by: Joshua Lock <josh@linux.intel.com>
2010-12-07 12:49:09 +00:00
Kevin Tian 4336d676d4 siggen.py: fix the wrong usage on BB_TASKHASH_WHITELIST
BB_TASKHASH_WHITELIST is expected to filter out native tasks from the
dependency list for target recipe's checksum. However current code
actually implements the opposite. All native sstate packages end up
to have empty task dependency while target sstate packages still have
native tasks counted into the checksum.

Signed-off-by: Kevin Tian <kevin.tian@intel.com>
2010-12-07 12:45:08 +00:00
Richard Purdie 7aacc310b5 bitbake/data_smart: Refactor _append/_prepend code to remove duplication
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-12-07 12:25:24 +00:00
Richard Purdie c354955d26 bitbake/data_smart: Fix append/prepend/override ordering issue
Where a variable name consisted of an append/prepend combined with an override
and there was also an append/prepend to the variable, the override could be lost
if the override was not in OVERRIDES.

For example:

FOO = "A"
FOO_append = "B"
FOO_append_virtclass-native = "C"

could result in "AB" even though virtclass-native was in OVERRIDES.

With this patch applied, the result is "ABC" as would be expected.

The problem was the deletion of the _append/_prepend flag was happening
if *any* append/prepend was procesed, the result should really be that
it should contain any unprocessed append/prepend.

Kevin Tian deserves credit for looking into this and working out the
problem here.

Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-12-07 12:16:16 +00:00
Kevin Tian ccd590bf9d siggen.py: set 'runtaskdeps' correctly
Now 'runtaskdeps' is assigned before taskhash whitelist is filtered out, and thus
always contains original task dependencies. This causes problem to diagnose sstate
related problem. So fix it by appending filtered dep to 'runtaskdeps' in the fly.

Signed-off-by: Kevin Tian <kevin.tian@intel.com>
2010-12-06 01:15:39 +00:00
Richard Purdie a0033b622e bitbake/utils.py: Improve traceback to be more helpful/clear to users
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-12-05 22:52:44 +00:00
Richard Purdie 1a0c39e050 bitbake/utils.py: Ensure the last lines of functions are printed in tracebacks
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-12-05 22:51:38 +00:00
Richard Purdie 58396a5d24 bitbake/runqueue: Teach scenequeue about noexec tasks (since they don't do anything we can assume they're always successful
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-11-28 13:54:32 -08:00
Richard Purdie d4125bcac0 bitbake: Simplfy pre and post cooker hooks
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-11-28 13:54:31 -08:00
Richard Purdie a39f335f59 bitbake/runqueue.py: Fix stamp handling for noexec tasks
This means the noexec messages are only shown once as the stamp files are now
correctly created.

Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-11-28 12:10:17 +00:00
Kevin Tian eaa188f3b9 siggen.py: fix comparison around runtaskdeps
two dep lists should be sorted before the comparison, or else it just
causes false-positive confusion.

Also fix 'taskdeps' to 'runtaskdeps' when printing out the difference.

Signed-off-by: Kevin Tian <kevin.tian@intel.com>
2010-11-15 23:04:06 +00:00
Joshua Lock 9545130576 bitbake-runtask-strace: simple shell script to wrap tasks in a strace call
Signed-off-by: Joshua Lock <josh@linux.intel.com>
2010-11-14 21:08:27 -08:00
Richard Purdie 05ba6fc7cb bitbake: Rewrite profiling code so its functional for both none and xmlrpc backends
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-11-13 21:23:54 +08:00
Richard Purdie 89929e1f28 bitbake/fetch: Allow checking of a single url at a time (massive sstate speed improvement)
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-11-13 20:10:53 +08:00
Richard Purdie caa7fe2284 bitbake/fetch: Fix handling of mirrors when checking for url validity
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-11-13 20:10:04 +08:00
Richard Purdie 7333328c48 Revert "bitbake/fetch/local: Also check DL_DIR for files since they could already exists there"
These changes were incorrect.

This reverts commit ae98f7eacb.
2010-11-13 20:03:13 +08:00
Richard Purdie aa81dda4d4 Revert "bitbake/fetch/local: Fix os.exists reference"
These changes were incorrect.

This reverts commit f8e3397935.
2010-11-13 20:02:38 +08:00
Richard Purdie f8e3397935 bitbake/fetch/local: Fix os.exists reference
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-11-13 10:23:15 +08:00
Richard Purdie ae98f7eacb bitbake/fetch/local: Also check DL_DIR for files since they could already exists there
[BUGID #533]

Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-11-13 07:28:55 +08:00
Richard Purdie 5ae5a145dd bitbake.runqueue.py: Fix no exec task handling to correct counter housekeeping
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-11-07 20:33:12 +00:00
Richard Purdie b80a8d28f8 bitbake: Add support for 'noexec' tasks
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-11-06 14:33:05 +00:00
Richard Purdie a59935fb3a bitbake/siggen: Allow siggen code to control final stampfile format
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-11-06 12:20:33 +00:00
Richard Purdie 88f6f3fadf bitbake/siggen: Provide better warnings about empty tasks
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-11-02 13:17:06 +00:00
Richard Purdie dc13a6197f bitbake/siggen: Fix finalise call arguments
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-11-01 10:23:09 +00:00
Richard Purdie c5fab99a6f bitbake/fetch: When fetchers return errors, ensure any partial download is cleared
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-10-23 00:22:54 +01:00
Richard Purdie 535a77a9b6 bitbake/fetch: Make URL checking slightly less verbose (distracting with the sstate code)
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-10-22 14:48:54 +01:00
Richard Purdie 36f1ae42fe pseudo/fakeroot: Move the pseudo directory creation into bitbake
If sstate was used to accelerate a build, the pseudo directory might not have
been created leading to subsequent task failures.

Also, sstate packages were not being installed under pseudo context meaning
file permissions could have been lost.

Fix these problems by creating a FAKEROOTDIRS variable which bitbake ensures
exists before running tasks and running the appropriate setscene tasks under
fakeroot context.

Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-10-22 13:00:40 +01:00
Richard Purdie 08cec50330 bitbake/fetch/git: Ensure fullclone repositories are fully fetched
The git fetcher was failing to pull in new branches into a git
repository mirror tarball as the git fetch command being used didn't
add new remote branches.

This patch uses "git fetch --all" for fullclones to ensure any
new remote branches are cloned correctly.

Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-10-19 17:17:41 +01:00
Richard Purdie 05c8ee60f1 bitbake/fetcher: Deal with a ton of different bugs
The more we try and patch up the fetcher code, the more things break. The
code blocks in question are practically unreadable and are full of corner
cases where fetching could fail. In summary the issues noticed included:

a) Always fetching strange broken urls from the premirror for "noclone"
   git repositories
b) Not creating or rewriting .md5 stamp files inconsistently
c) Always fetching git source mirror tarballs from the premirror even
   if they already exist but the checkout directory does now
d) Passing "None" values to os.access() and os.path.extsts() checks under
   certain circumstances
e) Not using fetched git mirror tarballs if the preexist and always
   try and fetch them.

This patch rewrites the sections of code in question to be simpler and
more readable, fixing the above problems and most likely other odd
corner cases.

Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-10-19 12:47:45 +01:00
Richard Purdie 1077021f70 bitbake/fetch/git.py: Fix git fetcher to correctly use mirror tarballs
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-10-19 00:19:34 +01:00
Richard Purdie 229b6630af bitbake-runtask: Ensure logging settings take effect in the worker process (and pass verbose setting)
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-10-14 20:07:15 +01:00
Joshua Lock 812f8835d2 bitbake/runqueue: make the runtask program configurable
Signed-off-by: Joshua Lock <josh@linux.intel.com>
2010-10-12 13:32:27 +01:00
Richard Purdie d14f9bf6c4 bitbake/build.py: Write logfiles on a per task basis, not per function
Per function logging made it hard to track down what was happening so
switch to logging on a per task basis.

[BUGID #383]

Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-10-11 12:53:54 +01:00
Richard Purdie ff539b11ac bitbake-runtask: Ensure logging options are inherited from the parent
[BUGID #291]

Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-10-11 12:52:57 +01:00
Richard Purdie 115514b5eb bitbake/fetch: Ensure SRCREV is still set correctly if the pn-X override has been expanded
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-10-09 17:24:16 +01:00
Richard Purdie bee7c21f6e bitbake/runqueue.py: Fix invalid variable reference fixing the -f option with setscene tasks
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-10-07 15:45:02 +01:00
Richard Purdie 24ad63979f bitbake/fetch/__init__.py: Abstract mirror variable handling and enhance to accept \n delimitation in variables
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-10-07 15:42:15 +01:00
Darren Hart 0a20caf090 Export ALL_PROXY for bitbake commands
We already export http,ftp,https proxy environment variables. Some environments,
GNOME for instance, place the socks proxy in ALL_PROXY and all_proxy. Export it
as well.

Signed-off-by: Darren Hart <dvhart@linux.intel.com>
2010-10-07 10:18:25 +01:00
Richard Purdie 35d095c282 bitbake/sstate: Implement a lookup function to speed up setscene processing
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-10-05 22:26:33 +01:00