Commit Graph

913 Commits

Author SHA1 Message Date
Richard Purdie 1b08a7eb8b bitbake/cache/runqueue.py: Move workload for recipe parsing to the child process
Parsing the recipe in the parent before forking off the child worker
can mean the parent doesn't hit the idle loop and becomes a bottleneck
when lauching many short lived processes.

The reason we need this in the parent is to figure out the fakeroot
environmental options. To address this, add the fakeroot variables
to the cache and move recipe loadData into the child task.

For a poky-image-sato build this results in about a 2 minute speedup
(1.8%).

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-02-28 20:48:08 +00:00
Joshua Lock 7738e86e8b bitbake/hob: only show one progress dialog when changing machine
Remove a spurious signal emission which resulted in two progress dialogs
being shown after changing the machine.

Signed-off-by: Joshua Lock <josh@linux.intel.com>
2011-02-25 15:24:32 -08:00
Richard Purdie 9166b9e32f bitbake/utils.py: Only try and add read access to a file if we don't have it
A file we're copying might be on a readonly filesystem so if we can already read
it, don't try and add read permission.

Fixes BUGID #771 in Yocto.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-02-25 17:24:34 +00:00
Richard Purdie 2069a29a82 bitbake/utils.py: Allow join_deps to return a list that isn't comman separated
Signed-off-by: Richard  Purdie <richard.purdie@linuxfoundation.org>
2011-02-25 16:34:38 +00:00
Joshua Lock 92d5e6c07f bitbake/codeparser: fix raising of ShellSyntaxError
To raise the ShellSyntaxError we need to import it's module and reference it
by namespace.

Signed-off-by: Joshua Lock <josh@linux.intel.com>
2011-02-25 15:38:29 +00:00
Joshua Lock 08c7a59b5a bitbake/cache: bump cache version after recent changes
Signed-off-by: Joshua Lock <josh@linux.intel.com>
2011-02-24 16:02:13 +00:00
Joshua Lock f258cedfe8 bitbake/progress: make progress dialog modal for parent window
Signed-off-by: Joshua Lock <josh@linux.intel.com>
2011-02-24 15:54:53 +00:00
Joshua Lock 6dbceb0be9 bitbake: Add new UI hob, a prototype Gtk+ GUI for creating images
Hob is a first stab at implementing an interactive GUI for BitBake.

Signed-off-by: Joshua Lock <josh@linux.intel.com>
2011-02-24 15:54:53 +00:00
Joshua Lock c2814caa5d bitbake/cooker: don't drop possible_world ref count
We need this if we want to run the buildWorldTargetList function more than
once, for example in a UI where we can change the MACHINE and DISTRO as much
as we like before triggering a build.

Signed-off-by: Joshua Lock <josh@linux.intel.com>
2011-02-24 15:54:53 +00:00
Joshua Lock fb62c54e13 bitbake/progress: add method to pulse the progress bar
When we're running a long operation with indeterminate duration it's useful
to use the gtk.ProgressBar's pulse method to show that something is happening
but we don't know how long it will take.

Signed-off-by: Joshua Lock <josh@linux.intel.com>
2011-02-24 15:54:53 +00:00
Joshua Lock 7f8aa691c5 bitbake/crumbs: update documentation header
Signed-off-by: Joshua Lock <josh@linux.intel.com>
2011-02-24 15:54:52 +00:00
Joshua Lock b14cda62d0 bitbake/event: fix some whitespace issues
Signed-off-by: Joshua Lock <josh@linux.intel.com>
2011-02-24 15:54:52 +00:00
Joshua Lock 9b43eaff99 bitbake: introduce crumbs.TaskListModel a gtk.ListStore subclass
Provide a gtk.ListStore subclass which includes a function,
populate(), which takes as input the data emitted by
bb.event.TargetsTreeGenerated and fills the ListStore model
appropriately.
Furthermore convenience functions are provided by which the caller can
get gtk.TreeModel subclasses which provide filtered views of the data.

Signed-off-by: Joshua Lock <josh@linux.intel.com>
2011-02-24 15:54:52 +00:00
Joshua Lock a13304e1c6 bitbake/[cooker|cache]: cache summary, license and group. Add to targets tree
Add summary, license and group metadata to RecipeInfo and the cache.
Unfortunately this impacts parse speed but gives us a much richer set of
metadata to expose through UI's which can be accessed via the
generateTargetsTree command.

Signed-off-by: Joshua Lock <josh@linux.intel.com>
2011-02-24 15:54:51 +00:00
Joshua Lock 3c1a3b0724 bitbake/cooker: reduce code duplication
Move runqueua and taskdata initialisation into a new function,
prepareTreeData(), so that generateDepTreeData() and
generateTargetsTreeData() are not duplicating the same logic.

Signed-off-by: Joshua Lock <josh@linux.intel.com>
2011-02-24 15:54:51 +00:00
Joshua Lock dcfc5ae7b1 bitbake/cooker: add generateTargetsTree method
The generateTargetsTree() command needs to return a model which includes more
metadata than the one generated by generateDepTree().

This patch adds a new method generateTargetsTreeData() to the cooker, based
on generateDepData(), and switches generateTargetsTree() to use it.

Signed-off-by: Joshua Lock <josh@linux.intel.com>
2011-02-24 15:54:51 +00:00
Joshua Lock 3939a216a5 bitbake: implement command to find configuration files for a config variable
Some configuration variables (MACHINE, MACHINE-SDK and DISTRO) set which
confguration files bitbake should use.
The added command , findConfigFiles, enables a UI to query which files are
suitable values for a specified parameter.

Signed-off-by: Joshua Lock <josh@linux.intel.com>
2011-02-24 15:54:51 +00:00
Joshua Lock 1b3eb0c35f bitbake: implement command to get all possible targets and their dependencies
Add a new command generateTargetsTree() which returns a dependency tree of
possible targets (tasks and recipes) as well as their dependency information.

Optional parameter 'klass' also ensures any recipes which inherit the
specified class path (i.e. 'classes/image.bbclass') are included in the model

Signed-off-by: Joshua Lock <josh@linux.intel.com>
2011-02-24 15:54:50 +00:00
Joshua Lock 920c402342 bitbake/cache: store a list of inherited files in the cache
Signed-off-by: Joshua Lock <josh@linux.intel.com>
2011-02-24 15:54:50 +00:00
Paul Eggleton edd64c7e86 bitbake/server/none: fix getEvent() to return events
In the none server, events don't get processed unless the idle_commands
function gets called, which previously wasn't happening with getEvent();
thus UIs that use this to get events were not working.

Fixes [BUGID #561]

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2011-02-24 15:53:58 +00:00
Paul Eggleton febb7f2632 bitbake/server/none: remove leftover XMLRPC bits from none server
Remove some comments, imports etc. to do with XMLRPC (inherited from xmlrpc.py
which this file was based upon.)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2011-02-24 14:32:01 +00:00
Richard Purdie 91c0b3a2e6 bitbake/build.py: Fix TaskFailed parameters
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-02-23 15:06:34 +00:00
Richard Purdie 101b599110 bitbake/data_smart: Improve Variable expansion error handling
If expanding a variable triggers an exception the caller currently has no
way to supress the error message or otherwise handle the siutation. An
example of where this is a problem is "bitbake -e" showing tracebacks and
errors for variables like SRCPV in OE/Poky.

Secondly in a chained expansion fails, log mesages are recorded for
every step of the expansion, not just the innermost error which is
where the real failure occured.

To fix this we introduce a new exception ExpansionError which callers
can handle as appropriate.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-02-23 13:13:31 +00:00
Richard Purdie 6ea24f04cd bitbake/data_smart: Improve the way lazyassignment works
Currently, if a variable has been set with ??= and the code looks it up
before the data finalisation phase, no value is found. This is causes
serious problems for anonymous python functions which manipulate data, or
for the fetcher revision handling code where revisions can be set with
??=.

There is also a significant performance implication for processing lazy
assignment in finalise.

Moving the check for a default value into getVarFlag addresses both
the timing issue and the performace. This change gives a 7% real time
performance improvement to parsing the Poky metadata. The cost of the
check at this point is minimal since we have all the data flags available.

This should also fix Yocto bug 752.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-02-23 11:48:01 +00:00
Richard Purdie 707132c60a bitbake/fetch2: Ensure SRCREV_pn-PN is checked for a revision when the SRC_URI is unnamed
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-02-23 11:47:50 +00:00
Richard Purdie 56a92105fe bitbake/cooker: Fix parsing failure zombie problem
When parsing if a SystemExit event is triggered, it causes the parsing thread to
exit and the main process hangs waiting for it to finish indefintely. Add code to
catch BaseExceptions and raise these with the main process gracefully instead
of just hanging indefinitely with zombie processes.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-02-23 11:09:07 +00:00
Richard Purdie c0f0a8ee0c bitbake/BBHandler: Improve handling of multiline comments and warn users of the change
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-02-22 21:45:28 +00:00
Richard Purdie 4b6de47e73 bitbake/runqueue: Drop PATH override. This was needed for the exec() change but not now we use fork()
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-02-21 17:26:18 +00:00
Chris Larson 05c4692320 siggen: add bb.data, bb.parse imports
(Bitbake rev: 8eafb12208fcd073f930c0c74f25831d02c02198)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-02-21 13:49:56 +00:00
Chris Larson c538800fd5 uihelper: import bb.build, kill commented lines
(Bitbake rev: 718448e96d714adf8aaecedac5cb77c7f36b9cdb)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-02-21 13:49:04 +00:00
Chris Larson 5f73918d99 runqueue: simplify RunQueueStats.copy
(Bitbake rev: 778571f155139fcf43d0e30e875c39d6592ae03f)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-02-21 13:48:52 +00:00
Chris Larson 0f42f6b640 runqueue: pass a copy of the RunQueueStats to events
This avoids cases where the stats are modified after the event is fired but
before it's dispatched to the UI.

(Bitbake rev: 1954f182687a0bd429175dda87f05d8a94bb403a)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-02-21 13:48:39 +00:00
Richard Purdie 359a4d8bf7 bitbake/runqueue.py: Move SIGCHLD handler to wrap none server sleep call
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-02-18 11:54:28 +00:00
Richard Purdie 3ef2d39dbf bitbake/fecth2: Ensure BB_FETCH_PREMIRRORONLY being set as false is handled correctly
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-02-16 12:35:35 +00:00
Richard Purdie 7e4c694c9f bitbake/runqueue.py: Ensure child has the default SIGCHLD handler restored
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-02-16 12:33:40 +00:00
Richard Purdie 974ea1a190 bitbake/runqueue: When we successfully fork off a task and there are more processes we can start do so immediately
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-02-15 23:20:54 +00:00
Richard Purdie 5a178b547b bitbake/runqueue: When we have successfully collected a pid value return True to ensure we don't sleep in the main loop
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-02-15 23:20:20 +00:00
Richard Purdie 03d40a4a7b bitbake/runqueue: Add a SIGCHLD handler to ensure we wake from sleeps for SIGCHLD events
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-02-15 23:19:43 +00:00
Richard Purdie 579ecccc22 bitbake/cooker: Ensure delays are accurately transfered to the idle loops from runqueue
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-02-15 23:18:39 +00:00
Richard Purdie 6cb8fd6def process.py: Avoid deprecation warning
bitbake/lib/bb/process.py:15: DeprecationWarning: BaseException.message has been deprecated as of Python 2.6

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-02-14 16:32:07 +00:00
Richard Purdie b9a97e07b4 bitbake/fetch2: Revert part of the unpack change until the issues with it are resolved
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-02-13 00:05:06 +00:00
Mark Hatle 3faa635fd4 fetch2: unpack revision
Revise the unpack function to have a way to disable the unpack.  This is
based on the work from "Andreas Oberritter <obi@opendreambox.org>", see
http://cgit.openembedded.net/cgit.cgi/openembedded/commit/?id=2bdfe8519eda8067845019a699acdf19a21ba380

In addition, the to_boolean function comes from the work of
"Chris Larson <chris_larson@mentor.com>", see
http://cgit.openembedded.net/cgit.cgi/openembedded/commit/?id=900cc29b603691eb3a077cb660545ead3715ed54

Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
2011-02-12 00:30:29 +00:00
Richard Purdie 029f19bd3d bitbake/fetch2: Add explict network access exception and handling to give users usable error messages
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-02-11 22:08:21 +00:00
Richard Purdie 4a226460d5 bitbake/fetch2: Fix message about md5 checksums when sha256 is incorrect
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-02-11 21:59:23 +00:00
Richard Purdie 132ec8d3b7 bitbake/fetch2: Fix patch merge error
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-02-11 16:24:26 +00:00
Richard Purdie a72d49478e bitbake/fetch2: Add some debug output so its clear when PREMIRRORS, upstream and MIRRORS are being used
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-02-11 16:16:51 +00:00
Richard Purdie 20afb21a37 bitbake/fetch2: Ensure network access status is reset for each loop iteration in download()
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-02-11 16:15:36 +00:00
Richard Purdie 05d8b6422d bitbake/fetch2/wget: Fix missing string parameter reference
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-02-11 12:16:44 +00:00
Richard Purdie ea70c4362f bitbake/fetch2: Ensure failed fetch attempts are logged in the debug logs
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-02-11 12:14:20 +00:00
Richard Purdie 7f30131faa bitbake/fetch2: Correctly handle git mirror tarball fetching
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-02-11 12:06:44 +00:00