Commit Graph

187 Commits

Author SHA1 Message Date
Richard Purdie 3059b09223 bitbake: bitbake: Update to version 1.20.0
(Bitbake rev: 2666865a27cedbffc82800b4cab1d6cff5a0222e)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-09-24 18:04:16 +01:00
Richard Purdie d28b1816e9 bitbake: bin/bitbake: Improve --help text
The --help text was rather inconsistent in style and plain incorrect in places,
using confusing terminology in others. I guess most people know what the options
do and don't read this but its confusing to new users.

This updates it to use the terms recipe and task consistently, remove
the references to stage, bbread and generally try and make the output
more useful.

[YOCTO #4856]

(Bitbake rev: 516311946c7bd14c84947dc44c3bb0563e5a9667)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-09-22 12:19:45 +01:00
Alexandru DAMIAN ba83eb315d bitbake: bitbake: cooker,xmlrpc,servers: implement CookerFeatures
Implementing feature set selection that allows a client
to enable specific features in the server at connection time.

Only enabling of features is supported, as there is
no way to safely remove data loaded into the cooker.
Once enabled, a feature will remain enabled for the
life of the cooker.

Client-server connection now supports specifying the feature
set required by the client. This is implemented in the Process
server using a managed proxy list, so the server cooker
will now load dynamically needed features based on what client
connects to it.

In the XMLRPC server the feature set is requested by
using a parameter for registerUIHandler function.
This allows observer-only clients to also specify features
for the server.

The server code configuration now is completly separated
from the client code. All hardcoding of client knowledge is
removed from the server.

The extra_caches is removed as the client can now specify
the caches it needs using the feature. The UI modules
now need to specify the desired featureSet. HOB is modified
to conform to the featureSet specification.

The only feature available is CookerFeatures.HOB_EXTRA_CACHES
which forces loading the bb.cache_extra:HobRecipeInfo class.

(Bitbake rev: 98e594837aab89ea042cfa9f3740d20a661b14e2)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-09-22 12:19:43 +01:00
Alexandru DAMIAN baff2630b3 bitbake: bitbake: read BBSERVER environement for remote server
This change allows bitbake to connect to a remote server
by reading the BBSERVER environment variable and effecting
the config. Basically a shortcut to the command line parameters.

(Bitbake rev: 81929f86c57ed0a4ad0cda7aaa820fceabaa61e9)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-06-17 16:09:10 +01:00
Alexandru DAMIAN f5cadbe005 bitbake: knotty, bitbake: add option to terminate a remote server
I add an option to terminate a remote server gracefully
as not to need a kill command.

(Bitbake rev: 7495f835666a9561c2c7d84da7aaa74e4df55b9a)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-06-17 16:09:10 +01:00
Alexandru DAMIAN 3ea9d647ec bitbake: knotty, xmlrpc: add observer-only mode
I add an observer only mode for the knotty UI and
the XMLRPC server that will allow the UI to register
a callback with a server in order to receive events.

The observer-UI is able to send read-only commands to the
server, and also is able to register as an event handler.

Read-only commands are the commands that do not change
the state of the server and have been marked as such in
the command module.

The observer can switch to a full client if it calls addClient
at any time, and the server has no other client running.

(Bitbake rev: 4de9ee21f1fa4d04937cc7430fb1fc8b7a8f61e2)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-06-17 16:09:10 +01:00
Richard Purdie 542d6a87db bitbake: bitbake: Update version to 1.19.1
(Bitbake rev: 6acd444a557bba977ae1772371fdadf5f510e3b6)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-06-14 12:52:58 +01:00
Alexandru DAMIAN 0ac7a182aa bitbake: bitbake: Improve remote server handling
Two fixes in bitbake related to running remote servers -
* can now specify correctly the bind port
* the information in print conforms to common server infrastructure

(Bitbake rev: b657208ee15ae065e5fcc2dd6e0051e03d246727)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-06-07 14:13:18 +01:00
Alexandru DAMIAN d0861b7a12 bitbake: bitbake: xmlrpc remote server
Added code in XMLRPC server that creates a stub local server
for a client-only connection and is able to connect to
a remote server, and receive events from the remote server.

Added the option to start a client with a remote server in
bitbake.

Original code by Bogdan Marinescu <bogdan.a.marinescu@intel.com>

(Bitbake rev: 25b2af76104d5aaf6435de8c158e0407512f97ce)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-05-30 10:44:00 +01:00
Alexandru DAMIAN b6b30095de bitbake: bitbake: move start server code in a separate function
This is a code sanitization targeted at making further
server-related changes easier (launch a server
separately or creating a mockup-server) to do.

(Bitbake rev: eac00258d213137ef73aed255c92b7981e2f1c75)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-05-30 10:20:32 +01:00
Alexandru DAMIAN e6a1b33bc8 bitbake: bitbake: move extra cache collection out of cooker
The collection of the extra caching data should not
be performed by the cooker, but supplied to it.
This patch will also streamline the code for launching servers
without a UI attached.

Based on a patch by Bogdan Marinescu <bogdan.a.marinescu@intel.com>

(Bitbake rev: f0b54280a6bce522508e4741e5f507bc284113a8)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-05-30 10:20:32 +01:00
Alexandru DAMIAN e09a159277 bitbake: bitbake: retrieve the entire ui module in launcher
Slight change in bitbake as to retrieve the entire UI module
for further processing instead of just the main function.

Based on a patch by Bogdan Marinescu <bogdan.a.marinescu@intel.com>

(Bitbake rev: f49341a9599d971829ef65b2b02732543740a3c9)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-05-30 10:20:31 +01:00
Richard Purdie 3fb25ef24b bitbake: server/bitbake: Remove launchUI method
With the removal of the none server type, we can remove the launchUI
method and simplify the code slightly.

(Bitbake rev: 9bef2f2dd0bcaa59528ebcb3c1ce053b7dff1ec6)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-05-30 09:54:35 +01:00
Richard Purdie e118ee3ad8 bitbake: server: Remove none server
The process server backend has been serving well as the default for a long
time now and the UI model is much better thought out that it used to be. With
the move to make bitbake a memory resident process, the none server is now
looking rather pointless and complicates the code needlessly. Lets therefore
now remove it.

(Bitbake rev: 9af03a89605e3db9bce3cea1e0f2d0b6cfaa6fe1)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-05-30 09:54:34 +01:00
Richard Purdie ba707d0ffa bitbake: bitbake/cookerdata: Explicitly specify cooker configuration options
As the code stands today its hard to know which configuration variables
are used by which parts of the system. Some are used by the UIs, some
by bin/bitbake itself, some by cooker.

This patch changes the configuration to just contain the variables cooker
uses, and changes bin/bitbake to access the variables it needs directly
which hopefully lets us start to untangle this mess.

(Bitbake rev: e57497a24b6157c92519a34accd66035a39ad1f8)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-05-22 12:10:14 +01:00
Richard Purdie f0930c8d63 bitbake: cooker: Move commandline parsing back into the UI/cookerdata
Building up a set of actions for the server is tricky since we depend upon the
commandline but fall back to values from the datastore. We should be able to build
a datastore without a commandline and vice versa. Ultimately the UI should send
the commands to the server.

This patch amounts to code rearranging, moving the heavy lifting to the UI, though
a helper in the configuration option. This will need further cleanup/tweaking but
this should be the only update needed to the UIs. The code now queries the server
for any missing data should it need to.

This code allows various knowledge of configuration variables to move to the UI side
only, partcularly pkgs_to_build but also all the command specifiers. It should also
be possible to move cmd eventually, I'm just unsure if any callers call the commands
expecting this to default to something sane right now.

(Bitbake rev: 2dbbb1d51dafd4451fef8fe16f095bcd4b8f1177)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-05-22 12:10:13 +01:00
Richard Purdie f242f5060b bitbake: bitbake: Create cookerdata splitting config from cooker and bin/bitbake
Currently the UI and server configuration is one big incestuous mess. To
start to untangle this we creater cookerdata, a new module which contains
various confiuration modules and the code for building the base datastore.

To start with we add a ConfigParameters() class which contains information
about both the commandline configuration and the original environment.

The CookerConfiguration class is created to contain the cooker.configuration
options. This means we can transfer new paramters to the server over something
like XMLRPC and then build a new configuration from these on the server.

Based on a patch from Alexandru Damian <alexandru.damian@intel.com>

(Bitbake rev: 35bd5997e8d8e74bc36019030cc10c560a8134f9)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-05-22 12:10:13 +01:00
Richard Purdie 308ae92100 bitbake: bin/bitbake: Improve exception handling
Due to the internal event processing, this excepting handler usually raises an
Empty error, masking the underlying failure. Ensure the original exception is
raised.

(Bitbake rev: 7d548568a55adfe84a976f2a549995e42da1afef)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-05-22 12:10:13 +01:00
Richard Purdie bd27cca7e6 bitbake: bin/bitbake: Fix queue import for pyhton 2 and 3
(Bitbake rev: 782bc0e797802224f06cb6f9098a50d658c86523)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-05-14 15:04:18 +03:00
Paul Eggleton fd8dcd7b88 bitbake: bin/bitbake: add a check for /dev/shm existing
Python multiprocessing needs /dev/shm, and if it doesn't exist (e.g. in
a chroot environment) or is not writable, you'll get
"OSError: [Errno 38] Function not implemented" or
"OSError: [Errno 13] Permission denied" with a traceback, which doesn't
really help to explain what is wrong.

Implements [YOCTO #4407].

(Bitbake rev: e7460d01f281f913a94192e1f4bd20688164bdd4)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-05-09 14:14:12 +01:00
Richard Purdie f58e82b9e9 bitbake: bin/bitbake: Ensure early errors are shown to the user
Currently if errors occur early in the init process, the errors may
not be shown to the user. This change ensures that if a failure does
occur, the messages are flushed from the queue and shown to the user.

(Bitbake rev: fda84c4285e0bc31c249b6dd5464aeb6ad622a9a)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-05-09 14:06:52 +01:00
Bogdan Marinescu 7709342e69 bitbake: bitbake: remove stack trace depth limitation
In case of an error in bitbake, the stack trace is limited to 5
items. This is an endless source of confusion and it makes bugs
reports impractical, since a full stack trace can't be included
in the bug report. This patch simply removes the depth limit.

(Bitbake rev: 02629c42fb09413d9da16cfe43e03338ce7db3ff)

Signed-off-by: Bogdan Marinescu <bogdan.a.marinescu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-05-03 16:24:13 +01:00
Richard Purdie c88bfe215f bitbake: Update to version 1.19.0 for master
(Bitbake rev: c47088a86fd8ad06c1810d04d9537c4cd01e8bef)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-05-03 16:24:05 +01:00
Richard Purdie ca6321fff4 bitbake: Update to version 1.18.0
(Bitbake rev: 94b54788cadabf6ebfb7711674646dbea6204805)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-04-10 16:17:22 +01:00
Richard Purdie b6d8f96ee5 bitbake: bitbake/utils: Improve environment handling to allow UIs access to original environment
We need to empty out the environment whilst we build the cooker but
we need the environment for the UIs since hob uses DISPLAY and other
session variables.

This patch adapts the utils functions to return removed environment
components so we can reinject them for use by the UI, allowing hob
to work again.

(Bitbake rev: fc330d810099c57fefd4e706159a73ad8401d97c)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-02-11 11:11:26 +00:00
Richard Purdie 945cd710cb bitbake: Update version to 1.17.0
(Bitbake rev: d96ef9398356b2a61f7cff765821616bf1e55492)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-02-06 13:13:01 +00:00
Richard Purdie 5d968368bc bitbake: bin/bitbake, bb/__init__: Move BBDEBUG handling to bin/bitbake
The method this older code was accessing doesn't exist. A library shouldn't
be changing behaviour based on the environment anyway when sepcific function
calls exist to correctly setup the logging. Fix the crash if BBDEBUG was used
by moving the code to bin/bitbake.

[YOCTO #3319]

(Bitbake rev: 3a1b75037fe1e5282ffd40b037e93353405faa45)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-12-14 12:36:04 +00:00
Richard Purdie c6dc0b9767 bitbake: Update version to 1.17.0
(Bitbake rev: cc7fdbdc607df530f5539b162831bf9998eb48d8)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-11-26 10:01:24 +00:00
Richard Purdie e450b10c4c bitbake: bitbake/server: Remove dead console log code
This code is dead and doesn't do anything so lets remove it.

(Bitbake rev: 8d45739f49618757a5d7d79782deda355e3981ec)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-11-01 11:46:21 +00:00
Richard Purdie 6494c05e80 bitbake: Update version to 1.16.0
(Bitbake rev: a579754a04bdcf450e6957dde614a15c11df39e2)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-10-02 11:40:56 +01:00
Jason Wessel 88429f018b bitbake: bitbake: Unbuffer stdout for log files
It is possible to lose critical log data when python exits in an
unorderly fashion via segmentation fault or certain types of crashes.
This is because the buffer characteristics are inherited from the top
level stdout, which should be set to unbuffered, for the purpose of
all the forked children.

This pushes the buffering to the OS, instead of having python managing
the buffers in its stream handler class.

This change is also to provide the ability to tail logs written from
processes in "real time" because they would be written in an orderly
fashion depending upon the OS characteristics for the file I/O.

(Bitbake rev: c6a367bc3224adafca698a4ffc5414ad83842c16)

Signed-off-by: Jason Wessel <jason.wessel@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-09-24 15:35:32 +01:00
Jason Wessel eed98e4666 bitbake: runqueue: Add --no-setscene to skip all setscene tasks
Mainly intended for the purpose of debugging or forcing builds
from source, the --no-setscene will prevent any setscene
tasks from running.

(Bitbake rev: 440e479f3e248482c38c149643403c6907ac7034)

Signed-off-by: Jason Wessel <jason.wessel@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-09-24 15:35:31 +01:00
Trevor Woerner d410ebb6cb bitbake: bin/bitbake: Clarify wording in help output for -s
The '-s' option shows (input) recipes, not (built/output) packages. Clarify
the help wording for this option so it is consistent with how bitbake defines
recipes and packages.

(Bitbake rev: 0cac6d647c58ae449323959220775fc1afd5bfb0)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-08-30 17:44:07 -07:00
Richard Purdie e781fb6250 bitbake: bin/bitbake: Remove knotty2 from the list of valid interfaces
(Bitbake rev: e0f91faf195be63803d39fb42fcd115ad558c79f)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-08-17 11:33:20 +01:00
Richard Purdie 26ace2c081 bitbake: Update version to 1.15.3
(Bitbake rev: befbb5c9428eaf98b2ff9c4b36b0b3f2dce7fe07)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-07-19 10:24:50 +01:00
Robert Yang 041a0e8dff bitbake: bitbake/cooker: Print which pkgs would be built in -g output
This is for giving the user a clear list to show which pkg would be
built, we have the "bitbake -g", but it is not easy to read for people,
it is for "dot". Improve the "bitbake -g" to also save a pn-buildlist:

$ bitbake -g core-image-sato
...
NOTE: PN build list saved to 'pn-buildlist'
[snip]

The contents of pn-buildlist:

busybox
shadow-native
pth
sysfsutils
qemu-helper-native
curl-native
ncurses-native
gdbm
xserver-xorg
linux-libc-headers
[snip]

[YOCTO #2404]

(Bitbake rev: 18aff925aece774d0172894e25584353519ca03f)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-07-09 17:19:32 +01:00
Paul Eggleton 25441f47ca bitbake: bitbake: tweak help text for -C slightly
What is specified is a command (or "cmd" as in the help text for -f)
rather than a task - i.e. you specify compile, not do_compile, so change
the sentence to reflect that and reorder it slightly so it makes sense.

(Bitbake rev: 9ab269410ef5cd4753fe11ec74759f421685bb7b)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-06-25 14:57:15 +01:00
Paul Eggleton c072fa8400 bitbake: bitbake: add -C option to invalidate a task and rebuild the target
This new command line option forces the specified task and all dependent
tasks up to the default task to re-run. This means that the following
single step:

bitbake -C compile somerecipe

is equivalent to the following two steps (with the recent change to -f):

bitbake -c compile -f somerecipe
bitbake somerecipe

Note that to work this option needs full hashing enabled (i.e.
BB_SIGNATURE_HANDLER must be set to a signature handler that inherits
from BasicHash). If this is not the case, -C effectively does nothing.

Based on a previous implementation of this option by Jason Wessel
<jason.wessel@windriver.com>.

Implements [YOCTO #2615].

(Bitbake rev: 2530e0faada5775897cfd1b93aba6925826dca73)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-06-21 13:10:32 +01:00
Enrico Scholz 358016f463 bitbake: support $BITBAKE_UI environment
Patch adds support for a $BITBAKE_UI environment variable which allows
to configure the preferred user interface. Although an '-u' option
(which will override the environment variable) exists already, it was
required to specify this option on every invocation of bitbake.

Because user interface is instanciated very early in the program it is
not possible to use bitbake.conf for setting up a default.  An
environment variable (which acts in a similar category like $PAGER or
$EDITOR) is a simple way for configuring the default.

(Bitbake rev: e3c213015953d1a0afb5ef4be59e1264990e5cee)

Signed-off-by: Enrico Scholz <enrico.scholz@sigma-chemnitz.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-05-23 11:35:11 +01:00
Richard Purdie 0c243d399a Update version to 1.15.2 (correspdoning to Yocto 1.2 release)
(Bitbake rev: 270a05b0b4ba0959fe0624d2a4885d7b70426da5)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-05-01 20:50:41 +01:00
Richard Purdie eeb0529e13 ui/knotty: Add a footer to the build output for interactive terminals as knotty2 UI
On terminals which support it, add summary information to the end of the
build output about the number of tasks currently running and how many tasks
we've run so far.

This provides a summary at a glace of what the current state of the build is
and what the build is currently doing which is lacking in the current UI.

Also disable echo of characters on stdin since this corrupts the disable,
particularly Crtl+C.

The "waiting for X tasks" code can be merged into this code too since
that is only useful on interactive terminals and this improves the
readability of that output too.

Improvements since v0:

* The tasks are ordered in execution order.
* The display is only updated when the list of tasks changes or there
  is output above the footer.
* Running task x oy y and package messages are supressed from the console

This UI can be accessed with "bitbake -u knotty2".

(From Poky rev: e38b4569648f2916c4370871c79e6a6090eb8bc1)

(Bitbake rev: 156189c799d2bb1f69bdaa04b5cd718fe7881425)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-03-29 19:44:01 +01:00
Dongxiao Xu 19089aca83 bitbake: add -B option to bind with interface
When start bitbake as a server only process, we need to assign certain
interface to it.

(Bitbake rev: 95b97d2dc6466ea3d99371f5b5bd68f6f3c99074)

Signed-off-by: Dongxiao Xu <dongxiao.xu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-02-23 22:52:16 +00:00
Richard Purdie 76c16c4fa8 Bump the version to 1.15.1
(Bitbake rev: 65c2eb1c095fe7ba3259060aefd4ba0e20cae319)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-01-23 09:01:36 +00:00
Dongxiao Xu ce9e6bad8f bitbake: add a new option "--server-only"
Create a new option "--server-only" for bitbake command, which allows
bitbake runs as a server, and let frontend connect the server itself.

"--server-only" should work with "-t xmlrpc", or bitbake will exit.

bitbake --server-only -t xmlrpc will print out the server address and
port information.

(Bitbake rev: 2677254cf9792cee083605267570d93b425cc2db)

Signed-off-by: Dongxiao Xu <dongxiao.xu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-01-06 15:55:57 +00:00
Richard Purdie 01f59c4169 Increment version post release
(Bitbake rev: 9c097704b4309dbe67fc360c8377edbedc354d00)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-11-04 15:43:27 +00:00
Richard Purdie 52253285fe Release 1.14.0
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-11-04 15:43:26 +00:00
Richard Purdie 7ee93b206a bitbake/logging: Overhaul internal logging process
At the moment it bugs me a lot that we only have one effective logging
level for bitbake, despite the logging module having provision to do
more advanced things. This patch:

* Changes the core log level to the lowest level we have messages of
  (DEBUG-2) so messages always flow through the core logger
* Allows build.py's task logging code to log all the output regardless
  of what output is on the console and sets this so log files now
  always contain debug level messages even if these don't appear
  on the console
* Moves the verbose/debug/debug-domains code to be a UI side setting
* Adds a filter to the UI to only print the user requested output.

The result is more complete logfiles on disk but the usual output to the
console.

There are some behaviour changes intentionally made by this patch:

a) the -v option now controls whether output is tee'd to the console.

Ultimately, we likely want to output a message to the user about where the
log file is and avoid placing output directly onto the console for every
executing task.

b) The functions get_debug_levels, the debug_levels variable, the
set_debug_levels, the set_verbosity and set_debug_domains functions are
removed from bb.msg.

c) The "logging" init function changes format.

d) All messages get fired to all handlers all the time leading to an
increase in inter-process traffic. This could likely be hacked around
short term with a function for a UI to only request events greater than
level X. Longer term, having masks for event handlers would be better.

e) logger.getEffectiveLevel() is no longer a reliable guide to what
will/won't get logged so for now we look at the default log levels instead.

[YOCTO #304]

(Bitbake rev: 45aad2f9647df14bcfa5e755b57e1ddab377939a)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-08-15 09:14:28 +01:00
Richard Purdie 05ca275693 Update version to 1.13.3
(Bitbake rev: f0b5d16426b983a67c51c47f3542162108bd4156)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-07-25 14:36:00 +01:00
Joshua Lock e51d542e78 bitbake|cooker: save a copy of the environment when BitBake is started
Create a data store in the cooker containing the values of the environment
from when BitBake is launched such that child processes can replicate
(and/or use values from) the host environment, rather than the cleaned up
environment that the main BitBake process uses.

(Bitbake rev: 54c7206165c0e7cfe5f7b243c80461baf5e7dfb1)

Signed-off-by: Joshua Lock <josh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-07-25 12:54:17 +01:00
Paul Eggleton bc4aefff98 adjust comments/messages for default server change
Default server is process, adjust comments and messages accordingly.

(Bitbake rev: 8ba4d0e98401cdb808f727703913ad8ba87f8e71)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-07-25 12:54:16 +01:00
Paul Eggleton 93cc53f842 add note to -b option indicating no dependency handling
The -b option doesn't handle dependencies so note this in the help.

(Bitbake rev: b133b2e2dd5bcde705397eb38fa20a5c4da6e3b3)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-07-25 12:54:16 +01:00
Richard Purdie 81545301d7 bitbake: Switch to use process as the default server
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-07-05 11:40:23 +01:00
Richard Purdie ea518846b8 bitbake: Add missing bracket, somehow lost by sync scripts
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-07-01 17:44:12 +01:00
Joshua Lock f3be8e9a7d bitbake: add -R option for loading configuration files after bitbake.conf
Useful if you want to load a configuration file that sets values which may
also be set in bitbake.conf or one of the files it includes.

(Bitbake rev: a8246ae5400c23df0d3ee29c36f4d9f257d1e6d1)

Signed-off-by: Joshua Lock <josh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-07-01 17:17:36 +01:00
Liping Ke cbd00b8927 bitbake: Make bitbake server type configurable.
Add -t options in bitbake for configuring server type.

(Bitbake rev: 5591329948648927154024bcb882c45766defac2)

Signed-off-by: Liping Ke <liping.ke@intel.com>
Signed-off-by: Joshua Lock <josh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-07-01 17:17:35 +01:00
Richard Purdie f0c5451e59 Update version to 1.13.2
(Bitbake rev: 5d41720d1550c04aac76275614ca15110c1c7f52)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-06-28 15:28:16 +01:00
Scott Garman 62d538fbe6 make exception handling syntax consistent
Update exception handling syntax to use the modern style:
except ExcType as localvar

(Bitbake rev: dbf5f42b06bef81749b13aa99945cc1292a6676d)

Signed-off-by: Scott Garman <scott.a.garman@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-06-15 11:13:13 +01:00
Richard Purdie a831dc120f Update version to 1.13.1
(Bitbake rev: c3c2ad6f22e35b893a353d4c21d0e923e46ad07b)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-06-09 22:44:55 +01:00
Richard Purdie e55dc39c37 Drop psyco support
At best it gave 1-2% improvement now, its 32 bit x86 only and isn't
supported after python 2.6.

PyPy is probably a better option now.

(Bitbake rev: 3c3bd0c2fa80d747f25401c17b785c7c2f3787ca)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-06-09 15:48:23 +01:00
Richard Purdie 355338c40c bitbake/cooker.py: Misc sync with upstream
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-06-08 20:38:27 +01:00
Richard Purdie 6c286ed9c5 bitbake/bin/bitbake: Return an error if exceptions occur
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-06-08 13:12:49 +01:00
Richard Purdie 658ba779ac bitbake/server/process: Update to new server API
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-06-08 11:38:47 +01:00
Richard Purdie cd3c4292e7 bitbake: Cleanup bitbake server init process to be clearer to follow
Create a standard format server class instance with method calls
for each step in the server setup. There should be enough hooks
for each of the different server types.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-06-08 11:37:26 +01:00
Richard Purdie e386fe4542 bitbake: Sync up with upstream cosmetic changes for bin/bitbake
Sync cosmetic differences between bitbake in poky and bitbake upstream
in preparation for resolving the real code differences in the server
handling.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-06-08 11:36:30 +01:00
Bob Foerster 2e0ef25a50 Resurrect alternative UIs
The various alternative UIs have been updated to once again be functional
with the latest bitbake internals.  Each of the UIs still have much room for
functional improvement.

In particular, they have been updated to:
 - interact with the new process based server
 - handle the current set of events and notifications fired from the server
   and its associated subsystems

(Bitbake rev: b947e7aa405966262c0614cae02e7978ec637095)

Signed-off-by: Bob Foerster <robert@erafx.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-05 11:13:48 +00:00
Chris Larson 9260c08e1d Ensure that the invalid UI error goes to stderr
(Bitbake rev: 76aeea2c23a518046d0b26f1a6fc43287a180a9d)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:50 +00:00
Chris Larson 23735bb765 Silence python 2.7 nested context manager warning
(Bitbake rev: 1c38f19fb8da2e6c55394162d0142e7146387e54)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:49 +00:00
Chris Larson 3d3af810fa Kill the uncaught exception handler
We now wrap the main() in a try/except, ensuring that both the main portion of
bin/bitbake and the UI raising an exception will be shown to the user.  For
the server and workers, we can ensure in the server itself that exceptions are
handled correctly.

(Bitbake rev: 240d4a7ae80a6636c302ae84266ddfed7a7fcedd)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:48 +00:00
Chris Larson c8d2dad049 Rename the ui 'init' method to 'main'
As these may run the UI in a blocking fashion and then return the exit code,
'init' was an inappropriate name, and 'main' is more appropriate.

(Bitbake rev: 4d081a0ed759bd526ab01849d650bd9e8d80ddd1)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:47 +00:00
Chris Larson 1e3b83f96b Error more pleasantly when trying to use python2.4
(Bitbake rev: 57402ee727c8c1d8f5017534fb0f5e0b20aaaa5b)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:46 +00:00
Chris Larson 7481c11e34 Don't show a traceback for a python version error
(Bitbake rev: 44c325306e55336b7b416fd88b39874012a3f603)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:46 +00:00
Chris Larson 6810622484 Use __file__, not sys.argv[0]
(Bitbake rev: 97e92abe49663eee189c89c1dc91fe69891faf73)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:45 +00:00
Chris Larson 0533a8465b Don't show uncaught exception message for KeyboardInterrupt
(Bitbake rev: 29634acd262b06fd14f6ef1e134346f274cf448f)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:34 +00:00
Chris Larson ecc68fa4fb Switch bitbake internals to use logging directly rather than bb.msg
We use a custom Logger subclass for our loggers

This logger provides:
- 'debug' method which accepts a debug level
- 'plain' method which bypasses log formatting
- 'verbose' method which is more detail than info, but less than debug

(Bitbake rev: 3b2c1fe5ca56daebb24073a9dd45723d3efd2a8d)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:33 +00:00
Chris Larson d3a45c7d41 Use logging in the knotty ui, and pass the log record across directly
This kills firing of Msg* events in favor of just passing along LogRecord
objects.  These objects hold more than just level and message, but can also
have exception information, so the UI can decide what to do with that.

As an aside, when using the 'none' server, this results in the log messages in
the server being displayed directly via the logging module and the UI's
handler, rather than going through the server's event queue.  As a result of
doing it this way, we have to override the event handlers of the base logger
when spawning a worker process, to ensure they log via events rather than
directly.

(Bitbake rev: c23c015cf8af1868faf293b19b80a5faf7e736a5)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:33 +00:00
Chris Larson 4855548ffb Use the python logging module under the hood for bb.msg
(Bitbake rev: 47ca82397bc395b598c6b68b24cdee9e0d8a76d8)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:33 +00:00
Chris Larson 5c50d43cfa Ignore python warnings that come from places we don't care about
Only shows warnings that come from bb, oe, or <string>

(Bitbake rev: 57018687f60b222ab220dd904c4bf870780171e9)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:33 +00:00
Chris Larson 151c4a7a33 Drop the 'ui failed to start' message, as the ui_init does more in knotty
(Bitbake rev: 39087138ffd5d427f07ecaa580a40885c5ffaff3)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:36:54 +00:00
Richard Purdie 05ba6fc7cb bitbake: Rewrite profiling code so its functional for both none and xmlrpc backends
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-11-13 21:23:54 +08:00
Joshua Lock 3e7db72216 bitbake: save the initial PATH for use when starting a subprocess
It was possible for bitbake-runtime to be run against a semi-installed
python-native resulting in tracebacks with ImportError's.

To prevent this we stash the initial PATH in the BBConfiguration when bitbake
is started and then set this in the env when launching bitbake-runtask through
subprocesses Popen() call.

Signed-off-by: Joshua Lock <josh@linux.intel.com>
2010-09-08 19:22:37 +01:00
Richard Purdie 43595fabbe bitbake: Implement signatures
Includes functionality to find out what changes between two different singature data dumps.

Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-09-03 16:08:32 +01:00
Bernhard Reutner-Fischer c4fde248b1 *: use print() as a function
to make python3 happy

(Bitbake rev: c82926ccdd4ec4e3ad6e78a381dacb96adf9b409)

Signed-off-by: Bernhard Reutner-Fischer <rep.dot.nop@gmail.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-07-16 15:10:40 +01:00
Chris Larson ef1de9ecaf Apply some 2to3 refactorings
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-07-02 15:41:37 +01:00
Chris Larson b9f8af16f4 Until the interactive mode is fixed, kill it from the valid options, to avoid confusion
(Bitbake rev: 65a51bb20677148a29b9e472f6d9d0a97798f024)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-07-02 15:41:37 +01:00
Chris Larson 526837e1c1 Move serverConnection.terminate() back into the finally
(Bitbake rev: 58e1b0a5dbd0f1ca137ad7ed1bd7ad9975a20fb7)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-07-02 15:41:37 +01:00
Chris Larson 2f0a485bd4 Limit the traceback length in the default exception handler
(Bitbake rev: 2b9dc5b55e24b4946ff03bf30ca52a48547caaad)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-07-02 15:41:37 +01:00
Chris Larson ce42c24d70 Resurrect the excepthook, this time just sending the info with bb.fatal
(Bitbake rev: 07757a1bf1129cc6a967a7cf1815d3d46a6bbdcf)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-07-02 15:41:36 +01:00
Cliff Brake 6c338660e8 remove return in finally statement
causes exceptions to not be handled after the finally statement

(Bitbake rev: 1a04610dea376340b9ea4c109f64995b4fd6ad02)

Signed-off-by: Cliff Brake <cbrake@bec-systems.com>
Signed-off-by: Chris Larson <clarson@kergoth.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-07-02 15:41:36 +01:00
Chris Larson 77a6052e39 Drop the non-debug traceback suppressor, at this point I think it does more harm than good
(Bitbake rev: 2c66ddc2713061ba31363ea69c24944f4a7cd3a8)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-07-02 15:41:36 +01:00
Chris Larson 75d49840cf Reorganize the main exception handling so an attribute exception raised in the ui isn't misinterpreted
(Bitbake rev: 49b8bab706c59e93287f8540187b62ba3ece0a11)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-07-02 15:41:35 +01:00
Chris Larson 0405011734 Adjust the exception handling based on Cliff Brake's patch
Rather than printing the exception ourselves, might as well re-raise and let
the default exception handler do it for us.

(Bitbake rev: acca7eb8450cbe30d65b5b266f1a642e7965df8a)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-07-02 15:41:35 +01:00
Chris Larson 6f52061844 Fix the debug level check in print_exception
(Bitbake rev: 9cb52edf9198fe6db735abbb61d0c4026c97a8d9)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-07-02 15:41:34 +01:00
Chris Larson 214d1f7433 Move the python version check into the python package
This ensures the check is run for anyone using the package, including
setup.py, though it's sad that it's necessary, since all it really wants is
__version__.

Ideally, we'd avoid the version check entirely in favor of checking for
functionality, but that's rather difficult with language constructs like
context managers, so this'll do for now.

(Bitbake rev: f936f1c0edf70860ea0f39ec1a500ba43b56f289)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-07-02 15:41:34 +01:00
Chris Larson 231d3a534c Drop an unnecessary second python version check and use the less ugly sys.version_info
(Bitbake rev: 8f4b7b10f8740a41c085beeca5105f15204b0979)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-07-02 15:41:34 +01:00
Chris Larson bdbeaf4760 Bump version to 1.11.0
(Bitbake rev: 6c89c332664a7789d02a701e7e530838ab778264)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-07-02 15:41:34 +01:00
Chris Larson 51864054da Move the python version check before the module imports, in case the modules utilize the newer python features
(Bitbake rev: 87c15dfcbeb10e7cafb14726209e4954671b3dbe)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-07-02 15:41:33 +01:00
Chris Larson ba6df176e3 Ignore deprecation warnings for now
(Bitbake rev: 661fe59445161c5bc6a2e73b7fcbf9d20c492e27)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-07-02 15:41:33 +01:00
Chris Larson 1c74fd768f Import fixups
(Bitbake rev: 4fa052f426e3205ebace713eaa22deddc0420e8a)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-07-02 15:41:32 +01:00
Chris Larson 7acc132cac Formatting cleanups
(Bitbake rev: 2caf134b43a44dad30af4fbe33033b3c58deee57)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-07-02 15:41:32 +01:00
Chris Larson 316ab4bb2b Bump minimum python version check to 2.6.
(Bitbake rev: 85c000cc7db47cfd489cf282c58db2dab0d87908)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-07-02 15:41:32 +01:00