Commit Graph

44 Commits

Author SHA1 Message Date
Paul Eggleton 614cbeddc8 bitbake: lib/bb/msg: introduce logger_create() function
We use this code to set up a logger with colour in a number of different
places, so create one function that does this and make some of bitbake's
utility scripts use it.

(Bitbake rev: b1ba7d1cc8ec33e2d081230287abd07f52136097)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-27 20:23:19 +01:00
Paul Eggleton 76feac37ce bitbake: knotty: add quiet output mode
Quiet output mode disables printing most messages (below warnings) to
the console; however these messages still go to the console log file.
This is primarily for cases where bitbake is being launched
interactively from some other process, but where full console output is
not needed.

Because of the need to keep logging all normal events to the console
log, this functionality was implemented within the knotty UI rather
than in bb.msg (where verbose mode is implemented). We don't currently
have a means of registering command line options from the UI end, thus
the option actually has to be registered in main.py regardless of the
UI, however I didn't feel like it was worth setting up such a mechanism
just for this option.

(Bitbake rev: db95cdef08e339dec7462bfde3ad7d75c1c60dd8)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-08 09:57:27 +01:00
Richard Purdie 0f2c59367a bitbake: bitbake: Convert to python 3
Various misc changes to convert bitbake to python3 which don't warrant
separation into separate commits.

(Bitbake rev: d0f904d407f57998419bd9c305ce53e5eaa36b24)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-06-02 08:24:02 +01:00
Richard Purdie a46d485856 bitbake: event/msg: Pass formatted exceptions
python3 can't cope with the previous approach we were using to pass
exceptions through the RPC. Avoid this by creating a formatted exception
on the sender side.

(Bitbake rev: d7db75020ed727677afbad07a90fb3eac0bf2c45)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-05-13 13:41:32 +01:00
Paul Eggleton 715d857174 bitbake: Fix default function parameter assignment to a list
With python you should not assign a list as the default value of a
function parameter - because a list is mutable, the result will be that
the first time a value is passed it will actually modify the default.
Reference:

http://docs.python-guide.org/en/latest/writing/gotchas/#mutable-default-arguments

(Bitbake rev: 7859f7388f2e3f675d0e1527cfde18625f36f637)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-19 18:05:44 +01:00
Richard Purdie cf97773394 bitbake: msg: Add stdout/stderr filters
Add logging filters which can be used to split output between stdout and
stderr, ERROR messages and above as passed by the Err filter, anything
below ERROR is passed by the Out filter. This is useful when trying to make
stderr more useful.

(Bitbake rev: d3e1419ef76be5e9ec976597361a5e14a7b6bcb6)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-03-27 10:17:20 +00:00
Richard Purdie 25e410b743 bitbake: event/msg: Add primitive server side UI log record filtering
Currently one of the bigger bottlenecks in bitbake is passing all the
log messages over IPC to the UI. This is worthwhile if the UI is going
to use them, pointless otherwise. The memory resident bitbake suffers
from this performance issue particularly badly.

This patch filters the log events on the server side with the global
log levels and hence reduces the traffic. This speeds up parsing
(18.5s down to 17s) and bitbake general command overhead is reduced
(7.3s for a NOP to 6.2s).

What isn't added here is general event filtering or the ability to
change the log levels once set. Provision is made for adding this
in a follow up patch though.

(Bitbake rev: 1bf0e88f57ba0bca62532e81d0d62cf88e2abcbb)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-08-23 17:29:54 +01:00
Richard Purdie 9be83df144 bitbake: knotty/msg: Avoid usage of curses initscr/endwin to avoid terminal corruption
Using curses initscr/endwin causes screen corruption if for example you
suspend bitbake and resume it. This changes the code to use a less
invasive approach to determining colour availability on the terminal.

(Bitbake rev: 4548a8f037eaf8d47a77052acc3e9ec264ac41e0)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-11-25 21:36:44 +00:00
Seth Bollinger cbdc81456b bitbake: knotty: Colorize knotty interactive console output
Add bold color output to log level name and standard color output to log msg
when bitbake is run from an iteractive console.  Color output is only
enabled
if the terminal supports color.

Used Jason Wessel's recommendation for transparency on verbose, note and
plain.

(Bitbake rev: 2734240da2cc150f811129a6adf6eb4b2161b204)

Signed-off-by: Seth Bollinger <seth.boll@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-11-20 15:31:57 +00:00
Paul Eggleton 0e39107047 bitbake: implement BB_VERBOSE_LOGS
Enable configuring whether "set +x" is added to all shell tasks rather
than forcing it; this is enabled by setting BB_VERBOSE_LOGS to 1.

(Bitbake rev: 659411b6bb30e1a8355afc1c29b8170a8f2b55ac)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-02-28 13:47:10 +00:00
Paul Eggleton dea2fd181a bitbake/lib/bb/msg.py: fix setting debug and verbosity levels
The debug and verbosity levels (as set by the -D and -v command line
options respectively)  were not being passed through within msg.py since
bitbake revision 45aad2f9647df14bcfa5e755b57e1ddab377939a due to
incorrect variable names.

Fixes [YOCTO #1513].

(Bitbake rev: c6e88b7c0e61f9586a275df53f48b90687c5f92f)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-09-25 16:25:27 +01:00
Richard Purdie a21ff559e7 Fixup remaining bb.msg.domain users
(Bitbake rev: d5abdacaf9ac604ef8d8c1bafb9b30617827cb4f)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-08-15 17:31:52 +01:00
Richard Purdie a6c48298b1 bitbake/msg.py: Drop manually created domain lists
This patch removes the majority of the domain related code from bb.msg
on the grounds that we now support dynamic creation of logging domains
so having this hardcoded is just error prone and less flexible. It also
makes the msg code overly and needlessly complex.

It also removes the bb.msg.debug/note/warn/plain/error functions since
we might as well remove them rather than try and fix them at this point.

(Bitbake rev: 7627b561cbcb1482b464d69db70f38ea663180f3)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-08-15 09:49:11 +01:00
Richard Purdie 7ee93b206a bitbake/logging: Overhaul internal logging process
At the moment it bugs me a lot that we only have one effective logging
level for bitbake, despite the logging module having provision to do
more advanced things. This patch:

* Changes the core log level to the lowest level we have messages of
  (DEBUG-2) so messages always flow through the core logger
* Allows build.py's task logging code to log all the output regardless
  of what output is on the console and sets this so log files now
  always contain debug level messages even if these don't appear
  on the console
* Moves the verbose/debug/debug-domains code to be a UI side setting
* Adds a filter to the UI to only print the user requested output.

The result is more complete logfiles on disk but the usual output to the
console.

There are some behaviour changes intentionally made by this patch:

a) the -v option now controls whether output is tee'd to the console.

Ultimately, we likely want to output a message to the user about where the
log file is and avoid placing output directly onto the console for every
executing task.

b) The functions get_debug_levels, the debug_levels variable, the
set_debug_levels, the set_verbosity and set_debug_domains functions are
removed from bb.msg.

c) The "logging" init function changes format.

d) All messages get fired to all handlers all the time leading to an
increase in inter-process traffic. This could likely be hacked around
short term with a function for a UI to only request events greater than
level X. Longer term, having masks for event handlers would be better.

e) logger.getEffectiveLevel() is no longer a reliable guide to what
will/won't get logged so for now we look at the default log levels instead.

[YOCTO #304]

(Bitbake rev: 45aad2f9647df14bcfa5e755b57e1ddab377939a)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-08-15 09:14:28 +01:00
Chris Larson 96e2ee1d73 Shift exception formatting into the UI
Now we use bb.exceptions to pass pickleable traceback entries to the UI, and
the UI is free to do whatever it wants to do with this information. By
default, the log formatter for the UIs formats it with bb.exceptions.  This
also means that all exceptions should now show 3 lines of context and limit to
5 entries.

(Bitbake rev: ee48d628ee038bd72e1cd94aa75f5ccbacbcee4c)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-06-08 20:38:25 +01:00
Chris Larson 0788081ac7 Enable some DeprecationWarnings
We'll be skipping the Pending Deprecation step given our release process.

(Bitbake rev: 67a55a6b45fec300bea42c18be41cf0a2f931072)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-05-06 18:24:13 +01:00
Richard Purdie 7586adb360 bitbake/msg: Ensure lower level debug messages have DEBUG prefix and reuse log level values from formatter
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-02-07 14:50:27 +00:00
Chris Larson 4428aabc92 msg: no need to hardcode the logging levels
(Bitbake rev: 8385bfb7da3a3b71f340a787d7f1502ba61c5b81)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-01-10 22:10:53 +00:00
Richard Purdie a13352cfaf bitbake/msg.py: Add named definitions for all logging levels
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-01-08 00:01:59 +00:00
Richard Purdie 0090a798eb bitbake: Sync a load of whitespace and other non-functionality changes with bitbake uptream
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:54 +00:00
Chris Larson 760b82e732 Fix logging level names for post-server-ui-split
(Bitbake rev: dc5a5c39291ec223cd761dce59d29eee7316cb70)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:52 +00:00
Chris Larson aaa55880ac build: send logging messages to the log file for python functions
(Bitbake rev: ee1cce6ab21ddda60a7a070d03e98ff8485a5e71)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:48 +00:00
Chris Larson fc27334d1a Drop 'fn' arguments to bb.msg functions
(Bitbake rev: 8341458e3d21b45db84e46bd32f8ad270000ce3c)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:34 +00:00
Chris Larson 55c0b36e36 Add pending deprecation warnings to the bb.msg functions
(Bitbake rev: 60293a42b5500b6139bcd912bf294f862ef9936b)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:34 +00:00
Chris Larson ecc68fa4fb Switch bitbake internals to use logging directly rather than bb.msg
We use a custom Logger subclass for our loggers

This logger provides:
- 'debug' method which accepts a debug level
- 'plain' method which bypasses log formatting
- 'verbose' method which is more detail than info, but less than debug

(Bitbake rev: 3b2c1fe5ca56daebb24073a9dd45723d3efd2a8d)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:33 +00:00
Chris Larson 4855548ffb Use the python logging module under the hood for bb.msg
(Bitbake rev: 47ca82397bc395b598c6b68b24cdee9e0d8a76d8)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:33 +00:00
Richard Purdie 616e75523b bitbake: Correctly route events from the worker to the server
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-08-20 12:26:19 +01:00
Richard Purdie b6bfe14205 bitbake: Switch to use subprocess for forking tasks and FAKEROOTENV to run shell and python under a fakeroot environment
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-08-19 22:36:25 +01:00
Bernhard Reutner-Fischer c4fde248b1 *: use print() as a function
to make python3 happy

(Bitbake rev: c82926ccdd4ec4e3ad6e78a381dacb96adf9b409)

Signed-off-by: Bernhard Reutner-Fischer <rep.dot.nop@gmail.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-07-16 15:10:40 +01:00
Chris Larson 3beecc66e9 Fix the silent exit when the specified task doesn't exist
(Bitbake rev: 2b64afa107dc24db510d0001b6eb58f0d5913d2f)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-07-02 15:41:35 +01:00
Chris Larson ad543e2e41 Apply the 2to3 print function transform
(Bitbake rev: ff2e28d0d9723ccd0e9dd635447b6d889cc9f597)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-07-02 15:41:33 +01:00
Chris Larson 1c74fd768f Import fixups
(Bitbake rev: 4fa052f426e3205ebace713eaa22deddc0420e8a)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-07-02 15:41:32 +01:00
Chris Larson 7acc132cac Formatting cleanups
(Bitbake rev: 2caf134b43a44dad30af4fbe33033b3c58deee57)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-07-02 15:41:32 +01:00
Chris Larson 1569c1ac85 Add a function to get the debug level, without poking into debug_levels manually
(Bitbake rev: 6ccc01a1a72ec354de52fef2bf5620384b8461ce)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-07-02 15:41:32 +01:00
Chris Larson c8498f59ec Fix invalid logging domain warning
(Bitbake rev: 8441fab6f27191fcc3c153a10753871e6aef08b1)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-07-02 15:41:32 +01:00
Chris Larson 350c3126e1 Make the default domain a proper domain, not a string
This lets you do -l Default on the commandline to bump the debug level of
messages with no domain, without bumping the level of everything else.

(Bitbake rev: bef33dfc36713d8c93e38e63ddac2e4f9858e787)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-07-02 15:41:32 +01:00
Chris Larson 943ece8610 Implement bb.msg.domain as a named tuple, drop the Enum class
Also fixes some bb.msg references from within bb.msg.

(Bitbake rev: db95af590f742c8186e84046ad9704fae1733720)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-07-02 15:41:32 +01:00
Chris Larson 146046bcda Immediately display messages if no UI handlers are installed yet
(Bitbake rev: 17c414d0c050c42d4beb3f1dd84573020aacb392)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-07-02 15:41:31 +01:00
Richard Purdie 22c29d8651 bitbake: Switch to bitbake-dev version (bitbake master upstream)
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-01-20 18:46:02 +00:00
Richard Purdie 7530674214 bitbake: Sync with upstream 1.8 branch
git-svn-id: https://svn.o-hand.com/repos/poky/trunk@2497 311d38ba-8fff-0310-9ca6-ca027cbcb966
2007-08-15 08:39:19 +00:00
Richard Purdie bfc70eb24e bitbake: Update along 1.8 branch
git-svn-id: https://svn.o-hand.com/repos/poky/trunk@2345 311d38ba-8fff-0310-9ca6-ca027cbcb966
2007-08-03 13:40:52 +00:00
Richard Purdie 7371e6323c bitbake: Update to 1.8.1 (inc. various bug fixes, epoch support)
git-svn-id: https://svn.o-hand.com/repos/poky/trunk@1419 311d38ba-8fff-0310-9ca6-ca027cbcb966
2007-04-01 15:04:49 +00:00
Richard Purdie f5665d5bfc bitbake: Sync with upstream.
* File licence headers were sanitised causing most of the diff. 
 * cooker.py was created from bin/bitbake. 
 * cvs fetcher port option was added
 * The -f force option was fixed to work correctly
 * Multiple entries in rrecrdeps are now handled correctly
   (allows adding do_deploy to image depends)
 


git-svn-id: https://svn.o-hand.com/repos/poky/trunk@1129 311d38ba-8fff-0310-9ca6-ca027cbcb966
2007-01-08 23:53:01 +00:00
Richard Purdie 306b7c7a97 bitbake: Upgrade from 1.4 -> 1.7.4ish
git-svn-id: https://svn.o-hand.com/repos/poky/trunk@863 311d38ba-8fff-0310-9ca6-ca027cbcb966
2006-11-16 15:02:15 +00:00