Commit Graph

124 Commits

Author SHA1 Message Date
Martin Jansa 9d0962491c bitbake: toasterui, knotty: don't print taskid followed by taskstring which are now in most cases identical
* unify the format how the task is described
* don't show taskid followed by taskstring as the taskstring is
  different only for setscene tasks (by _setscene suffix)
* the duplicated output was introduced by:
  2c88afb   taskdata/runqueue: Rewrite without use of ID indirection
  as reported and confirmed as a bug here:
  http://lists.openembedded.org/pipermail/openembedded-core/2016-June/123148.html
* show:
  NOTE: Running task 541 of 548 (/OE/build/oe-core/openembedded-core/meta/recipes-core/zlib/zlib_1.2.8.bb:do_package)
  instead of much longer:
  NOTE: Running task 541 of 548 (ID: /OE/build/oe-core/openembedded-core/meta/recipes-core/zlib/zlib_1.2.8.bb:do_package, /OE/build/oe-core/openembedded-core/meta/recipes-core/zlib/zlib_1.2.8.bb:do_package)

  and similarly for failed tasks:
  ERROR: Task (virtual:native:/OE/build/oe-core/openembedded-core/meta/recipes-core/zlib/zlib_1.2.8.bb:do_install) failed with exit code '1'
  instead of much longer:
  ERROR: Task virtual:native:/OE/build/oe-core/openembedded-core/meta/recipes-core/zlib/zlib_1.2.8.bb:do_install (virtual:native:/OE/build/oe-core/openembedded-core/meta/recipes-core/zlib/zlib_1.2.8.bb:do_install) failed with exit code '1'

(Bitbake rev: 696693d45f5eff1226866ed79dbfb67161d8cd3f)

Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-08-20 16:08:59 +01:00
Paul Eggleton 8a45291164 bitbake: knotty: don't show number of running tasks in quiet mode
There's not a whole lot of point showing how many tasks are running when
we're in quiet mode, it just looks a bit strange particularly when it's
not running any tasks.

(Bitbake rev: 5317200d9cd73c6f971bc1b0cfe8692749e27e3a)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-08-17 10:22:58 +01:00
Paul Eggleton ea0800049d bitbake: knotty: fix task progress bar not starting at 0%
If we have the task number here we need to subtract 1 to get the number
of tasks completed.

(Bitbake rev: 7c78a1cd3f0638ae76f7c7a469b7f667c7c58090)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-08-17 10:22:58 +01:00
Paul Eggleton 5f4559b2eb bitbake: knotty: don't display ETA for tasks with progress
It turns out that progress information we can extract from a task is
rarely apportioned closely enough to the time taken for the ETA to be
accurate, so showing it is going to be misleading most of the time for
anything but the most basic of examples. Let's just remove it and avoid
misleading (or worse, annoying) the user.

Fixes [YOCTO #9986].

(Bitbake rev: 235db4870b11db97250979e647b54cdb5ce4fbb6)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-26 08:10:35 +01:00
Paul Eggleton 23a551bea1 bitbake: knotty: fix some minor bugs in BBProgress
If you specify custom widgets then we don't want to assume where the
"extra" position is - you should have to specify it, and if it isn't
specified it shouldn't just wipe out the last widget or you can start to
see odd behaviour if you're modifying the code.

(Bitbake rev: 19e33c10feb1637589ceb05b5e8d58b1e012ccb8)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-26 08:10:35 +01:00
Richard Purdie 1b5688e0f0 bitbake: knotty: Handle process indicators more gracefully
Mistakes can happen with the generation of the progress events, change
knotty to be more tolerant of this rather than crashing, reporting to the
user when something unexpected happens. I haven't debugged why multiple
finish events appear to be triggered.

(Bitbake rev: 7dd06b1016b36420a9c55a45ff29dd64ae1dbcda)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-08 09:57:27 +01:00
Paul Eggleton 76feac37ce bitbake: knotty: add quiet output mode
Quiet output mode disables printing most messages (below warnings) to
the console; however these messages still go to the console log file.
This is primarily for cases where bitbake is being launched
interactively from some other process, but where full console output is
not needed.

Because of the need to keep logging all normal events to the console
log, this functionality was implemented within the knotty UI rather
than in bb.msg (where verbose mode is implemented). We don't currently
have a means of registering command line options from the UI end, thus
the option actually has to be registered in main.py regardless of the
UI, however I didn't feel like it was worth setting up such a mechanism
just for this option.

(Bitbake rev: db95cdef08e339dec7462bfde3ad7d75c1c60dd8)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-08 09:57:27 +01:00
Paul Eggleton 465f93968f bitbake: knotty: show task progress bar
In addition to the "currently running n tasks (x of y)" message, show a
progress bar for another view on how much of the build is left. We have
to take care to reset it when moving from the scenequeue to the
runqueue, and explicitly don't include an ETA since not all tasks take
equal time and thus it isn't possible to estimate the time remaining
with the information available.

(Bitbake rev: de682015a3fefeff36ddc4197641a700f3fb558d)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-08 09:57:27 +01:00
Paul Eggleton 8d56d596bb bitbake: knotty: add code to support showing progress for sstate object querying
Add support code on the BitBake side to allow sstate.bbclass in
OpenEmbedded to report progress when it is checking for availability of
artifacts from shared state mirrors.

Part of the implementation for [YOCTO #5853].

(Bitbake rev: 070ae856da0715dbaf4c560c837ea796ffc29f00)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-08 09:57:26 +01:00
Paul Eggleton ac5e720575 bitbake: lib: implement basic task progress support
For long-running tasks where we have some output from the task that
gives us some idea of the progress of the task (such as a percentage
complete), provide the means to scrape the output for that progress
information and show it to the user in the default knotty terminal
output in the form of a progress bar. This is implemented using a new
TaskProgress event as well as some code we can insert to do output
scanning/filtering.

Any task can fire TaskProgress events; however, if you have a shell task
whose output you wish to scan for progress information, you just need to
set the "progress" varflag on the task. This can be set to:
 * "percent" to just look for a number followed by a % sign
 * "percent:<regex>" to specify your own regex matching a percentage
   value (must have a single group which matches the percentage number)
 * "outof:<regex>" to look for the specified regex matching x out of y
   items completed (must have two groups - first group needs to be x,
   second y).
We can potentially extend this in future but this should be a good
start.

Part of the implementation for [YOCTO #5383].

(Bitbake rev: 0d275fc5b6531957a6189069b04074065bb718a0)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-08 09:57:26 +01:00
Paul Eggleton 1cf6e14a6c bitbake: knotty: import latest python-progressbar
Since we're going to make some minor extensions to it, it makes sense to
bring in the latest version of python-progressbar. Its structure has
changed a little but the API hasn't; however we do need to ensure our
overridden _needs_update() function's signature in BBProgress() matches
properly.

(Bitbake rev: c3e51d71b36cbc9e9ed1b35fb93d0978e24bc98a)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-08 09:57:26 +01:00
Paul Eggleton 481048cd2a bitbake: knotty: provide a symlink to the latest console log
If you're looking to find the latest console log repeatedly it can be a bit
tedious - let's just create a symlink just as we do with other logs to
make it easy to find.

(Bitbake rev: e9f41c0507a6527bf2ed86506813d4d4a89f8ebf)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-08 09:57:26 +01:00
Richard Purdie 0f2c59367a bitbake: bitbake: Convert to python 3
Various misc changes to convert bitbake to python3 which don't warrant
separation into separate commits.

(Bitbake rev: d0f904d407f57998419bd9c305ce53e5eaa36b24)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-06-02 08:24:02 +01:00
Richard Purdie 07eebc6689 bitbake: knotty: Fix output buffering issues
We need to flush the footer removal, else it may not be outputted until
the buffer is flushed as part of StreamHandler and this would lead to
it removing the ERROR output just printed which is extremely confusing.

Also ensure the footer is cleared before printing a summary as in
some cases it wasn't being removed, also leading to user confusion.

(Bitbake rev: 0e030c4d074c41859608dab5f3ad26b05f56b306)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-05-17 14:43:31 +01:00
Richard Purdie 7d06ec193e bitbake: knotty: Ensure consolelog file handle is closed
If we don't close the console log file handle, python prints a warning
about unclosed file handles upon exit which is annoying.

(Bitbake rev: 624dd92952b2fc736fd86abe5f2390b87b3a7dd3)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-05-13 13:41:32 +01:00
Richard Purdie 654eadfa30 bitbake: bitbake: Update logger.warn() -> logger.warning()
python deprecated logger.warn() in favour of logger.warning(). This is only
used in bitbake code so we may as well just translate everything to avoid
warnings under python 3. Its safe for python 2.7.

(Bitbake rev: 676a5f592e8507e81b8f748d58acfea7572f8796)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-05-11 10:34:30 +01:00
Richard Purdie 9f374c4e85 bitbake: providers: Add PREFERRED_RPROVIDER support
Sometimes you can end up in a situation where you need to specify that
a specific runtime entity should be provided by a specific entry.

An example of this is bluez where you could end up in a situation where
for example:

NOTE: multiple providers are available for runtime libasound-module-bluez (bluez4, bluez5)
NOTE: consider defining a PREFERRED_PROVIDER entry to match libasound-module-bluez
NOTE: multiple providers are available for runtime bluez-hcidump (bluez-hcidump, bluez5)
NOTE: consider defining a PREFERRED_PROVIDER entry to match bluez-hcidump

The only option here is to set something like PREFERRED_PROVIDER_bluez4 = "bluez4"
which is clearly not very informative.

I've actually held off adding RPROVIDER support for a long while as this
does have sigificant potential for misuse. It doesn't for example allow
multiple runtime providers of the same name to coexist, that simply isn't
supported. It therefore doesn't replace some of the name mappings such
as busybox verses coreutils that OE-Core faces as that is a different
problem with different constraints. This mechanism is simply to provide
bitbake with a hint to decide what the dependency tree should look like.

Also, this allows us to stop printing a confusing message telling the user
to set PREFERRED_PROVIDER when the setting needed would be rather ambiguous.

[YOCTO #5044]

(Bitbake rev: 62eb39d1474d024b204634689071700605c6095c)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-04-15 06:59:44 +01:00
Richard Purdie 8e7282c032 bitbake: cooker/knotty: Prefix parse logs with filename being parsed
We now prefix log messages coming from worker task context with the
PF and task info, however parsing messages all have to be manually
prefixed which is ugly and error prone. This change modifies the log
handler filter so this happens automatically, meaning we don't have
to change every message to include that information. This makes error
messages longer but more usable.

(Bitbake rev: 1af0ccaac81e182c4ca520037dda362d180e5605)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-04-15 06:59:44 +01:00
Christopher Larson a25858982a bitbake: bb.ui.knotty: prefix task messages with recipe/task
An example prefix: `perl-5.22.1-r0 do_compile:`

(Bitbake rev: 792b759e59e31d2e43d525a6e50d866b4f51f072)

Signed-off-by: Christopher Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-16 09:04:23 +00:00
Richard Purdie 8b217208cf bitbake: knotty: Set exit failure code on runQueueTaskFailed events
If the worker segfaults, we may never see a TaskFailed event from it, only
a runQueueTaskFailed event. In this case, return_value isn't getting set
leading to an incorrect exit code from bitbake. Fix by setting return_value
in both places.

(Bitbake rev: e5dd50e0d95d532fe31dde61f8c6b1a7a72321e9)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-15 16:48:12 +00:00
Richard Purdie 602da7c921 bitbake: knotty: Don't show errors for universe provider issues
When running universe builds, we don't expect an error exit code for
provider warnings. Change the error messages to warnings in this case.

This deals with errors causing problems on our autobuilders amongst
other issues.

(Bitbake rev: d4989fb0355476de172169f0698757f7360e9a1f)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-01 11:20:42 +00:00
Richard Purdie 5375e6431c bitbake: bitbake: Set process names to be meaninful
This means that when you view the process tree, the processes
have meaningful names, aiding debugging:

$ pstree -p 30021
bash(30021)───KnottyUI(115579)───Cooker(115590)─┬─PRServ(115592)───{PRServ Handler}(115593)
                                                ├─Worker(115630)───bash:sleep(115631)───run.do_sleep.11(115633)───sleep(115634)
                                                └─{ProcessEQueue}(115591)

$ pstree -p 30021
bash(30021)───KnottyUI(117319)───Cooker(117330)─┬─Cooker(117335)
                                                ├─PRServ(117332)───{PRServ Handler}(117333)
                                                ├─Parser-1:2(117336)
                                                └─{ProcessEQueue}(117331)

Applies to parse threads, PR Server, cooker, the workers and execution
threads, working within the 16 character limit as best we can.

Needed to tweak the bitbake-worker magic values to tell the
workers apart.

(Bitbake rev: 539726a3b2202249a3f148d99e08909cb61902a5)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-30 11:43:56 +00:00
Richard Purdie 46d62d076d bitbake: knotty: Use non-interactive mode as fallback for dumb terminals
TERM=dumb bitbake X

shows no output for task status which is suboptimal. Use the non-interactive
mode if the terminal doesn't support what we need for interactive mode giving
a better user experience. Also print a note to the console to say this has
happened.

[YOCTO #8768]

(Bitbake rev: 6f84cf4bd77f35fcd07e0b2f5149f1d6866a414d)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-08 13:58:08 +00:00
Richard Purdie 522dcaa0cb bitbake: knotty: Improve exception error message
Instead of:
"""
can only concatenate tuple (not "int") to tuple
"""
we now see:

"""
Traceback (most recent call last):
  File "/media/build1/poky/bitbake/lib/bb/ui/knotty.py", line 324, in main
    termfilter.updateFooter()
  File "/media/build1/poky/bitbake/lib/bb/ui/knotty.py", line 210, in updateFooter
    lines = 1 + int(len(content) / (self.columns + 1))
TypeError: can only concatenate tuple (not "int") to tuple
"""

which makes tacking down and fixing the problem much easier.

Also ensure we set an error exit code.

(Bitbake rev: d965bcae6cfd268406a3bd1ef77c5bb6c6e1c6d7)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-18 12:18:19 +00:00
Richard Purdie 01d67bfa77 bitbake: knotty: Fix row/column function return value issue
When the row handling was introduced, one of the callbacks was
missed resulting in:

TypeError: can only concatenate tuple (not "int") to tuple

Fix it.

(Bitbake rev: 0b77cea2bf5b5f5704e2650fb0332f5d78037781)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-18 12:18:19 +00:00
Richard Purdie f9384b0669 bitbake: knotty: Enforce terminal line limit to stop crazy scrolling
If there are more tasks running than there are lines on the terminal, the data
scrolls in ways the UI wasn't designed for. This patch adjusts the UI just to
show the processes which fit onto the number of rows in the terminal window.
You can see the total number running from the counter in the top left as usual
and this makes warning and errors messages scrolling from the top of the window
work as designed.

Ultimately, scrolling would be nice but is for another time, this fixes the
biggest UI issue on highly parallel machines.

(Bitbake rev: 67b77658e2bfa849f6f55c9c262cb11d6bfdb399)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-08 10:31:48 +00:00
Richard Purdie 5b19b71252 bitbake: bb/ui: Use getSetVariable command for BB_CONSOLELOG
Metadata can define BB_CONSOLELOG as containing ${DATETIME} and
this can get expanded to a different value each time the variable
is read. In the case of BB_CONSOLELOG, this behaviour is not
desireable.

The values of DATE/TIME are locked down at build time but this is too
late for the purposes of ensuring the system can figure out the real
value of BB_CONSOLELOG.

The best way to do this is to set the variable into the datastore, thereby
preserving its value.

[YOCTO #8411]

(Bitbake rev: 021f2eb55ab5863b57ed1b3f19f1b329bc1ad477)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-10-29 07:35:51 +00:00
Richard Purdie 4c2f28bf48 bitbake: cooker: properly fix bitbake.lock handling
If the PR server or indeed any other child process takes some time to
exit (which it sometimes does when saving its database), it can end up
holding bitbake.lock after the UI exits, which led to errors if you ran
bitbake commands successively - we saw this when running the PR server
oe-selftest tests in OE-Core. The recent attempt to fix this wasn't
quite right and ended up breaking memory resident bitbake. This time we
close the lock file when cooker shuts down (inside the UI process)
instead of unlocking it, and this is done in the cooker code rather than
the actual UI code so it doesn't matter which UI is in use. Additionally
we report that we're waiting for the lock to be released, using lsof or
fuser if available to list the processes with the lock open.

The 'magic' in the locking is due to all spawned subprocesses of bitbake
holding an open file descriptor to the bitbake.lock. It is automatically
unlocked when all those fds close the file (as all the processes terminate).
We close the UI copy of the lock explicitly, then close the server process
copy, any remaining open copy is therefore some proess exiting.

(The reproducer for the problem is to set PRSERV_HOST = "localhost:0"
and add a call to time.sleep(20) after self.server_close() in
lib/prserv/serv.py, then run "bitbake -p; bitbake -p" ).

Cleanup work done by Paul Eggleton <paul.eggleton@linux.intel.com>.

This reverts bitbake commit 69ecd15aece54753154950c55d7af42f85ad8606 and
e97a9f1528d77503b5c93e48e3de9933fbb9f3cd.

(Bitbake rev: a29780bd43f74b7326fe788dbd65177b86806fcf)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-01 07:34:09 +01:00
Leonardo Sandoval 3b5b74b430 bitbake: knotty.py: Make sure bitbake.lock is unlocked before exiting
Before exiting the UI, unlocks the bitbake.lock owned by cooker; this
way consecutive bitbake executions can lock it again without trouble.

[Yocto #7941]

(Bitbake rev: 69ecd15aece54753154950c55d7af42f85ad8606)

Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-09 18:00:21 +01:00
Robert Yang 0308f3101b bitbake: knotty.py: fix indent
It used 5 spaces as the indent.

(Bitbake rev: 162d35ed53d34b28b153adf643044e7f105fcff1)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-03-19 14:20:14 +00:00
Rob Woolley 855fc21ae9 bitbake: knotty: Do not log show_versions output
Every time the bitbake show versions command (bitbake -s) is run it creates
a 100k log file.

The consolelogfile is disabled for show environment and disabling show
versions would make the behaviour match.

(Bitbake rev: dee0ba94e39ea49c1e9261a5e8932356e3bb7c57)

Signed-off-by: Rob Woolley <rob.woolley@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-03-16 17:41:28 +00:00
Rob Woolley 0b5ab4d987 bitbake: knotty: Catch exceptions on broken pipes
Any exceptions that occur in calls to logging methods are automatically
suppressed, including exceptions due to broken pipes.

However, the knotty summary messages are printed directly to stdout, which
means that any broken pipes will cause an exception traceback in python.

By wrapping the summary section in a try / catch block we can check for
IOError exceptions caused by broken pipes and let them pass.

(Bitbake rev: 146e7e157f97b676858ecff583dd53800d997253)

Signed-off-by: Rob Woolley <rob.woolley@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-03-16 17:41:28 +00:00
Richard Purdie db45ddeeaf bitbake: command/cooker/knotty: Fix memres handling of command environment changes
If the environment changes, we need memory resident bitbake to adapt to those
changes. This adds in functionality to handle this alongside the configuration
option handling code. This means that the common usage:

MACHINE=X bitbake Y

now works with the memory resident server.

(Bitbake rev: 4d1343010da757a0c126bc22475354da44aaf8e3)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-01-21 14:37:39 +00:00
Richard Purdie ff5fba8462 bitbake: knotty: Ensure commandline parameters are updated in memres server
When using options like -k, -f, -v and so on with the memory resident
server, they'd currently only be set on the initial values passed to
the original command. This ensures they now match those specified
on the commandline for the options where this makes sense.

To make this work, a command to update the options on the server side
is required so this is added.

[YOCTO #5292]

(Bitbake rev: 1c75cc4d0c8b606c1fe76e6bf60bf6a32298b105)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-09-23 20:06:06 +01:00
Dan McGregor 50a8400b72 bitbake: knotty: ignore interrupted system calls
With the improved exception handling added in an earlier commit bitbake
now stops when recieving a SIGWINCH. This happens frequently when
disconnecting and reconnecting tmux sessions and bitbake didn't survive.
Restore old behaviour of ignoring interrupted system calls but keep
proper exception handling for other errors.

(Bitbake rev: 418358a595c75f45b8d15160ec42bbe569562d91)

Signed-off-by: Dan McGregor <dan.mcgregor@usask.ca>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-09-01 08:51:33 +01:00
Richard Purdie a4f077b1fe bitbake: knotty: Improve exception handling
Currently, IOErrors are just passed over due to the broken Exception
clause. A command like "bitbake X | <invalid command>" would break stdout
triggering a traceback. With these changes we print the exceptions, shut down
the server gracefully and exit which is a much nicer behaviour and is less
confusion to the user.

(Bitbake rev: 9544108f7b413038d871ce6ca88232de2f2434d9)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-08-23 09:31:42 +01:00
Namhyung Kim 70d0316d57 bitbake: knotty: Move second event check into a proper block
so that it doesn't check it twice for non-empty events

(Bitbake rev: e304ace6d8f7d0808ee401b7c01146b2798a81b4)

Signed-off-by: Namhyung Kim <namhyung@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-08-02 09:57:11 +01:00
Richard Purdie c0bdb8100f bitbake: knotty: Fix output printed to the user at debug log levels
If a user runs with the -v or -D options, its understandable they'd
expect to see log output from the workers yet right now a bug in the
log handling does not show this.

Fix the conditional to ensure such log output is shown on the terminal
when it has been requested. Ideally this data should always flow to
the logfiles but that is for another patch.

This also fixes the code to do what was always intended in the comments,
i.e. if the user specifies -D or -v, output is shown from the tasks,
otherwise notes are suppressed.

(Bitbake rev: 20a3c93d8572969e76563f29bff89400b93ffae7)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-05-11 13:15:12 +01:00
Peter Kjellerstedt 8f683bf782 bitbake: bitbake: knotty: Clear footer before outputting to stderr
With the recent change to split the log output to stdout and stderr,
error messages that appeared while the footer was printed got all
messed up. This was because the messages to stderr was output _after_
the footer, then clearFooter() tried to remove the footer but removed
the error message and parts of the footer.

(Bitbake rev: 4fafea4fa69542b491e84463f6eae0d5bf645673)

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-04-08 18:05:26 +01:00
Richard Purdie c4eeaa8e35 bitbake: knotty: Show a link to the logfile for failed setscene tasks
Its not immediately obvious to the user that a logfile exists for a failed setscene
task. Add code to knotty to display where that logfile is in those cases.

[YOCTO #6055]

(Bitbake rev: 0664fa15597785dd90cf205531a9801e6da6ba47)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-03-27 10:42:08 +00:00
Richard Purdie 6ef47ec5fd bitbake: knotty: Split error output onto stderr
When using bitbake -e in scripts, it would be helpful if the error
output appeared on stderr, not stdout. This change enables that building
upon the new bb.msg filters.

[YOCTO #5274]

(Bitbake rev: ebb797fc5c37d729e3cc8b2dc7156287d385c13b)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-03-27 10:17:20 +00:00
Richard Purdie 6f29f7d371 bitbake: knotty: Ensure the progress bar shows on stdout
I can understand why some programs would want the progress on stderr so
that real output can be captured on stdout. This is confusing for bitbake
since we don't show a progress bar at all in non-interactive cases.

Therefore make sure the progress bar goes to stdout, not the stderr default.

(Bitbake rev: 0529aa9966df5c56b07affe865efce18852efe5a)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-03-27 10:17:20 +00:00
Richard Purdie 8c1e43ca58 bitbake: cooker/event: Overhaul sanity test mechanism
Sanity tests are currently a pain as its hard to control when they run. This results
in issues where for example the bitbake -e output is not useful as the sanity tests
prevent it from executing. The sanity tests should run later than the base configuration.

This patch changes the sanity tests to always be event triggered with the option of
returning either events on the status, or raising errors. A new cooker feature is used
to change the behaviour depending on the controlling UI.

This does need a change to sanity.bbclass in the OE metadata but its worth the pain
for the increased flexibility and control this offers UIs and the improvement to the
user experience.

(Bitbake rev: 32e171bcc92c6e27fefee971e8714ddf8e1a8ac1)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-03-27 09:42:08 +00:00
Richard Purdie a445e03d8b bitbake: knotty: Remove latency when exiting
There is no point in waiting 0.25s for when we should be processing
the shutdown. This simply reordering removes latency from the
bitbake command.

(Bitbake rev: f147b41bcaf9d05b5ba3a70100f1ca799979aee7)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-03-10 11:10:01 -07:00
Richard Purdie 75b9f93638 bitbake: knotty: Add missing continue statement for runQueueExitWait event
The continue statement was missing for this event and the event was then
listed in the "known safe to ignore list". Clean this up.

(Bitbake rev: c4ee342300bf905e6e3bef581c61b86289461536)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-03-10 11:10:00 -07:00
Richard Purdie a7bc031f84 bitbake: knotty: Deal with exceptions not resetting terminal configuration
When an exception occurred, the terminal parameters (such as echo)
may not be reset correctly. This change ensures they do get
atexit time in all cases, avoiding the terminal corruption issues
that could sometimes occur.

(Bitbake rev: e1d89166f2dfe46412ff9a5610dd57b0cef74fe3)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-03-10 11:09:58 -07:00
Alexandru DAMIAN 24ae636fde bitbake: bitbake: event: adding generic event for metadata usage
Adding the generic bb.event.MetadataEvent that is
targeted specifically at metadata usage. This is
needed in order to let the metadata code send and receive
events during asynchrous execution without having
to define each event specifically in Bitbake.

Metadata code should subscribe to and fire the MetadataEvent
in order to communicate asynchronously, and identify
the object using event.type field, and parse the
data in the event.data field.

Knotty UI will ignore these event by default.

This deprecates RequestPackageInfo/PackageInfo, and that
event pair will be removed in the future.

(Bitbake rev: ae1ea51aaab73e010d1c3db39df058bebebc11dd)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-09-22 12:19:44 +01:00
Alexandru DAMIAN ba83eb315d bitbake: bitbake: cooker,xmlrpc,servers: implement CookerFeatures
Implementing feature set selection that allows a client
to enable specific features in the server at connection time.

Only enabling of features is supported, as there is
no way to safely remove data loaded into the cooker.
Once enabled, a feature will remain enabled for the
life of the cooker.

Client-server connection now supports specifying the feature
set required by the client. This is implemented in the Process
server using a managed proxy list, so the server cooker
will now load dynamically needed features based on what client
connects to it.

In the XMLRPC server the feature set is requested by
using a parameter for registerUIHandler function.
This allows observer-only clients to also specify features
for the server.

The server code configuration now is completly separated
from the client code. All hardcoding of client knowledge is
removed from the server.

The extra_caches is removed as the client can now specify
the caches it needs using the feature. The UI modules
now need to specify the desired featureSet. HOB is modified
to conform to the featureSet specification.

The only feature available is CookerFeatures.HOB_EXTRA_CACHES
which forces loading the bb.cache_extra:HobRecipeInfo class.

(Bitbake rev: 98e594837aab89ea042cfa9f3740d20a661b14e2)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-09-22 12:19:43 +01:00
Richard Purdie 926b60f6e4 bitbake: knotty: Cleanup error/interruption handling
Only display a CommandFailed ERROR: message if there is an error to display.

Only display an errors summary if we actually displayed errors.

(Bitbake rev: 568ea00acd226d48e725bb01d4f8c410ed1eaa61)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-09-14 08:20:38 +01:00
Richard Purdie 7663a52061 bitbake: cooker: Rename confusing 'stop' state to 'forceshutdown'
The shutdown state causes the server to finish what its doing, stop was
them meant to completely stop it. It doesn't mean the server is stopped
though. Renaming the current stop event for forceshutdown gives more
meaning to what it actually does. The stopped namespace then becomes
available to indicate a completely stopped server.

(Bitbake rev: 12e9d33bfae5294e3870dfd1202f63383ad05e92)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-09-14 08:20:38 +01:00