Rewritten toaster-eventreplay to make code working as expected,
more compact and readable.
[YOCTO #9585]
(Bitbake rev: 45370a860b24a761d1b6e08ba752079cc45f54da)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
class MockConfigParameters has only one attribute and only __init__
method. Replacing it with namedtuple makes code less nested and more
readable.
[YOCTO #9585]
(Bitbake rev: 5d4df14b0d38f6c89ca16de6dada58b4bb015d71)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Event objects are represented by base64-encoded strings in
the event file and can't be loaded by existing eventreplay code.
Fixed the code of loading events from file by decoding base64 strings
into the binary form and loading them with pickle.load.
[YOCTO #9585]
(Bitbake rev: a55c280c167f84caed6518119246e5a55f56cfd4)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Stored event mask list as self.eventmask for future use.
Fixed Exception: Command setEventMask not implemented.
[YOCTO #9585]
(Bitbake rev: 2e09074c70e89402de8f883dd402cd729118fc7e)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fixed AttributeError: 'MockConnection' object has no attribute
'getEventHandle'
[YOCTO #9585]
(Bitbake rev: 6deae30480a4288da0c8b3529e61f3495f260f24)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Command-line builds were displayed incorrectly, so that the
HTML elements for other builds were being "consumed" by the
command-line build elements due to incorrect positioning of
element end tags.
Fix by ensuring end tags close elements in the right places.
As the indentation was all over the place in the most recent
builds section template, it was almost impossible to see what the
problem was. So that was fixed, too.
[YOCTO #9842]
(Bitbake rev: 01659389813ad61d4f75b9f8d71528581322f0b0)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Worker needs input stream in binary mode as it reads binary content
from it. Current code does it by detaching a buffer from sys.stdin
and assigning it back to sys.stdin.
Detached buffer is io.BufferedReader in binary mode. This operation
is implicit as its purpose is not easily understandable from the code.
Replacing it with fdopen(sys.stdin.fileno(), 'rb') should make the
code more understandable.
Assigning the buffer to sys.stdin is not needed as worker doesn't
use sys.stdin. Moreover, it leads to difficult to debug issues down
the stack. For example, devpyshell doesn't work without reopening
sys.stdin in text mode. This is not needed anymore after this fix as
sys.stdin is not changed in worker code and remains in text mode.
(Bitbake rev: b26bcff4c4d72775f1def7e769015464953b955c)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Clean up and fix the most frequently built targets for the "Most built
recipes" section for the project configuration page.
[YOCTO #9846]
(Bitbake rev: 860475cfdd35301fb609ab3c89347566b0ca0adc)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Some pages contain other build buttons which may have the same class
attached. Make sure that we only select the buttons in the tables where
layerBtn is used.
[YOCTO #9841]
(Bitbake rev: ec083fd943c6996307beb3be3421403870d2f2b7)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add basic tests to validate the value, user types
in the text box for DL_DIR and SSTATE_DIR. Added
test case to validate the first char and inclusion
of space between the characters.
[YOCTO #9646]
(Bitbake rev: 1531e98c5ae1693d11d692c3589df29dff9364df)
Signed-off-by: Sujith H <sujith.h@gmail.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Validation logic on the project configuration page
prevented a user from adding ${variable} to these paths.
Update validation so a user can see a better message
when they type characters into the text inputs.
Two types of validation are implemented.
Either:
* The value should start with a "/", to allow absolute paths.
or
* The value should start with a "$", to allow bitbake
variables like ${TOPDIR}.
[YOCTO #9646]
(Bitbake rev: fca50153caef62a473d1812fd59ec7effe09f8c5)
Signed-off-by: Sujith H <sujith.h@gmail.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Remove SSTATE_MIRRORS from getting blacklisted. Hence the SSTATE_MIRRORS
should be removed from projectconf.html file so that users can know that
this variable is no more blacklisted.
[YOCTO #9598]
(Bitbake rev: b060eecae417521d8340748e7ace7a77488bd596)
Signed-off-by: Sujith H <sujith.h@gmail.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Remove SSTATE_MIRRORS variable from blacklists in views.py
This helps user to point mirrors using SSTATE_MIRRORS with
toaster.
[#YOCTO 9598]
(Bitbake rev: c3dd4fb193c80ed9a6dbddf7f6a1eadd0964ac8c)
Signed-off-by: Sujith H <sujith.h@gmail.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The init function of the parent class fires a progress event for 0
progress rather than a start event. UI code was assuming that progress
events should always have a start event first. This change ensures that
the start event is correctly generated.
This fixes crashes that were seen in knotty in some configurations.
(Bitbake rev: 9841651e050a3e9f395ab3c62545c51197734584)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Mistakes can happen with the generation of the progress events, change
knotty to be more tolerant of this rather than crashing, reporting to the
user when something unexpected happens. I haven't debugged why multiple
finish events appear to be triggered.
(Bitbake rev: 7dd06b1016b36420a9c55a45ff29dd64ae1dbcda)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When "Preparing RunQueue" shows up you can expect to wait up to 30
seconds while it works - which is a bit long to leave the user waiting
without any kind of output. Since the work being carried out during this
time is divided into stages such that it's practical to determine
internally how it's progressing, replace the message with a progress
bar.
Actually what happens during this time is two major steps rather than
just one - the runqueue preparation itself, followed by the
initialisation prior to running setscene tasks. I elected to have the
progress bar cover both as one (there doesn't appear to be much point in
doing otherwise from a user perspective). I did however describe it as
"initialising tasks".
(Bitbake rev: 591e9741e108487ff437e77cb439ef2dbca42e03)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add the ability to enter a mode where only a specified whitelist of
tasks can be executed outright; everything else must be successfully
provided in the form of a setscene task (or covered by a setscene task).
Any setscene failure outside of the whitelist will cause the build to
fail immediately instead of running the real task, and any real tasks
that would execute outside of the whitelist cause an immediate build
failure when it comes to executing the runqueue as well.
The mode is enabled by setting BB_SETSCENE_ENFORCE="1", and the
whitelist is specified through BB_SETSCENE_ENFORCE_WHITELIST, consisting
of pn:taskname pairs. A single % character can be substituted for the pn
value to match any target explicitly specified on the bitbake command
line. Wildcards * and ? can also be used as per standard unix file name
matching for both pn and taskname.
Part of the implementation of [YOCTO #9367].
(Bitbake rev: 624722c067a7fdd0c0f5d8be611e1f6666ecc4a2)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Quiet output mode disables printing most messages (below warnings) to
the console; however these messages still go to the console log file.
This is primarily for cases where bitbake is being launched
interactively from some other process, but where full console output is
not needed.
Because of the need to keep logging all normal events to the console
log, this functionality was implemented within the knotty UI rather
than in bb.msg (where verbose mode is implemented). We don't currently
have a means of registering command line options from the UI end, thus
the option actually has to be registered in main.py regardless of the
UI, however I didn't feel like it was worth setting up such a mechanism
just for this option.
(Bitbake rev: db95cdef08e339dec7462bfde3ad7d75c1c60dd8)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In addition to the "currently running n tasks (x of y)" message, show a
progress bar for another view on how much of the build is left. We have
to take care to reset it when moving from the scenequeue to the
runqueue, and explicitly don't include an ETA since not all tasks take
equal time and thus it isn't possible to estimate the time remaining
with the information available.
(Bitbake rev: de682015a3fefeff36ddc4197641a700f3fb558d)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add support code on the BitBake side to allow sstate.bbclass in
OpenEmbedded to report progress when it is checking for availability of
artifacts from shared state mirrors.
Part of the implementation for [YOCTO #5853].
(Bitbake rev: 070ae856da0715dbaf4c560c837ea796ffc29f00)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a class to help report progress in a task that consists of multiple
stages, some of which may have internal progress (do_rootfs within
OpenEmbedded is one example). Each stage is weighted to try to give
a reasonable representation of progress over time.
Part of the implementation for [YOCTO #5383].
(Bitbake rev: 751b75602872a89e8b1a7c03269bc0fdaa149c6f)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
For long-running tasks where we have some output from the task that
gives us some idea of the progress of the task (such as a percentage
complete), provide the means to scrape the output for that progress
information and show it to the user in the default knotty terminal
output in the form of a progress bar. This is implemented using a new
TaskProgress event as well as some code we can insert to do output
scanning/filtering.
Any task can fire TaskProgress events; however, if you have a shell task
whose output you wish to scan for progress information, you just need to
set the "progress" varflag on the task. This can be set to:
* "percent" to just look for a number followed by a % sign
* "percent:<regex>" to specify your own regex matching a percentage
value (must have a single group which matches the percentage number)
* "outof:<regex>" to look for the specified regex matching x out of y
items completed (must have two groups - first group needs to be x,
second y).
We can potentially extend this in future but this should be a good
start.
Part of the implementation for [YOCTO #5383].
(Bitbake rev: 0d275fc5b6531957a6189069b04074065bb718a0)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Since we're going to make some minor extensions to it, it makes sense to
bring in the latest version of python-progressbar. Its structure has
changed a little but the API hasn't; however we do need to ensure our
overridden _needs_update() function's signature in BBProgress() matches
properly.
(Bitbake rev: c3e51d71b36cbc9e9ed1b35fb93d0978e24bc98a)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If you're looking to find the latest console log repeatedly it can be a bit
tedious - let's just create a symlink just as we do with other logs to
make it easy to find.
(Bitbake rev: e9f41c0507a6527bf2ed86506813d4d4a89f8ebf)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Some services such as SourceForge seem to struggle to keep up under load, with
the result that over half of the autobuilder checkuri runs fail with
sourceforge.net "connection timed out".
Attempt to mitigate this by re-attempting once the network operation on failure.
(Bitbake rev: 54b1961551511948e0cbd2ac39f19b39b9cee568)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The progress patches change the output slightly, update the test to
deal with this.
(From OE-Core rev: 90dbd838fa97c89ace5cb147aa5cff39b94178b1)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The rootfs can be addressed also by referring to the PartUUID
value from the GPT.
This patch enables such type of reference.
(From OE-Core rev: 1ab2ca141d3defe4b80212e28ac7c3f2271e2515)
Signed-off-by: Igor Stoppa <igor.stoppa@intel.com>
Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
On some hardware platforms (Gigabyte, qemu), detection of USB devices
by the kernel is slow enough such that it happens only after the first
attempt to mount the rootfs. We need to keep trying for a while
(default: 5s seconds, controlled by roottimeout=<seconds>) and sleep
between each attempt (default: one second, rootdelay=<seconds>).
This change intentionally splits finding the rootfs (in the new
"rootfs") and switching to it ("finish"). That is needed to keep udev
running while waiting for the rootfs, because it shuts down before
"finish" starts. It is also the direction that was discussed on the OE
mailing list for future changes to initramfs-framework (like
supporting a "live CD" module, which would replace or further augment
mounting of the rootfs).
(From OE-Core rev: 2a50bb9ee8838e3d026c82dc09aaccb880a264f4)
Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This was a copy-and-paste of subprocess.check_output() from when we supported
Python <2.7, so simply delete it and use subprocess.check_output() instead.
(From OE-Core rev: b1f2d9ed8d4dc89c9e669f43f546463ccc2a76b9)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There is a typo in the logging parameters, "filname" is being used instead of "filename" for yocto-kernel, yocto-layer and wic scripts. This didn't cause issues before since python 2 didn't validate unused parameters but with python >= 3.4.3 an exception is thrown. This patch fixes this parameter name.
[YOCTO #9834]
(From OE-Core rev: 5d123a6ec0e97652b2ec4295428797a336ef357a)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
convert python2 based ptest of rt-tests package to python3
(From OE-Core rev: 2767cbe014f84c9ad42ed8ef1eca7caa862e5196)
Signed-off-by: Maxin B. John <maxin.john@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When using -m32 -mmusl options in this order, gcc hangs
in parsing the options decode_cmdline_options_to_array()
the reason is that we have broken the link when adding
mmusl options, the order of specifying libc was not kept
in order as a result it was unable to contruct the array
correctly and ended in parse hang.
We fix the options to specify the order properly.
(From OE-Core rev: b362cf6ef6d7fa22a525cf3e1d17943e897bd8aa)
Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(From OE-Core rev: 92cfdb2ba7e04e2b70986c6569f500dd2a48b5d1)
Signed-off-by: Michael Davis <michael.davis@essvote.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When detecting the second serial options we shouldn't append the
custom QEMU options otherwise we will end duplicating those.
(From OE-Core rev: 79798f20b2c0b98d84c3c4b14600635ff8ddfdad)
Signed-off-by: Otavio Salvador <otavio@ossystems.com.br>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
It fails to compile libpfm4 with misleading-indentation error:
| syst_count.c:346:3: error: this 'for' clause does not guard... [-Werror=misleading-indentation]
| for(c=cmin ; c < cmax; c++)
| ^~~
Add clauses to fix it.
(From OE-Core rev: 2857658ff5e477cb436cf2778a78fa9f192caab8)
Signed-off-by: Kai Kang <kai.kang@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Tasks for image recipes cannot be locked - there's nothing to restore
from shared state to cover them and as a result, if you had "live" in
IMAGE_FSTYPES the build would fail with "taskhash mismatch" errors for
do_rootfs and do_image_complete for the initramfs image recipe, since it
had to try to run those. We should probably catch that issue earlier in
the build and produce a proper error, but for now at least exclude these
signatures from the locked-sigs.inc file so that extensible SDK
installers built when IMAGE_FSTYPES includes "live". (It turned out we
already had code to find other image tasks in the task list in order to
generate the list of install targets.)
Follow-up fix for [YOCTO #9826].
(From OE-Core rev: a7133bf6bb650b944d29d01129f36a56282acd2b)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If you build an extensible SDK for an image and IMAGE_FSTYPES includes
"live" then the extensible SDK will fail to install with a bunch of
unexpected task execution errors, matching the missing items required to
build the live image. The issue was we were still depending on do_rootfs
rather than do_image_complete. The fix was slightly more complicated
than just changing the task name as do_image_complete's dependencies are
in the form of dependencies on tasks within the same recipe (represented
in the "deps" varflag rather than the "depends" varflag).
Fixes [YOCTO #9826].
(From OE-Core rev: 2b9c092e89b421bf7fd6a7c9604a83ae420d85ba)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Code cleanup, no functional changes - this code was never used.
(From OE-Core rev: 397b76c7f26e38e761b94b1f7987aafd55048e10)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We're opening source files with the default encoding (utf-8) but we
can't necessarily be sure that they are UTF-8 clean - for example,
recipetool create ftp://mama.indstate.edu/linux/tree/tree-1.7.0.tgz
prior to this patch resulted in a UnicodeDecodeError. Use the
"surrogateescape" mode to avoid this.
Fixes [YOCTO #9822].
(From OE-Core rev: 50fcd9d1b9a20d49bc873467a82a071f2f2f8b5a)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
do_configure for autogen-native would fail if build directory path
contains '-I' characters, which is caused by the unsuitable sed script
when processing libguile.
Error log:
[snip]
checking for GUILE... yes
awk: fatal: cannot open file `/buildarea1/poky/build /libguile/version.h' for reading (No such file or directory)
configure: error: in `/buildarea1/poky/build-Ice/tmp/work/x86_64-linux/autogen-native/5.18.10-r0/build':
configure: error: cannot determine Guile version
[snip]
(From OE-Core rev: 2004307044e958cdf508b72f180e238a3e297179)
Signed-off-by: Zhenbo Gao <zhenbo.gao@windriver.com>
Signed-off-by: Yi Zhao <yi.zhao@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We want world builds without GTK3DISTROFEATURES to work.
(From OE-Core rev: 0b2945588f7b539836369895dc7d3d2629abb431)
Signed-off-by: Joe Slater <jslater@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Make gettext-native in 0.16.1 recipe provides virtual/gettext-native like
what gettext-native in 0.19.6 recipe does. Otherwise we would fail to start
to do a world build if gettext and gettext-native are set to the low version.
Error message is like below.
ERROR: Multiple versions of gettext-native are due to be built
(From OE-Core rev: a0bdbd8f38d8d2ca0687f4353b715087f946247a)
Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
While the file in question is a command line utility
only, let's be clean and still fix it.
(From OE-Core rev: 40636c1ef853be01541b89d9b6f1321bbeefe8f3)
Signed-off-by: André Draszik <adraszik@tycoint.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Enforce the correct tag names across all of oe-core for consistency.
(From OE-Core rev: 606a43dc38a00cc243f933722db657aea4129f8e)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Ed Bartosh
Elliot Smith
Michael Wood
Sujith H
Richard Purdie
Paul Eggleton
Ross Burton
Christopher Larson
Igor Stoppa
Patrick Ohly
Humberto Ibarra
Maxin B. John
Khem Raj
Davis, Michael
Zhenhua Luo
Otavio Salvador
Kai Kang
Zhenbo Gao
Joe Slater
Chen Qi
André Draszik