In some cases there is a need to fire bb events into multiple
python threads so locking is needed (writing to a fd/socket).
Adding a helper functions for disable/enable by request to avoid
overhead.
[YOCTO #10330]
(Bitbake rev: a583dc0b296415ec904c081c4de96ceef46732a8)
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The value used for exclusion was always being expanded. This is actually
a bad idea since in most cases you'd want to exclude an unexpanded
value and makes it impossible to use the variable as intended.
This adjusts things so the value is not expanded and we can correctly
remove things from checksums much more easily.
(Bitbake rev: 81bc8201c475d2b6bef0168573915ad0140f6dad)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Error message
ERROR: Unprocessed MetadataEvent <bb.event.MetadataEvent object at 0x7f750e671a58>
doesn't give a lot of information about the event. It just prints
event object, which is always bb.event.MetadataEvent.
Including event type into the error message should make it more
informative:
ERROR: Unprocessed MetadataEvent TaskArtifacts
(Bitbake rev: 603c7c13536d3fa1786270e863688c1d2e511196)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
New MetadataEvent 'TaskArtifacts' causes this error.
Processing of this event will hopefully be implemented in future.
For now it should be enough to just skip it.
(Bitbake rev: 114a3fe3f23ef09782c5aa18f425d0d0dbdfdd35)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Due to the bug in processing EnvironmentError exception,
toasterui ignores it. As EnvironmentError is a base for OSError
and IOError this means that all OSError and IOError exceptions
were silently ignored.
(Bitbake rev: c8f4ca008bf9396b0ed45d44bfe2220c82a614a9)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Buildinfohelper assumes that all files mentioned in
manifest exist in deploy/ directory, which is not always
the case. Toaster crashes with OSError trying to
call os.stat on non-existing file.
Checking if file exists before processing it should
fix this.
[YOCTO #10185]
(Bitbake rev: 54565e7ca84d2722a2454e7fa52cda564b28b527)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Setting OEROOT in toaster script makes oe-init-build-env to
break with error:
bash: ../bitbake/bin/../../scripts/oe-buildenv-internal: No such file or directory
This happens because OEROOT contains path relative to build
directory.
Renamed OEROOT to OE_ROOT and unset it after it's used.
(Bitbake rev: 3b0967b160dc6123ef75e6f378221347bd923f1b)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Update the implementation of delete an imported layer so that it is
consistent with the other delete messages and wording. Also use the new
libtoaster way of setting a notification that the delete was successful.
(Bitbake rev: 0b8d3ac48b5a0984963d664ff5630e3b02c4ecd1)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Use the simpler libtoaster method of showing a notification about
successful import of a layer.
Also a number of whitespace clean ups.
(Bitbake rev: 89d3acbc32eadd2acf90030d8b9703ce193dff0c)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add front end modal and controls for deleting a build from the build
dashboard.
Also convert the Actions list to links instead of buttons as per the
design.
[YOCTO #6238]
(Bitbake rev: 93bca6d877e0b2b5b8ef6b27288c0987a6c899b1)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(Bitbake rev: cdc380c188fd17e55d1d270e5b468d931aa436b2)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Make sure that when we spawn a modal we clear any notifications and also
make sure that old notifications are cleared before showing a new one.
(Bitbake rev: c7f30a673ab973a2500092d2e981a47da05fbf12)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add confirm modal and api calls to delete a project from the project
dashboard.
[YOCTO #6238]
(Bitbake rev: e1cca28826dfa66d905dd4daf9964564c355207e)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We now have a number of places where we show change notifications based
on an event in a previous page (imported a layer, deleted a build,
deleted a project etc) and we show these notifications on various pages
so we add a simple notification utility to libtoaster.
(Bitbake rev: c8db313e907918b0df122006046b157d510ecc1d)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This view is specific to the builds dashboard rather than gernic api so
like ToasterTable and ToasterTypeAhead we class it as a widget as it has
a single purpose. Also clean up some flake8 identified issues.
Original author of the code moved in this commit is Elliot Smith.
(Bitbake rev: 05428514e64ec896faae4055619e149e98bc8f57)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Convert the project xhr calls into proper rest api and port the client
side calls to use the new API. Fix all the pyflakes identified issues
and clean up unused fields.
Also remove the api and client side code for changing release on the fly
as this is no longer supported.
[YOCTO #9519]
(Bitbake rev: 8b01767d6e787cdb09789116ebf57dfb70f521bc)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If the worker fails to launch, ensure the task is shown as failed rather
than a confusing "all succeeded" message.
Patch from Juro Bystricky
[YOCTO #10335]
(Bitbake rev: 0e9a2ff96d138641501874a1cd7aa6cc7e94d727)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When trying to obtain the current directory from a directory
that does not exist anymore, an exception is raised.
This patch handles such exception.
[YOCTO #10331]
(Bitbake rev: 4bcf77589312d9936340d8c308006c2fc9baf67c)
Signed-off-by: Juro Bystricky <juro.bystricky@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When calculating how many lines we'd printed we weren't properly taking
the progress bars into account, with the result that sometimes if the
last line printed on the terminal wrapped to the next line (which is
possible) we backed up less lines than we should have.
Additionally, we should always print a newline after updating the
progress bar - there's no need to check if there wasn't output (there
always will be courtesy of our overridden _need_update()) and we now
allow the line to wrap so we don't need to check the other condition
either.
Hopefully this will fix [YOCTO #10046].
(Bitbake rev: 326d18d96faf02675ba34ad3c3a20cd424b39b91)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Its often useful to know how long a task has been running for. This patch
adds that information to the task display, updating every 5s if there
were no other updates so the user can see how long tasks have been running
for.
[YOCTO #9737]
(Bitbake rev: 6c42025e5dd7761213be3f82f3252a7892d2239d)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
A "bitbake world -c unpack" currently breaks as not all tasks have an
unpack task. This change allows addition of world targets only if the
specified task exists which makes certain commands possible when otherwise
you just get errors which can't easily be avoided.
(Bitbake rev: ca4f5e6d01b5c8cf315f59bc86194d63c0d3d042)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently if you use the subdir parameter in a SRC_URI and pass an absolute path
then it gets appended to the unpack directory instead of being used directly.
This is inconvenient as it may be useful to use ${S} when you want to unpack a
file into the source tree.
Change this behaviour so that absolute paths are used directly instead of being
appended to the root directory. To ensure that recipes cannot write files to an
arbitrary location enforce that the subdir starts with the unpack root.
(Bitbake rev: c3873346c6fa1021a1d63bddd9b898a77c618432)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fixes [YOCTO #10293]
In the section about setting variables, I added a paragraph that
explains the use of single quotes when setting a variable. The
case covers when you must have the double quote charater as part
of your variable's value.
(Bitbake rev: 01e331cd0d612013badfb07df91151907f74903d)
Signed-off-by: Scott Rifenbark <srifenbark@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fixes [YOCTO #10100]
I added a new parent directory named "Functions You Can Call From
Within Python". This section contains a couple new sub-sections.
One is the existing "Functions for Accessing Datastore Variables".
The other is called "Other Functions", and it is used to point
or reference some commonly used functions that the user can call
from within Python.
(Bitbake rev: ecbcedd74125ef00599f4af384ee303dae8af5b7)
Signed-off-by: Scott Rifenbark <srifenbark@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We don't want a traceback for this exception, we need to catch it, fire
TaskFailed, and return failure.
(Bitbake rev: 63966ada459d44d3dc7817ad2a026a22e8f6700f)
Signed-off-by: Christopher Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If you "bitbake glibc-locale" then delete the libpcre-native sstate
and "bitbake glibc-locale -C package_write_rpm", it will fail with
rpmbuild missing the libprce library.
The reason is that libpcre-native fails to install from sstate (since
it isn't present) but doesn't get built and hence rpm-native tries to
run without its dependencies.
The simplest fix is not to add "covered" tasks which have failed to
install sstate. I can't help feeling there is more to this issue but
this does fix the current problem and shouldn't have adverse affects.
It is an unusual situation to have missing dependencies in sstate since
they're usually all present or not at all.
I've taken the opportunity to remove some old cruft from when we had
numeric task ids, the code can be simpler now.
(Bitbake rev: ba566b46d530b495f12f3a74f76434717b22a020)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When the layer is local source don't try and work out the location of
the layer by using the git information (getGitCloneDirectory)
[YOCTO #10199]
(Bitbake rev: 3dfea5214d4bd006e26630e5024774ecb84ea527)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: bavery <brian.avery@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Based on the same test as found in patchwork by Damien Lespiau
https://github.com/dlespiau/patchwork/blob/master/patchwork/tests/test_db.py
(Bitbake rev: 031cb194aaa1b6cc970fed3fa0d0dbd3ebac163f)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: bavery <brian.avery@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There is no need to lock build environment before changing
build status as this operation is very fast. However, there
is a need to unlock it after changing build status.
Explicitly unlocked BuildEnvironment after build reaches
final status SUCCEEDED, FAILED or CANCELLED. This should
allow runbuilds process to pickup next build faster.
(Bitbake rev: faa88272d656640c039572c5c8f3e6c56535b6f7)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fixed following pylint warnings:
C0330(bad-continuation): Wrong hanging indentation before block.
C0326(bad-whitespace): No space allowed around keyword argument assignment
C0326(bad-whitespace): Exactly one space required before assignment
C0301(line-too-long): Line too long
(Bitbake rev: 0eecd660e374a4dbcefe4c59f4c8654bf3a0e937)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If Toaster is stopped incorrectly there could be some
build requests and builds in incorrect state left from the previous run.
Running main processing function on start should take care of those.
(Bitbake rev: 6b9f8f6bb51d1aa2ca4effc34e076e331d0cb8d1)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Run main processing function 'runbuild' only if SIGUSR1 is
received. This signal is sent by Toaster when build status
is changed (either started, cancelled or finished).
This should stop continuous database polling as run_builds function
will be called only when needed, i.e. after build status is changed.
[YOCTO #8918]
(Bitbake rev: 62d598cc5aa01d23f1e9284e9e926bd55b1d1878)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Called signal_runbuilds API when build is scheduled, cancelled or
finished to notify runbuilds process about builds status change.
[YOCTO #8918]
(Bitbake rev: fe08f0fa4b328908e73695ebbceca87bc86a49f9)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This function reads pid of runbuilds process from
BUILDDIR/.runbuilds.pid and sends SIGUSR1 to it. signal_runbuilds
function will be used in Toaster code to notify runbuilds when
build is scheduled, finished or cancelled.
(Bitbake rev: 62955224a6d99e9f581d2bef924058070bfa4c43)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Toaster checks gir repository url is the same as locally cloned before
checking it out to existing local clone. This check can be skipped if
commit is 'HEAD' as in this case repository is not hard reset to
commit, so the local clone won't be changed.
[YOCTO #10163]
(Bitbake rev: 7e9a89e3fde5e71cb859799635974ec41790c44d)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The event handling 'Exception' was catching and triggering a backtrace. This
trace was obscuring any errors from an event handler that had raised the
BBHandledException, which should indicate do not print additional information.
(Bitbake rev: 51ca5193a5674b27d816140b0254f485912177a2)
Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add recommended layers to collection_depends[] so that dynamic
priority assignment will work for both depends and recommends.
Recommended layers do not cause an error or warning
if they are not in the collection list, but debug messages
are output for level 3 and above.
explode_dep_versions2 returns a dictionary, so we
change the variable deplist to depDict. The dictionary
values are lists which are either empty or contain only one
version specification.
(Bitbake rev: 20cdc3d609f8aea992f97c3db336574d3a549973)
Signed-off-by: Joe Slater <jslater@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In the custom recipe details page, the layer information is displayed
twice in the right hand column. Remove one of the layer entries, since
showing the layer information once should be enough.
[YOCTO #10037]
(Bitbake rev: e2b5dc3732781dc933c6bb10482926335720d110)
Signed-off-by: Belen Barros Pena <belen.barros.pena@linux.intel.com>
Signed-off-by: bavery <brian.avery@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The left navigation in the build history pages is not showing the active
item when you navigate directly to the errors or warnings information in
the build summary. Add a special case to make sure the "build summary"
item is highlighted.
[YOCTO #9864]
(Bitbake rev: f236d9ca28e45a270f50bb3edcd466b1bc8d2960)
Signed-off-by: Belen Barros Pena <belen.barros.pena@linux.intel.com>
Signed-off-by: bavery <brian.avery@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The build dashboard provides a count of tasks that were executed and not
executed, and of tasks that failed (if any). The number is a link to the
list of tasks.
Fix the links so that they filter the tasks table by the selected
criteria (executed, not executed or failed).
[YOCTO #9832]
(Bitbake rev: a75e70bbc9081f77f1e4aeeee8222b06112e4406)
Signed-off-by: Belen Barros Pena <belen.barros.pena@linux.intel.com>
Signed-off-by: bavery <brian.avery@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Task name was incorrectly added to the targets that already
contained :task suffix and fired with BuildInit event. This
caused Toaster to create incorrect Target objects and show
them in UI.
[YOCTO #10221]
(Bitbake rev: b7faf1af3bd3110fba347fbe6e432fc4ee66590a)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Users are surprised when dirs/cleandirs aren't acted upon for
empty functions. This reorders the code slightly so that those
flags are acted upon for empty functions as there are cases where
this is expected.
[YOCTO #10256]
(Bitbake rev: 5bf874673d75b5f4ff2b34f0ab8502558ee84d00)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The implementation of SRCREV_FORMAT has at least two issues:
1. Given two names "foo" and "foobar" and SRCREV_FORMAT = "foo_foobar",
"foo" might currently get substituted twice, and "foobar" not at
all.
2. If the revision substitued for some name happens to contain another
name as a substring, then that substring might incorrectly get
replaced.
Fix both issues by sorting the names with the longest ones first and
replacing all names at once with a regular expression. This was inspired
by
http://stackoverflow.com/questions/6116978/python-replace-multiple-strings.
(Bitbake rev: 8e6a893cb7f13ea14051fc40c6c9baf41aa47fee)
Signed-off-by: Ulf Magnusson <ulfalizer@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If we've already fetched a particular URL then we do not need to do so
again within in the same operation. Maintain an internal list of fetched
URLs to avoid doing that.
(Bitbake rev: b4705c80add1f618c11a9223cdd9578d763b50ec)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
All logging messages are printed on stdout when processing
UI event queue. This makes it impossible to distinguish between
errors and normal bitbake output. Output to stderror or stdout
depending on log level should fix this.
(Bitbake rev: 56ac0d4c7a5f47aeb707b15a0c305d9f73aae945)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Bitbake collects all events in special event queue when called with
-w option. However, it starts to write events to the eventlog only
after BuildStarted event is received. In some cases this event is
not received at all, e.g. when bitbake is run with --parse-only
command line option.
It makes sense to write all collected events when CookerExit event
received to make sure all events are written into the eventlog even
if BuildStarted event is not fired.
[YOCTO #10145]
(Bitbake rev: 57912de63fa83550c0ae658eb99b76e9cc91a8d1)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Toaster script kills runserver process 2 ways:
- sending signal to pid from .toastermain.pid.
- sending signal to pids found by grepping ps output:
ps fux | grep "python.*manage.py runserver"
Second approach is redundant and harmfull as it kills all django
development server running on the machine.
[YOCTO #7973]
(Bitbake rev: 0f47b17fe88dc660648d94b2d8d8286d87ae6295)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This file is not created anywhere, but handled in toaster
script code.
(Bitbake rev: 16f3cd3535c9eec71ea7594c1e3a83db00dba7ca)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There is no point of trying to kill django development server
when toaster starts because 'manage.py checksocket' command is already
used in the script code to check if development server port is occupied.
Even if Toaster is listening on another port, killing previous instance
looks quite implicit and doesn't solve anything as there are other
processes that might be still running.
(Bitbake rev: 0dab45e9815e8939219900264e86f569c714b7c6)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This patch fixes a typo in one of the IMAGE_FSTYPES values listed in
Toaster. It also updates the hardcoded list of values to match the
latest list in meta/classes/image_types.bbclass
[YOCTO #9447]
(Bitbake rev: 46db3279cb81b3ca6ce047204aee620f5ee51220)
Signed-off-by: Belen Barros Pena <belen.barros.pena@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When converting variable history file names to relative
paths, keep the layer directory's name so that the user
can distinguish between conf files with the same name.
[YOCTO #8188]
(Bitbake rev: 59561d652af91c2099b735084f0e44275d68e637)
Signed-off-by: David Reyna <david.reyna@windriver.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add forward slash to the list of special characters allowed in variable
names. Also update the list of allowed special characters in the error
messages.
[YOCTO #9611]
(Bitbake rev: 146f6f95a8753308edb31e952d7c440c8de11870)
Signed-off-by: Belen Barros Pena <belen.barros.pena@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Make sure the layer information disappears when the edit form shows, and
that the layer details come back when you click the 'cancel' button in
the edit form.
(Bitbake rev: bd08abe7c1f5fc96ee73c20b2c9d10a591a5f69c)
Signed-off-by: Belen Barros Pena <belen.barros.pena@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The layout of the import layer form was looking a bit awkward. This
commit tidies things up a bit.
(Bitbake rev: e5e51ca1394bc392eba99742c59d86b8e5fd5623)
Signed-off-by: Belen Barros Pena <belen.barros.pena@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The layout of the layer details page was looking a bit awkward. This
commit tidies things up.
(Bitbake rev: ce9a5f885f43bebf39d191309f48da83b31e60e0)
Signed-off-by: Belen Barros Pena <belen.barros.pena@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When you change the machine from the project configuration page, you get
some useful suggestions as you start typing a machine name. However, the
suggestions only include machines provided by the layers added to your
project. This is not necessarily clear from the design (yes, it should
be improved), which means you might be looking for a machine, not see it
in the suggestions, and assume the machine is not supported by
OpenEmbedded.
Since we are in no position to change the design of this page right now,
add some explanatory help text to address the situation.
[YOCTO #8034]
(Bitbake rev: 829c9bcb58f961c644e24b24265e0ef45f0fec57)
Signed-off-by: Belen Barros Pena <belen.barros.pena@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The 'Recipe version' column should not be part of the set of columns
shown by default in the tasks table. Set the hidden property for that
column to 'True' so that it doesn't show when you load that table
for the first time.
[YOCTO #10179]
(Bitbake rev: 249dd31fcaabbec32fdee30b0c84be90d4f92430)
Signed-off-by: Belen Barros Pena <belen.barros.pena@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
No functional changes, just make a couple of minor tweaks to the
comments for edit_metadata():
* There are four elements to be returned by the callback function
* Add an example return statement for when you don't want to modify the
value
(Bitbake rev: 99675c19375c96140bc8ae8f9fc3a1945a77cebb)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The correct name of the parameter is "version" not "ver" so ensure we
aren't misleading the user by giving the latter in an example.
(Bitbake rev: 14c045c6a20993d389b91ae2459d811a1430a7b2)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Allow using a top-level shrinkwrap file with one or more npm://
dependencies, i.e. if the module isn't found at the top level then look
one level down.
Part of the fix for [YOCTO #9537].
(Bitbake rev: f7de3f8b5f628dee043fe783148812914ab20813)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
"npmpkg" can be a default, but it should respect the subdir parameter as
with other FetchMethods. This allows us to have more than one npm://
entry in SRC_URI without nasty hacks.
Fix required in order to support [YOCTO #9537].
(Bitbake rev: e6a94d2091ec5d42f25102334a8492a731b8dec3)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
You cannot set a URL-specific value in an object-level variable on
the FetchMethod in urldata_init() or the result is the value specific to
the last URL will be the one that gets set. This prevented fetching more
than one npm:// URL correctly - the other tarballs would not download to
the correct location and do_unpack failed to find them as a result.
Fix required in order to support [YOCTO #9537].
(Bitbake rev: 1435b49ea7d0f9d4cc4a665fb2aa83d1eea7900f)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We were downloading into the current directory here, which is fine if
that current directory can be expected to be the right place - but
that's not true when called from recipetool within OE. We should
explicitly specify the directory to run the command in and then there
won't be a problem.
(Bitbake rev: 0ddaf725e5a0675b252b7f80b1706370e478175b)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The ud.pkgdir argument was being passed as the 'quiet' argument to
runfetchcmd, not the 'workdir' argument, resulting in fetching the svn module
into the root of DL_DIR, not where it belongs.
Cc: Matt Madison <matt@madison.systems>
(Bitbake rev: dc756510a95f88b192352be6fcd1d5d77852c348)
Signed-off-by: Christopher Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently, there is a check to remove the TOSTOP attribute from
a tty to avoid hangs. It assumes that sys.stdout will have a
file descriptor and this is not always true, some IO classes
will throw exceptions when trying to get its file descriptor.
This will add a check for such cases and avoid throwing an
exception.
[YOCTO #10162]
(Bitbake rev: cb4f8f6efa28ef2b13bc738a0118b876baa15b3e)
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We don't need to force everyone to use git for the method in which
openembedded-core is downloaded. For instance it could have been
downloaded and extracted as a tarball.
(Bitbake rev: 8b7180332691a41a013e07a52b26018402141b6a)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We no longer only deal with layers that have their source in a gir
repository, we also allow for local directories too so update the
BRLayer model to reflect this.
(Bitbake rev: a15f61f3ef5a87b87121457f76592c87f0ea5d7f)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add selenium tests for the new layer source switching functionality on
the layer details page. Edits the values for git repository and saves
and then edits the details for directory information and saves.
(Bitbake rev: acdfafdd753abe38a313c42e3a9d6211338b4e73)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We now have a dedicated file for the rest API so move and rework for
class based views. Also clean up all flake8 identified warnings.
Remove unused imports from toastergui views.
The original work for this API was done by Elliot Smith, Ed Bartosh,
Michael Wood and Dave Lerner
(Bitbake rev: 37c2b4f105d7334cdd83d9675af787f4327e7fe7)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Due to a copy paste error we managed to get some of the wrong
information in the oe fixture that provides a suggested default settings
for Toaster. This meant it tested correctly when it shouldn't have.
Fix:
- The use of local bitbake
- An incorrect call to realpath which didn't include its parent module.
- The field used for the local_dir of an existing openembedded-core
(Bitbake rev: d57a9124650e5367919668dfccf6aad4962a77f1)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
A few clean ups for the work done to integrate editing imported local layers
into the layer detail page.
(Bitbake rev: 092ef32e695b43c3337b7116722c4c6eba981396)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add an additional argument to the api to handle
local_source_dir which is the value user passes
to import non-git layers.
[YOCTO #9913]
(Bitbake rev: 2b5728fc5c0e578560506697f271605e80b5918f)
Signed-off-by: Sujith H <sujith.h@gmail.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This patch helps to implement the switching of layers
between directories and git repositories. Specifically
selection of git and local directory. Also enabling
form to view the selection.
[YOCTO #9913]
(Bitbake rev: 5c20834691f1b65cfc4a0c4ec12958f86b34bbeb)
Signed-off-by: Sujith H <sujith.h@gmail.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This patch updates the layerdetails html file to
add the feature of switching imported layers between
directories and git repositories.
[YOCTO #9913]
(Bitbake rev: 70319eb690a056b41b7e91d79560067edd623ee1)
Signed-off-by: Sujith H <sujith.h@gmail.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Because some image_license.manifest files contain multiple
FILES lines, and because those lines can sometimes not contain
a list of files (i.e. they look like "FILES:\n"), we were
resetting the list of kernel artifacts when we hit the second
"empty" line.
Fix by ignoring any FILES line which doesn't list files, and by
appending any files found in a valid FILES line, rather than
overwriting the existing list.
[YOCTO #10107]
(Bitbake rev: 927ec3524625ac731326b3c1c1361c2a4d2bd9e1)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If http basic auth creds were added to sstate mirrors like so:
https://foo.com/sstate/PATH;user=foo:bar;downloadfilename=PATH
The sstate mirror check would silently fail with 401 unauthorized.
This patch allows both the check, and the wget download to succeed by
checking for user credentials and if present adding the correct
headers, or wget params as needed.
[ YOCTO #9815 ]
(Bitbake rev: cea8113d14da9e12db80a5b6b5811a47a7dfdeef)
Signed-off-by: Stephano Cetola <stephano.cetola@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We can't execute the same task for the same package_arch multiple
times as the current setup has conflicting directories. Since
these would usually have the same stamp/hash, we want to execute in
sequence rather than in parallel, so for the purposes of task execution,
don't consider the "extra-info" on the stamp files. We need to add
a parameter to the stamp function to achieve this.
This avoids multiple update-rc.d populate_sysroot tasks executing in
parallel and breaking multiconfig builds.
(Bitbake rev: a9041fc96a14e718c0c1d1676e705343b9e872d3)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We need a separate fetcher cache per multiconfig as the revisions and other
SRC_URI data can potentially be different. For now, this is the simplest way
to achieve that and avoids linux-yocto kernel build failures when targeting
multiple machines for example.
(Bitbake rev: d98cc31d6668bc1d6372664593126b5e5132ef2c)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Parsing a recipe is such a common task for tinfoil-using scripts, and is
a little awkward to do properly, so add an API function to do it. This
should also isolate scripts a little from future changes to the internal
code. The first user of this will be the OpenEmbedded layer index update
script.
Part of the fix for [YOCTO #10192].
(Bitbake rev: 39780b1ccbd76579db0fc6fb9369c848a3bafa9d)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
To accommodate the OpenEmbedded layer index recipe parsing, we have to
have the ability to pass in a custom config datastore since it
constructs a synthetic one. To make this possible after the multi-config
changes, rename the internal _load_bbfile() function to parse_recipe(),
make it a function at the module level (since it doesn't actually need
to access any members of the class or instance) and move setting
__BBMULTICONFIG inside it since other code will expect that to be set.
Part of the fix for [YOCTO #10192].
(Bitbake rev: 5b3fedfe0822dd7effa4b6d5e96eaf42669a71df)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We should always shut down tinfoil when we're finished with it, either
by explicitly calling the shutdown() method or by using it as a
context manager ("with ...").
(Bitbake rev: 131e6dc4bbd197774d35d2b266bfb0816f6e6b1e)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Since calling the shutdown() function is highly recommended, make
tinfoil objects a little easier to deal with by adding context manager
support - so you can do the following:
with bb.tinfoil.Tinfoil() as tinfoil:
tinfoil.prepare(True)
...
(Bitbake rev: f59bc6be2b4af1acdcf6a1b184956b5ffd297743)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fixes [YOCTO #10196]
Added a new section named "Exporting Variables to the Environment".
This section provides a dedicated description for how to export
variables to the shell.
(Bitbake rev: b543458dd67d24a228fa2db0ecb4ddd20016a560)
Signed-off-by: Scott Rifenbark <srifenbark@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fixes [YOCTO #10141]
Section on Checksums (Signatures) had this variable referred to as
STAMP_DIR.
(Bitbake rev: 7dff6762148bc2ac8f81d89bbe595dfbfdf7b119)
Signed-off-by: Scott Rifenbark <srifenbark@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Now that the fetchers all preserve the current working
directory, the cwd changes in the try_mirror_url,
download, and checkstatus methods are no longer needed.
(Bitbake rev: 0ed8975c42718342a104a9764a58816f964ec4ea)
Signed-off-by: Matt Madison <matt@madison.systems>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(Bitbake rev: deab9a30987b225922490ca186c5307c15d45b82)
Signed-off-by: Jonathan Liu <net147@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fix the methods in all fetchers so they don't change
the current working directory of the calling process, which
could lead to "changed cwd" warnings from bitbake.
(Bitbake rev: 6aa78bf3bd1f75728209e2d01faef31cb8887333)
Signed-off-by: Matt Madison <matt@madison.systems>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fixed:
File "bitbake/contrib/dump_cache.py", line 39
print("Error, need one argument!", file=sys.stderr)
(Bitbake rev: 435c6fb838b9f38c0477bcc2f07c8ce22999132b)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* unify the format how the task is described
* don't show taskid followed by taskstring as the taskstring is
different only for setscene tasks (by _setscene suffix)
* the duplicated output was introduced by:
2c88afb taskdata/runqueue: Rewrite without use of ID indirection
as reported and confirmed as a bug here:
http://lists.openembedded.org/pipermail/openembedded-core/2016-June/123148.html
* show:
NOTE: Running task 541 of 548 (/OE/build/oe-core/openembedded-core/meta/recipes-core/zlib/zlib_1.2.8.bb:do_package)
instead of much longer:
NOTE: Running task 541 of 548 (ID: /OE/build/oe-core/openembedded-core/meta/recipes-core/zlib/zlib_1.2.8.bb:do_package, /OE/build/oe-core/openembedded-core/meta/recipes-core/zlib/zlib_1.2.8.bb:do_package)
and similarly for failed tasks:
ERROR: Task (virtual:native:/OE/build/oe-core/openembedded-core/meta/recipes-core/zlib/zlib_1.2.8.bb:do_install) failed with exit code '1'
instead of much longer:
ERROR: Task virtual:native:/OE/build/oe-core/openembedded-core/meta/recipes-core/zlib/zlib_1.2.8.bb:do_install (virtual:native:/OE/build/oe-core/openembedded-core/meta/recipes-core/zlib/zlib_1.2.8.bb:do_install) failed with exit code '1'
(Bitbake rev: 696693d45f5eff1226866ed79dbfb67161d8cd3f)
Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Introduce a new 'usehead' url parameter for git repositories. Specifying
usehead=1 causes bitbake to use whatever commit the repository HEAD is
pointing to. Usage of usehead=1 is only allowed for local git
repositories, i.e. it must always be accompanied with protocol=file url
parameter.
[YOCTO #9351]
(Bitbake rev: 2673fac5a9d06de937101e3fb2ddf1e60ff99abf)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Now you get the bb logger output for failed tests. This helps debugging
problems. Also, all stdout/stderr data for successful tests is silenced
which makes for less cluttered console output.
(Bitbake rev: ea19972a16f7639f944823d1d8a7728105460136)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Set this env variable to 'yes' to preserve temporary directories used by
the fetcher tests. Useful for debugging tests.
(Bitbake rev: 04132b261df9def3a0cff14c93c29b26ff906e8b)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This simplifies the script, and, gives new features. It is now possible
to run single test functions, for example. This is nice when writing new
test cases.
(Bitbake rev: 8c513580b9406b031674f799117eae7410f8e01c)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This patch adds the notion of supporting multiple configurations within
a single build. To enable it, set a line in local.conf like:
BBMULTICONFIG = "configA configB configC"
This would tell bitbake that before it parses the base configuration,
it should load conf/configA.conf and so on for each different
configuration. These would contain lines like:
MACHINE = "A"
or other variables which can be set which can be built in the same
build directory (or change TMPDIR not to conflict).
One downside I've already discovered is that if we want to inherit this
file right at the start of parsing, the only place you can put the
configurations is in "cwd", since BBPATH isn't constructed until the
layers are parsed and therefore using it as a preconf file isn't
possible unless its located there.
Execution of these targets takes the form "bitbake
multiconfig:configA:core-image-minimal core-image-sato" so similar to
our virtclass approach for native/nativesdk/multilib using BBCLASSEXTEND.
Implementation wise, the implication is that instead of tasks being
uniquely referenced with "recipename/fn:task" it now needs to be
"configuration:recipename:task".
We already started using "virtual" filenames for recipes when we
implemented BBCLASSEXTEND and this patch adds a new prefix to
these, "multiconfig:<configname>:" and hence avoid changes to a large
part of the codebase thanks to this. databuilder has an internal array
of data stores and uses the right one depending on the supplied virtual
filename.
That trick allows us to use the existing parsing code including the
multithreading mostly unchanged as well as most of the cache code.
For recipecache, we end up with a dict of these accessed by
multiconfig (mc). taskdata and runqueue can only cope with one recipecache
so for taskdata, we pass in each recipecache and have it compute the result
and end up with an array of taskdatas. We can only have one runqueue so there
extensive changes there.
This initial implementation has some drawbacks:
a) There are no inter-multi-configuration dependencies as yet
b) There are no sstate optimisations. This means if the build uses the
same object twice in say two different TMPDIRs, it will either load from
an existing sstate cache at the start or build it twice. We can then in
due course look at ways in which it would only build it once and then
reuse it. This will likely need significant changes to the way sstate
currently works to make that possible.
(Bitbake rev: 5287991691578825c847bac2368e9b51c0ede3f0)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If you don't do this, with Python 3 you get a warning on exit under some
circumstances.
(Bitbake rev: 49502685df3e616023df352823156381b1f79cd3)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
unset VAR
will clear variable VAR
unset VAR[flag]
will clear flag "flag" from var VAR
(Bitbake rev: bedbd46ece8d1285b5cd2ea07dc64b4875b479aa)
Signed-off-by: Jérémy Rosen <jeremy.rosen@openwide.fr>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Rather than passing in a datastore to build on top of, use the data builder
object in the cache and base the parsed recipe from this. This turns
things into proper objects building from one another rather than messy
mixes of static and class functions.
This sets things up so we can support parsing and building multiple
configurations.
(Bitbake rev: fef18b445c0cb6b266cd939b9c78d7cbce38663f)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There are some cases we want to parse recipes without any cache
setup or involvement. Split out the standalone functions into
a NoCache variant which the Cache is based upon, setting the scene
for further cleanup and restructuring.
(Bitbake rev: 120b64ea6a0c0ecae7af0fd15d989934fa4f1c36)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Rather that the current mix of static and class methods, refactor
so that the cache has the databuilder object internally. This becomes
useful for the following patches for multi config support.
It effectively completes some of the object oriented work we've been
working towards in the bitbake core for a while.
(Bitbake rev: 7da062956bf40c1b9ac1aaee222a13f40bba9b19)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Needing to access these static methods through a class doesn't
make sense. Move these to become module level standalone functions.
(Bitbake rev: 6d06e93c6a2204af6d2cf747a4610bd0eeb9f202)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Simple refactoring to allow for multiconfig support.
(Bitbake rev: 266b848da40904446eb1d084bbdc5307a9b45197)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
With the introduction of multi-config and the possibility of distributed
builds we need arrays of workers rather than the existing two.
This refactors the code to have a dict() of workers and a dict of
fakeworkers, represented by objects. The code can iterate over these.
This is separated out from the multi-config changes since its separable
and clearer this way.
(Bitbake rev: 8181d96e0a4df0aa47287669681116fa65bcae16)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The current codepaths are rather confusing. Stop passing these
as parameters and use the ones from when the object is created.
(Bitbake rev: 8c992c148d9619b10eeae8bbd9376ecf408037a5)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fixes [YOCTO #10117]
Applied a re-write to better clarify the behavior of dependencies.
(Bitbake rev: 28bb8ef7f737034055f3485795179cfdcdb9a41f)
Signed-off-by: Scott Rifenbark <srifenbark@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fixes [YOCTO #10095]
I added a third case to the "Conditional Metadata" section to
describe setting a variable for a single task.
(Bitbake rev: 24d648ce62b35f7d2b23fde732703c060579a0d2)
Signed-off-by: Scott Rifenbark <srifenbark@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fixes [YOCTO #10093]
Provided much more detail on how these functions work.
(Bitbake rev: dbe25523d899850f85acb6986eca98bf1b0ef52a)
Signed-off-by: Scott Rifenbark <srifenbark@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fixes [YOCTO #10071]
The use of any flags throughout the manual was very inconsistent.
I changed all references to any named flag in the text to be
formatted as code and to be enclosed in square brackets.
(Bitbake rev: be0fb616e64e54ae3e2420249f21f4edfd97d648)
Signed-off-by: Scott Rifenbark <srifenbark@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fixes [YOCTO #10071]
Provided more clear descriptions for these two flags.
(Bitbake rev: c85c9a468dc3ce606a5f8797e6be8b411a9f3bdb)
Signed-off-by: Scott Rifenbark <srifenbark@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This was added to enable the usage of git through proxies.
(Bitbake rev: 449fc52e483a3bf1cec1c5d8cf8c3946ec5292ab)
Signed-off-by: Francisco Pedraza <francisco.j.pedraza.gonzalez@intel.com>
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There's not a whole lot of point showing how many tasks are running when
we're in quiet mode, it just looks a bit strange particularly when it's
not running any tasks.
(Bitbake rev: 5317200d9cd73c6f971bc1b0cfe8692749e27e3a)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If we have the task number here we need to subtract 1 to get the number
of tasks completed.
(Bitbake rev: 7c78a1cd3f0638ae76f7c7a469b7f667c7c58090)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
A couple of fixes for the "Initialising tasks" progress bar behaviour:
* Properly finish the progress bar when using bitbake -S
* Finish the progress bar before calling BB_HASHCHECK_FUNCTION (so that
in OE when that shows its own "Checking sstate mirror object
availability" progress bar it gets shown on the next line as it
should).
(Bitbake rev: de6759d8e9990e426e6d6464a2e05381cd4c12d6)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The CUPS ipptool URL we were checking now redirects to github where the tarball
isn't present, so remove it from the test suite.
(Bitbake rev: 4b50895fb3462b21e3874a2e99c363c8d05e89e6)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The layer index update command has a special case for the
updating 'openembedded-core' layer, and it was missing reading
and updating the git web URL fields.
[YOCTO #8037]
(Bitbake rev: ce2f990a366d2d939e93e01f67688f12740c5fee)
Signed-off-by: David Reyna <david.reyna@windriver.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Adds handling of the non-git layers to create and update the
corresponding layer objects in Toaster.
(Bitbake rev: 0a9b5d7d9655dbb09d458fc6e330e932f0f9dab6)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Sync the BRLayer object with the new field added to the Layer object.
The BRLayer (BuildRequest Layers) are snapshots of the layers in the
project at build time and therefore need to mirror the required fields
of the layer object.
(Bitbake rev: a3112c922f036425977abffa0137b9133f61fcd6)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Update xhr_import_layer test to use locally imported
layer in local_source_dir.
[YOCO #9911]
(Bitbake rev: f855490dd04281beb5ae65d82430203d9fd263fc)
Signed-off-by: Sujith H <sujith.h@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Update layer branch and layer commit section
in the build configuration page for locally
imported layers. For locally imported layers
this secion goes as "Not applicable".
[YOCO #9911]
(Bitbake rev: 4ca3f602a955e01d445fb4789496e925f8d4234b)
Signed-off-by: Sujith H <sujith.h@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Update Layer branch and Layer commit
in recipe details page. For local layer
imported from toaster these fields are
not applicable.
[YOCO #9911]
(Bitbake rev: 35f7faabff17ca577fdd2e84bb25125047f66345)
Signed-off-by: Sujith H <sujith.h@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Update package details pages layer branch
and layer commit section for locally imported
layers. We add Not applicable to them along
with a helper text, which helps user understand
why its not applicable to those layers.
[YOCO #9911]
(Bitbake rev: 6978f65af4eabe91f65a6766799578c84a0988aa)
Signed-off-by: Sujith H <sujith.h@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Update the build tables section for locally
imported layers. Here we do not provide informations
such as branch or commit. Because those are locally
imported layer(s).
[YOCO #9911]
(Bitbake rev: 1b0934b45d3704de604405826de9fc1da51ceae1)
Signed-off-by: Sujith H <sujith.h@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Update table informations for pages:
1) Compatible layers
2) Compatible image recipes
Added Not Applicable to the fields of locally
imported layers because they are not git version.
[YOCO #9911]
(Bitbake rev: 7e73f8583a0f92d73b42410e1c960b6f2976a557)
Signed-off-by: Sujith H <sujith.h@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Collect the dir path imported from UI and make
sure that its getting added to bblayers.conf.
This patch exactly does the same job. Any layer
which is imported locally need not be cloned
again to _toaster_clones dir.
[YOCO #9911]
(Bitbake rev: 5669157994fc220a018e37927600988f3fd43271)
Signed-off-by: Sujith H <sujith.h@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a new field local_source_dir to model.
This will clearly differentiate us from the
vcs_url which is for git path.
Adding migration file 0010_layer_local_source_dir_path.py
along with this patch.
[YOCO #9911]
(Bitbake rev: 83763d89f9d0bc535e930a2094ba8201675d40be)
Signed-off-by: Sujith H <sujith.h@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Adding local directory to the database.
The local directory is added to vcs_url,
field of db.
[YOCO #9911]
(Bitbake rev: 2c3d48e7cd21a999ef145081352774f1759cd5e4)
Signed-off-by: Sujith H <sujith.h@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The helper text gets displayed accordingly when mouse
is hovered above the layers. If its a local directory
then no more branch is mentioned. Only directory path
is mentioned.
[YOCTO #9911]
(Bitbake rev: c1961c7844cab5b9c9144f3b4946197097f53e17)
Signed-off-by: Sujith H <sujith.h@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Update the css file for the layer import.
This changes will improve the UI for the
task.
[YOCO #9911]
(Bitbake rev: a41d0400478186c983334d8f83780639e7361c91)
Signed-off-by: Belen Barros Pena <belen.barros.pena@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This change will help us know if local_source_dir
is null or if there is value associated with this
field. This change will help us display the details
for duplicate layers tryied to import locally.
[YOCOT #9911]
(Bitbake rev: 4350b01f776e6e35745c3eb9440587973de00eec)
Signed-off-by: Sujith H <sujith.h@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Changes made in this patch will help user to select
either repo based on git and local. This patch also
improves the help provided to user so that user can
understand them and take necessary action to proceed.
[YOCOT #9911]
(Bitbake rev: 26bee4c2c05dc202857270cd5f56abff79674ca7)
Signed-off-by: Sujith H <sujith.h@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This patch will help to add local directory
to UI. The modification is made in importlayer.html
file. The radio buttons are created to separate
selection of git based repo and local directory.
Co-Author: Belen Barros Pena <belen.barros.pena@linux.intel.com>
[YOCOT #9911]
(Bitbake rev: 78e5edb8ed4bf6c8b0fb87fcf17a82485c145920)
Signed-off-by: Sujith H <sujith.h@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
manage.py lsupdates throws DataError exception if the recipe can't be
saved to the MySQL database:
django.db.utils.DataError: (1406, "Data too long for column 'license' at row 1"
Adding DataError exception to the list of exceptions should make
lsupdates to print a warning message and skip the recipe.
(Bitbake rev: 01891c13ae8dcba64e5cf79956f1b64c2afdaae3)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Many of the methods in toasterui and buildinfohelper rely
on the internal state of the buildinfohelper; in particular, they
need a Build object to have been created on the buildinfohelper.
If the creation of this Build object is tied to an event which
may or may not occur, there's no guarantee that it will exist.
This then causes assertion errors in those methods.
To prevent this from happening, add an _ensure_build() method
to buildinfohelper. This ensures that a minimal Build object
is always available whenever it is needed, either by retrieving
it from the BuildRequest or creating it; it also ensures that
the Build object is up to date with whatever data is available
on the bitbake server (DISTRO, MACHINE etc.).
This method is then called by any other method which relies on
a Build object being in the internal state, ensuring that the
object is either available, or creating it.
(Bitbake rev: 0990b4c73f194ec0be1762e4e48b1a525d8349fb)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Toaster occasionally records a task which depends on itself.
Add a test which checks that a task which depends on itself
can be displayed in the task page.
[YOCTO #9952]
(Bitbake rev: b7a699e701785b5bd8da97b6e1b760a1c6dd05f5)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Toaster occasionally records a task which depends on itself.
This causes a problem when trying to display that task if it
is "covered" by itself, as the code does the following: for
task A, find a task B which covers A; then, recursively
find the task which covers B etc. If B == A, this loop becomes
infinite and never terminates.
To prevent this, add the condition that, when finding a task B
which covers A, don't allow B == A.
[YOCTO #9952]
(Bitbake rev: 88c471c7e5995abb5bca62990b91650277b6c926)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add tests for ToasterTable UI table sort reverting, which can
only be exercised via the browser.
Check that if a table is sorted by a column, and that column
is hidden, then the sort reverts to the default for the table.
[YOCTO #9836]
(Bitbake rev: 5b016338478d784fd048ba2baae121c3e558090c)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When a ToasterTable is sorted by a column, and that column is
hidden from view, the sort doesn't revert to the default for the
table.
Modify the JS responsible for reloading the table data so that
it doesn't rely on clicking a table column heading (as this is
inflexible and error-prone). Instead, use a function to apply
the sort to the table; and call that function when column
headings are clicked.
This means that the ordering can be changed programmatically
to a specified default ordering when a column is hidden, without
having to click on a column heading.
Use this function when the current sort column is hidden, to
apply the default sort for the table.
[YOCTO #9836]
(Bitbake rev: a28377067b6f381bbc98db82f5c45fca6620f7ad)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Creating a custom image through the "New custom image" dialog
can sometimes result in a long pause between pressing the button
to create the image, and being transferred to the page showing
details of its content. This can make it appear as though pressing
the button had no effect.
To prevent this from happening, disable the button and text box
in the new custom image dialog after the "Create image" button is
pressed. Also show a loading spinner and "loading..." text on
the button to make it clear that the application is still responding.
[YOCTO #9475]
(Bitbake rev: dd8bede91e08c0b64b949ca98c74e6144da88fd1)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When one of the layer details tests fails (as it occasionally
does, if running on a machine under heavy load, due to sync issues),
the error message shown is misleading, as it is something like:
"Expected 'This was imported' in ['This was imported', ...]"
The string 'This was imported' is in the list shown in the message,
but the message suggests it isn't.
This is because the test compares the string with one list, but
then uses a different list in the fail message if the comparison
fails.
Fix the list shown in the message about the test failing.
(Bitbake rev: 34943b2278efe99c6744399e04a47cdda630468e)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add tests for the state transitions in the "most recent builds"
area of the all builds page.
[YOCTO #9631]
(Bitbake rev: b95681cf38475903ad4f73059313dda8c0dccef6)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Test that the build dashboard only shows a menu and a build
summary area if a build has properly "started" (i.e. has at least
one Variable object associated with it).
[YOCTO #8443]
(Bitbake rev: 9e16f76fb254ae967ded6c21251243b2af9b16b6)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When a build fails, it shouldn't have links on its build time in the
recent builds area or in the all builds table.
[YOCTO #8443]
(Bitbake rev: 3d7b247512eb01607741f5f6ce7cb01d241e49e7)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Remove the side bar and build details modules for failed builds.
[YOCTO #8443]
(Bitbake rev: 9d68a5bfdccd399791e1af048fefa73b5df90bcb)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a property to the Build model which records whether
the BuildStarted event has occurred for the build.
The proxy for this event is the presence of variables recorded
against the Build: as the buildinfohelper only saves variables
when the BuildStarted event occurs (as the variables aren't
available on the bitbake server before that point), we can
tell whether BuildStarted has happened by counting Variable
objects on the Build.
This can then be used to determine whether a Build "properly"
started, enabling a different dashboard display (left-hand menu
hidden) if the build didn't record any useful information (e.g.
if it had a bad target).
[YOCTO #8443]
(Bitbake rev: aa151a4d2de4a54fe3075a8c56a4935158398a18)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Failed builds don't have any time data recorded for them,
so the time field in the builds table, the time shown
in the recent builds area, and the build time shown in
the build dashboard should not be links for failed builds.
[YOCTO #8443]
(Bitbake rev: 6f9c472d95ee800da079f6b828b956d9f8c67ce6)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Help icons need to have the Bootstrap tooltip() method called on
them so that the popups are correctly styled.
Ensure that the colour of the help/error/warning icons is correct,
depending on the build state.
Fix pluralisation of errors and warnings shown.
Add a div around the build state area so it's easy to pick up
where the state is going to display (e.g. in tests).
[YOCTO #9631]
(Bitbake rev: 98a923ff14188832ac44e0dbafc73bcba10e25da)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
To prevent showing a "0% of tasks complete" message for a long time,
don't show the progress bar until the first task has finished.
While waiting for that first task, show a message about tasks
starting instead.
[YOCTO #9631]
(Bitbake rev: 5529bcd860d2932b967a064ae28690ac5a725342)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Modify buildinfohelper and toasterui so that they record the
recipe parse progress (from ParseProgress events in bitbake)
on the Build object.
Note that because the Build object is now created at the
point when ParseStarted occurs, it is necessary to set the
build name to the empty string initially (hence the migration).
The build name can be set when the build properly starts,
i.e. at the BuildStarted event.
Then use this additional data to determine whether a Build
is in a "Parsing" state, and report this in the JSON API.
This enables the most recent builds area to show the recipe
parse progress.
Add additional logic to update the progress bar if the progress
for a build object changes.
[YOCTO #9631]
(Bitbake rev: f33d51d46d70e73e04e325807c1bc4eb68462f7b)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The most recent builds area of the all builds and project builds
table needs to update as a build progresses. It also needs
additional functionality to show other states (e.g. recipe parsing,
queued) which again needs to update on the client side.
Rather than add to the existing mix of server-side templating
with client-side DOM updating, translate all of the server-side
templates to client-side ones (jsrender), and add logic which
updates the most recent builds area as the state of a build changes.
Add a JSON API for mostrecentbuilds, which returns the state of
all "recent" builds. Fetch this via Ajax from the build dashboard
(rather than fetching the ad hoc API as in the previous version).
Then, as new states for builds are fetched via Ajax, determine
whether the build state has changed completely, or whether the progress
has just updated. If the state completely changed, re-render the
template on the client side for that build. If only the progress
changed, just update the progress bar. (NB this fixes the
task progress bar so it works for the project builds and all builds
pages.)
In cases where the builds table needs to update as the result of
a build finishing, reload the whole page.
This work highlighted a variety of other issues, such as
build requests not being able to change state as necessary. This
was one part of the cause of the "cancelling build..." state
being fragile and disappearing entirely when the page refreshed.
The cancelling state now persists between page reloads, as the
logic for determining whether a build is cancelling is now on
the Build object itself.
Note that jsrender is redistributed as part of Toaster, so
a note was added to LICENSE to that effect.
[YOCTO #9631]
(Bitbake rev: c868ea036aa34b387a72ec5116a66b2cd863995b)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In situations where a bitbake run fails before the build
properly starts and BuildStarted is fired, a UI has no way
to get at the targets passed to the build. This makes it
difficult for the UI to report on the targets which failed.
Fire a BuildInit event before running buildTargets() or
buildFile(). This enables a UI to capture targets passed to
buildTargets(), even if the build fails (e.g. the targets
themselves are invalid).
[YOCTO #8440]
(Bitbake rev: ac02fda870965bf7d44ff5688eda54d2d11ab9c7)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This has been replaced using django's inbuilt loaddata.
Django command documented at:
https://docs.djangoproject.com/en/1.8/ref/django-admin/#django-admin-loaddata
(Bitbake rev: 3edd33aff2b9eef82090dc4f9b5461f901aec8be)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Adds a spinner so that you know that the parse and http fetch from the
layerindex is in progress.
(Bitbake rev: e1c1c8827f3892551084bf1c0909c1b33f0dca83)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Call django's inbuilt loaddata command to load the appropriate fixtures.
We also attempt to load a fixture called "custom" and fail silently if
we don't have one. This is where initial customisations can be done to
load particular settings or data into Toaster (for example layers or
default values for variables)
Make sure the value for TEMPLATECONF is available to checksettings so
that we can have a go a working out which default data to load.
(Bitbake rev: 7d14ca8cbabbb893e507a66e4cc6e3e77c1e8c84)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
These fixtures provide a recommended default configuration of toaster for
either using bitbake and oe-core or as part of poky.
They can be used as the sample configuration for writing custom configurations.
(Bitbake rev: d2f4ffbf061e874a7731441f8346fdc568e9f07f)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When using firefox and selenium we get an exception generated when a
disabled button click is attempted. This should happen in the test but
we need to catch the exception to make sure it doesn't cause the test to
fail.
[YOCTO #10056]
(Bitbake rev: 06f74fe91f8e162f49a4e856a62a97093b48262a)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If the user explicitly passes in "--help" then it should return 0. This
is the convention follow by the typical application. This allows the
user to check for options without triggering an error.
(Bitbake rev: 1d5102fe6c932dad1c2b975385e10a33a91ba1a4)
Signed-off-by: Randy Witt <randy.e.witt@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The git annex fetcher needs git annex to be initialized. Previously
it was using 'git annex sync' to do this, but that has the downside
of moving the checkout to the tip of the default branch. This means
that tags, SRCREV, etc don't work in the gitannex case.
(Bitbake rev: c1a57e2dd7fc96834643be5591a96f239215481a)
Signed-off-by: Terry Boese <terry.boese@vecima.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Revision d0f904d407f57998419bd9c305ce53e5eaa36b24 accidentally broke
items() and values() and made them cause stack overflows. Undo that
breakage.
(Bitbake rev: 88c5beca705efa7df4a96fb2aaf3f13c336ac328)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Starting from tar 1.29 the --exclude option won't work
anymore if is not used before the path. There are some
fetch modules that copy the ptest using tar and --exclude
option. This fixes these for bitbake.
[YOCTO #9763]
(Bitbake rev: cc71d5d9da71ea5f21d02f3b2fbf119bd2d794f0)
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fixes [YOCTO #9985]
Fixed an operator typo from ":=" to "+=" in the note
at the bottom of the section.
(Bitbake rev: 319d3d387161182069e6d1b3db17ccb539d097ed)
Signed-off-by: Scott Rifenbark <srifenbark@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fixes [YOCTO #7718]
In the "Executing a List of Task and Recipe Combinations"
section, I changed the improper bitbake command example to
use the correct syntax. This change was review feedback.
(Bitbake rev: c79fab1be4898ec88bfddd4c4f84e76dc3a3125b)
Signed-off-by: Scott Rifenbark <srifenbark@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fixes [YOCTO #10011]
Added paragraphs near the end to describe the role of virtual
targets.
(Bitbake rev: fc47bb99dbc1972dfb3a83c0f4c479963da64bcc)
Signed-off-by: Scott Rifenbark <srifenbark@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fixes [YOCTO #10003]
I added a small paragraph explaining what happens when expansion
of a variable that does not exist occurs.
(Bitbake rev: 8006da3f229d0227215ccd59cd273edacf72f9ce)
Signed-off-by: Scott Rifenbark <srifenbark@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fixes [YOCTO #9985]
Fixed some problems with the changes. A small typo for an
example and added a clarifying operator in another sentence.
(Bitbake rev: d8ae3775eefe3f7b62fc26cae5b742ae83850c13)
Signed-off-by: Scott Rifenbark <srifenbark@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fixes [YOCTO #9984]
Added a small note to the bottom to help clarify.
(Bitbake rev: 64bf49826088c56b739ed971251f05b4564c712e)
Signed-off-by: Scott Rifenbark <srifenbark@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
I had inconsistent usage of the "_append" style operator syntax
in the chaper. I was using a mix of <filename>_append</filename>
and "_append". I changed to "_append" for consistency.
(Bitbake rev: 5c5b88e402376268baa15c5d04d2592f28d76751)
Signed-off-by: Scott Rifenbark <srifenbark@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fixes [YOCTO #9985]
Made the following changes:
* Section Removal (Override Style Syntax): Added a small
qualifying sentence at the end to further define behavior
* Added new section "Override Style Operation Advantages":
This section provides some rationale behind the "_append"
style operations.
* Section "Examples": Changed an example to use the "="
operator rather than the "+=" operator.
(Bitbake rev: 797d9627baad9ccd3d55e825c0d705311f631f78)
Signed-off-by: Scott Rifenbark <srifenbark@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fixes [YOCTO #9984]
Added more detail to the examples that show the effects of
variable expanison.
(Bitbake rev: 480096ca93c0a649ebfff68dfc7d9bbe8eb2ea2d)
Signed-off-by: Scott Rifenbark <srifenbark@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This reverts commit 6f6cd0674fd1595f4e74b7da692e0c348b2660c6 as it was
a duplicated commit.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The link to the Chromedriver downloads page is dead, so
put in the correct URL.
(Bitbake rev: f0e6832bc33df2bb1b2f4b7f436ffbf023e24b13)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
For the latest Firefox versions, WebDriver requires a download of a
separate binary and an additional capability to be defined on it.
Modify our tests so that when "marionette" is set as the browser,
this capability is defined on the Firefox driver. Also add a note to the
README about the additional installation steps required.
(Bitbake rev: f6011d986f9a573a39e7b98af0aefe6cc88461ad)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If using OE's externalsrc with a source tree that is not tracked by git
and contains broken symlinks, you can receive "TypeError: unorderable
types: NoneType() < str()" within the file checksum code due to:
checksums.sort(key=operator.itemgetter(1))
Don't add files with no checksum to the checksums list in order to avoid
this.
(Bitbake rev: 484fe5a3f5b840e5422cbdff0eef9aecfe944a19)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If better_exec() throws a subprocess.CalledProcessError then show the output to
the user as it likely contains useful information for solving the problem.
(Bitbake rev: 8a6424ed871c3cbacd21cae8bc801197f83d67a6)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
While switching from master to krogoth build with a common download directory,
got a large number of warnings like the one listed below:
WARNING: freetype-2.6.3-r0 do_fetch: Couldn't load checksums from
donestamp /home/maxin/downloads/freetype-2.6.3.tar.bz2.done: ValueError
(msg: unsupported pickle protocol: 4)
These warnings are caused by the difference in pickle module
implementation in python3(master) and python2(krogoth). Python2 supports
3 different protocols (0, 1, 2) and pickle.HIGHEST_PROTOCOL is 2 where as
Python3 supports 5 different protocols (0, 1, 2, 3, 4) and
pickle.HIGHEST_PROTOCOL is obviously 4.
My suggestion is to use 2 since it is backward compatible with python2
(all the supported distros for krogoth provides python2 which supports
pickle protocol version 2)
(Bitbake rev: cc67800f279fb211ee3bb4ea7009fdbb82973b02)
Signed-off-by: Maxin B. John <maxin.john@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
It's possible that the logging FIFO doesn't do a complete read (or the sender a
complete write) with the result that an incomplete message is read in bitbake.
This used to result in silently truncated lines but since 42d727 now also
results in a warning as the start of the rest of the message isn't a valid
logging command.
Solve this by storing incoming bytes in a bytearray() across reads, and parsing
complete messages from that.
[ YOCTO #9999 ]
(Bitbake rev: 508112793ee7ace613f07695222997309a2ca58f)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add tests for the tasks and recipes sub-pages of the build
dashboard.
[YOCTO #9833]
(Bitbake rev: cecee440a76950f2824ea34b88e84185be493337)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
None of the columns in the built recipes table are marked
as not hideable, so it is possible to remove all the columns
and make the table disappear.
Set the recipe name and version columns as not hideable.
Also rename the "Name" column to "Recipe", for consistency with
the design and with other recipe tables.
[YOCTO #9833]
(Bitbake rev: 3abd0ac300462e6d1335018cf2d0420de7cc8b76)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The task, recipe and order columns in the build tasks table
should not be hideable. If they are, it's possible for the
table to have all of its columns hidden so that it no longer
displays.
Set the hideable property to prevent these columns from being
hidden.
[YOCTO #9833]
(Bitbake rev: dc1781e3783724823fd6b0c2d65f6b2771e8d0be)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The task display template formatting had split the Django
url template tag across two lines and broken it. This resulted
in a gibberish URL for task logs.
Fix by placing the tag and its arguments on a single line.
[YOCTO #9837]
(Bitbake rev: d6e88b7b410b6b99b47b031111a1126da9fd31b3)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Partially add back a revised version of the layersource handling so that
we can continue to support the old toasterconf.json and it's setup of
the local project.
(Bitbake rev: cc1a1bc2ea6ae058278d4ecf483f5ea00502c6cb)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If the migration didn't get the release conversion right for say, a
local or imported layer it would be handy to be able to edit this
in the django admin page.
Also useful for developers to be able to tweak layers on the fly.
(Bitbake rev: 0b23b6919ef1162a0c7fb5b5a961a24653c51eb2)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We don't need to keep track of layerindex data in our database. And
using branch==release is very confusing in the schema. Instead use the
existing Release definition to keep track of which release a
layer_version is for.
Remove the Branch model and all references to it.
Create a migration path to convert from up_branches to their
corresponding releases.
(Bitbake rev: f8f4cffe6fd371f3a7e63690c68f3fcb5dc1f297)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Adds basic progress % information and provides better description of
what is happening.
(Bitbake rev: 6393dbf97d450d2521c0bc9429da0987bb7720ec)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Remove and replace layersource model references in the tests and test
data. Remove the orm/test as this only tested LayerSource interactions
which have now been removed.
(Bitbake rev: 61a47cbc92c856690cb0e8da7102b2e669eaee0a)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Replace references to the now deprecated layersource models across
Toaster with the new enums for layer source types.
(Bitbake rev: 48c09c62eb979d840132e58144f0d81ffee675b1)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Move and refactor the layerindex layer source update mechanism so that
we don't have to track the layerindex objects in the toaster database.
Move this out of the orm and into the management command.
Paves the way for future improvement to allow you to specify a layer
index server as an argument to the command.
(Bitbake rev: f83527edc6d52a34cd73a9c3650ee484407e2e0c)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We had a lot of complexity around different layer sources to allow for
multiple sources and different priorities for these source. This was
implemented using rigged abstract classes which represented the
different layer sources when in fact just an enum/flag on the
layer_version object is sufficient for our needs.
Remove the LayerSourcePriority object as this is not needed. We no longer
have a problem of multiple layers coming from multiple sources so this
is not needed. Two migrations are added to first remove the child models
which represented layersources. Then a second migration is needed to
remove the LayerSource model it's self as Django can't understand the
non-standard base class dependency. Triggering this issue:
https://docs.djangoproject.com/en/1.8/topics/migrations/#dependencies
Clean up a number of flake8 warnings in classes which were modified.
[YOCTO #9853]
(Bitbake rev: 26624740418de95adb1f4a970a30a5f1149ebb79)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Remove the LayerSource admin from django admin interface. LayerSources
are not going to be manageable from the admin interface.
(Bitbake rev: 1c20ffcf88c5b73f6cf5e9b69b81e6bc8cd2493d)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We don't need to configure layer sources in the initial configuration as
this information is provided by the models.
(Bitbake rev: fd56c152699bc4c2d22b87728d0fefbff5209135)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In MultiStageProgressReporter, set a guard when we start the progress
so that it can't happen more than once. This fixes "Initialising
tasks.." being shown twice in succession when running bitbake in
non-interactive terminal mode.
(Bitbake rev: 923e68e069127ee7f6e11b91eb1cfa09d502a110)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
It turns out that progress information we can extract from a task is
rarely apportioned closely enough to the time taken for the ETA to be
accurate, so showing it is going to be misleading most of the time for
anything but the most basic of examples. Let's just remove it and avoid
misleading (or worse, annoying) the user.
Fixes [YOCTO #9986].
(Bitbake rev: 235db4870b11db97250979e647b54cdb5ce4fbb6)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If you specify custom widgets then we don't want to assume where the
"extra" position is - you should have to specify it, and if it isn't
specified it shouldn't just wipe out the last widget or you can start to
see odd behaviour if you're modifying the code.
(Bitbake rev: 19e33c10feb1637589ceb05b5e8d58b1e012ccb8)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When using a PREMIRROR with plain (non-unpack) files, a SRC_URI like
SRC_URI = "file://devmem2.c"
will cause devmem2.c to be a symlink in the WORKDIR pointing to the
local PREMIRROR.
Trying to apply a patch on this file will either modify the file on
the PREMIRROR or will fail due to sanity checks:
ERROR: devmem2-1.0-r7 do_patch: Command Error: 'quilt --quiltrc /cache/build-ubuntu/sysroots/x86_64-oe-linux/etc/quiltrc push' exited with 1 Output:
Applying patch devmem2-fixups-2.patch
File devmem2.c is not a regular file -- refusing to patch
(Bitbake rev: cfd481fe9799e7a4c6bfac32e56cc91cfcd81088)
Signed-off-by: Enrico Scholz <enrico.scholz@sigma-chemnitz.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
For some reason the data written in this way is coming back out the
files out of order. I've not been able to simplify the test case to a
point where this was standalone reproducible. Simplify the code and
write out the cache files sequentially since this seems to avoid the
errors and makes the code more readable.
(Bitbake rev: 14ec47f5f0566dbd280fae8a03160c8500ad3929)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We've seen cache corruption where the pairs come out in a different
order to the way we saved them for unknown reasons. Add better sanity
checking to give a more user friendly error rather than a crash/traceback.
Also allows the system to reparse and recover.
(Bitbake rev: 4be4a15491530bd6dc018033ad3d4b2562ab6e23)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Since we no longer have random data like version fields in these structures
and we can assume any extra cache data subclasses our class, simplify the
code.
This is mostly reindenting after removal of the pointless type checks.
(Bitbake rev: 5eb36278ac9975de1945f6da8161187320d90ba7)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Firstly, don't store the versions fields in memory in the cache objects
data store. This just complicates the code for no good reason.
Secondly, write the version fields to all cache files, not just the
core one. This makes everything consistent and easier.
(Bitbake rev: cb666262b2f986b5d9331dfb30458ef1a151fa4d)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If an "extras" cache file is corrupted, the system would not notice
and later fail with errors about missing entries. Add a test for this
which means we can fall back to re-parsing in those cases.
[YOCTO #9902]
(Bitbake rev: 51843d8f2bbe2e54db7593ca61984abe70423ef6)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Otherwise you can look at the log and wonder why parsing isn't happening
when it really is due to other code paths.
(Bitbake rev: b48d95677a4d285a77cda2892179965f7f8f06dd)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Idle timeout can be specified either by -T/--idle-timeout option or
by sessing BBTIMEOUT environment variable. Bitbake xmlrpc server
will unload itself when timeout exprired, i.e. when server is idle
for more than <idle timeout> seconds.
[YOCTO #5534]
(Bitbake rev: 5fa0b3a19e7d0f40790f737abeddf499d53f1f6a)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This option makes bitbake xmlrpc server to run in foreground.
It should be useful for debugging purposes.
(Bitbake rev: 9d4254be5853a546a346bf0d19919dcfba12773d)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Needed a section on executing a list of task and recipe combinations.
(Bitbake rev: 6f6cd0674fd1595f4e74b7da692e0c348b2660c6)
Signed-off-by: Scott Rifenbark <srifenbark@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Added a minor tweak to the third paragraph to be more inclusive
of the topic.
Fixes [YOCTO #9970]
(Bitbake rev: a5d36e8bccf35ffbca41a4facaa041d36f587529)
Signed-off-by: Scott Rifenbark <srifenbark@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In the BBDEBUG variable description, the "-d" parameter needed to
be "-D". Fixed it.
Fixes [YOCTO #9950]
(Bitbake rev: 475e7611f6c5b884d86152cb0b334e9d96849608)
Signed-off-by: Scott Rifenbark <srifenbark@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fixes [YOCTO #9909]
Added some technical clarifications to the existing note to be clear
about what is causing the limitations in this case. Applied some
formatting fixed to for the use of the include word.
(Bitbake rev: e9ad2e0f6d2681dd793cc39c468eb86e57fd6f48)
Signed-off-by: Scott Rifenbark <srifenbark@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Applied formatting to a "after" word that should be
<filename>after</filename>. Also added a new clarifying paragraph in
the "Recursive Dependencies" section.
Fixes [YOCTO #9861]
(Bitbake rev: ada90f68afd17cb85cb5957f3f91b537ee3599f6)
Signed-off-by: Scott Rifenbark <srifenbark@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Added a new Perforce Fetcher section in the same spirit as the existing
sections for other supported fetchers. Changes included the new section,
removal of the bulleted item that mentioned this fetcher as an
"additional" fetcher, and the creation of a new variable in the glossary
named P4DIR.
(Bitbake rev: 47e03b1789ee1c18407dbac33a6c235752019865)
Signed-off-by: Scott Rifenbark <srifenbark@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fixes [YOCTO #9909]
Added a note that talks about how the BBCLASSEXTEND mechanism
adds recipe variants.
(Bitbake rev: 185c9c5fc205fe9c9daf5238f11b92bd0954f5c6)
Signed-off-by: Scott Rifenbark <srifenbark@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fixes [YOCTO #9861]
In the "Dependencies internal to the .bb File" section, I placed a
note providing more detail on how recipes are built regarding task
dependency.
(Bitbake rev: c2e72928fbd21d622860a54a55f4239ba27c07a2)
Signed-off-by: Scott Rifenbark <srifenbark@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fixes [YOCTO #9861]
I updated the section on dependencies to give a couple of examples
for task dependencies within a single recipe and dependencies for
tasks between two individual recipes.
(Bitbake rev: 231deeb2b0b3847cba971ca30c16e29357100ae6)
Signed-off-by: Scott Rifenbark <srifenbark@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There is no default.
Fixes [YOCTO #9683]
(Bitbake rev: 886afd6d876c9ae04cf681931db8f1774dc53096)
Signed-off-by: Scott Rifenbark <srifenbark@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Needed a section on executing a list of task and recipe combinations.
(Bitbake rev: 17f7f366df3a5951ae88e24c43aecf3d65d83c14)
Signed-off-by: Scott Rifenbark <srifenbark@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In the runqueue cleanup/conversion, "dep" was mistakenly used where "tid" should
be leading to incorrect task-depends.dot files and causing general confusion.
Fix this, its clearly incorrect looking at the code.
(Bitbake rev: 689730dbb068c5ea3593e7b92fe5d5e5c0c3760a)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In order to reuse the Selenium helper outside a Django
environment (for functional testing), add a new module
containing the base class SeleniumTestCaseBase, which only
inherits unittest.TestCase
Add a class SeleniumTestCase with multiple inheritance of
StaticLiveServerTestCase and SeleniumTestCaseBase to prevent
existing tests from breaking.
(Bitbake rev: 1177b3f368d8b7f8557eb649adb2e327b6df801c)
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If environment variable BBSERVER == 'autostart' bitbake will
automatically load server if it's not running yet.
If host and port are in bitbake.lock then bitbake tries to check
if server is running and responses to commands and starts new
server only if this check fails.
[YOCTO #5534]
(Bitbake rev: 89c6e625d47303b2aad8e6645762f17aee01b2d4)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
All environment variables that are not in the list returned by
preserved_envvars_exported are cleaned by bb.utils.clean_environment.
Added BBSERVER to the list as we need to access it in bb/main.py
after the call of bb.utils.clean_environment.
(Bitbake rev: 15c4ea679f4fe097a9f21cccfc82907b5f39a4e4)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Implemented check_connection function. The purpose of this function
is to check if bitbake server is accessible and functional.
To check this this function tries to connect to bitbake server and
run getVariable command.
This API is going to be used to implement autoloading of bitbake
server.
[YOCTO #5534]
(Bitbake rev: 1a18f5ceb478f766b53850451549333f655621ea)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Doing both os.unsetenv(foo) and then del os.environ[foo] is pointless as del
will call unsetenv automatically.
(Bitbake rev: a4463e2ff3c7d234320176d671719243292f1af0)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Implement progress reporting support specifically for the fetchers. For
fetch tasks we don't necessarily know which fetcher will be used (we
might initially be fetching a git:// URI, but if we instead download a
mirror tarball we may fetch that over http using wget). These programs
also have different abilities as far as reporting progress goes (e.g.
wget gives us percentage complete and rate, git gives this some of the
time depending on what stage it's at). Additionally we filter out the
progress output before it makes it to the logs, in order to prevent the
logs filling up with junk.
At the moment this is only implemented for the wget and git fetchers
since they are the most commonly used (and svn doesn't seem to support
any kind of progress output, at least not without doing a relatively
expensive remote file listing first).
Line changes such as the ones you get in git's output as it progresses
don't make it to the log files, you only get the final state of the line
so the logs aren't filled with progress information that's useless after
the fact.
Part of the implementation for [YOCTO #5383].
(Bitbake rev: 4027649f422ee64b1c4e1ad8d48ac295050afbff)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If a build doesn't produce any image files, the package count
and size shouldn't be shown.
Also add some metadata to build dashboard elements so it is clear
what they're for, and so they can be queried by the tests.
(Bitbake rev: 3d5090af4475b1d0bb56911a8e30abf9097c1b3c)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: bavery <brian.avery@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add tests for display of image, kernel and SDK artifacts on the
build dashboard, checking that the "Images" option in the left-hand
menu and the "Build artifacts" section display correctly for
different types of build.
Also add metadata to elements on the build dashboard so it's clearer
what they represent, and to assist in finding them in the tests.
Add a method to the test helper to make it more convenient to check
whether a single element matching a selector exists.
[YOCTO #8556]
[YOCTO #8563]
[YOCTO #9500]
(Bitbake rev: 644a888ce5a2141f2e6e1c22430e196b65cb1313)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: bavery <brian.avery@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The build dashboard customise image button (for creating a new
custom image based on an image recipe used by a build) shouldn't
rely on targets: whether a new custom image can be created or not
depends on whether any of the recipes used by the build are image
recipes.
Modify the method used to determine whether a build has customisable
images to look at the image recipes used during the build, rather
than whether the targets run by the build refer to image recipes.
(Bitbake rev: 6648876c91134bda8498b4f8d7ace9147ec0d985)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: bavery <brian.avery@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Store the path to the *.rootfs.manifest file for targets which
generate images.
A link to the package manifest is displayed in the build dashboard
for targets which produce image files.
Like the license manifest path, if a target would have produced
the package manifest (but didn't, because it already existed), that
path is copied from the target which did produce the package
manifest.
(Bitbake rev: 79b8e349a0da2ea6b97ad82daa5837e6dfffe0af)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: bavery <brian.avery@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
SDK targets (populate_sdk) produce SDK artifacts but no image files.
Currently, these targets appear under the "Images" heading in the
build dashboard, even though they aren't strictly image targets.
Change the heading to "Build artifacts". Also remove the section
which states that a build produced no image files: this is not
correct for populate_sdk targets (those targets don't produce
image files under any circumstances); and other changes mean
that all targets which do produce images will now show those
files.
The check for whether to display the "Build artifacts" section also
needs to change, as we show targets here which didn't produce any
images but did produce SDK artifacts.
[YOCTO #8556]
(Bitbake rev: b4dce68045c4615e7a6a474e952f670721a3b54e)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: bavery <brian.avery@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In the build dashboard, add a section for manifests to each target,
which will contain the license and package manifests for a target.
As we don't record the package manifest (yet), just move the
license manifest link to that section for now.
(Bitbake rev: 58cb685b567a081834a605530957f165d35fcf5e)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: bavery <brian.avery@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a has_image_targets() method to Build, and use that to hide
the "New custom image" button on the build dashboard if a build
has no targets which build images.
[YOCTO #9514]
(Bitbake rev: 3c4b053e44ea512ef2ced67289a7b0161db6ce9b)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: bavery <brian.avery@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In the build dashboard, we had issues with showing images correctly,
as we were using the is_image property of targets to determine
whether a target would have image files. This property can
be set to True if a target refers to an image recipe
(e.g. "core-image-minimal"), even if the task used in the build
didn't produce any image files.
By adding has_images() methods to the Target and Build objects,
which count associated Target_Image_File objects,
we can correctly determine whether a target has image files
associated with it, and if any of the targets for a build has
image files. This means that we can screen out the left-hand
"Images" menu options for builds which contained image-related
targets (e.g. "core-image-minimal") but which didn't produce
any images (e.g. "rootfs" task).
[YOCTO #9500]
[YOCTO #9784]
(Bitbake rev: f6bba0ff254d5ed3163151d4b938f3a43c9acb0a)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: bavery <brian.avery@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If a target is built which is classified as an "image" target
(e.g. "core-image-minimal"), Toaster reads the list of files in
the image (from the files-in-image.txt file).
However, Toaster continues to do this for builds which don't
produce images, if the recipe providing the target is an
image recipe. This can result in a list of files in the image
being attached to a target which didn't produce an image (e.g.
rootfs).
When associating files with an image, ensure that only targets
with a task which produces an image have "files in the image"
associated with them.
[YOCTO #9784]
(Bitbake rev: 44375d0c2a88e0070b8067c9285b89c54eaf3152)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: bavery <brian.avery@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
SDK artifacts were previously picked up by toaster.bbclass and
notified to buildinfohelper (via toasterui). The artifacts
were then added to the Build object, so that it wasn't clear
which artifact went with which target; we were also unable
to attach SDK artifacts to a Build if they had already been
attached to a previous build.
Now, toaster.bbclass just notifies the TOOLCHAIN_OUTPUTNAME when
a populate_sdk* target completes. The scan is moved to buildinfohelper,
where we search the SDK deploy directory for files matching
TOOLCHAIN_OUTPUTNAME and attach them to targets (not builds).
If an SDK file is not produced by a target, we now look for a
similar, previously-run target which did produce artifacts.
If there is one, we clone the SDK artifacts from that target
onto the current one.
This all means that we can show SDK artifacts by target, and should
always get artifacts associated with a target, regardless of whether
it really build them.
This requires an additional model, TargetSDKFile, which tracks
the size and path of SDK artifact files with respect to Target
objects.
[YOCTO #8556]
(Bitbake rev: 5e650c611605507e1e0d1588cd5eb6535c2d34fc)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: bavery <brian.avery@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When buildinfohelper records the targets for a build, it looks
up any existing targets for a build and creates them if they
are not present. This is because in the case of Toaster-triggered
builds, the Target objects have already been created (inside
triggerBuild()) and don't need to be recreated; but in the case
of cli builds, the Target objects have to be created by
buildinfohelper.
The issue is that the code for retrieving an existing target for
a build only looks for Targets with a matching target and build,
e.g. Targets for build X with target "core-image-minimal". But it
is perfectly legitimate to call bitbake with a command like
"bitbake core-image-minimal:do_populate_sdk
core-image-minimal:do_populate_sdk_ext". In such a case, the
code which looks for matching targets finds two objects, as it
doesn't filter by task.
Add the task into the filter for the Target so that only one
Target object is be returned. Note that a command
line like "bitbake recipe:task recipe:task" will still cause an
error as bitbake doesn't de-duplicate the command line arguments
and will run the recipe:task combination twice.
(Bitbake rev: 1c0a689fdaae6469d4afb98583161073d32ea50b)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: bavery <brian.avery@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The bzImage and modules files were previously attached to a build,
rather than to the target which produced them. This meant it was
not possible to determine which kernel artifact produced by a
build came from which target; which in turn made it difficult to
associate existing kernel artifact with targets when those
targets didn't produce artifacts (e.g. if the same machine + target
combination was built again and didn't produce a bzImage or modules
file because those files already existed).
By associating kernel artifacts with the target (via a new
TargetArtifactFile model), we make it possible to find all
the artifacts for a given machine + target combination. Then, in
cases where a build is completed but its targets don't produce
any artifacts, we can find a previous Target object with the same
machine + target and copy its artifacts to the targets for a
just-completed build.
Note that this doesn't cover SDK artifacts yet, which are still
retrieved in toaster.bbclass and show up as "Other artifacts",
lumped together for the whole build rather than by target.
[YOCTO #8556]
(Bitbake rev: 9b151416e428c2565a27d89116439f9a8d578e3d)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: bavery <brian.avery@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Refactor retrieval of suffix from image file path, making it a
a method on Target_Image_File. This makes it easier to use this
in the build dashboard for individual images, plus reduces the
complexity of the code required to get all of the image file
suffixes for a build.
(Bitbake rev: 9c38de3dec74c122c2060cad37331bdafc6858ec)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: bavery <brian.avery@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Move the image and artifact scan code from toaster.bbclass and
consolidate its logic with the existing logic in buildinfohelper.
Remove handler setup for events which used to be fired from
toaster.bbclass but which are now handled directly by buildinfohelper.
[YOCTO #8556]
(Bitbake rev: f0085cd554604cfff4a3f40a34825fbb6878004f)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: bavery <brian.avery@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The build dashboard was showing the targets for the build in the page
heading and title as "Target object".
Add a filter which extracts the "target" from each Target object
as a string so that the heading and title display correctly.
Also sort the image file suffixes alphabetically.
[YOCTO #8556]
(Bitbake rev: 33a24992139e7dc05d14bbe1da60299b732945aa)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: bavery <brian.avery@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Runqueue errors direct the user to view the "failure below",
but no additional error message is available.
Log the stacktrace so that the user can see what went wrong.
Also fix a typo in the log message.
(Bitbake rev: e191f401e372ee181bc02250232ad9cb9a0e9477)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: bavery <brian.avery@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The OrderedDict's item is sorted by insertion order, there might be a
problem when build the same recipe again, for example:
- First build of acl:
Depends: libattr1 (>= 2.4.47), libc6 (>= 2.24)
- Second build of acl:
Depends: libc6 (>= 2.24), libattr1 (>= 2.4.47)
They are exactly the same depends, but tools like "diff" doesn't think
so. Return sorted OrderedDict will fix the problem.
(Bitbake rev: a392f19f16ef8202ce3c12afbeb186a02438da17)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In a few places we use the fetcher code to fetch files outside of a
task, for example uninative in OE. In that case the pid of the event is
0 and that was causing an error in BBUIHelper.eventHandler(). Check the
pid and do nothing if it's 0.
(Bitbake rev: 59cb919e5cd5c653fb4d69b2d6a4320648443e10)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When bitbake executes a shell or Python function it can cd/chdir() into a
directory before executing the task. If no directory is specified then the
default of $B is used. However $B is an OpenEmbedded variable and BitBake
shouldn't be aware of it.
To solve this change the semantics slightly so that if no directory is
specified, the current working directory isn't changed. There's also a sanity
check that emits a warning if a Python task does os.chdir() without restoring
the old path, and the previous working directory is restored.
This does change semantics: whereas before a function in OE would have $B as the
working directory unless specified, now the working directory is the top of the
build tree. Any breakage this causes can be solved by either adding
do_some_task[dirs] = "${B}" or by using absolute paths in the task.
[ YOCTO #4634 ]
(Bitbake rev: 67a7b8b021badc17d8fdf447c250e79d291e75f7)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The messaging FIFO is UTF-8, so decode the command as UTF-8 as well as the value
as otherwise "bberror" != b("bberror") and none of the messages from shell
functions are ever displayed.
Also add an else to the command parser so unhandled commands are noticed.
[ YOCTO #9947 ]
(Bitbake rev: 42d727743fa599e0a3c5ad2c29a1e6ede1a918bb)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
With Python 3 create_string_buffer needs a bytes() not a str() but as we were
catching all exceptions nobody noticed.
[ YOCTO #9910 ]
(Bitbake rev: 6576a9a95486c28a01d4211b4a33cc3e2c55a7cc)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add the front end feature to delete a layer from the layer details page.
[YOCO #9184]
(Bitbake rev: 91815229f60eb9deba7d299f05c69b52ff1df59c)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Also clean up flake8 warnings in XhrBuildRequest
(Bitbake rev: aa0845242eda8650a97180bf6675551c26554cde)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Update, clean up and move the api for updating a layerversion from the
views to api. Also update the layerdetails page to include the
layerversion id in the url getter.
[YOCTO #8952]
(Bitbake rev: 20f4e23bc86290f0a42881a7cac44c41eafa86fc)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When using toaster-eventreplay to run a bitbake event file
through toasterui/buildinfohelper, errors occur when the
tasks are updated with buildstats info:
RuntimeWarning: DateTimeField Task.started received a naive
datetime (2016-07-06 09:15:22.070000) while time zone support
is active.
This is because a method in buildinfohelper returns a naive
datetime, but Django is expecting timezone-aware datetimes.
Ensure that datetimes used to set the started/ended times on
tasks are converted to timezone-aware datetimes.
(Bitbake rev: df9f4337bec87024ea6a43138c6080a755eb7fab)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Rewritten toaster-eventreplay to make code working as expected,
more compact and readable.
[YOCTO #9585]
(Bitbake rev: 45370a860b24a761d1b6e08ba752079cc45f54da)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
class MockConfigParameters has only one attribute and only __init__
method. Replacing it with namedtuple makes code less nested and more
readable.
[YOCTO #9585]
(Bitbake rev: 5d4df14b0d38f6c89ca16de6dada58b4bb015d71)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Event objects are represented by base64-encoded strings in
the event file and can't be loaded by existing eventreplay code.
Fixed the code of loading events from file by decoding base64 strings
into the binary form and loading them with pickle.load.
[YOCTO #9585]
(Bitbake rev: a55c280c167f84caed6518119246e5a55f56cfd4)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Stored event mask list as self.eventmask for future use.
Fixed Exception: Command setEventMask not implemented.
[YOCTO #9585]
(Bitbake rev: 2e09074c70e89402de8f883dd402cd729118fc7e)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fixed AttributeError: 'MockConnection' object has no attribute
'getEventHandle'
[YOCTO #9585]
(Bitbake rev: 6deae30480a4288da0c8b3529e61f3495f260f24)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Command-line builds were displayed incorrectly, so that the
HTML elements for other builds were being "consumed" by the
command-line build elements due to incorrect positioning of
element end tags.
Fix by ensuring end tags close elements in the right places.
As the indentation was all over the place in the most recent
builds section template, it was almost impossible to see what the
problem was. So that was fixed, too.
[YOCTO #9842]
(Bitbake rev: 01659389813ad61d4f75b9f8d71528581322f0b0)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Worker needs input stream in binary mode as it reads binary content
from it. Current code does it by detaching a buffer from sys.stdin
and assigning it back to sys.stdin.
Detached buffer is io.BufferedReader in binary mode. This operation
is implicit as its purpose is not easily understandable from the code.
Replacing it with fdopen(sys.stdin.fileno(), 'rb') should make the
code more understandable.
Assigning the buffer to sys.stdin is not needed as worker doesn't
use sys.stdin. Moreover, it leads to difficult to debug issues down
the stack. For example, devpyshell doesn't work without reopening
sys.stdin in text mode. This is not needed anymore after this fix as
sys.stdin is not changed in worker code and remains in text mode.
(Bitbake rev: b26bcff4c4d72775f1def7e769015464953b955c)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Clean up and fix the most frequently built targets for the "Most built
recipes" section for the project configuration page.
[YOCTO #9846]
(Bitbake rev: 860475cfdd35301fb609ab3c89347566b0ca0adc)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Some pages contain other build buttons which may have the same class
attached. Make sure that we only select the buttons in the tables where
layerBtn is used.
[YOCTO #9841]
(Bitbake rev: ec083fd943c6996307beb3be3421403870d2f2b7)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add basic tests to validate the value, user types
in the text box for DL_DIR and SSTATE_DIR. Added
test case to validate the first char and inclusion
of space between the characters.
[YOCTO #9646]
(Bitbake rev: 1531e98c5ae1693d11d692c3589df29dff9364df)
Signed-off-by: Sujith H <sujith.h@gmail.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Validation logic on the project configuration page
prevented a user from adding ${variable} to these paths.
Update validation so a user can see a better message
when they type characters into the text inputs.
Two types of validation are implemented.
Either:
* The value should start with a "/", to allow absolute paths.
or
* The value should start with a "$", to allow bitbake
variables like ${TOPDIR}.
[YOCTO #9646]
(Bitbake rev: fca50153caef62a473d1812fd59ec7effe09f8c5)
Signed-off-by: Sujith H <sujith.h@gmail.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Remove SSTATE_MIRRORS from getting blacklisted. Hence the SSTATE_MIRRORS
should be removed from projectconf.html file so that users can know that
this variable is no more blacklisted.
[YOCTO #9598]
(Bitbake rev: b060eecae417521d8340748e7ace7a77488bd596)
Signed-off-by: Sujith H <sujith.h@gmail.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Remove SSTATE_MIRRORS variable from blacklists in views.py
This helps user to point mirrors using SSTATE_MIRRORS with
toaster.
[#YOCTO 9598]
(Bitbake rev: c3dd4fb193c80ed9a6dbddf7f6a1eadd0964ac8c)
Signed-off-by: Sujith H <sujith.h@gmail.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The init function of the parent class fires a progress event for 0
progress rather than a start event. UI code was assuming that progress
events should always have a start event first. This change ensures that
the start event is correctly generated.
This fixes crashes that were seen in knotty in some configurations.
(Bitbake rev: 9841651e050a3e9f395ab3c62545c51197734584)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Mistakes can happen with the generation of the progress events, change
knotty to be more tolerant of this rather than crashing, reporting to the
user when something unexpected happens. I haven't debugged why multiple
finish events appear to be triggered.
(Bitbake rev: 7dd06b1016b36420a9c55a45ff29dd64ae1dbcda)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When "Preparing RunQueue" shows up you can expect to wait up to 30
seconds while it works - which is a bit long to leave the user waiting
without any kind of output. Since the work being carried out during this
time is divided into stages such that it's practical to determine
internally how it's progressing, replace the message with a progress
bar.
Actually what happens during this time is two major steps rather than
just one - the runqueue preparation itself, followed by the
initialisation prior to running setscene tasks. I elected to have the
progress bar cover both as one (there doesn't appear to be much point in
doing otherwise from a user perspective). I did however describe it as
"initialising tasks".
(Bitbake rev: 591e9741e108487ff437e77cb439ef2dbca42e03)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add the ability to enter a mode where only a specified whitelist of
tasks can be executed outright; everything else must be successfully
provided in the form of a setscene task (or covered by a setscene task).
Any setscene failure outside of the whitelist will cause the build to
fail immediately instead of running the real task, and any real tasks
that would execute outside of the whitelist cause an immediate build
failure when it comes to executing the runqueue as well.
The mode is enabled by setting BB_SETSCENE_ENFORCE="1", and the
whitelist is specified through BB_SETSCENE_ENFORCE_WHITELIST, consisting
of pn:taskname pairs. A single % character can be substituted for the pn
value to match any target explicitly specified on the bitbake command
line. Wildcards * and ? can also be used as per standard unix file name
matching for both pn and taskname.
Part of the implementation of [YOCTO #9367].
(Bitbake rev: 624722c067a7fdd0c0f5d8be611e1f6666ecc4a2)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Quiet output mode disables printing most messages (below warnings) to
the console; however these messages still go to the console log file.
This is primarily for cases where bitbake is being launched
interactively from some other process, but where full console output is
not needed.
Because of the need to keep logging all normal events to the console
log, this functionality was implemented within the knotty UI rather
than in bb.msg (where verbose mode is implemented). We don't currently
have a means of registering command line options from the UI end, thus
the option actually has to be registered in main.py regardless of the
UI, however I didn't feel like it was worth setting up such a mechanism
just for this option.
(Bitbake rev: db95cdef08e339dec7462bfde3ad7d75c1c60dd8)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In addition to the "currently running n tasks (x of y)" message, show a
progress bar for another view on how much of the build is left. We have
to take care to reset it when moving from the scenequeue to the
runqueue, and explicitly don't include an ETA since not all tasks take
equal time and thus it isn't possible to estimate the time remaining
with the information available.
(Bitbake rev: de682015a3fefeff36ddc4197641a700f3fb558d)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add support code on the BitBake side to allow sstate.bbclass in
OpenEmbedded to report progress when it is checking for availability of
artifacts from shared state mirrors.
Part of the implementation for [YOCTO #5853].
(Bitbake rev: 070ae856da0715dbaf4c560c837ea796ffc29f00)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a class to help report progress in a task that consists of multiple
stages, some of which may have internal progress (do_rootfs within
OpenEmbedded is one example). Each stage is weighted to try to give
a reasonable representation of progress over time.
Part of the implementation for [YOCTO #5383].
(Bitbake rev: 751b75602872a89e8b1a7c03269bc0fdaa149c6f)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
For long-running tasks where we have some output from the task that
gives us some idea of the progress of the task (such as a percentage
complete), provide the means to scrape the output for that progress
information and show it to the user in the default knotty terminal
output in the form of a progress bar. This is implemented using a new
TaskProgress event as well as some code we can insert to do output
scanning/filtering.
Any task can fire TaskProgress events; however, if you have a shell task
whose output you wish to scan for progress information, you just need to
set the "progress" varflag on the task. This can be set to:
* "percent" to just look for a number followed by a % sign
* "percent:<regex>" to specify your own regex matching a percentage
value (must have a single group which matches the percentage number)
* "outof:<regex>" to look for the specified regex matching x out of y
items completed (must have two groups - first group needs to be x,
second y).
We can potentially extend this in future but this should be a good
start.
Part of the implementation for [YOCTO #5383].
(Bitbake rev: 0d275fc5b6531957a6189069b04074065bb718a0)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Since we're going to make some minor extensions to it, it makes sense to
bring in the latest version of python-progressbar. Its structure has
changed a little but the API hasn't; however we do need to ensure our
overridden _needs_update() function's signature in BBProgress() matches
properly.
(Bitbake rev: c3e51d71b36cbc9e9ed1b35fb93d0978e24bc98a)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If you're looking to find the latest console log repeatedly it can be a bit
tedious - let's just create a symlink just as we do with other logs to
make it easy to find.
(Bitbake rev: e9f41c0507a6527bf2ed86506813d4d4a89f8ebf)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Some services such as SourceForge seem to struggle to keep up under load, with
the result that over half of the autobuilder checkuri runs fail with
sourceforge.net "connection timed out".
Attempt to mitigate this by re-attempting once the network operation on failure.
(Bitbake rev: 54b1961551511948e0cbd2ac39f19b39b9cee568)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Otherwise the function like d.getVarFlag(e, 'task', True) which is used by
do_listtasks will still get it, and list the deleted tasks.
(Bitbake rev: 779d73619daf59f76f5b0313e7fb5409f6e82553)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Restructured EventWriter code to make it more readable:
- got rid of init_file method as it's called only once
- renamed exception variable e -> err
- renamed event variable e -> evt
- simplified main 'if' structure of send method
(Bitbake rev: 31977e7bb98f676197c6cee66f6ab4c12d4dcbde)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
class EventLogWriteHandler is a simple wrapper class with only one
class member. Replacing it with namedtuple makes code less nested and more
readable.
(Bitbake rev: 7c5b6812d32d173df36e7f9fc1d877329e79f994)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There is no need to remove output file as it gets rewritten by
open(self.eventfile, 'w') anyway.
(Bitbake rev: 1fc9957837b7038dfb983217a3fcd880f143e3a4)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
pickle converts python objects into the binary form that can't be
decoded to text and therefore can't be converted to JSON format.
Attempt to convert event objects raises this error:
TypeError:
b'\x80\x03cbb.runqueue\nrunQueueTaskSkipped\nq\x00)...
is not JSON serializable
Encoded pickled event objects to base64 to be able to convert data
structure to JSON.
[YOCTO #9803]
(Bitbake rev: f18055237e6084f90f6221442e3ba021dcc59c50)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
EventLogWriteHandler object was created and used in
BBCooker.initConfigurationData.
This causes creation of multiple EventLogWriteHandler objects
and results in duplicated entries in the output event file
as BBCooker.initConfigurationData is called multiple times.
Added eventlogfile parameter to EventLogWriteHandler to avoid using
global variable DEFAULT_EVENTFILE.
Moved EventLogWriteHandler to the module level.
Created EventLogWriteHandler object in BBCooker.__init__ to ensure that only
one handler object is created.
(Bitbake rev: d3ad8eee850ec2df54aa09fae44cc7e69c12f32a)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The code is still a bit icky (and should be refactored to not use
Gdk.threads_enter/leave) but it should work about as reliably as
it did with Gtk+2.
Based on earlier patches by Maxin and Joshua.
(Bitbake rev: 8eee64a64144e27b5b8c2aca88e138882c3deab7)
Signed-off-by: Jussi Kukkonen <jussi.kukkonen@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Remove unused css file that still had the bootstrap v2 license in it.
(Bitbake rev: a8fe4177a6303aa57301c977c1daf7d4ff6ec586)
Signed-off-by: bavery <brian.avery@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In 2c88afb6 find_chains()'s taskid argument was renamed to tid but
taskid is still used as key to explored_deps dictionary. Use tid instead
of taskid.
(Bitbake rev: 29a34ae8f5306d2779bcc761c52f1f9d13a0c0c5)
Signed-off-by: George McCollister <george.mccollister@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In recipes which use the perforce fetcher, enable use of SRCREV to
specify any of: ${AUTOREV}, changelist number, p4date, or label. This
is more in-line with how the other fetchers work for source control
systems.
Allow p4 to use the P4CONFIG env variable to define the server URL,
username, and password if not provided in a recipe.
This does change existing perforce fetcher usage by recipes and will
likely need those recipes which use the perforce fetcher to be updated.
No recipes in oe-core use the perforce fetcher.
References [YOCTO #6303]
(Bitbake rev: 6298696bb94a127cdec7964315f6891ba92cd026)
Signed-off-by: Andrew Bradford <andrew.bradford@kodakalaris.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Python 3 changed the return value of check_output to binary rather than
a string. This fix decodes the binary before calling splitlines, which
requires a string.
(Bitbake rev: 1072beefe172423873a22a10c7171e10d0401e1e)
Signed-off-by: Stephano Cetola <stephano.cetola@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The iterator used to create a search query refers to a
variable "x" which isn't set, causing an "'x' is not defined" error
and preventing table searches (on non-ToasterTables) from working.
Use the "field" variable instead, which contains the name of the
field to add to the query.
[YOCTO #9749]
(Bitbake rev: a3ebeb37f7bd4cffe6707c634b4f0a0ea52ccc45)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
First parameter of traceback.format_exc is a 'limit' - a number
of stracktraces to format.
Passing exception object to format_exc is incorrect, but it works in
Python 2 as this code from traceback module works:
while tb is not None and (limit is None or n < limit):
Comparing integer counter n with the exception object in Python 2
always results in True. However, in Python 3 it throws exception:
TypeError: unorderable types: int() < <Exception type>()
As format_exc is used in except block of handling another
exception this can cause hard to find and debug bugs.
(Bitbake rev: a9509949d7e2adba6e3cd89f97daa19a955855b5)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When parsing, we should reset the event handlers we registered when
done. If we don't do this, parse order may change the build, depending
on what the parse handlers do to the metadata.
This issue showed up as a basehash change:
ERROR: Bitbake's cached basehash does not match the one we just generated (
/media/build1/poky/meta/recipes-core/meta/nativesdk-buildtools-perl-dummy.bb.do_unpack)!
This is due to the eventhandler in nativesdk.bbclass being run, despite
this .bb file not inheriting nativesdk.bbclass. The parse order was
different between the signature generation and the main multithreaded
parse.
Diffsigs showed:
bitbake-diffsigs 1.0-r2.do_unpack.sigbasedata.*
basehash changed from 887d1c25962156cae859c1542e69a8d7 to cb84fcfafe15fc92fb7ab8c6d97014ca
Variable PN value changed from 'nativesdk-buildtools-perl-dummy' to '${@bb.parse.BBHandler.vars_from_file(d.getVar('FILE', False),d)[0] or 'defaultpkgname'}'
with PN being set by the event handler.
(Bitbake rev: 0219271d4130c1f4cf071c7577a4101c54c04921)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Another patch added migrations in the correct location (different date) so
these in the wrong directory can be removed.
(Bitbake rev: 8aa84abc952835792db1614b6fe2305ab9dca9e1)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
I'm not sure what possesed me when I wrote this code originally but its
indirection of everyting to use numeric IDs and position dependent lists
is horrific. Given the way python internals work, its completely and
utterly pointless from performance perspective. It also makes the code
hard to understand and debug since any numeric ID has to be translated
into something human readable.
The hard part is that the IDs are infectous and spread from taskdata
into runqueue and even partly into cooker for the dependency graph
processing. The only real way to deal with this is to convert everything
to use a more sane data structure.
This patch:
* Uses "<fn>:<taskname>" as the ID for tasks rather than a number
* Changes to dict() based structures rather than position dependent lists
* Drops the build name, runtime name and filename ID indexes
On the most part there shouldn't be user visible changes. Sadly we did
leak datastructures to the setscene verify function which has to be
rewritten. To handle this, the variable name used to specifiy the version
changes from BB_SETSCENE_VERIFY_FUNCTION to BB_SETSCENE_VERIFY_FUNCTION2
allowing multiple versions of bitbake to work with suitably written
metadata. Anyone with custom schedulers may also need to change them.
I believe the benefits in code readability and easier debugging far
outweigh those issues though. It also means we have a saner codebase
to add multiconfig support on top of.
During development, I did have some of the original code coexisting with
the new data stores to allow comparision of the data and check it was
working correcty, particuarly for taskdata. I have also compared
task-depends.dot files before and after the change. There should be no
functionality changes in this patch, its purely a data structure change
and that is visible in the patch.
(Bitbake rev: 2c88afb60da54e58f555411a7bd7b006b0c29306)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Using positions in lists for flags is an odd choice and makes the code
hard to maintain. Maintaining a list is slow since list searches are
slow (watch bitbake -n slow massively with it) but we can use a set()
instead.
This patch uses python sets to maintain the lists of tasks in each state
and this prepares for changing the task IDs from being integers.
(Bitbake rev: 8c1ed57f6ea475b714eca6673b48e8e5f5f0f9c3)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fix a number of selectors which have changed after the port to
bootstrap3. Also fix the modal wait_until_visible and returning of the
text for the radio buttons in the modals for edit custom image and new
custom image on the build dashboard.
(Bitbake rev: 5f80dac65f419825bd81a734273a2465d5a01bab)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Use underscore instead of dash in the file name for the test settings.
Also fix the import of the settings module.
(Bitbake rev: 0bdfcafdd1e2ebc10dc0cd343c8bb77f09a71c90)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fix the selectors after changes made for bootstrap3 and table links being
removed.
(Bitbake rev: fccc8869dc465b49f236c15a9aa14b7d06694b8e)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
- Update the string for recognising a duplicate image recipe
- Fix an incorrect relative import
(Bitbake rev: 830743b1aa29a5ac220141e9c24b2592d6cebb29)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Get the ttTypeahead object on the input to see if it's been initialised
correctly.
(Bitbake rev: d382b2afb2939ae1b69a80ad083fb4c47303f8d0)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
First parameter of traceback.print_exc and traceback.format_exc APIs is
a 'limit' - a number of stracktraces to print.
Passing exception object to print_exc or format_exc is incorrect, but
it works in Python 2 and causes printing only one line of traceback.
In Python 3 comparison of integer and exception object throws exception:
TypeError: unorderable types: int() < <Exception type>()
As these APIs are usually used in except block of handling another
exception this can cause hard to find and debug bugs.
(Bitbake rev: c5a48931ac8db9e56f978c50861c19d0d0c808e3)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Changed python to python3 in shebang to ensure that
manage.py is always run by python3.
(Bitbake rev: 3638b8e5390c36076e14c181e955505750031571)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fix deprecation warning specify the parser used for creating the
BeautifulSoup object.
(Bitbake rev: d34546e88881e89588206877ebaea506cda4f6c2)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The view code for downloading image files used the "r" flag
to read the file, then used the open file object to form the
HTTP response.
While this worked in Python 2, Python 3 appears to be more strict
about this sort of thing, and Django throws a UnicodeDecodeError
when a file opened this way is used in a response.
Open the file with the "b" flag (binary mode) so that Django can
correctly convert the binary file handle to an HTTP response.
(Bitbake rev: c4d67968d0ec1d5ff53cdc0dccf6a7869c89597b)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Previously, we didn't specify a specific version of Selenium.
When upgrading to Python 3 and installing Selenium to work with it,
the JS unit test broke, as the report format produced by Selenium
had changed.
Modify the test so that it works with the latest Selenium report
format.
Add a note to the README that the given Selenium version should
be used to prevent unexpected test failures.
(Bitbake rev: 571c2b70d3c123614618672ce7532bb5f4c36630)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The code previously imported urllib to make use of querystring
quoting, but was modified to support Python 3. During this
process, the reference to urllib was not fixed, which resulted
in table filters breaking.
Remove the reference to urllib (which is no longer imported)
and instead reference the imported unquote_plus() function.
(Bitbake rev: f66ee686d4e3002a4a38a7424bcc605a6df9db0b)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Removed unneeded code as it causes the following error in Python 3:
TypeError: expected bytes, bytearray or buffer compatible object
(Bitbake rev: 1626e986820da626c19aa8c664155a2cb1db8fd9)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Python 3 complains about 'wa' mode this way:
ValueError: must have exactly one of create/read/write/append mode
Fixed by using 'a' mode.
[YOCTO #9584]
(Bitbake rev: eb23a84e079125a08d0c8e910f7035ad9584c432)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Decoded response content to str to pass it to json.load as it breaks
in Python 3 with this error:
TypeError: expected bytes, bytearray or buffer compatible object
[YOCTO #9584]
(Bitbake rev: 3566ecc3ae682358d1fc58bdc0bed5cbed5e9608)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Explicitly use python3 so that the modules for python3 are checked.
(Bitbake rev: e7951541c34c5561187110ba0ec69b9c45022747)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In the 'outcome' filter for tasks, remove the 'not applicable' option,
since it should not be exposed to users.
(Bitbake rev: 4e21817b0d7a91e634bdb2069850627c38fde053)
Signed-off-by: Belen Barros Pena <belen.barros.pena@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The explanatory message in the empty state of the packages built table
was missing some spaces.
(Bitbake rev: 70f600f86ec4d536004d968919e86d2afa58d585)
Signed-off-by: Belen Barros Pena <belen.barros.pena@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Make sure the current page is always highlighted in the left navigation.
(Bitbake rev: 3d7f5d6ae843bcc3b5af1a9414ab5cb2759d0853)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Belen Barros Pena <belen.barros.pena@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Port all the pages in the build analysis area to bootstrap version 3.
(Bitbake rev: f963b73f0bf32db2df39dd79d8d85184c280cda0)
Signed-off-by: Belen Barros Pena <belen.barros.pena@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Design change to have no links for non-named fields in BuiltPackagesTable.
BuiltRecipesTables and BuildTasksTable.
Additionally:
- Fix class and elements for Bootstrap3 in help text.
- Change title of InstalledPackages table to Packages Included.
- Change which columns are default shown/hidden columns.
(Bitbake rev: 7c377e161a90184a4786e33951d8c0e30688ba3b)
Signed-off-by: Belen Barros Pena <belen.barros.pena@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
'k' was replaced by 'key' at some point but not fixed in the
body of the loop. This caused a failure when the the query
was constructed for a filtered queryset, due to the variable
not being defined.
(Bitbake rev: 37fb9a364e645baadda30cc74a18baa565f39857)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In some pages of the UI, the UI tests failed because parts of
the UI which should be visible were being reported as not
visible by the Selenium PhantomJS driver.
On investigation, it turns out that PhantomJS uses a very narrow
default window. This meant that some parts of the UI were being
clipped and were thus not "visible" to the driver, causing test
failures (specifically, on the new custom image page).
Ensure that the window is maximized before running tests to
prevent this happening.
(Bitbake rev: 3f61bef9359926211be7a3177a2071963726ead5)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
After porting the build table to a unified mechanism for showing
dependencies in tables it highlighted that the dependencies selected to
be shown were un-filtered. i.e. all dependencies from all contexts were
shown. The context for a package's dependencies is based on the target
that they were installed onto, or if not installed then a "None" target.
Depending on where the template for the dependencies are show we need to
switch this target which is why a filter and utility function on the
model is added.
Additionally to use the same templates in the build analysis we also
need to optionally add links to the build data for the packages being
displayed as dependencies.
Customising a Custom image recipes may or may not have a target
depending on whether they have been built or not, if not we do a best
effort at getting the dependencies by using the last known target on
that package to get the dependency information.
[YOCTO #9676]
(Bitbake rev: 31e7c26cc31a7c8c78c1464fa01581683bfd2965)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If a cell template rendering causes an exception catch it and carry on,
this at least allows the table to show the rest of the data if nothing
else. Also improve the error logging so that it's possible what the
offending template snippet was.
(Bitbake rev: 566b4fbe9cfe4c6c0605c4f6444083ee4cf8b958)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
- Add new build tables to be tested
- Add required data into the fixture and clean up a few empty fields
- Fix the SoftwareRecipesTable specific test so as not to rely on two
particular defined recipes
(Bitbake rev: 7cf23671659666b27b5629fecd5f947f9bdb94e0)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
As in the old build tables it's useful to jump and highlight a
particular row in the table using the #hash in the URL.
(Bitbake rev: 927bfc26ffcb0eb6c0a5b3c0905b7efc2f19b9a2)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Port the Task based tables to ToasterTable. This is the Task, Time, CPU
usage and Disk I/O tables.
(Bitbake rev: bebcef7a4bf08b10e472475435ddc7a524364adb)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This is the table that displays all the packages built in the build.
Build -> Packages. Adds a template snippet for the git revision popover.
(Bitbake rev: df62f38ff4e634544c9b1e97c5f6ca45e84a4f1e)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Make sure we can create empty states for toaster tables that include
actions for users to get out of the empty state. Allows a template to be
used as an empty state.
(Bitbake rev: 9f21a1781345a2d2757616a9ab6f34ec48e9e93d)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Belen Barros Pena <belen.barros.pena@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Convert all the HTML templates, JS and CSS in the project parts of
toaster to use bootstrap 3.
(Bitbake rev: 69527a731eada699d3f604ff8f3ae9410981ba9b)
Signed-off-by: Belen Barros Pena <belen.barros.pena@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The Twitter typeahead.js library expects the developer to use
a source which does a local search for matching suggestions, then
falls back to a remote search if that doesn't return enough
results.
However, in Toaster, we don't do any caching of the suggestions
for a typeahead, so our source only works in asynchronous mode.
Consequently, we see fewer than the expected number of suggestions
if the typeahead has already shown suggestions matching a query.
For example, searching for "meta-n" in the layers typeahead will
show the results for this query; but when the query changes to
"meta-ne", a new set of results is fetched, which mostly overlaps
with the results for "meta-n". The typeahead assumes that the
overlapping items are locally cached and have been delivered
synchronously, and just appends the new results which don't
overlap with the previous query. But because we don't provide any
results synchronously, we just end up with the single
non-overlapping result in the drop-down.
This can be fixed by hacking typeahead.js so that instead of
appending asynchronous results, we always overwrite and redraw
the whole typeahead menu.
This is a temporary fix, and should be properly fixed (when we
have time), perhaps by using typeahead.js's associated Bloodhound
library.
Added a note about the hack to the license file as an explanation
of why the unminified JS file is included in Toaster.
(Bitbake rev: afbaf326e1123c92952fa71e0e820a4ff83488ca)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Migrate from Bootstrap 2's built-in typeahead to Twitter's
typeahead library.
This is to facilitate moving to Bootstrap 3, which doesn't have
a typeahead.
(Bitbake rev: 0748177b40188a6fb735fe1ba1c17294afa4a3d0)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a helper class for running build tests. Subclass this and call the
build method to get setup for running tests on the resulting data from a
build.
(Bitbake rev: 2cafb16033b2f38156e6e0a74c02120f87a4fa92)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* In the custom images table, show the recipe file and download icon
only when the recipe file exists and can be downloaded.
* Also in the custom images table, make sure the download icon tooltip
shows on hover.
* In the custom image details page, show the correct icon (download)
next to the recipe file in the right hand column.
* In the custom image details page, show the recipe file and download
icon in the right hand column only when the recipe file exists and can
be downloaded
* Also in the custom image details page, simplify the help text we show
when the 'download' button is disabled: it gave so much information about
what's actually happening under the hood that it was a bit hard to
follow.
(Bitbake rev: 5d4172bc005efe209de595f6a151ae9ed062d051)
Signed-off-by: Belen Barros Pena <belen.barros.pena@linux.intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When the tests are complete add some marker elements to the test page
DOM which can then looked at for a selenium browser test.
(Bitbake rev: 81798ab046b0717614e28e0a339172f3803e451b)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fixed:
DeprecationWarning: The 'warn' method is deprecated, use 'warning' instead
(Bitbake rev: a3f464d202dafef4538e66c008cdecb7b8709ed1)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If a line like:
foo=${@' '.join([d.getVar('D', True) + x for x in (' '.join([d.getVar('FILES_bash-' + p, True) or '' for p in ['lib', 'dev', 'staticdev', 'doc', 'locale', 'ptest']])).split()])}
is added to a function like do_install, it fails with Exception name 'd'
is not defined. This is due to a change of behaviour in python 3 compared
to python 2. Generator expressions, dict comprehensions and set comprehensions
are executed in a new scope but list comprehensions in python 2.x are not. In
python 3 they all use a new scope.
To allow these kinds of expressions to work, the easiest approach is
to add 'd' to the global context. To do this, an extra optional parameter
is added to better_eval and we use that to add 'd'.
(Bitbake rev: 8f74881037bb01013d3d439dc0c269909a198c1c)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The functionality of BBPOSTCONF and BBPRECONF was added in
commit 21b314d4d1 but there
was a typo in the variable name that raises an exception
in bitbake.
[YOCTO #9235]
(Bitbake rev: 6d1379c8818400e5cdc442e6142f08a110fd5b95)
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
"hash() is randomised by default each time you start a new instance of
recent
versions (Python3.3+) to prevent dictionary insertion DOS attacks"
which means we need to use hashlib.md5 to get consistent values for
the codeparser cache under python 3. Prior to this, the codeparser
cache was effectively useless under python3 as shown by performance
regressions.
(Bitbake rev: 12d43cf45ba48e3587392f15315d92a1a53482ef)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
No functionality change, just avoids function call overhead in a
function which loops heavily.
(Bitbake rev: 633c0c19f87a92497a7e9771811cdc953e1b7047)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
deb packages in modern Debian versions have the data tarball compressed
with xz rather than gzip, and thus explicitly extracting data.tar.gz
fails. Unfortunately ar doesn't support wildcards matching items to
extract, so we have to find out what the name of the file is first and
then extract it, relying on tar to figure out how to unpack it based on
the filename rather than doing it with pipes and making that
determination ourselves.
(Bitbake rev: 17ff08d225a8fa7faffd683c028369574954fba9)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Previously the code used to match a reference to its SHA-1 in
_latest_revision() used the Python "in" operator, which made it match
if the reference matched the beginning of an existing tag or
branch. This test, however, must be exact. I.e., either the reference
matches a tag or branch exactly, or it does not match at all.
(Bitbake rev: e5986c78a6108fd7578989c20efcbf0b81c97e03)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie
Aníbal Limón
Ed Bartosh
Michael Wood
Juro Bystricky
Paul Eggleton
Ross Burton
Scott Rifenbark
Christopher Larson
Mark Hatle
Joe Slater
Belen Barros Pena
Markus Lehtonen
Ulf Magnusson
David Reyna
Mariano Lopez
Sujith H
Elliot Smith
Stephano Cetola
Matt Madison
Jonathan Liu
Robert Yang
Martin Jansa
Jérémy Rosen
Francisco Pedraza
Randy Witt
Ulrich Ölmann
Terry Boese
Maxin B. John
Mario Domenech Goulart
Enrico Scholz
Andrew Bradford
Jussi Kukkonen
bavery
George McCollister
Ed Bartosh
Peter Kjellerstedt