Fixed pylint warnings:
No space allowed around keyword argument assignment
No space allowed after bracket
No space allowed before bracket
(Bitbake rev: c39770239f7b61217501782b9c5e9d3211355d42)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Environment variables BBSERVER, BBTOKEN and BBEVENTLOG silently
overwrite bitbake command line arguments. This is confusing and
can cause issues that are difficult to debug. It's better to use
them as default values instead.
Used environment variables BBSERVER, BBTOKEN and BBEVENTLOG to set
default values for command line arguments.
Changed setting default value of --ui command line argument from
BITBAKE_UI to look similar way.
(Bitbake rev: 87040be4ff54cd460961318224deef8f2ea4c85a)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Bitbake uses set of environment variables to set command line
options, e.g. seeting BBTOKEN variable has the same effect
as using --token command line option.
Added new environment variables BBPRECONF and BBPOSTCONF that
are equivalents of --read and --postread command line options.
They can be used by high level scripts to append or prepend
configuration files to conf/local.conf
[YOCTO #9235]
(Bitbake rev: bf604ec1ca4eb4d0b22bcc72249963e6d7445f34)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If the based on recipe has not yet been checked out/cloned we cannot
generate the custom image recipe file that uses it. So disable/remove
the option to download it.
[YOCTO #9425]
(Bitbake rev: f08651a954358e9c6ce4d69de285e4cd9e1408ae)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This function returns the base recipe file path only if it currently
exists. This allows us to know whether we can proceed at this point with
generating a custom image recipe. It also enables us to call this
function from the templates to enable visual indication of this state.
Some whitespace fixes also added in generate_recipe_file_contents
(Bitbake rev: bc30d1b235b9ecacef5b2eaa851b9247d857f317)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Some users may want to use authenticated SSH connections with credentials stored
in a keyring, such as gnome-keyring. These typically need a DBus session bus
connection, so pass DBUS_SESSION_BUS_ADDRESS into the fetcher environment.
To avoid the user needing to set it in their local.conf (which wouldn't be
usable) or adding it to the environment-cleansing whitelist (which would
potentially impact builds) allow the variables being passed to the fetchers to
come from the data store (first) or the original environment (second).
(Bitbake rev: 20ad1ea87712d042bd5d89ce1957793f7ff71da0)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Convert existing tests to Selenium.
Add basic tests to check that the modal contains radio buttons to select
a custom image to edit when a build built multiple custom images, and
to create a new custom image from one of the images built during
the build.
[YOCTO #9123]
(Bitbake rev: c07f65feaba50b13a38635bd8149804c823d446a)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add functionality to the placeholder button on the build dashboard
to open a modal dialog displaying editable custom images, in cases
where multiple custom images were built by the build. Where there
is only one editable custom image, go direct to its edit page.
The images shown in the modal are custom recipes for the project
which were built during the build shown in the dashboard.
This also affects the new custom image dialog, as that also has
to show custom image recipes as well as image recipes built during
the build. Modify the API on the Build object to support both.
Also modify and rename the queryset_to_list template filter so that
it can deal with lists as well as querysets, as the new custom image
modal has to show a list of image recipes which is an amalgam of two
querysets.
[YOCTO #9123]
(Bitbake rev: 8c2aea3fa8e1071de60390e86e2536904fa9b7c0)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When a build is viewed in the dashboard, enable users to edit
a custom image which was built during that build, and/or create
a new custom image based on one of the image recipes built during
the build.
Add methods to the Build model to enable querying for the
set of image recipes built during a build.
Add buttons to the dashboard, with the "Edit custom image"
button opening a basic modal for now. The "New custom image"
button opens the existing new custom image modal, but is modified
to show a list of images available as a base for a new custom image.
Add a new function to the new custom image modal's script which
enables multiple potential custom images to be shown as radio
buttons in the dialog (if there is more than 1). Modify existing
code to use this new function.
Add a template filter which allows the queryset of recipes for
a build to be available to client-side scripts, and from there
be used to populate the new custom image modal.
[YOCTO #9123]
(Bitbake rev: 4c49ffd28e41c4597bdac34d5e54c125571a4b95)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The Selenium helper's enter_text() method doesn't cause
keyup events to trigger unless the element where text is
being entered has been clicked.
Prefix all text entry with a click() on the element to ensure
that keyup events fire.
(Bitbake rev: cea34880ad3847bd0e24c9b650eb816e1757cf2b)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Test adding a new custom image when:
1. No custom images are in the project yet.
2. User tries to add custom image which duplicates the name of
an existing custom image.
3. User tries to add custom image which duplicates the name
of a non-image recipe.
[YOCTO #9209]
(Bitbake rev: 21c1f8f8e30ef868ea6fd861eea1389f149f1049)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Project.release can be null. This causes an exception when calling
get_all_compatible_layer_versions(), as the query to fetch
the layer versions references release.branch_name.
Add a guard to the function so that an empty queryset is returned
if the release isn't set for a project.
(Bitbake rev: 6919a2b2e412a9e7b652a6bc191e7c1bed035222)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We currently prevent the same name being used for multiple custom
images, but make the check across all projects. This means that
custom image names have to be unique across all projects in
the Toaster installation.
Modify how we validate the name of a custom image so that we
only prevent duplication of custom image names within a project,
while ensuring that the name of a custom image doesn't duplicate
the name of a recipe which is not a custom image recipe.
[YOCTO #9209]
(Bitbake rev: 9abbb46e799c06757e03addd54e3f5d3c0fe2886)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In the import layer page, the "Add layer" button in the layer dependencies
section doesn't accurately reflect whether the layer name in the
corresponding input can be added. A partial or empty layer name can
leave the button active, such that when it is clicked, a
previously-selected layer can be accidentally added.
Fix by keeping track of the items currently available in the typeahead,
only activating the "Add layer" button when the input matches the name
of one of those items.
[YOCTO #8511]
(Bitbake rev: dbb4f0282ded361baf9e5a0346e134bece5314b9)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When hiding a column in the build tasks or build packages included
table, if the column is set as the current order by for the table, the
order by is not reset to the default. The result is that the table
stays sorted by the hidden column.
Set the default_orderby for these two tables correctly to ensure the
corresponding table is re-sorted when a column is hidden, if that column
was being used as the order by.
[YOCTO #9011]
(Bitbake rev: b99e1012f0ad1dc82a769df15a232280c8e57b9e)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If a ToasterTable is ordered by an optional column and that
column is subsequently hidden, the table ordering switches back
to the default ordering for the table. However, the table headings
don't update to reflect the new ordering. This is because the
code which sets the heading weight and hides/shows the caret symbols
only runs when the table is first loaded.
Store the default order by and re-apply it when the data is updated.
[YOCTO #9011]
(Bitbake rev: 820761e664cd2d62cc6c333a0e59580b0c4a034d)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fixed typo that caused incorrect processing of BBEVENTLOG
environment variable. Even if variable is set it was ignored
by bitbake.
(Bitbake rev: 2705b5f59aef4a070e2df2752d27bd04ea747057)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Added information about using lower-case characters and no
underscore characters when using OVERRIDES.
(Bitbake rev: 4397a023f8bf738913d71a38b3e06503207311e2)
Signed-off-by: Scott Rifenbark <srifenbark@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fixes [YOCTO #9343]
I updated the inherit directive section to note that you can
inherit multiple classes using a single line.
(Bitbake rev: 3f29ebec593f20894f7a2a0b6d50ee22474aee75)
Signed-off-by: Scott Rifenbark <srifenbark@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Sometimes you can end up in a situation where you need to specify that
a specific runtime entity should be provided by a specific entry.
An example of this is bluez where you could end up in a situation where
for example:
NOTE: multiple providers are available for runtime libasound-module-bluez (bluez4, bluez5)
NOTE: consider defining a PREFERRED_PROVIDER entry to match libasound-module-bluez
NOTE: multiple providers are available for runtime bluez-hcidump (bluez-hcidump, bluez5)
NOTE: consider defining a PREFERRED_PROVIDER entry to match bluez-hcidump
The only option here is to set something like PREFERRED_PROVIDER_bluez4 = "bluez4"
which is clearly not very informative.
I've actually held off adding RPROVIDER support for a long while as this
does have sigificant potential for misuse. It doesn't for example allow
multiple runtime providers of the same name to coexist, that simply isn't
supported. It therefore doesn't replace some of the name mappings such
as busybox verses coreutils that OE-Core faces as that is a different
problem with different constraints. This mechanism is simply to provide
bitbake with a hint to decide what the dependency tree should look like.
Also, this allows us to stop printing a confusing message telling the user
to set PREFERRED_PROVIDER when the setting needed would be rather ambiguous.
[YOCTO #5044]
(Bitbake rev: 62eb39d1474d024b204634689071700605c6095c)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Back in history the code did depend on previous build results. This was
bad for determinism and we no longer do that. Update comments to match
the current behaviour.
(Bitbake rev: c3fa7e561c22786d3ac57d04c367aa50f1b3b820)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We now prefix log messages coming from worker task context with the
PF and task info, however parsing messages all have to be manually
prefixed which is ugly and error prone. This change modifies the log
handler filter so this happens automatically, meaning we don't have
to change every message to include that information. This makes error
messages longer but more usable.
(Bitbake rev: 1af0ccaac81e182c4ca520037dda362d180e5605)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
An invalid task causes bitbake to exit incorrectly, firing a
CommandCompleted event rather than a CommandFailed one. This
means that clients listening for CommandFailed events are
unable to detect the build failure even though one occurred.
Passing an exception string to finishAsyncCommand when a task
fails causes the CommandFailed event to be fired correctly.
[YOCTO #9087]
(Bitbake rev: 98a2c37e077b16e3bc8bb102bd18b293130d15a4)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When multiple recipes which both provide something are being built, bitbake
informs us that most likely one of them provides something the other doesn't,
which is usually correct, but unfortunately it's rather painful to figure out
exactly what that is.
This patch dumps two sets of information, one is the provides information for
each recipe, filtered so only common components are removed. The other is a list
of dependees on the recipe, since sometimes this can easily identify why something
is being built.
Its not straightforward for bitbake to obtain the information but since the
warning/error code path isn't the normal one, we can afford to go through some
less than optimal processing to aid debugging.
Also provide the same information even if we're showing a warning since its still
useful.
[YOCTO #8032]
(Bitbake rev: 96fc889b8e62ba4463c71158c4b7286c48d68cd8)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
For a custom image, if a search for a package results in no packages
found, then additional information should be presented to the user.
This is different than a 'no results' found for a search in other
contexts, for example, a search for a package in a non-customised build.
For a custom image, a package search failure can happen because the
package was not added to the custom image. This commit presents more
information to the user, suggesting why the package was not found in the
custom image.
The generic table view handling js changes to handle a new div
element no-results-special-... such that, if present, that template
section is shown rather than the default no-results-... section.
[YOCTO #9154]
(Bitbake rev: 66b7c7ef61058b52031d71b10effcfe69afbd57b)
Signed-off-by: Dave Lerner <dave.lerner@windriver.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Bitbake variables don't include ":" characters so exclude these from the variable
expansion regexp.
This assists when parsing shell code which does A=${B:-C} as we don't want a
dependency on a variable called "B:-C".
(Bitbake rev: 14ed41a292123374d94f5c786a619881f2ddea42)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There were no tests that verified the value of origvalue in the callback
routines used by edit_metadata(). This patch adds one for a simple
multiline variable.
(Bitbake rev: ece3a4d02d8162dee78c2062c10291b5fd625c36)
Signed-off-by: Randy Witt <randy.e.witt@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
edit_metadata() would corrupt a variable that was multiline, but
had the ending quotes on the same line as the last value. For example:
TEST_VAR = " foo \
bar"
would become " foo ba" because the code would always delete the last
character on the line and then do it again if the line ended in the
quote. This however doesn't show up if you have:
TEST_VAR = " foo \
bar \
"
which is how all the test cases were written.
This patch fixes that bug and adds and fixes a test that matched the bugs
behavior rather than the expected behavior.
(Bitbake rev: 14f05cbdc2ad8d59a94af1c8816567d93c39c88c)
Signed-off-by: Randy Witt <randy.e.witt@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
QUnit is now included in Toaster's test suite and distributed
with its source code.
(Bitbake rev: 540c0c8d6aeec5391aa4f76614db6905afabba2c)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(Bitbake rev: 81ccbf243050a5a9245d2de4c1de342771c09a59)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add set a MACHINE if needed and add a layer. When we're running in the
context of the django unit tests we don't have these defaults setup for
the project so add them.
(Bitbake rev: a0c1432f32930a17e10d50c08c2aa84a0659514b)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(Bitbake rev: 6288a3bd6678ed1c7863cfde6eb33b7c28207777)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently we're using a hosted version of quint however this means that the
testing has to be online or have a cached version of quint. Add the
files to Toaster to be able to use offline.
(Bitbake rev: fc5024e2ed0e4ec2ee234a42fd0403c70c3f819e)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a column to the custom image pages that shows the reverse
dependencies in a format matching the dependencies column:
- either blank or a button showing the count of reverse dependencies,
- when the button is clicked, a popover appears showing the list
of reverse dependencies, with each package's size, and the total
size of all of the reverse dependencies.
The implementation adds a packages table method to retreive the reverse
dependency total size, and adds a separate 'popover' html template. Both
of these changes follow the pattern for the dependencies column.
[YOCTO #9163]
(Bitbake rev: 2f978dccaa1ec82c7ad350bdc1cd9500000984eb)
Signed-off-by: Dave Lerner <dave.lerner@windriver.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We have been seeing UnicodeDecodeErrors when handling the
ImagePkgList MetadataEvent in ORMWrapper's
save_target_file_information() if the event includes filenames
that include non-ASCII characters.
In the short term work around this by converting paths to the
unicode type when passing them to Django's ORM. This is a bit of
a hack but it's too late in the cycle to do anything more invasive.
[YOCTO #9142]
(Bitbake rev: f50fff03b3de02e73a3cc2eb9935f7c345dbddc4)
Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
buildinfohelper stores current Build object in its internal
state. Any changes to Build object will be lost if internal
state is not updated as current buildinfohelper code
saves Build object from internal state when build is
completed.
This bug causes incorrect build state when build is cancelled.
Updating internal state should fix it.
Note, that this commit updates internal state after status of
the build is changed to Build.CANCELLED. There are several other
places in the code where Build object is updated without updating
internal state. They should be carefully analyzed and fixed.
(Bitbake rev: d056cf40fc55530cb1736aedfb9a3c355884991e)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When bitbake doesn't need to build anything it still sends
ImagePkgList event with empty 'pkgdata', 'imgdata' and 'filedata'
fields. This causes crash in buildinfohelper code as it's assumed
that above mentioned fields always have data keyed by build target:
ERROR: u'core-image-minimal'
Traceback (most recent call last):
File "toasterui.py", line 423, in main
buildinfohelper.store_target_package_data(event)
File "buildinfohelper.py", line 1218, in store_target_package_data
imgdata = BuildInfoHelper._get_data_from_event(event)['imgdata'][target.target]
KeyError: u'core-image-minimal'
Fixed this by using dict.get method with empty dictionary as default
return value instead of trying to get value without checking if target
key is in the data.
(Bitbake rev: c39cc463e6d9594bf2c5ac8bb74e834f6f2cf7c8)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
It was incorrectly assumed in the current code that bitbake is in
../bitbake/bin/ directory. It's not always the case.
Using bitbake from $BBBASEDIR should be .
(Bitbake rev: 6c9e3375d278bee712c41f07428bc82108b9aaae)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This variable will be used in localhost controller code
to determine correct bitbake location, so it has to be
exported.
(Bitbake rev: 429d47325aadb74d476e5b7f4738f2dfe26cde95)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Modified the projectconf.html to include DL_DIR and
SSTATE_DIR. Updated the script section in the html
to handle the changes made by the user on DL_DIR
and SSTATE_DIR. Included validation check for the
folder names.
[YOCTO #8422]
(Bitbake rev: bd9f8973d4c9c0722874a058466b1b911112500e)
Signed-off-by: Sujith H <sujith.h@gmail.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Update toaster's views.py to support DL_DIR and SSTATE_DIR for
page projectconf.html. Removed DL_DIR and SSTATE_DIR from
blacklist. Initial value of DL_DIR and SSTATE_DIR comes from
BuildEnvironment.
[YOCTO #8422]
(Bitbake rev: 9f672d7ba503d17175eef37ec03a5779e4c9f792)
Signed-off-by: Sujith H <sujith.h@gmail.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If client xmlrpc token is not provided in the command line, bitbake
generates random token. Server token in --server-only mode is always
empty. This doesn't allow clients with non-empty tokens to connect
to the server. Specifying empty token should stop generation of
random token and make it possible for clients to communicate with
the server.
(Bitbake rev: b54ec2d7cb9f2fb4ff1e8af11b6ecf97d181272c)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
- Organise the imports into logical groups
- Fix 80 col wrapping
- Remove catch all exceptions
- Log to the toaster log
- Use QuerySet functions such as .first() and Q()
(Bitbake rev: c382f550c62437954205c7dd9c7e1891d5d03945)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add handlers to make sure we remove the BuildEnvironment LOCK when we
have cancelled a build.
(Bitbake rev: 23d0a7f9664450a09c2610631b38590a09b33744)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Don't count CANCELLED builds when returning the number of builds.
(Bitbake rev: c3c29fd4eb5116b771e8e16281d4e3cdf4fae165)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add the cancel build button to the mrb section template and add the event
handlers to cancelABuild.
Also clean up the calls to startABuild to use the updated libtoaster methods
and to make the code consistent with it's cancelABuild counterpart.
Co-Author: Sujith H <sujith.h@gmail.com>
(Bitbake rev: 6b82ffca1aa9ca2d0feec64b15466bc8ba160011)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Exclude cancelled builds from showing in the builds table
[YOCTO #6787]
(Bitbake rev: a724d6aa7515e712a1d656e46e1d0f3bf7d4cea9)
Signed-off-by: Sujith H <sujith.h@gmail.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When a build is cancelled the build (action) is complete if it has been
caused the request being cancelled then update the build outcome
accordingly.
(Bitbake rev: d94d12914d351bf560b06d6f4e45c294b04ecaa3)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
To accurately reflect the state of a build request we also need a
cancelling state. This is set when we've started a build and then for
whatever reason cancel it, cancelling is not instantaneous so we have
this state to indicate that a cancel is in progress.
Also add a state transition guard. As the state of a BuildRequest can
currently be modified by three processes; Toastergui,
Runbuilds/bldcontrol and the buildinofhelper we cannot say for sure
which process will be running at the time of cancellation so in order to
avoid one of these processes making an incorrect transition only allow
transitions of state to increase.
e.g. CREATED -> QUEUED -> INPROGRESS
And to ignore such requested changes such as
INPROGRESS -> CREATED
(Bitbake rev: 449598c8e6be75bd0c9d59e7bdf859d1d6f83858)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
A new state CANCELLED is introduced to, distinguish
the state of build.
[YOCTO #6787]
(Bitbake rev: 404f406fecae879703bcfe96f3b65086b115fa8a)
Signed-off-by: Sujith H <sujith.h@gmail.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Remove getBBController function from BuildEnvironmentController.
The constructor of BitbakeController is updated appropriately so that
call can be made to connect to running server.
The call to startBBServer is removed from bbcontroller and handledin
localhostbecontroller.
[YOCTO #6787]
(Bitbake rev: e15151106aae21d3b164ca868be42bd63905f0a1)
Signed-off-by: Sujith H <sujith.h@gmail.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Update the implementation of startABuild and cancelAbuild to reflect
changes to the backend api. We now have a dedicated endpoint to make
calls into so add this url to libtoaster.ctx and allow passing null in
as a url value to indicate that we want to use the current project
Also:
- Fix some documentation comments
- Add the convenience of passing in an array of targets to startABuild
(Bitbake rev: 61a21d96abab113cbd13376cdb8b08a426b50538)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Update the implementation of the backend api for cancelling builds with
the new cancelling BuildRequest state and cancelled Build state.
Also added some docstring about general usage.
Co-Author: Sujith H <sujith.h@gmail.com>
(Bitbake rev: 0d76084f5d896e4199e1446e2d6d43190a4fcc3a)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Move the backend xhr implementation of the build request changes
into it's own file and out of the ToasterTable definition.
It used to live in the views.py but in a hope of starting to collate logical
groups of views move this to a new file called api.
(Bitbake rev: 29572f0e6bd3b5e8315f3b93d55bdb8967b86bc3)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add forceShutDown function to BitbakeController class. This function
provides a mechanism to cancel the build from toaster. An API which
can be used safely to cancel build and hence shutdown running bitbake
server.
[YOCTO #6787]
(Bitbake rev: 89dc2ee8f5b7255538082ce4f6cb5277839875a8)
Signed-off-by: Sujith H <sujith.h@gmail.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
toasterui exits event loop on one of the following events:
CommandCompleted, CommandFailed or CommandExit.
Unfortunately none of them come from bitbake when build fails.
This is normai if toasterui runs in observer mode. However, if it's
in build mode this causes toasterui to stuck in the infinite loop
waiting for new events.
The only event we can rely on is BuildCompleted as it always
comes from bitbake unlike 3 above mentioned events.
Modified the code to always shutdown toasterui in build mode
on BuildCompleted event.
(Bitbake rev: 9cd60f98b13cf7b1c518851a51e1cbaa596d8f81)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Toaster can't run builds on Ubuntu as default /bin/sh points
to dash there. The reason is that oe-init-build-env can't
be sourced under dash for various reasons. It can be fixed
or work arounded, but it wouldn't fix builds for older releases.
Explicitly using bash to start builds should fix the issue.
(Bitbake rev: e4352603b21d43a8b5d9ee417c036fff0a263d40)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The keys 'started', 'ended', 'cpu_time_user', 'disk_io_read' and
'disk_io_write' were added to the event recently, so they don't
exist in the events generated by bitbake server from older releases.
Checking if task_to_update structure has these keys before using
them should fix build of older releases.
(Bitbake rev: 79611d0ea742263074fbb0bf5f1e39df75fd9f55)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Added 'INHERIT+="toaster buildhistory"' line to the conf/local conf
when Toaster starts. It should make commandline builds to provide
all required information to Toaster backend.
(Bitbake rev: 1271cf430087c66f87c46689b37b8a3538c35739)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Bitbake server is used only during the build. There is no need
in keeping server running after the build.
Running bitbake -m in the subshell after the build should stop
the server.
(Bitbake rev: 062c68e3e7d00834118fe07ceb0899874be714ae)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Added 'nowait' parameter to _shellcmd method to support
running chain of commands in a subshell.
This is going to be used to stop bitbake server after
the build.
(Bitbake rev: 89fae3c8712bfaac48686c58b480e40c1abdcfdc)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Reimplemented triggerBuild method to support one build directory
per project:
- start bitbake server from the cloned repository
- don't run observer
- run bitbake build directly instead of triggering it
through xmlrpc
[YOCTO #7880]
[YOCTO #9058]
[YOCTO #8958]
(Bitbake rev: aba8e19bf609196f749e185d43571f706707e408)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Removed updating of bblayers.conf. It will be done in runBuild method.
Changed return value: return list of layers.
Removed _updateBBLayers method.
(Bitbake rev: 198bf7e6b8d7f847f2619b71c6bd86a9a76156c9)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Called triggerBuild with brbe parameter instead of adding TOASTER_BRBE
variable to the database and fetching it in triggerBuild.
(Bitbake rev: 669bb3f6865d8bb89cd94ef9d0eba3346c47d7b2)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This API is quite dangerous as it removes build directory.
It's not used anywhere and most probably will not be used in future
as toaster is going to have one build directory per project.
(Bitbake rev: ef9e126227d3bdee86c1592e2baa43d21e6a32df)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We still will have to run bitbake server, but it will be done
different way and the code will be in triggerBuild function.
Removed startBBServer API from BuildEnvironmentController and
LocalhostBEController classes.
(Bitbake rev: a8f986d73a84f28fa8b116b91fb30c5d2cd4edbe)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
buildinfohelper.brbe is lost when buildinfohelper is closed.
This causes incorrect report of brbe when build is done.
Saved brbe attribute before closing buildinfohelper and used
it to report correct brbe.
Got rid of useless and confusing 'ToasterUI build done 1'
log message.
(Bitbake rev: 5d7cce0d0ed70f6b3ebd6cbad300d86964a13398)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
DepTreeGenerated event doesn't contain 'providermap' data in jethro.
Modified buildinfohelper to handle events without this data. This
should make it possible to handle jethro events coming from jethro
bitbake server by the latest buildinfohelper.
(Bitbake rev: f6dcb1c9967f042beae024146781cb8235a9e1f2)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Return value of self.BBServer.registerEventHandler differs between
jethro and master. To be able to build jethro toaster should be
able to communicate with jethro bitbake server i.e. it must work
with both old and new registerEventHandler call.
(Bitbake rev: f356c154016c428a3b53af61a075de6f14d9d1d9)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In current toaster code BRBE(build request:build environment) value
is passed from toaster to buildinfohelper through the 'SetBRBE' event.
Passing it through environment variable is easier as it doesn't
involve rpc communication between toaster and bitbake server.
It also eliminates the need in running bitbake observer process.
Added parameter 'brbe' to BuildInfoHelper.__init__
Used environment variable TOASTER_BRBE to set brbe for
buildinfohelper object.
(Bitbake rev: a0c8e2b309055e5927a8ff729d292ccaa69d0575)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Set BITBAKE_UI variable to 'toastergui' for command line builds
to use toasterui as a default ui module for bitbake.
(Bitbake rev: 9ad6393d30cb6196cf7c9a5adcf33febd724d294)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
noui command line option doesn't makes sense anymore as toaster doesn't
run bitbake. It should be safe to to remove it.
The purpose of this option was to skip running bitbake observer process.
This was never used before as it's not possible to run toaster build
without running observer.
(Bitbake rev: 7506719090e8bb39231cf389c4a5b47f1b37a01f)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
>From now on toaster script will not run bitbake server.
It will be started by runbuilds and stopped after the build.
(Bitbake rev: 3fbd8534149e87c5a5d1bc1691711cfca05cafd1)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
It was used for workaround git 1.7.9.2 which was released in 2012 which
should not be existed on nowadays host, so remove it to avoid
confusions.
(Bitbake rev: 6140d0cc9aecf1029ca16fed47071dfcc92f4269)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Python 2's sqlite3 module defaults to returning Unicode strings for SQL
text queries, which could trickle down to other parts of bitbake code and
cause unexpected Unicode conversions. Using byte strings avoids this issue.
For example, the git fetcher's AUTOREV support caches HEAD SHA1's using
bb.persist_data, so sometimes the git command strings passed to fetch2's
runfetchcmd() were unicode, potentially causing UnicodeDecodeErrors when
it appended the values of environment variables containing non-ASCII chars.
[YOCTO #9382]
(Bitbake rev: 09623a0811c613a47a01ae465b822d8156faca30)
Signed-off-by: Daniel Klauer <daniel.klauer@gin.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
sstate.bbclass for example writes siginfo files to a separate location
but we need to read taint data from the standard path.
(Bitbake rev: da444c9761ee15a59ea8880e3f812a5d3f1a1aaa)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The taint values need to be passed from the server to the workers to
ensure they see the same stamp values. Also ensure that the "nostamp:"
prefix isn't included in the checksum value to match the server
calculation. This ensures the checksums are all consistent.
(Bitbake rev: f80ba20e90f3746f7faee3e0ff7f249025fec8ee)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In theory all the information to recalcuate the task signatures was written
into the siginfo/sigdata files. In reality, some of the information was
written into the filename.
Firstly this patch duplicates that info into the file itself just for easy
of use since its small.
Secondly, we abstract out the existing "calculate the checksum" code for
the taskhash, and add a function to calculate the bashhash based on the
informaiton within the file.
Finally, we call these functions when we're writing out the data to check
that the data we're writing is consistent. I've found a couple of places
it wasn't and its good to know about these in advance, rather than having
a siginfo/sigdata file which a given hash in its filename but a contents
which give a different result.
This should all combine to avoid a certain class of checksum bugs making
it into world, and identifying problems in advance.
(Bitbake rev: 0f50a18d7a0ea0d68edd8e5217e29111f4b1ea0b)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When I enabled debugging of the checksum code, I found the value calculated
from siginfo/sigdata files for do_fetch tasks never matched. This was due
to an error in the way the data was being stored for these, it wasn't ordered
correctly. This patch fixes things so the checksums calculated from
siginfo/sigdata files is correct when file checksums are present.
(Bitbake rev: 046c1be7594fae2eec3d1f242ba3e9a85f1a1880)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The real method is a few lines later, this one is incorrect and
just causing confusion. Remove it.
(Bitbake rev: a896f263300f069400eae533be0daf5dedf41c95)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The tests were skipped when running without network even though they
didn't require network. This commit also adds a test case for URLs with
ports in them (the ports should not be considered when doing trusted
network checks).
(Bitbake rev: 77747de6b20538063eef3b188489a35bef225359)
Signed-off-by: Olof Johansson <olof.johansson@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Bitbake would fail to classify the following URL as belonging to a
allowed network, because of the port number in the url.
BB_ALLOWED_NETWORKS = "*.example.com"
SRC_URI = "http://git.example.com:8080/foo.tar.gz"
Since protocols aren't specified in the BB_ALLOWED_NETWORKS variable,
it's reasonable to believe that this should work regardless of protocol
being used.
(Bitbake rev: ff603df23037e10fb2cfdf150429cba3f65072cd)
Signed-off-by: Olof Johansson <olof.johansson@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We were doing a string search for the literal string "require" and
assuming that a match meant the recipe uses the require keyword to
include the contents of another recipe. This test isn't very smart (it
should at least have matched on "require ") and triggers on the word
require anywhere in the recipe, i.e. the word "required" in the
DESCRIPTION of core-image-lsb.bb, and then breaks as the much smarter
regex fails to match anything.
Instead always run the regex search and only try to update the
require entry when the regex matches.
(Bitbake rev: a6add0e95d3d1e9a6a9fcabd73543bc5c278915f)
Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The help text said to append a semicolon and a task name to a target to
run a task other than do_build. What you need to append is a colon.
[YOCTO #9326]
(Bitbake rev: 691419e23a94a94129b177e71d2c728b12689139)
Signed-off-by: Belen Barros Pena <belen.barros.pena@linux.intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add additional metadata to the layer created for build history to be
able to identify the layer and recipe later on. Specifically this is the
branch and release to which the recipe and layer are associated with
enabling differentiation of two recipes which are local release and
master and 'master' release.
[YOCTO #8528]
[YOCTO #8545]
(Bitbake rev: 3deebd887bddbbd02fd9829a180aab494b1af7c4)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(Bitbake rev: 20ce1e1b39a9b602eb51ca0ba3954ea3e999c874)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(Bitbake rev: 0e5f45d68e423f8462937879eed3253db31b2bb5)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(Bitbake rev: 31204937f71a7e0aa08361c3e20d02d063788a86)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(Bitbake rev: 4fda6be831d10e6d266b11975a0e9a35a7f35a77)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Create a new folder for Selenium tests.
Add a new base Selenium testcase class and a helper which
instantiates a webdriver for a given browser.
Add a sample Selenium test case which can be used as a template
for creating new tests.
(Bitbake rev: b7a377aa2ab36390d619e2a0436ccb4b8d186c23)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There appears to have been a lot of copy and pasting of the code
which prints tracebacks upon failure and limits the stack trace to
5 entries. This obscures the real error and is very confusing to the user
it look me an age to work out why some tracebacks weren't useful.
This patch removes the limit, making tracebacks much more useful for
debugging.
[YOCTO #9230]
(Bitbake rev: 5549748a200b5df259fc7352477ec59471b87b2f)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
It's very possible that we added layer as:
BBLAYERS += "/path/to/meta/"
then there would be warning:
WARNING: No bb files matched BBFILE_PATTERN_core '^/path/to/meta//'
This patch can fix the problem.
(Bitbake rev: 2b1cb21d18fb18399e682021b866babeced9a4aa)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
With the code as it stands today it not possible to execute a python function
and get "normal" python exception handling behaviour. If a python function
raises an exception, it forces a traceback to be printed and the exception
becomes a FuncFailed exception.
This adds in a parameter 'pythonexception' which allows standard python
exceptions to be passed unchanged with no traceback. Ultimately we may want
to change to this convention in various places but at least it means we can
start to add sane functions now.
(Bitbake rev: 85cf22fd0ed26bb7dc7738ef2a10441891f38ae2)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There are some codepaths where the file checksum is verified and can
be found to mismatch but the 'rename' logic doesn't kick in. If code
relies on the presence of a file for the checksum having been checked
(e.g. uninative.bbclass) then it can be used when the checksum hasn't
matched.
Therefore rename the file whenever an invalid checksum is encountered.
(Bitbake rev: 69ef6c8a9db02bfa0e3fac72481ec26586a29a01)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Prevent a hang when shutdown() is called during parsing (e.g. after
SIGINT). We must not append 'None' to the jobs queue. Otherwise the
worker loop inside Parser.realrun() may break out at the wrong point,
causing the results queue thread blocking bitbake indefinitely.
[YOCTO #9319]
(Bitbake rev: 7ebea3e9a60232222efa8a546a0ff28a53029949)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently bbappend files in a layer are applied in the order they're
found on disk (as reported by glob) which means things are not
deterministic.
By sorting the glob results, the order becomes deterministic, the parsing
order for .bb files also should be deterministic as a result of this change.
[YOCTO #9138]
(Bitbake rev: 3f8febc4212fbd3485ac9bdd4ac71b8fb0a05693)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Before this patch, directory symlinks mathcing filename pattern (either
a file name or a glob pattern) were followed. However, directory
symlinks deeper in the search chain were omitted by os.walk(). Now
directory traversal behaves consistently, ignoring syminks on all
levels.
One reason for choosing not to "walk into" directory symlinks is that
dir symlinks in externalsrc.bbclass in oe-core are causing problems in
source tree checksumming.
[YOCTO #8853]
(Bitbake rev: 66dff37ebcd1dd14ebd6933d727df9cf0a641866)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If local.conf contains an invalid line, e.g.:
APPEND += " igor"
(note the leading space) then nasty tracebacks are shown which confuse the
user. Change so the parse error is simply shown without a traceback, improving
the user experience.
[YOCTO #9332]
(Bitbake rev: 148aa1fb45dcb37a756a08301a7daf270e753180)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When prefix is part of the version directory it need to ensure that
only version directory is used so remove previous directories if exists.
Example: pfx = '/dir1/dir2/v' and version = '2.5' the expected result
is 'v2.5' instead of '/dir1/dir2/v2.5'.
[YOCTO #8778]
(Bitbake rev: c760531c6dbf88135ab9f8e6f0784ccbf2cce1e4)
Signed-off-by: Aníbal Limón <limon.anibal@gmail.com>
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Little improvement for reference tokens by names instead of index.
(Bitbake rev: e8ea15eeb1857ed4bb6337836bd2fb1f5dbb1bdf)
Signed-off-by: Aníbal Limón <limon.anibal@gmail.com>
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We recently dropped lockfiles for file:// urls which in itself makes
sense.
If a file url redirects to something like an http:// mirror, we'd have
no lock taken for the original file and could race against others
trying to download the file. We therefore need to ensure there is a
lock taken in the mirror handling code.
This adds code to take such a lock, assuming it isn't the same lock
as the parent url.
(Bitbake rev: 913b6ce22cd50eac96e8937c5ffc704bfce2c023)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
For customised image package removal change behavior.
From:
Only display the immediate dependents of the requested package
to remove, not the full dependent list, that is dependents of
dependents ...
Do not remove the displayed dependents, just notify the user
of the list.
To:
Display the complete dependent tree, traversing all reverse
dependencies starting from the package to be removed and then it's
dependents.
Change the modal dialog to note that all of these dependents will
be removed automatically.
[YOCTO #9121]
(Bitbake rev: 1185a5bfe1b05a1b63a927c9583dfc031fdac8a9)
Signed-off-by: Dave Lerner <dave.lerner@windriver.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
For customised image package removal, show a different modal dialog that
lists ALL of the packages dependent on this package, with a Remove All
button - implying that all of the dependents will be removed.
[YOCTO #9121]
(Bitbake rev: 768e6bb90c433687b0d52f256b7115499ca3418b)
Signed-off-by: Dave Lerner <dave.lerner@windriver.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When a package is to be removed, show the full list of packages that are
dependent on that package, telling user that these packages will also be
removed.
[YOCTO #9121]
(Bitbake rev: f5cb59b6b10a714b18b1c00b9a8598dd855c84b5)
Signed-off-by: Dave Lerner <dave.lerner@windriver.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This fixes a problem when the repository contains multiple levels of submodules via a resursive submodule init.
(Bitbake rev: dbafbe229360ffe5908b106a9c10e274712b9b17)
Signed-off-by: Derek Straka <derek@asterius.io>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently xmlrpc server implicitly sets itself into single use mode
when bitbake server is started with anonymous port (0) or no port is
provided in command line. In this mode bitbake shuts down xmlrpc server
after build is done. This assumption is incorrect in some cases.
For example Toaster uses bitbake in this mode and expects xmlrpc server
to stay in memory.
Till recent changes single use mode was always unset due to the bug.
When the bug was fixed it broke toaster builds as Toaster couldn't
communicate with bitbake server in single use mode.
Reimplemented logic of setting single use mode. The mode is explicity
set when --server-only command line parameter is not provided to bitbake.
It doesn't depend on the port number anymore.
[YOCTO #9275]
[YOCTO #9240]
[YOCTO #9252]
(Bitbake rev: afc0dd5c532684f6201b1e12bbf4c226ea19062d)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
I added wording to note that include/require checks the current
directory in addition to BBPATH
Fixes [YOCTO #8566]
(Bitbake rev: bdad45ebe551912b055ffa349d254ae5947a3ba6)
Signed-off-by: Scott Rifenbark <srifenbark@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fixes [YOCTO #9283]
Updated the description to document conditional inherits. Provided
several examples.
(Bitbake rev: 07f97f4d913cf1c8233995152105fff6c6c7b9a0)
Signed-off-by: Scott Rifenbark <srifenbark@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
bb.event.ParseStarted event is not processed by toasterui, but
present in event list. This causes the following error:
WARNING: Unknown event: <bb.event.ParseStarted object at ...
and non-zero return code:
WARNING: Return value is 1
(Bitbake rev: 1cc102f3d83d9467a3a3c422254333796ba95605)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Remove the very verbose log dump from toasterui. This generates several
megabytes of not that useful debug information and actually hinders
finding the original exception.
(Bitbake rev: a21dc134bdce2c9eb5e47c770094660f0c45c398)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This makes it possible to prevent a recipe to be cached, and thus,
parsed every time.
Use with care.
[YOCTO #8853]
(Bitbake rev: 78335c1fbe5266116700c2413aac28b00423a75b)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Try a secondary file path if the first does not exist. When we get the
recipe paths and layer information from the layer index it is not a
complete path but we are usually able to reconstruct it. If the complete
real path has been discovered by building then use this instead.
[YOCTO #9206]
(Bitbake rev: 238db2e03405d259d48dfc477a276191e6a47698)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Move the generated layer for custom recipes into the build directory.
The build directory makes more sense as this layer/recipe is generated
for a particular build/project.
(Bitbake rev: 77f3728ee6ea379bffcf73f33c26e982cb0795f4)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We don't need to skip file:/// uri type locations for git repositories.
If you're using a file:/// uri you should know that it has to be a local
path.
[YOCTO #9200]
igned-off-by: Michael Wood <michael.g.wood@intel.com>
(Bitbake rev: 4d0e5804103c2d98d038b3c490cac37a73f3cc47)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Use a constant to define the name for the toaster custom images layer;
this constant is then used to identify this layer in various places.
(Bitbake rev: 2540969ec71612af7f9041cadcc401513e9b357b)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We may have a recipe which is based on a custom image recipe that has
already been built so keep the recipe file around so that it can be read
by the generate recipe function.
(Bitbake rev: 502148fe498fe0a41ae89c9f649d3cb1253b0487)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Update the upstream url used for testing cups versions after upstream website
changes.
(Bitbake rev: 5f06041d4936fc22297945bbbad7020bfa9083c6)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fixes [YOCTO #9231]
npm when given an invalid registry URL with --registry actually goes and
fetches from the default registry, but this commit makes sure it goes to the
specified one.
(Bitbake rev: 7c849be7c70a5db4f66fe3041486abb923b5e4ee)
Signed-off-by: Brendan Le Foll <brendan.le.foll@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Instead of trying one time with a timeout of 20 seconds try 4 times with
a timeout of 5 seconds, to account for a slow server start.
(Bitbake rev: 4a7fe63126dd8177baa5ad21e59e0bebeea8c596)
Signed-off-by: Lucas Dutra Nunes <ldnunes@ossystems.com.br>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When returning the compatible layers make sure that we are only
listing: All the layers which are for this release && configuration
layers (i.e. aren't part of the build history) and which aren't an
imported layer OR are this project's imported layer(s).
[YOCTO #8944]
(Bitbake rev: de8baedaccb451c12bc3f642449db3f64aed6bf7)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The data available from buildstats is now more fine grained than
previously, so take advantage of that to enrich the data we save
against tasks:
* Store the CPU usage for user and system separately, and display
them separately.
* Disk IO is now measured in bytes, not ms. Also store the
read/write bytes separately.
* Store started and ended times, as well as elapsed_time. This
will enable future features such as showing which tasks were
running at a particular point in the build.
There was also a problem with how we were looking up the Task
object, which meant that the buildstats were being added to
new tasks which weren't correctly associated with the build. Fix
how we look up the Task (only looking for tasks which match the
build, and the task and recipe names in the build stats data) so
the build stats are associated with the correct task.
[YOCTO #8842]
(Bitbake rev: efa6f915566b979bdbad233ae195b413cef1b8da)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When user enters a non-ascii character in the project
name of toaster, the build doesn't get triggered.
Use force_bytes to fix this.
Also deal with non-ascii project names when logging the
build request in runbuilds.
[YOCTO #9071]
(Bitbake rev: b6141c4d170885d3bdf63074afcb1e41fde0a8f0)
Signed-off-by: Sujith H <sujith.h@gmail.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
For some reason, the values for SRC_URI[md5sum] and SRC_URI[sha256sum]
were not being expanded. That lead to the following code not working
as expected:
SRC_URI = "http://.../${PN}-${PV}.tar.gz"
MD5SUM = "123abc..."
SHA256SUM = "abcd1234..."
SRC_URI[md5sum] = "${MD5SUM}"
SRC_URI[sha256sum] = "${SHA256SUM}"
(Bitbake rev: ba011470df0ea8bd89f01c0b02ec4b3969e60ce7)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
XMLRPCServer.single_use attribute was always set to False.
This caused xmlrpc server to keep running after build is done as
BitBakeServerCommands.removeClient only shuts down server if its
single_use attribute is set to True.
(Bitbake rev: 0a60b0928a0a746a60d2c2f294ff1903963c7086)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Without this you get a rather odd traceback instead of the proper
exception message.
(Bitbake rev: 2fe1826d3077eeda6cde433d3a1e6620f74e08dd)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The output of "npm view dependencies" isn't entirely JSON if there are
multiple results, but the code here was just discarding the output if
the entire thing didn't parse as JSON. Split the output into lines and
iterate over it, parsing JSON fragments as we find them; this way we end
up with the last package's dependencies since it'll be last in the
output.
Digging further, it seems that the dependencies field reported by "npm
view" also includes optional dependencies. That wouldn't be a problem
except some of these optional dependencies may be OS-specific; for
example the "chokidar" module has "fsevents" in its optional
dependencies, but fsevents only works on MacOS X (and is only needed
there). If we erroneously pull in fsevents, not only is it unnecessary
but it causes "npm shrinkwrap" to throw a tantrum. In the absence of a
better approach, look at the os field and discard the module (along with
any of its dependencies) if it isn't for Linux.
As part of this, we can reduce the calls to npm view to one per package
since we get the entire json output rather than querying twice for two
separate fields. Overall the time taken has probably increased since we
are being more thorough about dependencies, but it's not quite as bad as
it could have been.
(Bitbake rev: 436d67fe7af89ecfbd11749a6ae1bc20e81f2cc8)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
"2 || 3" is a valid version specification for a dependency in an npm
package.json file, but of course that looks like something else when
sent to a shell. Quote the version value to avoid this.
(Bitbake rev: bea0246831a46d943d2e27d6b38f6e498bd3413c)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Continue after processing BuildStarted event to fix
WARNING: Unknown event: <bb.event.BuildStarted object at 0x2554150>
(Bitbake rev: 12f1fb8c9b70fea0c9145f881bcceb8af32df6af)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Toasterui exits only if bitbake observer shuts down.
In build mode it should exit when build is done.
Made toasterui exit on bb.command.CommandCompleted,
bb.command.CommandFailed and bb.command.CommandExit events
when it's running in build mode.
(Bitbake rev: b11f9d6d3c2eb615335901e1dcea699daf3afb4c)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently toasterui works only in observer mode. This is
artificial limitation which was made to support current toaster
design. As we decided to stop using bitbake server we'll
need to run toasterui also in build mode.
[YOCTO #7880]
(Bitbake rev: d4b5796899c3ca5c7becd7322291afd8afb35a31)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently toasterui ignores return value of setEventMask
command, which created confusing difference between set of
events set by this command and the real set used in the code.
Checked if setEventMask succeeded. Print error message and
exit if it's not.
(Bitbake rev: 6e3f13ffb47102b5df2da91fbc3f5da3179245b2)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Executing setEventMask command when bitbake server is in readonly
mode causes runCommand to fail with the following error:
'Not able to execute not readonly commands in readonly mode'
Set readonly attribute for setEventMask command to make it working
for Toaster UI. This should not do any harm as this command doesn't
influence cooker state.
(Bitbake rev: 8a47d30b2555255fbf6049c5ed69b29664c32b17)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Removed events not used in the code from the list.
Added events that are used in the code.
(Bitbake rev: 16b14ec16049cc2040a60ad5fc95f6e19dda91a6)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Reformatted and reordered list of events to make changes
easily and see them clearly in the diffs.
(Bitbake rev: 42a2d1115f2b23dc063a3172285ca3be73cf70bb)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The code of this module is broken for a long time.
The functionality of it can be easily achieved by running
'manage.py runbuilds.py' on remote machine.
[YOCTO #8806]
(Bitbake rev: 975081eefdd7041a6b4bef6842c1bac9799a1b44)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Removed usage of sshbecontroller from bbcontroller, models, tests
and database schema.
(Bitbake rev: 3ee06eb7e96de5dba539ad52201867e77d06a53e)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Raised NotImplementedError instead of Exception to be
able to catch it.
This is a preparation for removing sshbecontroller module.
It has to be done as code in bldcontrol/tests.py imports custom
NotImplementedException from sshbecontroller.
(Bitbake rev: c243ab6c83fe12d84777e4c3a18fd393827b9327)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This patch e0fd96442a removed the
"use strict" directive from the projectpage.js file by mistake.
This patch makes amends.
(Bitbake rev: b8044ce60af3f0b064cfba76c577503cc896e358)
Signed-off-by: Belen Barros Pena <belen.barros.pena@linux.intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
I've received some feedback on the 'Revision' label we use in the import
layer page. It is not quite communicating that what's required is a Git
revision. Changing it to 'Git revision' to make it a bit more specific.
The change applies not only to the import layer page, but to all pages
showing revision information in the project configuration section.
For more on the feedback received, check
https://bugzilla.yoctoproject.org/show_bug.cgi?id=8429#c3
(Bitbake rev: 09392f36a4f115c2432302125e8cac48a9aa304f)
Signed-off-by: Belen Barros Pena <belen.barros.pena@linux.intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
As we do with the popover snippet which shows dependencies inline in the
table also show dependencies which have both TYPE_TRDEPENDS and
TYPE_RDEPENDS.
Also remove obsolete comment
(Bitbake rev: d3b5f3b7ba4550e7cd03a37ca19ccd2fc0042b2d)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
[YOCTO #9156]
For a customized image when adding a dependent package X that depends on
dependency package Y, in addition to adding X to appends_set and Y to
includes_set, make sure that Y is no longer in the excludes_set. Y may
have been added to the excludes_set by a prior package removal.
(Bitbake rev: 6b29d3297de2ae48a3ac5529ba0d22f895276b56)
Signed-off-by: Dave Lerner <dave.lerner@windriver.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Tarballs that are fetched down via npm repositories seem to often have
unknown headers. This doesn't affect our ability to extract the contents
though so we don't really care to see those warnings.
(Bitbake rev: b38975103e52a0c25e9ad9032c8cca1c47cbdcc2)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
npm allows you to specify a dependency as a Github username+path, or
omit the version entirely. You can hit these if you don't use a
shrinkwrap file, with the result that the code later fails due to the
output of "npm view" being empty; so handle this lazily by just ignoring
this part of the dependency if it's not really a version.
(Bitbake rev: 7b7a65c44dbdd5ba9366d4e2093f76df8758d546)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
No code changes, just fix to use four spaces.
(Bitbake rev: 66a9ee7d54ca9c25209f72da079f260ccdcc872a)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fixes [YOCTO #9147]
Added the d.expand(expression) fuction to the list of functions
in the "Accessing Datastore Variables Using Python" section.
(Bitbake rev: 19507b80f35d37dc4b1614bd390b8e261dd4a2bd)
Signed-off-by: Scott Rifenbark <srifenbark@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fixes [YOCTO #9148]
Added a note about Variable expressions (e.g. ${X}) are no longer
expanded within Python functions.
(Bitbake rev: 4ebe55cb8c8db1c5cf5d127e213487c5a453a68a)
Signed-off-by: Scott Rifenbark <srifenbark@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fixes [YOCTO #8567]
Updated the variable list describing the variables that affect
how environment variables are handled. Also updated the section
on how those variables are passed.
(Bitbake rev: 91cb52b1e77bba9d046239933b5c0513d01e6824)
Signed-off-by: Scott Rifenbark <srifenbark@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This cleanup fixes a few issues:
1) Removes the superflous code to set toasterconf.json
- this isn't used and referenced meta-yocto
2) Changes exit to return so we don't surprise the user by exiting their shell
- this is necessary because it is being sourced
3) Removes the last references to the old TOASTER_MANAGED variable
- this is historical and no longer used.
4) Adds -t parameter to lsof
- This stops it from dying on odd filesystems and is much
faster since all we are using are processes anyway
5) Handles start and stop as params
- it was easy to confuse the script especially
if we were calling it with parameters.
- if start/stop isn't specified, it will still toggle
(Bitbake rev: 88fddbe80f56828026bf93560037af52b5dab628)
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This ui does not work in master, nor has it been updated for several years.
[YOCTO #9178]
(Bitbake rev: 9fad1d13eed1f725971e6d12d3977cd31e07019a)
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We've been gearing up the Toaster web UI to replace the Hob (GTK+ based) UI
for some time now; Hob has basically been on life support for the past few
releases. As of late last month in master, Toaster has the capability to
select the packages in an image, removing the last thing that Hob could do
that Toaster couldn't.
To recap, the reasons why Hob is being removed include:
- The code is tightly woven into BitBake, making it fragile. This means it
needs significant QA and maintenance on an ongoing basis.
- Some of the implementation is not ideal; we'll be able to remove some cruft
from BitBake and OE-Core at the same time.
- It's GTK+ 2 based, not the current GTK+ 3.
- Toaster is now a much more capable UI and is being actively maintained
The discussion about removing hob can be found at:
http://lists.openembedded.org/pipermail/openembedded-architecture/2016-February/000082.html
(Bitbake rev: be2cceea159c6ca9111eff3df87b98513eab6d72)
Signed-off-by: bavery <brian.avery@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
npm-shrinkwrap will sometimes resolve a git URL which instead of a http url, in
this case go and grab the dist.tarball via npm instead of using the resolved
URL.
(Bitbake rev: eb53b927ff59aa19cf28bc46beb9f9a185a59990)
Signed-off-by: Brendan Le Foll <brendan.le.foll@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The previous commit breaks absolute pathnames in file:// urls, this
fixes it.
(Bitbake rev: b8113a1800687a37a26ac28deafdbafd74cc138e)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When in SRC_URI appears file://dir;subdir=foo unpacker copies 'dir' to ${WORKDIR}, not
${WORKDIR}/foo as it should be.
These changes are fixing following bugs as well:
Bug 6128 - Incorrect wildcard unpack behaviour in fetcher
Bug 6129 - Local directories unpack to a different location than local files
(Bitbake rev: e659a3b0c2771679057ee3e13cd42e6c62383ff2)
Signed-off-by: Alexander Shashkevich <alex@stunpix.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
For some reason the enablement piece of the patch went missing, add it.
(Bitbake rev: 0270b5a3873ed0aeca3a66198c87a6164fb644b8)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
PN can contain '_', e.g. gcc-cross-x86_64 and an override cannot
hence we do this manually rather than use OVERRIDES.
(Bitbake rev: 7a6baf02617d1edced4eaff235e73d746e2a3b68)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
For local files, there are no races with downloads, we don't need ".done"
stamps and we don't need lockfiles.
This considerably cleans up DL_DIR and all the pointless ".done" files
as well as removes stalls over local files with the same name.
(Bitbake rev: 48e903745db578d9b9b425a8d411c1369df0eb94)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Rather than create ".lock" and ".done" files with no name, error,
forcing us to fix the cases where this is a problem.
(Bitbake rev: 81158071508cc68c39db7d501370872f44d335cc)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If we specify urls such as npm://somehost;someparams the fetcher currently
does a poor job of handling mirrors of these urls due to deficiencies in the
way decodeurl works. This is because "somehost" is returned as a path, not
a host.
This tweaks the code so that unless its a file url, the host is returned
correctly.
This patch also adds test cases for these urls to the exist set of test
urls.
We need to tweak the URI() class since this thinks this is a relative url
which is clearly isn't. We also need to handle the fact that encodeurl will
error if passed a url of this form (it would want the path to be '/'.
(Bitbake rev: 83203cd2e677706e0111892a7843b83263cb8bd9)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If we wget something which looks directory like we end up with lock files
and done stamps without names, they also all use the same lockfile.
This change ensures that we use separate lock files based on the url
and avoid creating the mysterious ${DL_DIR}/.done files.
(Bitbake rev: 20bc82086018832e047345a672d74b6c1c113650)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
I have received quite a few complaints about the use of the word
'delete' for layer removal, so change it to 'remove'. That also matches
the language we use for packages in image customisation.
[YOCTO #9165]
(Bitbake rev: 3c5ac2ddfb3f5ecd3f3218de0d6564e5f3842b98)
Signed-off-by: Belen Barros Pena <belen.barros.pena@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
I quite dislike the 'Run again' label we use in the button that rebuilds
things. Changing it to 'Rebuild', which is shorter and more specific.
(Bitbake rev: 865a2015e86a1bc5cc7d63308f27c292d1ca98eb)
Signed-off-by: Belen Barros Pena <belen.barros.pena@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Correct formatting of the banner message after adding a package to a
custom image.
[YOCTO #9101]
(Bitbake rev: da233005eb8cfa7842cd1a768c16e42aaaa55fad)
Signed-off-by: Dave Lerner <dave.lerner@windriver.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The default "Command line builds" project does not have a Configuration
page. It therefore needs a custom breadcrumb where the project name goes
to the project builds page, instead of the project configuration page.
(Bitbake rev: 5545acf6703a25ee46776138bbbd804615add89c)
Signed-off-by: Belen Barros Pena <belen.barros.pena@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Returns a script (string) that reconstructs the state of the
entire database at the time this function is called. The script
language is defined by the backing database engine, which is a
function of server configuration.
Returns None if the database engine does not support dumping to
script or if some other error is encountered in processing.
The SQLite3 implementation in db.py calls iterdump() [1] to generate
a script. iterdump() is the library equivalent of the `sqlite3 .dump`
shell command, and the scripts are compatible. Execute the script in
an empty SQLite3 database using the sqlite3 utility to restore a backup
of prserv.
Use case: Backup a live PR server database in a non-racy way, such
that one could snapshot the entire database after a set of bitbake
builds all using a shared server. I.e. All changes made prior to
the start of a dump_db() operation should be committed and captured
in the script. Subsequent changes made during the backup process are
not guaranteed to be captured.
Testing: ~7MB database backs up in ~1s while PR server is under load
from 32 thread bitbake builds on two separate machines.
[1] https://docs.python.org/2/library/sqlite3.html#sqlite3.Connection.iterdump
(Bitbake rev: 004003daf6bd0f0233ce5c2d95f1d7d64ab91bb3)
Signed-off-by: Haris Okanovic <haris.okanovic@ni.com>
Reviewed-by: Ken Sharp <ken.sharp@ni.com>
Reviewed-by: Bill Pittman <bill.pittman@ni.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fix the view of 'Image Recipes' under 'Configuration' to only show
image recipes that are not customised since custom images have their
own page.
[YOCTO #9111]
(Bitbake rev: 18a93b360301a5497d5c8ef74ab71f374f2ad210)
Signed-off-by: Dave Lerner <dave.lerner@windriver.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The build dashboard doesn't show image and artifact files correctly,
as it shows the full filename for images and the filename plus
path relative to DEPLOY_DIR for artifacts.
Instead, show just the suffix for image files, and the basename
for artifact files.
(Bitbake rev: 8084dcdc283b4dc170f066c202f89d56ce1abbef)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In the 'New custom image' page, each image recipe name listed should
link to the corresponding image recipe details page, so that users can
look into what packages are installed by a certain image, and decide
based on that if they want to customise it or not.
This patch adds that missing link.
(Bitbake rev: a481af693bfef0171732c18c298e285986b82de3)
Signed-off-by: Belen Barros Pena <belen.barros.pena@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The search field at the top of our tables was using one of the Bootstrap
classes for text field sizing. Those classes are a bit rigid, resulting
in text fields too wide that made other table controls wrap.
Setting a maximum width to the search form using one of the span classes,
combined with a % width css declaration, make for text fields that
adapt a bit better to the horizontal space available in each table.
(Bitbake rev: 7833fab2e03f2d9a01ab9ad0a13c190382098b5e)
Signed-off-by: Belen Barros Pena <belen.barros.pena@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Our project pages have 4 tabs: builds, configuration, import layer and
new custom image. Even though we treat the 'configuration' as the
default tab, it comes second after the builds tab.
That's a bit strange: the default tab should be the first one listed.
This patch changes the tab order to put 'configuration' first.
(Bitbake rev: ccb90019489c2c324c2a5a60295e02280a2ec18f)
Signed-off-by: Belen Barros Pena <belen.barros.pena@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The existing breadcrumb does not always provide a link to the project
configuration page. When you are in the build history pages, you must go
back to the builds information first, and from there access the project
configuration. That feels very long.
Change the breadcrumb so that the project name item always provides a
link to the project configuration.
(Bitbake rev: 9910f3f292d35fc91215d550c5f123dcf18ab35d)
Signed-off-by: Belen Barros Pena <belen.barros.pena@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Now that we use modal dialogs to display dependency information for
packages, we are hitting their maximum height relatively often. It is set
by default to 400px, which makes it a bit tight at a 1280x800 viewport
size.
Reduce the maximum height to 300px to make things a bit more
comfortable.
(Bitbake rev: e36001d61768979d66cba0f3d4f5a2aaf4af2cb7)
Signed-off-by: Belen Barros Pena <belen.barros.pena@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The 'add layer' button in the project configuration page remains enabled
after you add a layer. If you click it again, the same layer you just
added is added again.
This patch disables the 'add layer' button on click, to avoid this bit
of weirdness.
[YOCTO #8905]
(Bitbake rev: 63705f60035884a810fdd36e5a3fe10e411f23c7)
Signed-off-by: Belen Barros Pena <belen.barros.pena@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The form for naming new custom images shows you an error message when
the name already exists or you include an invalid character in it. But
when an error appears, the input field was missing the red highlight.
This patch applies the right class to the form controls whenever an
error message is shown.
(Bitbake rev: df342e7662179410467c47cd870180ea75f863d4)
Signed-off-by: Belen Barros Pena <belen.barros.pena@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The placeholder text in the form where you name your new custom images
didn't display fully. This patch fixes the styles so that the text shows
properly. It also changes the text itself to make it a bit shorter.
[YOCTO #9122]
(Bitbake rev: 9df802182b0b96295b148a1681c2265e72d8306b)
Signed-off-by: Belen Barros Pena <belen.barros.pena@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Although the support for building more than one release and how we
handle the build directories is the subject of lively discussion, we all
seem to agree on removing the ability to change the release of a
project. The feature is currently not working but exposed to users,
which is not a happy state of affairs.
This patch comments out the controls that give access to the release
changing functionality to hide them from users, but does not touch
anything else. Once all moving pieces start to settle down, we can make
a final decision regarding this feature, and clean up the code
accordingly.
[YOCTO #8917]
(Bitbake rev: 3a8c6f7155517cd61a160595b81e7bed84ba4eaf)
Signed-off-by: Belen Barros Pena <belen.barros.pena@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
npm fetcher with support for shrinkwrap files and lockdown files to easily
download and install an npm package with strict dependency resolution.
The SRC_URI should be in the format of:
SRC_URI = "npm://registry.npmjs.org/;name=${PN};version=${PV}"
To add a shrinkwrap and lockdown file use:
NPM_SHRINKWRAP := "${THISDIR}/${PN}/npm-shrinkwrap.json"
NPM_LOCKDOWN := "${THISDIR}/${PN}/lockdown.json"
(Bitbake rev: dec75bbc5d075acb322dad8b1c40d6bd518dc9fd)
Signed-off-by: Brendan Le Foll <brendan.le.foll@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This is useful as npm-lockdown uses sha1 because npm releases the sha1 of
packages and whilst this is undocumented it seems no other algorithm is
supported
(Bitbake rev: fd5d9011f6dd7029895b64d8a02d33185b9aa8ae)
Signed-off-by: Brendan Le Foll <brendan.le.foll@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This test was passing but because the assertion was the wrong way round
and should have expected the first one to pass and second one to fail,
in reality both were failing as the method for checking the invalid char
was incorrect.
(Bitbake rev: 932a92b8130d4815656dc885f0c6e4afa4502022)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When showing the package list for the image recipe details only show the
packages which are installed in the image rather than all the packages
which are produced.
[YOCTO #9108]
(Bitbake rev: be7dca6c7607c0d13151c2d3f7ad7adcdf365076)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Instead of having to manually write up in the setUp function all the
different objects to create in the database, just specifiy them in a
fixture and load it. This has the advantage that it can be used on a
live server by using the ./manage.py loaddata command and can then
be inspected.
(Bitbake rev: 8a653ad8b075892d24d15696b3070c5a28a4fbe2)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Make sure the default orderby for the SoftwareRecipesTable is applied
(Bitbake rev: 1688608b537d8de840c6d1e4802ae41ca872e5bf)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We messed up the migrations by squashing some of the image
customisation model definitions into the initial migration which
has meant some irreversible operations on mysql took place.
This deletes, re-orders and fixes the migrations.
If your schema is up to date you may want to use ./manage migrate
with --fake or --fake-initial to avoid re-applying migrations.
[YOCTO #9116]
(Bitbake rev: 19bd63fc3a28dcbd0f531a5b06a037da34568bac)
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
At some point in the future, getVarFlag should expand by default. To
get there from the current position, we need a period of time where the
expand parameter is mandatory.
This patch starts that process. Clear errors will result from any code
which doesn't provide this. Layers can be fixed with an expression
like:
sed -e 's:\(\.getVarFlag([^,()]*, [^,()]*\)):\1, False):g' -i `grep -ril getVar *`
(Bitbake rev: aa3faebdf6af66ab34f74d328b2113de0b08c7ee)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
At some point in the future, getVar should expand by default. To get
there from the current position, we need a period of time where the
expand parameter is mandatory.
This patch starts that process. Clear errors will result from any code
which doesn't provide this. Layers can be fixed with an expression
like:
sed -e 's:\(\.getVar([^,()]*\)):\1, False):g' -i `grep -ril getVar *`
(Bitbake rev: fab717d303df0bcef737661f6917f275f35215a4)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Define a new bitbake configuration variable BB_HASH_CHECKSUM_CACHE_FILE
that can be used to define the cache file to use for file checksum
cache.
(Bitbake rev: a965b390d6240e279c190b92b17c0573e9bd604c)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If no cache file name is given a default from class variable is used,
like before.
(Bitbake rev: 2602a312818f564961de7dfa63c429d45ff9e5ac)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Move the local file checksum functionality from bb.fetch2 into
bb.checksum module.
(Bitbake rev: 4f60933283f377d68f191db849dac6c1dc7a0aed)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Before this patch the usage of cache was quite useless as the file
checksums were not actually cached on disk but re-calculated every time.
This patch utilises the new writeout_file_checksum_cache() method of the
SignatureGenerator class to do the job.
(Bitbake rev: 5ac9cbf405841ed3f65e6f99a3cee032567fb182)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Extend the API in order to be able to write out the file checksum cache
onto disk. SignatureGeneratorBasic class now implements a method that
update the fetcher local files checksum cache with the task file
dependency checksums.
(Bitbake rev: ecdabd321d48fa367b89ebffc00aa525b6eaa95c)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Drop unused 'd' argument from the cache save methods, simplifying the
API.
(Bitbake rev: 81bc1f20662c39ee8db1da45b1e8c7eb64abacf3)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The current behavior of Hob is that there is a "Run Image" button which becomes visible only for qemu images.
My suggested change is:
- if an image is selected and it is qemu-compatible, let the "Run image" button be named "Run qemu image"
- if an image is selected and it is not qemu-compatible, let the same button show up with the name "Run custom image", and besides that, an option shows-up to allow the selection of the custom script (by default it points out to runqemu script) to be used for launching this custom image
Note: in case there is more than one toggled image (qemu runnable or deployable), when the user clicks the "Run custom image" button, a dialog will be presented, allowing to choose between any of the existing images.
[YOCTO #8940]
(Bitbake rev: cc4cfc2370297b8feb2dc39d4262e73adf06c09a)
Signed-off-by: Mirela Rabulea <mirela.rabulea@nxp.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
An example prefix: `perl-5.22.1-r0 do_compile:`
(Bitbake rev: 792b759e59e31d2e43d525a6e50d866b4f51f072)
Signed-off-by: Christopher Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This lets us filter and use -l to show messages from that source specifically.
(Bitbake rev: 7946927156dec33364418988eb921ddb273660eb)
Signed-off-by: Christopher Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If the worker segfaults, we may never see a TaskFailed event from it, only
a runQueueTaskFailed event. In this case, return_value isn't getting set
leading to an incorrect exit code from bitbake. Fix by setting return_value
in both places.
(Bitbake rev: e5dd50e0d95d532fe31dde61f8c6b1a7a72321e9)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If there is a missing provider and we're using "-k" mode alongside "-w",
we could get a traceback since there was no provider. Add tests to avoid this.
(Bitbake rev: 90a4805e4e770a433b4394ea99792731e9a4b546)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We're seeing issues where the self test, which uses tinfoil doesn't
notice the changed contents of include files. The issue is
cached_statements in the parser being reused when the files have changed.
Whilst looking at this, I realised there were some other issues:
* We need to also invalidate the mtime cache when cooker restarts
* We should pass full filenames to the file invalidation code
* We should process cached_statements as part of inotify invalidation
With these fixes, the caching is more reliable for memory resident
bitbake too. It does raise some questions about cache validation and
lifecycles and indicates bitbake does need more work in the area,
preferably with the removal of the globals. This at least highlights
and works around some of the current issues.
(Bitbake rev: 3f507ff8bc467fba936cf3f31bb8ea8e02f168e8)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
CookerParser.shutdown code doesn't do all required work to shutdown
parser processes. As a result bitbake hangs if interrupted during
parsing. Putting None into the parser_quit queue should fix this issue
as it makes parsers to quit main loop.
(Bitbake rev: f67307977e8f089ce6d208d3e9de2a6a1768757e)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The brbe variable is set on the bitbake server when the Toaster
UI starts. This enables Toaster to associate events with the
correct build and build environment.
However, the brbe variable is also used when a build starts to
identify whether a new build needs to be created, or an existing
one looked up. This causes a bug for command-line builds which
happen after a Toaster-triggered build: because the brbe variable
is never unset on the server or the buildinfohelper, the new
command-line build events are treated as originating from the
previous build.
Ensure the brbe variable is reset when the buildinfohelper "closes"
a build, so that each build then either sets the brbe variable
(Toaster-triggered builds) or leaves it blank (command-line builds).
Also modify the localhostbecontroller so that the brbe variable
is not set on the server and not looked up from the server. This
ensures that it is only set when the triggerBuild() method is
called, and that it remains as None for command-line builds.
[YOCTO #9021]
(Bitbake rev: 4a6a8d0074f62208d843b06344be31ae73d9b745)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Remove symlinks in the UI tests folder as they are causing
problems for bitbake upstream.
[YOCTO #8787]
Signed-off-by: Mihail Stanciu <stanciux.mihail@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If something fails in a exec_func_python() the current stack trace shows
incorrect filenames and linenumbers. For example:
The stack trace of python calls that resulted in this exception/failure was:
File: '/media/build1/poky/meta/recipes-sato/images/core-image-sato.bb', lineno: 200, function: <module>
0196: chksum = bb.utils.sha256_file(fn)
0197: f.write('%s\t%s\n' % (chksum, os.path.relpath(fn, baseoutpath)))
0198:
0199:
*** 0200:copy_buildsystem(d)
0201:
File: '/media/build1/poky/meta/recipes-sato/images/core-image-sato.bb', lineno: 9, function: copy_buildsystem
0005:IMAGE_FEATURES += "splash package-management x11-base x11-sato ssh-server-dropbear hwcodecs"
0006:
0007:LICENSE = "MIT"
0008:
*** 0009:inherit core-image
0010:
0011:IMAGE_INSTALL += "packagegroup-core-x11-sato-games"
File: '/usr/lib/python2.7/subprocess.py', lineno: 535, function: check_call
0531: The arguments are the same as for the Popen constructor. Example:
0532:
0533: check_call(["ls", "-l"])
0534: """
*** 0535: retcode = call(*popenargs, **kwargs)
0536: if retcode:
0537: cmd = kwargs.get("args")
0538: if cmd is None:
0539: cmd = popenargs[0]
The problem is the use of "FILE" to obtain the current filename. Instead,
we therefore inject the function being executed into the methodpool which
allows us to correct its linenumber and filename information. We can then
clearly mark the initial piece as autogenerated and the rest of the linenumber
and filename information should be correct. Afterwards the trace starts:
The stack trace of python calls that resulted in this exception/failure was:
File: 'exec_python_func() autogenerated', lineno: 2, function: <module>
0001:
*** 0002:copy_buildsystem(d)
0003:
File: '/media/build1/poky/meta/classes/populate_sdk_ext.bbclass', lineno: 66, function: copy_buildsystem
0062: import glob
0063: import oe.copy_buildsystem
0064: import subprocess
0065:
*** 0066: subprocess.check_call("foo")
0067:
0068: oe_init_env_script = d.getVar('OE_INIT_ENV_SCRIPT', True)
0069:
0070: conf_bbpath = ''
File: '/usr/lib/python2.7/subprocess.py', lineno: 535, function: check_call
0531: The arguments are the same as for the Popen constructor. Example:
0532:
0533: check_call(["ls", "-l"])
0534: """
*** 0535: retcode = call(*popenargs, **kwargs)
0536: if retcode:
0537: cmd = kwargs.get("args")
0538: if cmd is None:
0539: cmd = popenargs[0]
We can't inject into methodpool at parsing time, since there may be
_append or other override operations against the function before its
execution.
(Bitbake rev: fae153095d23157dd7e72c29f683f86149ee33a8)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Right now, if you have some python code like:
X = "a"
def somefunction(d):
d.setVar("X", "b")
d.setVar("Y", "${X}")
then any sane person would expect that Y = "b" at the end of the
function. This is not the case, Y = "a".
This is due to the python function being expanded before execution, the
executed code would read d.setVar("Y", "a"). This understandably
confuses people, it also makes it near impossible to write ${} in a
python function without unintended things happening.
I think there is general agreement we should fix this and standardise
on non-expansion of python functions. We already don't expand anonymous
python (mostly).
I've checked OE-Core with buildhistory before and after this change and
there were a small number of issues this exposed which I've sent
patches for.
I propose we default to not expanding python code and then deal with
any consequences from that if/as/where identified. This will improve
new user understanding and usability of the system, it also allows
several long standing weird expansion issues to be fixed.
(Bitbake rev: 8bf33a8e92c0e188fa392030025756196c96fcbb)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We don't want to expand python functions since they aren't expanded
at execution time (e.g. anonymous python). They can also have side
effects.
This function is primarily used by toaster for variable dumps for later
display. The lack of expansion of python functions won't matter in this case
and actively helps some variable handling (e.g. SRCPV).
(Bitbake rev: 3f5520b4844a4bdd615046479ba08ed192bdc8cd)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Expanding python functions for variable dependencies doesn't really make sense,
not least since this causes execution of any inline python, it also makes it
impossible to write expressions like d.expand("${X}") of d.setVar("X", "${Y}")
which may have the wrong values if expanded now.
This starts to standardise the approach across bitbake for handling python code.
(Bitbake rev: 765a2480dbe288f64562a9611dd93b6b6dd0a64e)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We don't expand anonymous python before execution, so nor should
we do this when calculating checksums for them.
(Bitbake rev: 5f10987edda35b08970a6dd6ccf9febad271ce3e)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Revert "toaster: models.py do not use transactions".
This reverts commit e248a503ba
(Bitbake rev: 48d0d510816346073f135bb86fb7904fdb50bb27)
Manually managing the database transactions caused more
problems than it temporarily solved so we return control back to
Django's orm.
[YOCTO #8796]
(Bitbake rev: 25c531915b6f8f79a0822996ceb97f90483e359f)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* Display always the first target in alphabetical order to match what we
do in the breadcrumbs and the build dashboard heading
* Remove the extra space between the '+' and the additional number of
targets
* Make sure the tooltip with the full target list takes the Bootstrap
tooltip styles
* Replace the word 'targets' in the tooltip with 'recipes', since that's
how we call build targets everywhere else in Toaster
(Bitbake rev: 3b8747d0af4b9164e973940ed97751c951e74110)
Signed-off-by: Belen Barros Pena <belen.barros.pena@intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The pages in the build data section of Toaster showed different
breadcrumbs: in some pages the machine was displayed, but not in others.
For builds with more than one target, some pages showed the first
alphabetical target (the correct behaviour), others didn't.
This patch removes the inconsistencies, showing exactly the same
breacrumb across all pages in the section.
The patch also removes the extra space between the '+' and the number of
targets when the builds have more than one target.
Remove an unneeded debug message
(Bitbake rev: 9cdbb543311b6f4a8a88c27fc157d998242444ee)
Signed-off-by: Belen Barros Pena <belen.barros.pena@intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The functionality overlap between these two functions is significant and
its clearer to handle both things together since they are intimately
linked. There should be no behaviour change, just clearer code.
(Bitbake rev: 391aa4afc91be90d8d3ee47e1bf797d6ebe61a71)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When we use functions from the data store, they now have correct line number
and filename information. This function would attempt to correct line numbers
which doesn't need correcting, leading to misleading messages to the user.
Therefore remove this code as being obsoleted.
(Bitbake rev: 918bec86bc8ee94feb82380ff410d9fdcbe9e720)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Extended the dot styling of dependencies created by bitbake -g in dot syntax to differentiate between the various kinds.
depends: solid
rdepends: dashed
rrecommends: dotted
The change observed is that depends get an explicit style which is the same as dot default behavior and the runtime recommends get
dotted while before they were dashed. This helps to distinguish them graphically as well as eases post processing by script.
(Bitbake rev: 86e78e0ca7aa5452411f35239942ecee3d8824ec)
Signed-off-by: Henning Schroeder <henning.schroeder@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add the dependencies of packages which are added to the
CustomImageRecipe. Currently just handle the first tier of dependencies
as this is what we show in the UI.
(Bitbake rev: 5c44609a9bf9fb23241b7dd7c58b08901d75008d)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Allow package groups in our custom image recipe. Excluding them creates
more undefined behaviour than including them at this stage. Also update
to use convenience method for returning all packages.
(Bitbake rev: 8c2e8a13badacb816c4b1178b6661600008b38af)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add link to the last successful build if there is one and add
conditionals for the ancillary recipe metadata.
(Bitbake rev: 4660aaf1c6775270f8f3d0afbb7fa2ee7a2a1563)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a method to update the packages included list from the last
build, this effectively "synchronises" the package list from what
we think will happen at the Customise image stage with what
actually was produced with a build. It's not ideal to have this
function here but we also need to make sure that no race condition
of the user accessing this list and it being updated occurs.
(Bitbake rev: 8cf6e67a955574b33856a082bdadf3194f2b6ba4)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Field to keep track of when the package list for the CustomImageRecipe
was last updated from a build.
(Bitbake rev: 4bd4e49f13a7625997a43f3b2e67ed42c3c8e08b)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a convenience method to get the last successful build
target for a CustomImageRecipe.
(Bitbake rev: 4dde3d830cd38bbe306d83629dcb80da5fc9b027)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In the dependencies popover just show direct dependency in the list
rather than recommends, conflicts etc
(Bitbake rev: e69b00532b011327bc2495a6fb52cfe98f0f897d)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This convenience method returns just the Package_Dependency for
the package which are regular dependencies i.e. not RECOMMENDS or
any other types.
(Bitbake rev: bd76c22fe2aa06690b4ee25de69219ac0bf6b4d6)
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Instead of keeping the original dependency information for the pool of
CustomImagePackage reset it with each new build.
(Bitbake rev: a0b97ffc7a468bad081ce3276c74728bf6830250)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add front end handling of validation response from create new
CustomImageRecipe api.
(Bitbake rev: eff66b502df8e001cd0abc25bcbd742687169619)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Check that the name for a new CustomImageRecipe doesn't already exist in
the project or in the database of existing recipes (e.g. from the layer
index). Also restrict the characters entered for the recipe naming
convention.
(Bitbake rev: f290d428460a07e73050ff613bc222cc8c04f5ec)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add the file size of the packages and the total to the JSON response.
(Bitbake rev: bbbd304c49b0940a695d15273934edff95d70836)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Whem a m2m field changes we need to clear the ToasterTables cache as
this can affect the state of items in ToasterTables.
For example the CustomImagePackages being added or removed from a custom
image recipe.
(Bitbake rev: c9d7b68ee0186a71e8e75a5d87122a0328001515)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Update the states of the packages in the package selection UI to reflect
whether it's likely that 1st level dependencies for the package will be
also added.
(Bitbake rev: 119569d83c3fb1d1bd162624819b3f9c63a791c4)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Instead of defining this as a string it's sufficiently large enough to
warrant its own file.
(Bitbake rev: 6b39423fe5a3ed30289a8b303329a5725f7d273b)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If we're breaking up an image recipe's packages we will also need to
override IMAGE_FEATURES to make sure the customisation is not altered
(Bitbake rev: 9fd7b05dc0cf9240f7c8e3dc77b009064fd2b0cb)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When the packages associated with a custom image recipe are
shown in the customrecipe editing page, locale packages are
shown in the same way as all other packages. This gives the false
impression that these packages can be removed, when in fact
they are automatically added due to the IMAGE_LINGUAS build
variable.
Modify the customrecipe page so that locale packages cannot be
removed, and provide some help text explaining why.
[YOCTO #8927]
(Bitbake rev: b2208e53c00a67a7d0345e7378e6806b8ae40fb4)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The custom image editing page doesn't show locale and packagegroup
packages: they are filtered out of the queryset used to populate
the ToasterTable.
Rather than filtering these packages out, include them in the list
of packages which are shown.
(Bitbake rev: 38a753e7b2e9ede326856b830b25e13bdd6d0d9b)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The custom image package selection filters were using the old
ToasterTable filter approach, which caused the table filter to fail.
Amend the table to use the new ToasterTable filtering API to fix
this.
(Bitbake rev: 72a4cb30842fd053e46dc56df222729cbe735162)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The progress updater for the recent builds section makes a JSON
call to the project view URL to get progress for each build.
However, conversion of the builds pages to ToasterTable broke this,
as the JSON response no longer contained the data necessary to
populate the progress bars.
Move the recent builds query to the Build model, so that it is
accessible to the ToasterTables using it ("project builds" and
"all builds"), as well as to the "project" view.
Modify the code in the recent builds template to use the slightly
different objects returned by the recent builds query on Build.
(Bitbake rev: 5189252635ddc7b90c9a43aaed9f196c31e1dcad)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When applying migrations, Django shows this warning:
"Your models have changes that are not yet reflected in a migration,
and so won't be applied."
This is because the customimagerecipe model has changed, but those
changes are not covered by a migration.
Add the missing migration to clear this warning.
(Bitbake rev: df8185fcbd84061976d91b03b2a9268b319a6184)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Django detects a conflict between a migration added to support
image customisation and another migration which supports PROVIDES.
Add a merge migration to resolve the conflict (as suggested by
Django).
(Bitbake rev: a26bfd9d2490dc0fd90bf6d1690e63ac26001559)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a special case for when the recipe we have based a custom image
recipe on requires another recipe.
In this case we need to adjust the file location to be able to
require
the recipe when we're in the toaster-custom-images layer.
For example:
"require core-image-minimal.bb" is changed to:
"require recipes-core/images/core-image-minimal.bb"
(Bitbake rev: 26025e1ea49b3ebfcfd508d1608fa8c9e722ad03)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Poll the server for the project build progress value. This is something
that will need to be re-done once we have a proper API for this on the
server side.
[YOCTO 8328]
(Bitbake rev: ec467e43c39eadf02412b89db10c09ed78a5a9f5)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We need to know the location of the based_on recipe's layer on the
file system before we try and generate the custom image recipe. As
we read the recipe to make the custom version.
(Bitbake rev: e6a7cacbddd1df5bac0b79384199cf7264c5bbd5)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Make sure we send the current list of layers to the customise button to
be able to know whether it should be set as an add layer button or a
customise button on the New custom image page.
(Bitbake rev: 5ddb35c98b609d85f97d482b54cabe3a2812afe6)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add additional data to the setUp to be able to test all the tables for
Image Customisation. Also add the name of the table being tested to the
num of rows assertion.
(Bitbake rev: dfcbcf789cf3f0733ca26b0601fdf97ce4291674)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Rename the recipe_id to custrecipeid to avoid confusion about which type
of object we're going to be accessing. This means that in the unit tests
for tables we can pass a different kwargs for custom recipes vs normal
recipes.
(Bitbake rev: ae3301a1047b3efb4b340b50a10d5d585b7333da)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Update test for adding and removing a package from a CustomImageRecipe
so that it uses the CustomImagePackage and correct fields for the packages
included. Change the test for error condition to use an invalid package
id as ManyToMany remove() on package that isn't in the relationship does
not throw an error.
(Bitbake rev: daccb2978f833a9e7af270160331da3e9a158219)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>