Commit Graph

4769 Commits

Author SHA1 Message Date
Harald Welte fa0881442f make bitbake git fetcher use https:// instead of git:// protocol by default
especially since github git:// access is disables as of today, we should
default to https:// instead.
2022-01-11 12:29:08 +01:00
Richard Purdie 69d985c0c3 bitbake: fetch2: Change git fetcher not to destroy old references
It looks like we're about to see a lot of changes in branch names in repos. If
we have the prune option here, those old names are lost, the changes propagate
to our source mirrors and our old releases break.

We have the force option so any replaced references should be replaced, its only
orphaned branches which will now be preserved.

I believe this behaviour will cause us fewer problems given the changes that
look likely to happen.

(Bitbake rev: 08aafc53ba911337d2d3558654b06f294ae634e2)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-09-08 14:46:38 +01:00
Richard Purdie b0ede5088b bitbake: tests/fetch: Update gnome.org urls after upstream changes
(Bitbake rev: 98b8ea103390d5bc3583efb2e72dbac6529822a5)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-08-29 10:44:26 +01:00
Peter Kjellerstedt 604d82bc0e bitbake: fetch2/__init__: Disable pseudo in runfetchcmd()
If a fetcher, e.g., git, is run when pseudo is active it will think it
is running as root. If it in turn uses ssh (as git does), ssh too will
think it is running as root. This will cause it to try to read root's
ssh configuration from /root/.ssh which will fail. If ssh then needs to
ask for credentials it will hang indefinitely as there is nowhere for it
to ask the user for them (and even if there was it would not access the
correct private keys).

The solution to the above is to temporarily disable pseudo while
executing any fetcher commands. There should be no reason for them to be
executed under pseudo anyway so this should not be a problem.

RP Ammendum:

We finally did get more information about how to reproduce this problem,
something needs to trigger bb.fetch2.get_srcrev() in a pseudo context,
for example when AUTOREV is in use or the recipe doesn't have a defined
SRCREV. That SRC_URI needs to be using protocol=ssh. This would trigger
an ls-remote of the remote repo and if that happens under pseudo, the
wrong ssh credentials may be attempted which can hang.

[YOCTO #12464]

(Bitbake rev: 5d42dce5e612060f4181c14e1cfdec2388bb7adf)

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-02 11:56:15 +01:00
Tzu Hsiang Lin 6aefc80585 bitbake: fetch2: fix import error for Python 3.6.5
When running bitbake command with Python 3.6.5 always result in
import error causing by the change of distutils module.
This patch replaces the method to search executable in PATH by
"/usr/bin/env <command>".

(Bitbake rev: 86a4cc7248afd3e561dc70f3490f45549265470a)

Signed-off-by: Tzu Hsiang Lin <t9360341@ntut.org.tw>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-02 11:44:05 +01:00
Richard Purdie 518b1f3de5 bitbake: providers: Fix determinism issue
We saw builds where runtime providers were sometimes changing order and the
build result was therefore non-deterministic. For example it could show:

DEBUG: providers for lib32-initd-functions are: ['lib32-lsbinitscripts', 'lib32-initscripts']
or
DEBUG: providers for lib32-initd-functions are: ['lib32-initscripts', 'lib32-lsbinitscripts']

which could cause a test to pass or fail.

This change ensures we don't rely on the random order of dictonaries in
memory and act deterministically.

(Bitbake rev: eddab52a4e7c49bfcdc2c26004d0bc777a1e5c25)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-03-09 06:12:09 -08:00
David Reyna 2021e95bcd bitbake: toaster: allow dots in user path names
The dot '.' character should be allowed in the user paths for
local non-git layers, DL_DIR, and SSTATE_DIR.

[YOCTO #10650]

(Bitbake rev: ca9b9ffc250eb3ece5af3d64ff5febef69d555b0)

Signed-off-by: David Reyna <David.Reyna@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-11-05 22:41:27 +00:00
David Reyna ecb828a68f bitbake: toaster: reserve HEAD from imported layers
The HEAD reference in Toaster layers are reserved for the
"Local Yocto Project" layers, stored at the top directory.
Imported layers are not allowed to use this since they are
managed differently - for example the 'remotes' will collide.

Fix the add layer handler to not drop the data fields when it
is a git repo.

Explicitly inform the user when an internal Toaster error is
returned via AJAX, so that they know why clicking the layer add
button did not do anything.

[YOCTO #9924]

(Bitbake rev: 90fb89e5e3ada9e36ab0b2a5a0dbfd74f9be279f)

Signed-off-by: David Reyna <David.Reyna@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-11-05 22:41:27 +00:00
Ross Burton 9351217eb7 bitbake: tests/fetch: handle network failures gracefully
If there is a network failure the return value from latest_versionstring() is
('','') which later causes an exception when comparing versions.

Improve this by checking the return value and failing the test early.

[ YOCTO #12053 ]

(Bitbake rev: 4284d918dc7451f6a4d00be065369e86bbaec4d2)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-11-05 22:37:29 +00:00
Mikko Rapeli 4490625088 bitbake: bitbake: Fix return value checks from subprocess.call()'s
Python function subprocess.call() returns the return value of the
executed process. If return values are not checked, errors may
go unnoticed and bad things can happen.

Change all callers of subprocess.call() which do not check for
the return value to use subprocess.check_call() which raises
CalledProcessError if the subprocess returns with non-zero value.

https://docs.python.org/2/library/subprocess.html#using-the-subprocess-module

All users of the function were found with:

$ git grep "subprocess\.call" | \
  egrep -v 'if.*subprocess\.call|=\ +subprocess\.call|return.*subprocess\.call'

Tested similar patch on top of yocto jethro. Only compile tested
core-image-minimal on poky master branch.

(Bitbake rev: aac956a0950c05bf58af1885474cd89bb9c8fbd1)

Signed-off-by: Mikko Rapeli <mikko.rapeli@bmw.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-11-05 22:37:29 +00:00
Andre Rosa a475795953 bitbake: Replace deprecated git branch parameter "--set-upstream"
Since 2017-08-17 (git version 2.14.1.473.g3ec7d702a) using deprecated
git branch parameter "--set-upstream" causes a fetcher error. Replace
it by "--set-upstream-to".

https://git.kernel.org/pub/scm/git/git.git/commit/?id=52668846ea2d41ffbd87cda7cb8e492dea9f2c4d
says, it's deprecated since 2012-08-30 so hopefully all still supported
host distributions have new enough git to support "--set-upstream-to".

ERROR: PACKAGE do_unpack: Fetcher failure: ...;
git -c core.fsyncobjectfiles=0 branch --set-upstream master origin/master failed with exit code 128, output:
fatal: the '--set-upstream' option is no longer supported. Please use '--track' or '--set-upstream-to' instead.

ERROR: PACKAGE do_unpack: Function failed: base_do_unpack

(Bitbake rev: 698e36430761cd3cf81283d58ca5bb3426bd24dd)

Signed-off-by: Andre Rosa <andre.rosa@lge.com>
Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-11-05 22:37:29 +00:00
Richard Purdie b1f14ecddf bitbake: toaster/highlight.pack.js: Fix corrupted file
The newly added file in the last commit was corrupted, fix it.

(Bitbake rev: 64e4f407bc235b774acf55f959b699ad3a5c9fa8)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-10-10 11:07:11 +01:00
David Reyna 15eb02516b bitbake: toaster: Remove prettify
Remove "prettify.js" and "prettify.css" due to license issues with Apache2.
Replace with "highlight.pack.js" with its BSD3 License.

[YOCTO #12206]

(Bitbake rev: 8c86ec28abd406f6d62dee9f917ed1df47f7f301)

Signed-off-by: Jason Wessel <jason.wessel@windriver.com>
Signed-off-by: Brian Avery <brian.avery@intel.com>
Signed-off-by: David Reyna <david.reyna@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-10-10 09:42:04 +01:00
Paul Eggleton 30613467d8 bitbake: cooker: add BB_CMDLINE to enable access to UI command line with memres
In OpenEmbedded's buildhistory class we want access to the bitbake
command line that launched the build, and up to now we were simply using
sys.argv from within the event handler to get that. Unfortunately that
doesn't work in memory resident mode, since the event handler is
naturally executing within the server and thus will give you the command
that launched the bitbake server which is much less interesting. Add a
dynamic variable BB_CMDLINE to provide access to this, set from sys.argv
within the UI process in updateToServer().

(Note that BB_CMDLINE isn't currently passed through to the worker, so
this is only really readable from event handlers plus any explicit
getVariable calls - in theory an observe-only UI could read it for
example.)

Part of the fix for [YOCTO #11634].

(Bitbake rev: 85596c9af3bb6407159c6c8de229cbe275aa74ea)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-09-21 17:20:11 +01:00
Mark Hatle b5bef4ea31 bitbake: cooker.py: Fix layer priority processing
If you have a layer with a blank BBFILE_PATTERN the layer was ignored
when processing the list of layers with priorities.  This list is not
only used for processing recipes, but also by additional programs such
as bitbake-layers show-layers.

Without this change, a layer that provides configuration or classes
only does now show up in show-layers, which is used by the
yocto-compat-layer.py script.  This causes a failures in the compatibility
check.

(Bitbake rev: 1db8336cee8cd22aaaa67e3dde5b9218840ba4a1)

Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-09-21 16:51:07 +01:00
David Reyna 372f72f2ee bitbake: toaster: recipe links broken for default layers
The default layers are missing the recipe link definitions in
the fixture files, and because they are predefined they do not
get the updated information from the Layer Index.

[YOCTO #12006]

(Bitbake rev: b408dfae3685494ae34417e72c586b9eb0ddace9)

Signed-off-by: David Reyna <David.Reyna@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-09-21 16:51:07 +01:00
David Reyna 17b4f4e731 bitbake: toaster: edit column list not sorted
The list of columns in the many 'Edit Columns' pop-ups became unsorted
with the 'Toaster Table' implementation. These entries need to be
gathered and sorted in the column processing.

[YOCTO #12004]

(Bitbake rev: d06a8ff74be11e4315feac6768064355ec15611f)

Signed-off-by: David Reyna <David.Reyna@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-09-21 16:51:07 +01:00
Olaf Mandel 8ef614ee77 bitbake: toaster: set default pokydirname if no external layers
If no external layers are defined, pokydirname is not set. Rectify
this by taking the 'be.sourcedir' as the pokydirname.

[YOCTO #12015]

(Bitbake rev: 4bcf15abbd6c399b7094762394393d41e0f347e7)

Signed-off-by: Olaf Mandel <o.mandel@menlosystems.com>
Signed-off-by: David Reyna <David.Reyna@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-09-21 16:51:07 +01:00
Olaf Mandel 9566ec3c3a bitbake: toaster: debug message for lists layers missing separators
One of the debug messages during build contains a list of all layers
but without spaces or other separators between them. Use pformat
instead.

[YOCTO #12014]

(Bitbake rev: 00fb88c9c969ae9b38004adf236c057628b3227d)

Signed-off-by: Olaf Mandel <o.mandel@menlosystems.com>
Signed-off-by: David Reyna <David.Reyna@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-09-21 16:51:07 +01:00
Awais Belal f744d7c496 bitbake: toaster: Order column in Tasks selectable
The build page provides tables related to performance that
cover build time, CPU time and disk IO. The "Edit columns"
drop down does not allow selection of the Order column and
makes it hidden as well which is not accurate from user
interaction point of view.
This patch enables the hideable property for the Order column
so it is hidden by default but the user can enable it through
the drop down if need be.

[YOCTO #11040]

(Bitbake rev: a3063e3c7e3ae21bcebe23fcf0a4fda559252fb5)

Signed-off-by: Awais Belal <awais_belal@mentor.com>
Signed-off-by: David Reyna <David.Reyna@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-09-21 16:51:07 +01:00
David Reyna 4c246b822a bitbake: toaster: display error when the fstype select is empty
There must be at least one FSTYPE selected in the Toaster bitbake
variable editor page. When the user deselects all the "Save"
button gets disabled, but the error message is missing.

[YOCTO #8126]

(Bitbake rev: 407bb26eebe4ec8a68547fd98b0a15cb18895d9a)

Signed-off-by: David Reyna <David.Reyna@windriver.com>
Signed-off-by: David Reyna <David.Reyna@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-09-21 16:51:06 +01:00
Paul Eggleton 9034b2c57b bitbake: cooker: ensure monkey-patching in collect_bbfiles() gets undone on error
In collect_bbfiles() we're monkey-patching os.listdir in order to find
which directories to watch, and then undoing that when we're finished -
however if an exception occurred for any reason there was nothing to
ensure the latter occurred. This may not have caused any issues, but as
this kind of thing really ought to be secured using try...finally just
in case, so do that.

(Bitbake rev: 021e0eda63bd629af56b5e1c380c5f98868f7332)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-09-21 16:51:06 +01:00
Paul Eggleton 0704bbdaeb bitbake: cooker: fix watching empty directories
The code that was supposed to watch directories along BBFILES for
creation of new files wasn't working in the case where the directory did
not initially contain any matching files - since in updateCache() we are
passing the directory path to add_filewatch() and the latter function
calls os.path.dirname() on the path on the assumption that it is a file
path, and thus the parent of the directory got watched but not the
directory itself. (If the directory wasn't empty everything worked fine
since add_filewatch() was called elsewhere with the path to one of the
files in that directory, and thus the directory got watched). Add a
parameter to add_filewatch() to tell it we are passing it directory
path(s) rather than file path(s).

(Bitbake rev: b0b36ff889715227e394c59a36326c31ea21b96b)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-09-21 16:51:06 +01:00
Richard Purdie 200362645b bitbake: cooker: Track directories searched for bbappend/bb files
Some of the directories searched by BBFILES are not currently being added
to the inotify watch list. This can mean that added append files are not
noticed leading to misleading metadata results when using
BB_SERVER_TIMEOUT != 0.

We use glob to expand the BBFILES references and without writing our own
implentation, figuring out the directories it searches is hard. We use
some horrible hacks here to intecept the listdir calls, I'm open
to better ways to handle this but this does ensure we have the right
watches set.

(Bitbake rev: a161ea021a1e309c25d1adc09e6e3fc58442c893)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-09-21 16:51:06 +01:00
David Reyna 4b35d0a012 bitbake: toaster: get_last_build_id not called correctly
The method "get_last_build_id" missing the "()" in several calls. This
has been failing silently with Django 1.8 but intermittently crash with
Django 1.9.

[YOCTO #11570]

(Bitbake rev: 98e424c2cd3ce8bc7df247263e44a6d6d9ede3e6)

Signed-off-by: David Reyna <David.Reyna@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-07-27 22:36:46 +01:00
David Reyna b161cb0c35 bitbake: toaster: add getMessage to MockEvent
The MockEvent needs to not only stand in for Toaster and Bitbake quick events,
it also needs to stand in for LogRecord, and for that it needs to provide
the new getMessage method.

[YOCTO #11440]

(Bitbake rev: e95ed33726b2a6d03382e07c6efe2ab574f84ee8)

Signed-off-by: David Reyna <David.Reyna@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-07-27 22:36:46 +01:00
David Reyna 55f8db0246 bitbake: toaster: fail on layers with sub-layer
For layers directories with sub-layers (for example meta-intel),
Toaster breaks when trying to exact match a dependency event for
that sub-layer against the top level layer directory paths.

Given that top the level layer directory paths are unique, adding
a test to see if the dependency path is a subset of a top level
path is also unique.

Also, since a warning was issued the processing should not fail
on the assert but instead should continue and gather the recipes
and tasks.

[YOCTO #11149]

(Bitbake rev: e1c769eed2258bb2697e9558f484dd62a3da7399)

Signed-off-by: David Reyna <David.Reyna@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-07-27 22:36:46 +01:00
David Reyna f505d9fad9 bitbake: toaster: add ID's to build menu links
Add IDs to the build page menu lines, for example "Tasks", "Recipes",
"Time", "CPU Time", "Disk I/O", and so forth. This will support test
automation.

[YOCTO #11337]

(Bitbake rev: 0bd1fcdbfa6b9bf4642f1a2b1d46767c7c15a5d1)

Signed-off-by: David Reyna <David.Reyna@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-07-27 22:36:46 +01:00
David Reyna 1a8d911989 bitbake: toaster: add ID's to navigation links
Add IDs to the navigation bar links "All builds", "All projects",
and "Documentation. This will support test automation.

[YOCTO #11335]

(Bitbake rev: 4734001672cb35789be5a8d5c8b91ca57821bdba)

Signed-off-by: David Reyna <David.Reyna@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-07-27 22:36:45 +01:00
Richard Purdie c11f1ff584 bitbake: siggen: Make calc_taskhash match get_taskhash for file checksums
The code in these two functions is meant to be equivlanet in behaviour
but isn't. Add in code to ensure files that don't exist are handled
consistently by both functions. Users did report being able to generate
tracebacks otherwise.

(Bitbake rev: 1b66c57d0f8a9bd9f9feb2a85759e18d9a1d674b)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-06-27 13:46:50 +01:00
Joshua Lock 18341b9dc9 bitbake: fetch: fix handling of files with incorrect checksums from a premirror
Ensure that when an item fetched from a premirror has an invalid checksum the
fetcher falls back to the usual logic of trying the upstream and any configured
mirrors.

(Bitbake rev: cc52b9b12c60810142252b9cb5d4268e42371b8e)

Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-06-27 13:45:24 +01:00
David Reyna c519dc1cd4 bitbake: toaster: move release lookup from morty to pyro
Move the release name lookup into the layer index
from 'morty' to the 'pyro'.

Move the bitbake branch from 1.32 to 1.34.

[YOCTO #11377]

(Bitbake rev: 21d963149b5d97452420230a252101115b708d85)

Signed-off-by: David Reyna <David.Reyna@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-20 07:59:40 +01:00
Paul Eggleton d45f5be977 bitbake: lib/bb/command: fix overrides in remote datastores where there is no default
If you have, for example, no value set for a variable VARIABLE and a
then VARIABLE_pn-something = "value" and then you parse something.bb,
you expect getVar('VARIABLE') on the resulting datastore to return
"value", but the code here assumed that if the variable wasn't set
without overrides then we didn't need to return the overridedata and
thus we didn't get the overridden value.

In OE this affected the ability to get RECIPE_MAINTAINER for a recipe
in a script using tinfoil (since this is only set from an inc file with
_pn- overrides for each recipe, and no default is set).

(Bitbake rev: b3d2c9917c5fd8278878328794daa107ddf79b64)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-20 07:59:40 +01:00
Richard Purdie 1423508b29 bitbake: bitbake: Update version to 1.34.0 for stable release
(Bitbake rev: b4da94a368c6c44c6e5b6e6e9a1e041ed84b4554)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-19 10:42:12 +01:00
Juro Bystricky 00d49a05a6 bitbake: command.py: multiconfig support for findBestProvider
In a multiconfig environment, a tinfoil call such as

    tinfoil.parse_recipe("multiconfig:arduino-101-sss:gcc")

can fail with an error such as:

  File "/data/master/poky/bitbake/lib/bb/tinfoil.py", line 373, in get_recipe_file
    raise bb.providers.NoProvider('Unable to find any recipe file matching "%s"' % pn)
bb.providers.NoProvider: Unable to find any recipe file matching "multiconfig:arduino-101-sss:gcc"

The culprit is findBestProvider, which does not handle multiconfig.
This patch fixes the error and in the case mentioned above the tinfoil call returns:

  "multiconfig:arduino-101-sss:/data/master/poky/meta/recipes-devtools/gcc/gcc_6.3.bb"

[YOCTO#11210]

(Bitbake rev: e9c03fbfd7b057b28645affa263cb4aebfa24b04)

Signed-off-by: Juro Bystricky <juro.bystricky@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-19 10:20:24 +01:00
Nathan Rossi bf0fc2da66 bitbake: lib/bb/{data, siggen}: Don't warn on SkipRecipe exceptions
Recipes/variables that raise a SkipRecipe exception are intentionally
skipped, and should not generate warnings.

[YOCTO #11319]

(Bitbake rev: edf6e6094a9f7ad4b2ba06eef8fd34756edbedce)

Signed-off-by: Nathan Rossi <nathan@nathanrossi.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-12 15:16:05 +01:00
Richard Purdie 000b7d3d1c bitbake: data_smart: Ensure _remove operations on newly set variables are cleared
We clear append/prepend on newly set variables, we should also clear
remove operations. If we don't do this, there is no way we can actually
delete a remove operation. Bitbake internally uses parsing=True to avoid
these side effects when making its own internal calls.

Also add a testcase to bitbake-selftest to ensure we remain consistent going
forward from here.

(Bitbake rev: 3a319f079d699c870d8531e051ab65e6278d1fa5)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-12 15:09:58 +01:00
David Reyna b06f7cbb94 bitbake: toaster: fix SDK artifact capture
Use the TaskArtifacts event to scan the SDK and ESDK manifests
to cleanly collect the respective artifact files.

The previous method was broken when the SDK file deployment moved
from the do_populate_sdk[_ext] tasks to their sstate tasks. That
method is disabled (but not yet removed) in preparation for the
rest of refactor work for the parent #10283 work.

[YOCTO #10850]

(Bitbake rev: 1360d7b847cc01031edb2f4b289fac9560d72fa7)

Signed-off-by: David Reyna <David.Reyna@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-11 18:05:09 +01:00
Markus Lehtonen fc6370626e bitbake: cache: prevent division-by-zero in load_cahefile()
[YOCTO 11315]

(Bitbake rev: 227c5acd4b40154bc61202e7bb67a13818a7d727)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-11 18:05:09 +01:00
David Reyna 2a40a0587e bitbake: toaster: fix add layer button for Machines
Fix a missing close quote in the layer add button, and
use the layer index for the xhr_layer call.

[YOCTO #11213]

(Bitbake rev: 54d61e7c64c97799de85b3563119f0f5d051675e)

Signed-off-by: David Reyna <David.Reyna@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-11 18:05:09 +01:00
David Reyna d8be835e28 bitbake: toaster: Toaster filters are broken
A syntax fix inadvertantly broke the Toaster filter feature.

[YOCTO #11317]

(Bitbake rev: 29c9fa31d0f03553e24391568d2a5cb588595420)

Signed-off-by: David Reyna <David.Reyna@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-11 18:05:09 +01:00
Paul Eggleton b372e85d9b bitbake: bitbake-diffsigs: colourise output
If the output is a TTY, add colour to the output in order to make it
easier to read. At the moment this is fairly basic, just add colour to
the "titles" of each change and to the diff output.

I tried to introduce this without changing the code too much - rather
than moving everything over to the new python formatting style, I've
introduced a color_format() function which takes care of the colour
formatting, either accepting additional format arguments or
alternatively leaving the caller to use the old-style formatting (%) to
insert values.

(Bitbake rev: 04a023c8fdea1e1812fcdcaf00345aab59f9abe1)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-10 23:00:32 +01:00
Paul Eggleton 5f7bf1f66d bitbake: lib/bb/siggen: show word-diff for single-line values containing spaces
If a variable value has changed and either the new or old value contains
spaces, a word diff should be appropriate and may be a bit more readable.
Import the "simplediff" module and use it to show a word diff (in the
style of GNU wdiff and git diff --word-diff).

Also use a similar style diff to show changes in the runtaskhashes list.
I didn't use an actual word-diff here since it's a little different - we
can be sure that the list is a list and not simply a free-format string.

(Bitbake rev: 20db6b6553c80e18afc4f43dc2495435f7477822)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-10 23:00:32 +01:00
Paul Eggleton 5d8b89fc0b bitbake: lib/bb/siggen: add collapsed mode to compare_sigfiles()
If we just want to drill down to the actual differences then we don't
need to see certain things in the output, e.g. basehash changing or the
signature of dependent tasks. This will be used for comparing signatures
within buildhistory-diff in OE-Core; the default mode as used by
bitbake-diffsigs and bitbake -S printdiff remains unchanged for the
moment.

(Bitbake rev: 6543a59b1ebd3194a7c6421cffc66ebe31a67c62)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-10 23:00:32 +01:00
Paul Eggleton 8c015e45ff bitbake: lib/bb/siggen: don't show unchanged runtaskdeps list
If the runtaskdeps list hasn't actually changed (but the signatures of
some of the tasks did) then it doesn't make sense to print out the old
and new lists as they are both the same and may be very long, e.g. for
do_rootfs in OE.

(Bitbake rev: cb170543605288b3e8badfac3a54c588f4c95413)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-10 23:00:32 +01:00
Paul Eggleton 445e622b23 bitbake: lib/bb/siggen: show a diff when dumping changes to multi-line values
When dumping changes to signatures e.g. output of bitbake -s printdiff,
if for example a function has changed, it's much more readable to see a
unified diff of the changes rather than just printing the old function
followed by the new function, so use difflib to do that.

Note: I elected to keep to one item in the returned list per change,
rather than one line per line of output, so that the caller can still
look at changes individually if needed. Thus I've added some handling to
bitbake-diffsigs to split the change into lines so that each line is
displayed indented.

(Bitbake rev: 4d254ae63a35231c98e3f73f669b040ed1144042)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-10 23:00:32 +01:00
Paul Eggleton f3b010b39a bitbake: lib/bb/siggen: add missing path separator to cleaned paths
Printing "pbzip2pbzip2_1.1.13.bb" is ugly, we need to add a separating
slash so that we get "pbzip2/pbzip2_1.1.13.bb" instead.

(Bitbake rev: 55cd4045a37afc954f4d5091f524756b266064fc)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-10 23:00:32 +01:00
David Reyna 123ec6ebc2 bitbake: toaster:test:Create selenium tests for project dashboard page
Added 7 new testcases that verify the UI interface and elements of the project detail page.
This testcases can be found on testopia in the links:

Verifies that the project is created and that you get redirected to the configuration page
https://bugzilla.yoctoproject.org/tr_show_case.cgi?case_id=1514

Verifies that the left side bar menu, all links are clickable and they show on the UI
https://bugzilla.yoctoproject.org/tr_show_case.cgi?case_id=1515

Verifies that after creating a project the default project configuration is created
https://bugzilla.yoctoproject.org/tr_show_case.cgi?case_id=1516

Verifies that the default machine is set, once creating the project
https://bugzilla.yoctoproject.org/tr_show_case.cgi?case_id=1517

Verifies the built recipes information of the project detail page
https://bugzilla.yoctoproject.org/tr_show_case.cgi?case_id=1518

Verifies the default release information of the project
https://bugzilla.yoctoproject.org/tr_show_case.cgi?case_id=1519

Verifies that the default layers are assigned to the project
https://bugzilla.yoctoproject.org/tr_show_case.cgi?case_id=1520

Verifies that the links to the Configuration, Builds, Import layer and New Custom Image are present and work.

[YOCTO #9808]

(Bitbake rev: eaeddaf96efb8079b307652eac208f4ab5019ad4)

Signed-off-by: Libertad Cruz <libertad.cruz@intel.com>
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: David Reyna <David.Reyna@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-10 23:00:32 +01:00
Belal, Awais f45a5a5de8 bitbake: toaster: toastergui: implement machine name validation
Valid machine names cannot include spaces anywhere
in the name and doing so will result in a build failure.
This implements a mechanism to alert the user against
such a misconfiguration and does not allow input of
such machine names.

[YOCTO #8721]

(Bitbake rev: 6fb642935a3787659aa316ca906025d2d87964cf)

Signed-off-by: Awais Belal <awais_belal@mentor.com>
Signed-off-by: David Reyna <David.Reyna@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-10 23:00:32 +01:00
Richard Purdie eb7430e5b9 bitbake: Bump version to 1.33.4
(Bitbake rev: fe5d8a0fdaec99721455dead626ecfb72bdeb616)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-10 23:00:32 +01:00