Commit Graph

1095 Commits

Author SHA1 Message Date
Enrico Scholz 448d0be779 wic: accept '-' in bitbake variables
'-' is valid and common in bitbake variables (e.g. 'FOO_pn-bar'). Accept
it and other characters when reading the .env file.

Also, allow variables to be empty.

(From OE-Core rev: e688ac8e92d2bc451d8b2d437596f630bedccd2c)

(From OE-Core rev: 2a69250abf61e51f633033ddb672e8f459191899)

(From OE-Core rev: aa367a3583f6e1fb0e55f7ec46ddad13291f1c6f)

Signed-off-by: Enrico Scholz <enrico.scholz@ensc.de>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-11-05 22:39:46 +00:00
Stanley Cheong Kwan, Phoong 0ffe27ffef recipetool: git reformat URI mangling & parameter stripped
recipetool seems to be mangling and stripping out the parameters for git
URI. This will fix this issue as well as resolve the conflict of
protocol parameter added by user. If a user adds their own protocol as
an argument, it'll be honored.

[YOCTO #11390]
[YOCTO #11391]

(From OE-Core rev: e3c832e49a9596537198a46075ed3d6794639953)

Signed-off-by: Stanley Cheong Kwan, Phoong <stanley.cheong.kwan.phoong@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
(cherry picked from commit 0cd2fc8ca278ebaa76de95545eef26a07b350c8e)
Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-07-27 22:36:45 +01:00
Patrick Ohly bc3fa28e93 yocto-compat-layer.py: make signature check code reusable
This moves the main content of test_signature into a helper
function. It can be reused by arbitrary tests that need to do
a before/after signature comparison. Long-term this might even
be useful in oeqa itself.

(From OE-Core rev: 23939d0acb563e27a1419d3f532a47c3d6a59a3e)

Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-07-27 22:36:44 +01:00
Patrick Ohly 026586965a yocto-compat-layer.py: allow README with suffix
It may be useful to append a suffix denoting the file format. For
example, README.rst is rendered differently when viewed on Github, and
also helps editors to switch to a mode more suitable for the format.

The tests uses a file pattern to find the README file(s) and treats
the one with the shortest name as the main one which must not be
empty.

(From OE-Core rev: 0abc29f135d7f870168d9676395f3a4bb77174b8)

Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-07-27 22:36:44 +01:00
Patrick Ohly 8f20c03dd6 yocto-compat-layer.py: add test_world
"test_signatures" ignores wold build breakage for the sake of
reporting differences also when a world build is broken. Therefore we
need a dedicated test that a world build at least theoretically can
proceed without obvious parse time problems (dependencies, parse
errors, dangling .bbappends, etc.).

This is similar to the BSP test_machine_world. The difference is
that test_world doesn't change the MACHINE.

(From OE-Core rev: da18bea6808aaddf3fa3fe72ac7e2d87d7e78b95)

Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-07-27 22:36:44 +01:00
Patrick Ohly 14d5932c20 yocto-compat-layer.py: apply test_signatures to all layers
Software layers were previously allowed to change signatures, but
that's not desired for those layers either. The rule that a layer
which is "Yocto Compatible 2.0" must not change signatures unless
explicitly requested holds for all kinds of layers.

However, as this is something that software layers might not be able
to do right away, testing for signature changes in software layers can
be disabled. It's on by default, as that was Richard's
recommendation. Whether that should change needs further discussion as
part of finalizing "Yocto Compatible 2.0".

As it might still change, the tool now has both a with/without
parameter so that users of the tool can choose the desired behavior
without being affected by future changes to the default.

(From OE-Core rev: e4dce65ce604a74da0f09ee2742cf8b13cf96c8e)

Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-07-27 22:36:44 +01:00
Patrick Ohly 6d7302b53c yocto-compat-layer.py: tolerate broken world builds during signature diff
The "test_signatures" test ignored a broken world build when getting
signatures, but the code which then tried to analyze a difference
found by the test didn't, which prevented printing the difference.

(From OE-Core rev: f2190e7e81f86735510c6ab10d3ac781146113f9)

Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-07-27 22:36:44 +01:00
Patrick Ohly 7546daec4f yocto-compat-layer.py: avoid adding layers more than once
add_layer_dependencies() might get called more than once, or one of
the layer dependencies might already be present. The function should
not add layers again because doing so can cause warnings like:

  WARNING: Duplicate inclusion for .../meta-openembedded/meta-oe/conf/distro/include/meta_oe_security_flags.inc in .../meta-openembedded/meta-oe/conf/layer.conf

(From OE-Core rev: 9821cec1ca52deee444ae3ff14dc548c8312ba3c)

Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-07-27 22:36:44 +01:00
Paul Eggleton 645467e03f scriptutils: fix fetch_uri() to work with RSS
Since recipe-specific sysroots were implemented, devtool add and devtool
upgrade operations that fetch from a URL that requires native sysroot
dependencies will fail to work as there is no recipe-specific sysroot
set up for them during fetching. An example was any URL pointing to a
tarball compressed with xz, e.g. devtool upgrade on gnutls.

The most expedient way to fix this is to set up a dummy recipe-specific
sysroot to use for the fetch/unpack operations. We do this in the same
manner as bitbake -b does, so we're just taking all of the sysroot
components available and creating a sysroot from those rather than
ensuring the correct dependencies are there - this means that we're
still going to have problems if e.g. xz-native hasn't been built yet,
but that issue will be trickier to solve and is tracked separately.

Fixes [YOCTO #11474].

(From OE-Core master rev: 559151e783759af78b5cdd76cdbb9ce325a391e6)

(From OE-Core rev: 9e7905c7bada1bafda661fc4a6177afeef0f5015)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-05-30 00:26:17 +01:00
Kristian Amlie e3e78df1b2 wic: Fix typo in help screen.
This was overlooked when f6a064d969 was merged.

(From OE-Core rev: 29a209822488ab687abdb1ceffdd9c7af5b3db68)

Signed-off-by: Kristian Amlie <kristian.amlie@mender.io>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-29 11:17:22 +01:00
Paul Eggleton bce0b509bf devtool: extract: drop erroneous bb.event.TaskStarted
This is a non-existent event - we already have the actual
bb.build.TaskSucceeded further down in the list hence why it wasn't
noticed earlier.

(From OE-Core rev: 4e059a5ceb6f44401154e89e37f56de1d664a7cb)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-14 09:47:08 +01:00
Paul Eggleton 02c39d3fad devtool: extract: fix handling of failed tasks
If a task such as do_fetch fails when we're extracting source for a
recipe (within devtool modify / upgrade / extract / sync) then we should
naturally stop processing instead of blundering on; in order to do that
we need to be listening for the TaskFailed event. Thanks to Richard
Purdie for noticing and fixing this.

(From OE-Core rev: 9174b845bf6a6be7753bf6b921959b1f3f2dcbc0)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-14 09:47:08 +01:00
Paul Eggleton 2338032d0b devtool: modify: add --keep-temp option for debugging
Most of the other extract-based commands have this option but oddly I
left it out for modify - I guess because if I was debugging an issue here
I just used devtool extract to do so, but there's no reason why we can't
have it here and it is useful.

(From OE-Core rev: 98fbc46e1a51237213bd7825a922389d3ab2ad9b)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-14 09:47:08 +01:00
Patrick Ohly 68079d0f87 yocto-compat-layer: better handling of per-machine world build breakage
It is fairly common that BSP layers enable recipes when choosing
machines from that layer without checking whether the recipe actually
builds in the current distro. That breaks "bitbake world", retrieving
signatures and thus the test_machine_signatures test.

It's better to let that test continue with the signatures that can be
retrieved and report the broken world build separately. Right now, the
new test_machine_world iterates over all machines. More elegant and
useful in combination with a (currently missing) selection of which
tests to run would be to generate one test instance per machine. But that
is not straightforward and has to wait.

The "-k" argument alone was not enough to proceed despite failures,
because bitbake then still returns a non-zero exit code. The existance
of the output file is taken as sign that the bitbake execution managed
was not fatally broken.

(From OE-Core rev: 02f5d7836b726e40fef82b50b8145acc839b360b)

Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-13 10:54:10 +01:00
Patrick Ohly de76d1cc94 yocto-compat-layer: test signature differences when setting MACHINE
Selecting a machine is only allowed to affect the signature of tasks
that are specific to that machine. In other words, when MACHINE=A and
MACHINE=B share a recipe foo and the output of foo, then both machine
configurations must build foo in exactly the same way. Otherwise it is
not possible to use both machines in the same distribution.

This criteria can only be tested by testing different machines in combination,
i.e. one main layer, potentially several additional BSP layers and an explicit
choice of machines:
yocto-compat-layer --additional-layers .../meta-intel --machines intel-corei7-64 imx6slevk -- .../meta-freescale

To simplify the analysis and limit the amount of output, mismatches
are sorted by task order such that tasks that run first are also
reported first. Following tasks for the same recipe and set of
machines then get pruned, because they are likely to be different
because of the underlying task (same approach as in
test_signatures). The difference here is that we get information about
all machines. The task order in the base configuration serves as
heuristic for sorting that merged list.

The test has already found issues in go-cross (depended on
tune-specific libgcc) and gdb-cross (had a tune-specific path
unnecessarily), so it is also useful to uncover issues that are not
caused by the BSP layer itself.

(From OE-Core rev: cb0d3de4540e412cfcb7804b4b1689141c80e3a1)

Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-13 10:54:10 +01:00
Paul Eggleton 40d1771944 recipetool: create: hide missing npm error when called from devtool
If devtool is called with a URL to a source repository containing a
node.js module, we don't know that until recipetool has fetched it, and
due to the structure of the code we have to exit with a special code in
order to let devtool know it needs to build nodejs-native. We also want
to suppress the error message that recipetool would normally print under
these circumstances; there is already a mechanism for this but it wasn't
operative in the case where we're pointed to a source repository rather
than an npm:// URL, so create some plumbing so that we know to hide the
message.

(From OE-Core rev: 0c2d0fbb1c6c5b82183799eb7ef80074f86bcfc4)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-13 10:54:10 +01:00
Paul Eggleton 48f8d1201a devtool: add: prevent repeatedly running recipetool
If recipetool returns with exit code 14 this means devtool needs to
build nodejs-native and then call it again. If recipetool returns exit
code 14 again then clearly something has gone wrong and we should just
quit with an error.

(From OE-Core rev: 8d7cced6e06d7c2037f5ab75ac859f501129532e)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-13 10:54:10 +01:00
Paul Eggleton bb8f141d0c devtool: add: fix node.js/npm handling with recipe specific sysroots
The change over to recipe specific sysroots means that we can no longer
get a known location simply from configuration for the npm binary - we
need to get the recipe sysroot for nodejs-native, look there for npm if
we need to check it's present, and add that to PATH when calling out to
npm. Unfortunately this means anywhere we need to get that path we have
to have parsed all recipes, otherwise we have no reliable way of
resolving nodejs-native. Thus we have to change recipetool create to
always parse all recipes (the structure of the code does not allow us to
do this conditionally).

In the worst case, if npm hasn't already been added to its own sysroot
and we are fetching from a source repository rather than an npm
registry, this gets a bit ugly because we end up parsing recipes three
times:
1) recipetool startup, which then fetches the code and determines it's
   a node.js module, finds that npm isn't available and then exits with
   a specific error to tell devtool it needs to build npm
2) when we invoke bitbake -c addto_recipe_sysroot nodejs-native
3) when we re-invoke recipetool

This code is badly in need of refactoring, but now is unfortunately not
the time to do that, so we're going to have to live with this ugliness
for now.

Fixes [YOCTO #10992].

(From OE-Core rev: acfdbd796c99882b8586023c8c6b848716105c8d)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-13 10:54:10 +01:00
Paul Eggleton 0440cbccda recipetool: create: fix for regression in npm license handling
OE-Core commit c0cfd9b1d54b05ad048f444d6fe248aa0500159e added handling
for AND / OR in license strings coming from npm, but made the assumption
that an & would always be present in the license value. Check if it's
there first so we don't fail if it isn't.

(From OE-Core rev: abe2955df2dc558de6068d9373dfcb47d690704b)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-13 10:54:09 +01:00
Patrick Ohly e93a2ab3e3 yocto-compat-layer: also determine tune flags for each task
locked-sigs.inc groups tasks according to their tune flags (allarch,
i586, etc.). Also retrieve that information while getting signatures,
it will be needed to determine when setting a machine changes tasks
that aren't machine-specific.

(From OE-Core rev: 67f9a8759f47680dbf349797801b2a1e8d149377)

Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-12 15:09:58 +01:00
Patrick Ohly 937c1ea974 yocto-compat-layer: include bitbake-diffsigs output
After filtering out potential false positives, it becomes feasible to
include the output of bitbake-diffsigs for those tasks which
definitely have a change.

Depends on bitbake-diffsigs with the "--signature" parameter.

Enhanced output now is:

   AssertionError: False is not true : Layer meta-xxxx changed 120 signatures, initial differences (first hash without, second with layer):
      gstreamer1.0-plugins-base:do_fetch: 76973f19f2e30d282152bdd7e4efe5bb -> e6e7c6fa9f2bd59d7d8d107f7c6ca1ac
         Task dependencies changed from:
         ['PV', 'SRCREV', 'SRC_URI', 'SRC_URI[md5sum]', 'SRC_URI[sha256sum]', 'base_do_fetch']
         to:
         ['GST_IMX_PATCHES_TO_APPEND', 'PV', 'SRCREV', 'SRC_URI', 'SRC_URI[md5sum]', 'SRC_URI[sha256sum]', 'base_do_fetch']
         basehash changed from d679d30bd1ea41c56e57419b57587f3c to 090a79b45f5fa26d10f9d34e2ed7a1e6
            List of dependencies for variable SRC_URI changed from '{'PV', 'SRC_URI[md5sum]', 'SRC_URI[sha256sum]'}' to '{'GST_IMX_PATCHES_TO_APPEND', 'PV', 'SRC_URI[md5sum]', 'SRC_URI[sha256sum]'}'
         changed items: {'GST_IMX_PATCHES_TO_APPEND'}
         Dependency on variable GST_IMX_PATCHES_TO_APPEND was added
         Variable SRC_URI value changed:
         "     http://gstreamer.freedesktop.org/src/gst-plugins-base/gst-plugins-base-${PV}.tar.xz     file://get-caps-from-src-pad-when-query-caps.patch     file://0003-ssaparse-enhance-SSA-text-lines-parsing.patch     file://0004-subparse-set-need_segment-after-sink-pad-received-GS.patch     file://encodebin-Need-more-buffers-in-output-queue-for-bett.patch     file://make-gio_unix_2_0-dependency-configurable.patch     file://0001-introspection.m4-prefix-pkgconfig-paths-with-PKG_CON.patch     file://0001-Makefile.am-don-t-hardcode-libtool-name-when-running.patch     file://0002-Makefile.am-prefix-calls-to-pkg-config-with-PKG_CONF.patch     file://0003-riff-add-missing-include-directories-when-calling-in.patch     file://0004-rtsp-drop-incorrect-reference-to-gstreamer-sdp-in-Ma.patch [--] {+${GST_IMX_PATCHES_TO_APPEND}+}"

      pulseaudio:do_install: 6bb6fe23e11a6d5fef9c3a25e73e4f9c -> 3f54ea75673a792e307197cfa6ef2694
         basehash changed from ac4efcfa783bd04a5a98a2c38719aedd to 37679d99623a37c8df955da3a01415a5
         Variable do_install value changed:
         @@ -1,3 +1,7 @@
              autotools_do_install
           	install -d ${D}${sysconfdir}/default/volatiles
          	install -m 0644 ${WORKDIR}/volatiles.04_pulse  ${D}${sysconfdir}/default/volatiles/volatiles.04_pulse
         +    if [ -e "${WORKDIR}/daemon.conf" ] && [ -e "${WORKDIR}/default.pa" ]; then
         +        install -m 0644 ${WORKDIR}/daemon.conf ${D}${sysconfdir}/pulse/daemon.conf
         +        install -m 0644 ${WORKDIR}/default.pa ${D}${sysconfdir}/pulse/default.pa
         +    fi

[YOCTO #11161]

(From OE-Core rev: 312edd42b6cc553de4d476c76e8e36a882e11cdd)

Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-12 15:09:58 +01:00
Ed Bartosh 8838dd2dbd oe-selftest: test wic sparse_copy API
Added new parameter 'api' to sparse_copy function to specify
underlying filemap API to use. By default sparse_copy will
try both available APIs.

Added test case for sparse_copy to wic test suite.

(From OE-Core rev: 88701cef6ba399e82f96ed1b0eef9a44ed8c1687)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-12 15:09:57 +01:00
Robert Yang 7792688ac2 qemux86-directdisk.wks: vda -> sda
Previously, runqemu grep root=/dev/sd or root=/dev/hd on the image, and
would use vda if no grep result, now we have set QB_DRIVE_TYPE to
"/dev/sd" by default, and the device will be /dev/sda, so use sda to
replace vda in the test case.

(From OE-Core rev: 3f1f42984021d87ac43aaa16f38b706c2c965e02)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-12 15:09:57 +01:00
Patrick Ohly 7d4fa58c5c yocto-compat-layer: limit report of signature changes
Typically a single change cascades through the entire task dependency
chain. Developers had to figure that out themselves, based on hard to
read and interpret output (not sorted, no indention, no explanations):

   $ yocto-compat-layer.py -n meta-xxxx
   ...
   AssertionError: True is not false : Layer meta-xxxx changed signatures.
   webkitgtk:do_install changed fe2edc9082bc0da98f9cb1391c52f565 -> b3a44684c5cd9aacd3f7c6ed88eefab5
   gstreamer1.0-plugins-good:do_configure changed 3b2f8211be3fe08422bf6087f3af16d1 -> 7d80e42fa1f4f01ff4dfe2ea4477d382
   pulseaudio:do_package_qa changed 5d0a58ada66ff17f5576555302ac319a -> 0e13bcb96143d1ae54c451bc3de0aa30
   epiphany:do_prepare_recipe_sysroot changed 29e1b277dbcb005bd54950594c50d91b -> d3c45527b37677a0668ce483c6db3052
   ...
   gst-player:do_packagedata changed 9ce6efdd357dd74919bc4957458b1e95 -> d0c083ce629f37adfc9c4ba9eff81f83
   gstreamer1.0-plugins-base:do_install changed 1161cd867d15bea63e5dd5d9abf0519c -> 5bf2b652a2d77fee3eedb35af2f201a0
   gstreamer1.0-rtsp-server:do_packagedata changed 6781dc3070f80b843ed1970d74dd323e -> 454620c2e3b9fea87e525d14b6ed0344
   alsa-plugins:do_packagedata changed 1808c3f737cb805b169d004e948ea19c -> 480124b7fa5eab1f73bf96440d725231

Now the tool automates the problem analysis: it retrieves the depgraph
using the tinfoil API and only reports those tasks with modified
signatures whose dependencies have not changed, i.e. those tasks which
definitely introduce a change.

>From the previous example, that just leaves two tasks that need to be
checked:

   AssertionError: False is not true : Layer meta-xxxx changed 120 signatures, initial differences (first hash without, second with layer):
      gstreamer1.0-plugins-base:do_fetch: 76973f19f2e30d282152bdd7e4efe5bb -> e6e7c6fa9f2bd59d7d8d107f7c6ca1ac
      pulseaudio:do_install: 668eb1e30af129df9806b0aa0d7c10cd -> 1196bdb88eef56eeee4613bb06b9387e

This pruning might be a bit too aggressive in the sense that tasks
which inherit a change and then add more changes themselves won't be
reported initially. They will be found when fixing the reported tasks
and re-running the check.

For a developer it seems better to have something listed which
definitely is a problem and needs fixing instead of everything,
including the tasks which don't need fixes.

(From OE-Core rev: 7ab0e09de75bfd7e7498bfa72d1f2f5d02a96747)

Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-10 23:00:42 +01:00
Patrick Ohly efd3b0ee30 yocto-compat-layer: fix also other command invocations
In commit 5b9ac62ab535d, one place was fixed where a command was
invoked such that failures caused double stack traces and stderr was
lost. The same problem also occurs elsewhere, triggered for example by
a layer with parsing problems.

Now a new utility method is used instead of repeating the code.

(From OE-Core rev: b6c72c0d169473e2626938be2ee59f850624612e)

Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-10 23:00:42 +01:00
Chen Qi 9cacf8488c wic: improve error message
When using `wic create mkefidisk -e core-image-minimal', the following
error message appeared.

  Please bake it with 'bitbake parted-native' and try again.

However, following this command doesn't do any help. The same problem
still appeared.

The problem is that when we 'bitbake parted-native', it doesn't have
anything to do with core-image-minimal. And the required tool 'parted'
is not under core-image-minimal's recipe-sysroot-native directory.

Improve the error message so that following it could get things done.

(From OE-Core rev: f0425c0a0f1c98f65bf61fd9aa7e023ed41a35fa)

Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-05 23:22:13 +01:00
Ed Bartosh b5bc885ba7 Revert "filemap: remove FilemapSeek class"
FIEMAP API is not supported by tmpfs file system, but
SEEK_HOLE/SEEK_DATA is supported.

Returned back FilemapSeek class that implements support
of SEEK_HOLE/SEEK_DATA API to make sparse_copy API working
on tmpfs again.

This reverts commit 6b80c13f7a.

(From OE-Core rev: e75bd6a7dd5c1b4bad039c35cf4a2ffc2f77c60a)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-05 23:22:13 +01:00
Markus Lehtonen df20260016 scripts/oe-build-perf-report: fix 'charts ready' console message
The javascript console log messages are used in scraping, when
converting an html test report to html email. Before this patch a
console message indicating that all charts have been drawn was not
correctly sent if the last test failed (or didn't have chart data for
some other reason) which, in turn, caused oe-build-perf-report-email.py
script to fail with a timeout.

(From OE-Core rev: 79b90ae02257002ea831a48f6798794b7711c1f8)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-05 23:22:13 +01:00
Mark Hatle 6afe0e07ec yocto-compat-layer.py: Fix the signature validation
The initial signatures need to be collected -after- the dependency layers have
been added to the system.  Otherwise changes that happen due to dependencies,
outside of the layer being scanned, will show up as signature problems.

The add_layer function was split into two pieces so that we can process
the dependencies first, and then add the layer itself for the comparison.

(From OE-Core rev: 4eb0932e755b7cb582a8db811aeed1397ecb92cc)

Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-05 23:22:13 +01:00
Mark Hatle d2d019a11b compatlayer/__init__.py: Allow add_layer to process recursive deps
When processing a layer for dependencies, you have to process the layer
itself, it's dependencies, the dependencies dependencies and so forth until
all items have been processed.

i.e.:  LayerA requires LayerB requires LayerC requires layerD

The end result should be LayerB, LayerC and LayerD are all dependencies of
LayerA.

(From OE-Core rev: 6e0a268b750fb6701604dd936cd2cf3b47a6e804)

Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-05 23:22:13 +01:00
brian avery 0ffc7b8d2e devtool: point runqemu to correct native bindir
devtool/runqemu.py was relying on STAGING_BINDIR_NATIVE to find the host
tools it needed like qemu-system-<arch>.  In the post RSS world, this no
longer exists. This patch points it to
{STAGING_DIR}/{BUILD_ARCH}/{bindir_native}.

[YOCTO #11223]

(From OE-Core rev: 1910f9e9336bfedc8278a3bc02e7e7f934a4fc86)

Signed-off-by: brian avery <brian.avery@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-05 23:22:12 +01:00
Markus Lehtonen 034702f520 scripts/contrib: add oe-build-perf-report-email
Script for sending build perf test reports as an email. Mangles an html
report, generated by oe-build-perf-report, into a format suitable for
html emails. Supports multipart emails where a plaintext alternative can
be included in the same email.

Dependencies required to be installed on the host:
- phantomjs
- optipng

[YOCTO #10931]

(From OE-Core rev: 9e97ff174458f7245fc27a4c407f21a9d2e317ab)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-01 23:28:20 +01:00
Markus Lehtonen 9f299876f7 scripts: add oe-build-perf-report script
A new tool for pretty-printing build perf test results stored in a Git
repository. The scripts is able to produce either simple plaintext
report showing the difference between two commits, or, an html report
that also displays trendcharts of the test results. The script uses
Jinja2 templates for generating HTML reports so it requires
python3-jinja2 to be installed on the system.

[YOCTO #10931]

(From OE-Core rev: 3b25404f0f99b72f222bdca815929be1cf1cee35)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-01 23:28:20 +01:00
Saul Wold 19e99786be yocto-bsp/i386 machine.cfg: Explicitly disable 64BIT
Since we do not set the 64 bit flags, newer kernels seem to build 64bit
config files by default. This is due to a hard-coded uname -m check that
selects the KBUILD_DEFCONFIG based on the host, not the cross target.

Similar to e9ec769926b2378e63380bd7762ce7ce201af151 in the yocto-kernel-cache repo

(From meta-yocto rev: e35017cc67f6d3c5cc00488d3460de0dcec773b3)

Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-01 08:15:32 +01:00
Daniel Schultz 28997864e2 wic: partition: Update fsck parameters
These parameters are copied from the ext image class.

-D will let fsck perform further directory optimizations
-v might be helpful for debugging purpose

(From OE-Core rev: be93e378506a85772503005294503cfc348a552c)

Signed-off-by: Daniel Schultz <d.schultz@phytec.de>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-01 08:14:58 +01:00
Ed Bartosh 67ec097bc3 wic: remove prepare_empty_partition_squashfs
There is not much sense in creation of empty squashfs
partition. It's also not possible to create empty squashfs
partition of specified size.

Even more, prepare_empty_partition_squashfs method is
absolutely broken. It raises exception when called and
even its signature differs from the rest of of similar
methods. It means that nobody uses it and it's safe
to remove it.

Removed prepare_empty_partition_squashfs method and
testing of empty squashfs partition.

(From OE-Core rev: 9152960f250cb4df1e559d747fb09005675a0d75)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-01 08:14:58 +01:00
Ed Bartosh d62a97ec54 wic: fix list of supported fstypes in help content
Added vfat and msdos to the list of supported fstypes in
'wic help kickstart' output.

[YOCTO #11137]

(From OE-Core rev: ab7f2a77e124d8859002619e7ba3117e8a165df7)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-01 08:14:58 +01:00
Ed Bartosh 1586663fb3 wic: set correct system id for msdos partitions
Explicitly set system id 0x6(FAT16) for msdos partitions.

Removed old code that attempts to achieve the same result
using 'parted ... lba off'.

(From OE-Core rev: 230452faf151e277bfb2b49526923f8097755b35)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-01 08:14:58 +01:00
Ed Bartosh e5959eb480 wic: set FAT 16 for msdos partitions
Used '-F 16' parameter for mkdosfs to create FAT16 partitions for
'msdos' partition type.

[YOCTO #11137]

(From OE-Core rev: b6243a03ced9a719a5801afcee014b03313cc43c)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-01 08:14:58 +01:00
Ed Bartosh 88e1d55de2 wic: support 'msdos' fstype
Added prepare_empty_partition_msdos and prepare_rootfs_msdos
methods to support 'msdos' filesystem type.

Created aliases prepare_empty_partition_vfat and prepare_rootfs_vfat
to continue supporting creation of vfat patitiions.

(From OE-Core rev: f06c507078da72f616f45effe5005cc01615a17c)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-01 08:14:57 +01:00
Ed Bartosh b4d15e0713 wic: don't silently skip unknown fstypes
Fixed wic code that loops through hard-coded list of known fstypes
to find prepare_rootfs_<fstype> or prepare_empty_partition_<fstype>
methods and silently skipping unknown fstypes.

(From OE-Core rev: ebb8fb5f81f473156c9aa4bf1965e538492a851b)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-01 08:14:57 +01:00
Ed Bartosh 54cd064c66 wic: allow only supported fstypes
Restricted possible values of --fstype to the list of
supported types. This should catch incorrect values
when .wks file is being parsed.

Removed checks for empty fstype and mentioning of
unsupported fstype 'ontrackdm6aux3'.

(From OE-Core rev: 21af89a6d44ccea6aef975ffd2483a8fad1231de)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-01 08:14:57 +01:00
Ed Bartosh ad1bce56c4 wic: remove unused code from runner module
Removed unused APIs 'outs' and 'quiet'.
Removed 'catch' parameter from runner.runtool API as wic
uses only one value of it. Removed the code that handles
unused values of 'catch' parameter.

[YOCTO #10618]

(From OE-Core rev: 1e45a4f72b16c7ab64f46907d2d2ee9cd749dc23)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-01 08:14:57 +01:00
Ed Bartosh d39a158855 wic: remove runner.show API
Replaced runner.show call to exec_cmd call in bootimg-pcbios
plugin. Removed runner.show API as it's not used anywhere else.

[YOCTO #10618]

(From OE-Core rev: 9749336c37249af99c92478c3e4dc8821cb9a816)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-01 08:14:57 +01:00
Ed Bartosh 2122dd7718 wic: use wic-tools STAGING_DATADIR as bootimg_dir
If bootloader artifacts are not found in default bootimg_dir
use wic-tools sysroot for the same purpose. This should
prevent wic from failing if bootloader artifacts can't be
found in image native sysroot.

(From OE-Core rev: 9674bbd0585fc25ccd362f233b83d07ff8f6ff53)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-01 08:14:57 +01:00
Ed Bartosh 3a53361084 wic: remove fsimage plugin
Removed fsimage plugin and prepare_rootfs_from_fs_image API as
they duplicate functionality of rawcopy plugin. Fsimage plugin makes
wic to remove the image artifact from deployment directory, which
can cause problems too.

[YOCTO #10618]

(From OE-Core rev: 9a470752f5698f791f8f78e28d163a0b9c695186)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-01 08:14:57 +01:00
Ed Bartosh 6b80c13f7a filemap: remove FilemapSeek class
FIEMAP API was added to Linux kernel 2.6.28 back in 2008
SEEK_HOLE and SEEK_DATA API was added much letter.
As FIEMAP is used by filemap module as a default API it's
safe to remove FileMpSeek class as it's never used.

[YOCTO #10618]

(From OE-Core rev: 44e9406ea6e3263d2fb95e9d534a21f74f318480)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-01 08:14:57 +01:00
Aníbal Limón 0dae983296 scripts/lib/compatlayer: detect_layers always use realpath's
If you are using relative paths and change to other folder for
execution it will fail, so use realpaths always.

[YOCTO #11164]

(From OE-Core rev: 14283700f8ec9dcb29cbc00c92d76173f1601bf5)

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-31 10:10:29 +01:00
Aníbal Limón 7effe18700 scripts/yocto-compat-layer.py: Add dependency validation in add_layer
Some layers don't have dependencies so add a validation to avoid
exception when trying to None.split().

(From OE-Core rev: 39103285029a0bb7b64dc5a305c484988b4c651a)

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-29 16:37:27 +01:00
Leonardo Sandoval e6adeac7aa recipes-kernel: Skip kernel version check on kernel templates
Currently, SRCREV is set to AUTOREV, causing mismatch between PV and the kernel
version shown in the makefile (see below to see the bitbake log). The solution
is to skip this check, suggesting to remove it once SRCREV is locked.

ERROR: linux-yocto-4.10+gitAUTOINC+01f18cba44_53be19cad6-r0.1 do_kernel_version_sanity_check: Package Version (4.10+gitAUTOINC+01f18cba44_53be19cad6) does not match of kernel being built (4.10.5). Please update the PV variable to match the kernel source or set KERNEL_VERSION_SANITY_SKIP="1" in your recipe.
ERROR: linux-yocto-4.10+gitAUTOINC+01f18cba44_53be19cad6-r0.1 do_kernel_version_sanity_check: Function failed: do_kernel_version_sanity_check (log file is located at /home/lsandov1/poky2/build/tmp/work/myqemu-poky-linux/linux-yocto/4.10+gitAUTOINC+01f18cba44_53be19cad6-r0.1/temp/log.do_kernel_version_sanity_check.26904)
ERROR: Logfile of failure stored in: /home/lsandov1/poky2/build/tmp/work/myqemu-poky-linux/linux-yocto/4.10+gitAUTOINC+01f18cba44_53be19cad6-r0.1/temp/log.do_kernel_version_sanity_check.26904
ERROR: Task (/home/lsandov1/poky2/meta/recipes-kernel/linux/linux-yocto_4.10.bb:do_kernel_version_sanity_check) failed with exit code '1

[YOCTO #11064]

(From meta-yocto rev: 79bfd911f826c6c7e26827bb200dd69e6f567b17)

Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-28 08:52:14 +01:00