Commit Graph

45625 Commits

Author SHA1 Message Date
Patrick Ohly 68079d0f87 yocto-compat-layer: better handling of per-machine world build breakage
It is fairly common that BSP layers enable recipes when choosing
machines from that layer without checking whether the recipe actually
builds in the current distro. That breaks "bitbake world", retrieving
signatures and thus the test_machine_signatures test.

It's better to let that test continue with the signatures that can be
retrieved and report the broken world build separately. Right now, the
new test_machine_world iterates over all machines. More elegant and
useful in combination with a (currently missing) selection of which
tests to run would be to generate one test instance per machine. But that
is not straightforward and has to wait.

The "-k" argument alone was not enough to proceed despite failures,
because bitbake then still returns a non-zero exit code. The existance
of the output file is taken as sign that the bitbake execution managed
was not fatally broken.

(From OE-Core rev: 02f5d7836b726e40fef82b50b8145acc839b360b)

Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-13 10:54:10 +01:00
Patrick Ohly de76d1cc94 yocto-compat-layer: test signature differences when setting MACHINE
Selecting a machine is only allowed to affect the signature of tasks
that are specific to that machine. In other words, when MACHINE=A and
MACHINE=B share a recipe foo and the output of foo, then both machine
configurations must build foo in exactly the same way. Otherwise it is
not possible to use both machines in the same distribution.

This criteria can only be tested by testing different machines in combination,
i.e. one main layer, potentially several additional BSP layers and an explicit
choice of machines:
yocto-compat-layer --additional-layers .../meta-intel --machines intel-corei7-64 imx6slevk -- .../meta-freescale

To simplify the analysis and limit the amount of output, mismatches
are sorted by task order such that tasks that run first are also
reported first. Following tasks for the same recipe and set of
machines then get pruned, because they are likely to be different
because of the underlying task (same approach as in
test_signatures). The difference here is that we get information about
all machines. The task order in the base configuration serves as
heuristic for sorting that merged list.

The test has already found issues in go-cross (depended on
tune-specific libgcc) and gdb-cross (had a tune-specific path
unnecessarily), so it is also useful to uncover issues that are not
caused by the BSP layer itself.

(From OE-Core rev: cb0d3de4540e412cfcb7804b4b1689141c80e3a1)

Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-13 10:54:10 +01:00
Paul Eggleton 40d1771944 recipetool: create: hide missing npm error when called from devtool
If devtool is called with a URL to a source repository containing a
node.js module, we don't know that until recipetool has fetched it, and
due to the structure of the code we have to exit with a special code in
order to let devtool know it needs to build nodejs-native. We also want
to suppress the error message that recipetool would normally print under
these circumstances; there is already a mechanism for this but it wasn't
operative in the case where we're pointed to a source repository rather
than an npm:// URL, so create some plumbing so that we know to hide the
message.

(From OE-Core rev: 0c2d0fbb1c6c5b82183799eb7ef80074f86bcfc4)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-13 10:54:10 +01:00
Paul Eggleton 48f8d1201a devtool: add: prevent repeatedly running recipetool
If recipetool returns with exit code 14 this means devtool needs to
build nodejs-native and then call it again. If recipetool returns exit
code 14 again then clearly something has gone wrong and we should just
quit with an error.

(From OE-Core rev: 8d7cced6e06d7c2037f5ab75ac859f501129532e)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-13 10:54:10 +01:00
Paul Eggleton bb8f141d0c devtool: add: fix node.js/npm handling with recipe specific sysroots
The change over to recipe specific sysroots means that we can no longer
get a known location simply from configuration for the npm binary - we
need to get the recipe sysroot for nodejs-native, look there for npm if
we need to check it's present, and add that to PATH when calling out to
npm. Unfortunately this means anywhere we need to get that path we have
to have parsed all recipes, otherwise we have no reliable way of
resolving nodejs-native. Thus we have to change recipetool create to
always parse all recipes (the structure of the code does not allow us to
do this conditionally).

In the worst case, if npm hasn't already been added to its own sysroot
and we are fetching from a source repository rather than an npm
registry, this gets a bit ugly because we end up parsing recipes three
times:
1) recipetool startup, which then fetches the code and determines it's
   a node.js module, finds that npm isn't available and then exits with
   a specific error to tell devtool it needs to build npm
2) when we invoke bitbake -c addto_recipe_sysroot nodejs-native
3) when we re-invoke recipetool

This code is badly in need of refactoring, but now is unfortunately not
the time to do that, so we're going to have to live with this ugliness
for now.

Fixes [YOCTO #10992].

(From OE-Core rev: acfdbd796c99882b8586023c8c6b848716105c8d)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-13 10:54:10 +01:00
Paul Eggleton 0440cbccda recipetool: create: fix for regression in npm license handling
OE-Core commit c0cfd9b1d54b05ad048f444d6fe248aa0500159e added handling
for AND / OR in license strings coming from npm, but made the assumption
that an & would always be present in the license value. Check if it's
there first so we don't fail if it isn't.

(From OE-Core rev: abe2955df2dc558de6068d9373dfcb47d690704b)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-13 10:54:09 +01:00
Robert Yang 9dc4707035 blacklist.bbclass: fix for multilib
* Fixed:
  The netmap has been blacklisted in
  meta-networking/recipes-kernel/netmap/netmap_git.bb, but lib32-netmap still can
  be built (suppose it doesn't depend on another broken recipe netmap-modules, it
  is a little complicated, will talk below):
  $ bitbake lib32-netmap

  This is because of the old code masks on bb.event.ConfigParsed which can only
  handle global blacklist, netmap sets blacklist in the recipe, so it can't be
  handled, and lib32-netmap can be built. which was incorrect:
  blacklist_multilib_eventhandler[eventmask] = "bb.event.ConfigParsed"

  Move multilib code into multilib.bbclass can fix the problem easily:
  $ bitbake lib32-netmap
    ERROR: Nothing PROVIDES 'lib32-netmap'
    ERROR: lib32-netmap was skipped: Recipe is blacklisted: BROKEN: <foo>

* Not fixed
  Another problem is netmap-modules has also been blacklisted in the recipe, and
  the recipe inherits module.bbclass, so multilib.bbclass doesn't handle it as the
  code shows:
    # There should only be one kernel in multilib configs
    # We also skip multilib setup for module packages.
    provides = (e.data.getVar("PROVIDES") or "").split()
    if "virtual/kernel" in provides or bb.data.inherits_class('module-base', e.data):
        raise bb.parse.SkipPackage("We shouldn't have multilib variants for the kernel")

  And netmap-modules provides lib32-netmap-modules which is handled in
  multilib_global.bbclass, so bitbake lib32-netmap-modules can't show
  the blacklist message:
  $ bitbake netmap-modules
  ERROR: Nothing PROVIDES 'netmap-modules'
  ERROR: netmap-modules was skipped: Recipe is blacklisted: BROKEN: <foo>
  ERROR: netmap-modules was skipped: We shouldn't have multilib variants for the kernel

  $ bitbake lib32-netmap-modules
  ERROR: Nothing PROVIDES 'lib32-netmap-modules'. Close matches:
    netmap-modules
    netmap-modules
    lib32-fbset-modes

  Note the different messages between netmap-modules and lib32-netmap-modules.
  This is because multilib.bbclass doesn't handle the "module" recipe so
  there is no PN called lib32-netmap-modules, therefore blacklist.bbclass can't
  handle it.

  Note, there are two "netmap-modules" which needs to be fixed later.

(From OE-Core rev: c8749ed1edcbb544f6656ee5da80f2cf647c405a)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-13 10:54:09 +01:00
Scott Rifenbark 94467d5087 dev-manual, ref-manual: Edits to the "Incrementing Binary Package" section.
Fixes [YOCTO #10995]

I applied some revisions to the related sections for incrementing
a binary package version and revision sections.  Mainly clarified
terminology and pulled the sections into one single section head.

I also cleaned up referencing to all these sections throughout the
YP manual set (i.e. the ref-manual).

(From yocto-docs rev: 7deda18dd496cc383c5de71a10f2b11b1ca0688f)

Signed-off-by: Scott Rifenbark <srifenbark@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-12 19:35:00 +01:00
Scott Rifenbark f9e7fba02f dev-manual, ref-manual: Created new section on linear revision for packages
Fixes [YOCTO #10995]

For the dev-manual, I added a new section on incrementing package
revision numbers that explains how the build system uses various
variables to create linearly numbered revisions for packages.

For the ref-manual, I cross-referenced into the new section from the
AUTOSRC and SRCREV variables.

(From yocto-docs rev: 53c3092e0f0293f4db0a88659abb4cf1c93f208c)

Signed-off-by: Scott Rifenbark <srifenbark@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-12 19:35:00 +01:00
Scott Rifenbark 7f47526f5c ref-manual: Initial draft of 2.3 migration section.
(From yocto-docs rev: c4c995aa94e5100e0d306eb91f490c23726ea91f)

Signed-off-by: Scott Rifenbark <srifenbark@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-12 19:35:00 +01:00
Scott Rifenbark beb301c03a dev-manual: Added section on using headers to interface with devices
Fixes [YOCTO #8596]

Added a new section to describe the right way to use headers to
interface to a device or custom Linux kernel API.  Too often a
user wants to modify the libc header file, which absolutely wrong.
This new section provides some guidance.

(From yocto-docs rev: 960c49bf6446398a9efb2d018d09d63b49e97196)

Signed-off-by: Scott Rifenbark <srifenbark@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-12 19:35:00 +01:00
Scott Rifenbark e2232e6813 dev-manual, ref-manual: Added PACKAGE_WRITE_DEPS and concept support
Fixes [YOCTO #11274]

When a post-installation or pre-installation script uses tools that
have dependencies, you need to specify the tools using the new
PACKAGE_WRITE_DEPS variable.  I added this information at the end
of the "Post Installtion Scripts" section in the dev-manual.

I also added a new variable entry for the PACKAGE_WRITE_DEPS variable
in the glossary.

(From yocto-docs rev: 4d130b63cba5ebabd4af52b62421d387a4b54353)

Signed-off-by: Scott Rifenbark <srifenbark@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-12 19:35:00 +01:00
Scott Rifenbark 1d5745f959 sdk-manual: Fixed link to Neon download site.
Fixes [YOCTO #11263]

(From yocto-docs rev: c7e68a533045ff482b5fc022886df58e199dd4b7)

Signed-off-by: Scott Rifenbark <srifenbark@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-12 19:35:00 +01:00
Scott Rifenbark e0a4972087 dev-manual: Added cross-reference to QA and Test Infrastructure section.
Fixes [YOCTO #10887]

I placed a small paragraph at the end of the introduction to the
"Performing Automated Runtime Testing" section that points into the
new section on QA and testing infrastructure in the ref-manual.

(From yocto-docs rev: 493051e1770e07b89c19a1f235f1dd226610c77a)

Signed-off-by: Scott Rifenbark <srifenbark@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-12 19:35:00 +01:00
Scott Rifenbark 43fefd1211 ref-manual: Added section on test and QA infrastructure
Fixes [YOCTO #10887]

There needed to be an overview of the test and QA infrastructure
used by the YP development team for releases and that is visible
to the developer.  I added a new section for this information into
the existing section on YP release processes.

(From yocto-docs rev: 17b3d6898b37bdbfbd23a333a4d659a9746696d0)

Signed-off-by: Scott Rifenbark <srifenbark@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-12 19:35:00 +01:00
Scott Rifenbark c53ba427ef dev-manual: Added YP Compatibility feature
Fixes [YOCTO #10596]

No information existed for the version 1.0 YP Compatibility Program
for layers.  Additionally, the version 2.0 flavor is adding a script
as part of the application process.  I put in a new section on the
whole topic.  Also, integrated the information into relevant sections
with some cross-referencing.

(From yocto-docs rev: 6341705d4c08901b2657c489f4e995968db00d85)

Signed-off-by: Scott Rifenbark <srifenbark@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-12 19:34:59 +01:00
Nathan Rossi 97f30c70bd busybox: In defconfig enable ASH_CMDCMD for built-in 'command'
It is common for *ash shells to have 'command' available as a built-in
function. POSIX 2008 also documents the availability of this command.

Additionally the /etc/profile of base-files requires this command to be
available as of commit e77cdb7611 ("base-files: profile: Do not assume
that the tty command exists"). If it is not available the following
message is output during login on a image using busybox.

    -sh: command: not found

It however should be noted that tcsh and csh do not provide 'command'
(built-in or otherwise).

(From OE-Core rev: e41c90b852167bbcb434da57b84280e855acae33)

Signed-off-by: Nathan Rossi <nathan@nathanrossi.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-12 15:18:46 +01:00
Nathan Rossi bf0fc2da66 bitbake: lib/bb/{data, siggen}: Don't warn on SkipRecipe exceptions
Recipes/variables that raise a SkipRecipe exception are intentionally
skipped, and should not generate warnings.

[YOCTO #11319]

(Bitbake rev: edf6e6094a9f7ad4b2ba06eef8fd34756edbedce)

Signed-off-by: Nathan Rossi <nathan@nathanrossi.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-12 15:16:05 +01:00
Richard Purdie 000b7d3d1c bitbake: data_smart: Ensure _remove operations on newly set variables are cleared
We clear append/prepend on newly set variables, we should also clear
remove operations. If we don't do this, there is no way we can actually
delete a remove operation. Bitbake internally uses parsing=True to avoid
these side effects when making its own internal calls.

Also add a testcase to bitbake-selftest to ensure we remain consistent going
forward from here.

(Bitbake rev: 3a319f079d699c870d8531e051ab65e6278d1fa5)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-12 15:09:58 +01:00
Ed Bartosh 64a379b325 wic-image-minimal: stop using core-image-minimal
Removed dependency to core-image-minimal rootfs from the
recipe and .wks file. This was a bad idea as recipes are
not allowed to use other recipe's rootfs or anything else
from workdir.

This should also fix test_qemu test case when rm_work is
enabled.

(From OE-Core rev: 0f1879982ab6006f14ac2c9f5546a7b1f79460cd)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-12 15:09:58 +01:00
Patrick Ohly 31e53430f1 yocto-compat-layer: add --additional-layers
The new --addditional-layers parameter takes a list of layer
directories and adds them to the build configuration before starting
testing. The resulting base configuration then more closely matches
a full distro.

This is relevant in two cases:
1. some layers like meta-freescale dynamically enable more recipes
   in their layer.conf depending on which other layers are active,
   so testing only against OE-core might miss problems which occur
   only when also some other layers are active
2. BSP layers might be fine in combination with machines from
   OE-core, but might break in combination with some other machines

As before, test_signatures only warns about signature changes
introduced by the layer which is under testing, and not those changes
introduced by the additional layers.

(From OE-Core rev: 0e8528f7c6201e8a5d2799123241c0e1b85081ce)

Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-12 15:09:58 +01:00
Patrick Ohly e93a2ab3e3 yocto-compat-layer: also determine tune flags for each task
locked-sigs.inc groups tasks according to their tune flags (allarch,
i586, etc.). Also retrieve that information while getting signatures,
it will be needed to determine when setting a machine changes tasks
that aren't machine-specific.

(From OE-Core rev: 67f9a8759f47680dbf349797801b2a1e8d149377)

Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-12 15:09:58 +01:00
Patrick Ohly 937c1ea974 yocto-compat-layer: include bitbake-diffsigs output
After filtering out potential false positives, it becomes feasible to
include the output of bitbake-diffsigs for those tasks which
definitely have a change.

Depends on bitbake-diffsigs with the "--signature" parameter.

Enhanced output now is:

   AssertionError: False is not true : Layer meta-xxxx changed 120 signatures, initial differences (first hash without, second with layer):
      gstreamer1.0-plugins-base:do_fetch: 76973f19f2e30d282152bdd7e4efe5bb -> e6e7c6fa9f2bd59d7d8d107f7c6ca1ac
         Task dependencies changed from:
         ['PV', 'SRCREV', 'SRC_URI', 'SRC_URI[md5sum]', 'SRC_URI[sha256sum]', 'base_do_fetch']
         to:
         ['GST_IMX_PATCHES_TO_APPEND', 'PV', 'SRCREV', 'SRC_URI', 'SRC_URI[md5sum]', 'SRC_URI[sha256sum]', 'base_do_fetch']
         basehash changed from d679d30bd1ea41c56e57419b57587f3c to 090a79b45f5fa26d10f9d34e2ed7a1e6
            List of dependencies for variable SRC_URI changed from '{'PV', 'SRC_URI[md5sum]', 'SRC_URI[sha256sum]'}' to '{'GST_IMX_PATCHES_TO_APPEND', 'PV', 'SRC_URI[md5sum]', 'SRC_URI[sha256sum]'}'
         changed items: {'GST_IMX_PATCHES_TO_APPEND'}
         Dependency on variable GST_IMX_PATCHES_TO_APPEND was added
         Variable SRC_URI value changed:
         "     http://gstreamer.freedesktop.org/src/gst-plugins-base/gst-plugins-base-${PV}.tar.xz     file://get-caps-from-src-pad-when-query-caps.patch     file://0003-ssaparse-enhance-SSA-text-lines-parsing.patch     file://0004-subparse-set-need_segment-after-sink-pad-received-GS.patch     file://encodebin-Need-more-buffers-in-output-queue-for-bett.patch     file://make-gio_unix_2_0-dependency-configurable.patch     file://0001-introspection.m4-prefix-pkgconfig-paths-with-PKG_CON.patch     file://0001-Makefile.am-don-t-hardcode-libtool-name-when-running.patch     file://0002-Makefile.am-prefix-calls-to-pkg-config-with-PKG_CONF.patch     file://0003-riff-add-missing-include-directories-when-calling-in.patch     file://0004-rtsp-drop-incorrect-reference-to-gstreamer-sdp-in-Ma.patch [--] {+${GST_IMX_PATCHES_TO_APPEND}+}"

      pulseaudio:do_install: 6bb6fe23e11a6d5fef9c3a25e73e4f9c -> 3f54ea75673a792e307197cfa6ef2694
         basehash changed from ac4efcfa783bd04a5a98a2c38719aedd to 37679d99623a37c8df955da3a01415a5
         Variable do_install value changed:
         @@ -1,3 +1,7 @@
              autotools_do_install
           	install -d ${D}${sysconfdir}/default/volatiles
          	install -m 0644 ${WORKDIR}/volatiles.04_pulse  ${D}${sysconfdir}/default/volatiles/volatiles.04_pulse
         +    if [ -e "${WORKDIR}/daemon.conf" ] && [ -e "${WORKDIR}/default.pa" ]; then
         +        install -m 0644 ${WORKDIR}/daemon.conf ${D}${sysconfdir}/pulse/daemon.conf
         +        install -m 0644 ${WORKDIR}/default.pa ${D}${sysconfdir}/pulse/default.pa
         +    fi

[YOCTO #11161]

(From OE-Core rev: 312edd42b6cc553de4d476c76e8e36a882e11cdd)

Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-12 15:09:58 +01:00
Patrick Ohly 3e903cb42f recipes/*-cross recipes: ignore TARGET_ARCH sstate hash
"yocto-compat-layer.py --machines" showed that shared packages like
gcc-cross-powerpc64 have a sstate signature that depends on
TUNEFLAGS. As a result, there are unnecessary rebuilds and potential
conflicts in a multiconfig.

That's due to the way how TARGET_ARCH is set. Richard Purdie suggested
setting TARGET_ARCH[vardepvalue] as fix, which works. It would be
shorter to do that in cross.bbclass instead of repeating the relevant
line in different recipes, but Richard was concerned about potential
side-effects in other usages of cross.bbclass.

TARGET_GOARM as used in go.inc is still causing signature differences
for go-cross-powerpc64 and machines b4420qds-64b and p5020ds-64b. This
needs further investigation.

(From OE-Core rev: 39bfa0dd3237cbca47e7fca1075d521f9d073f25)

Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-12 15:09:58 +01:00
Patrick Ohly fc43f10a51 gdb-cross: avoid tune specific paths
gdb-cross used to be specific to the tune flags, but isn't
anymore. Therefore it is enough to use TARGET_SYS instead of
TUNE_PKGARCH to create a unique path.

Fixes a sstate signature difference that was found via
yocto-compat-layer.py's test_machine_signatures check. In practice it
probably showed up as unnecessarily rebuilding gdb-cross when
switching between machines like intel-corei7-64 and qemux86-64.

(From OE-Core rev: f346473a4868563db7fb63665e808c3fe25a8b58)

Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-12 15:09:58 +01:00
Patrick Ohly 30686e2fd6 go-cross: avoid libgcc dependency
libgcc gets compiled differently depending on the tune flags for the
target. That dependency would make go-cross also tune specific and
prevent sharing it between different machines using the same
architecture.

For example, MACHINE=intel-corei7-64 and MACHINE=qemux86-64 shared the
same go-cross-x86_64, but compiled libgcc differently.

The libgcc dependency gets inherited from go.inc, but does not seem to
be necessary for go-cross (compiling go-helloworld still succeeds).
The dependency is left in go.inc conditionally, just in case that it
really is relevant for the various on-target recipes which inherit
that.

Because go-cross*.bb includes go*.bb, moving the DEPENDS into a .inc
file that only gets included for the target recipes doesn't
work. Reshuffling the content of three .bb files seems too intrusive
at this point.

(From OE-Core rev: 58149a7be4172074349951aaf5af95fa40fd4bdb)

Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-12 15:09:58 +01:00
Maxin B. John a9f119382b ed: update SRC_URI to yoctoproject mirror
Upstream has removed the 1.14.1 release from ftp.gnu.org and
moved to the latest 1.14.2. Since we don't want to upgrade at
this point of time, temporarily move the SRC_URI to yoctoproject
mirror.

(From OE-Core rev: a2f1026b3d8c9f9810cb4389a8a93fabb04e15a4)

Signed-off-by: Maxin B. John <maxin.john@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-12 15:09:58 +01:00
Jussi Kukkonen db1f1adace native/nativesdk: Use fixed DISTRO_FEATURES
There seems to be little advantage to letting distro features affect
native builds. There is a significant disadvantage: a change to
DISTRO_FEATURES will trigger a lot of unnecessary native tasks. In a
test like this:
  $ bitbake core-image-minimal
  # append " systemd" to DISTRO_FEATURES
  $ bitbake core-image-minimal
The latter build takes 44 minutes (28%) of cpu-time less with this
patch (skipping 135 native tasks). Sadly wall clock time was not
affected as glibc remains the bottleneck.

Set native distro features to DISTRO_FEATURES_NATIVE appended with
an intersection of DISTRO_FEATURES and DISTRO_FEATURES_FILTER_NATIVE.
Current default values (baitbake.conf) are
* DISTRO_FEATURES_FILTER_NATIVE ?= "api-documentation" (as gtk-doc-native
has much less dependencies when built without it)
* DISTRO_FEATURES_NATIVE ?= "x11" (to enable native UIs even if target
does not containe them)

Do the variable setting in native_virtclass_handler() because otherwise
it could still be overridden by appends and the feature backfilling.
Shuffle the early returns so DISTRO_FEATURES gets set as long as
the packagename ends with "-native".

Add similar variables for nativesdk.

To make nativesdk work we need to enable the locale options so
nativesdk-glibc-locales can build and to avoid the init manager check
in the nativesdk case so add those fixes.

(From OE-Core rev: 731744d5538e315702be828e6f2bd556309dee07)

Signed-off-by: Jussi Kukkonen <jussi.kukkonen@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-12 15:09:58 +01:00
Ian.Arkver f2e41a197f package_manager.py: Add a name for the generated dnf repo.
This adds repo_name as the name, which is the same as the
repo ID and hence a bit pointless, but it stops dnf from
complaining that the repo doesn't have a configured name.

(From OE-Core rev: c92cff23893e87cfb7ef9f77b16761310104543a)

Signed-off-by: Ian.Arkver <ian.arkver.dev@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-12 15:09:58 +01:00
Robert Yang 620655697d selftest/wic.py: vda -> sda
Previously, runqemu grep root=/dev/sd or root=/dev/hd on the image, and
would use vda if no grep result, now we have set QB_DRIVE_TYPE to
"/dev/sd" by default, and the device will be /dev/sda, so use sda to
replace vda in the test case.

(From OE-Core rev: a722016df3f452f7a870157a99a1abb7d97d8280)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-12 15:09:57 +01:00
Ed Bartosh 8838dd2dbd oe-selftest: test wic sparse_copy API
Added new parameter 'api' to sparse_copy function to specify
underlying filemap API to use. By default sparse_copy will
try both available APIs.

Added test case for sparse_copy to wic test suite.

(From OE-Core rev: 88701cef6ba399e82f96ed1b0eef9a44ed8c1687)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-12 15:09:57 +01:00
Robert Yang 7792688ac2 qemux86-directdisk.wks: vda -> sda
Previously, runqemu grep root=/dev/sd or root=/dev/hd on the image, and
would use vda if no grep result, now we have set QB_DRIVE_TYPE to
"/dev/sd" by default, and the device will be /dev/sda, so use sda to
replace vda in the test case.

(From OE-Core rev: 3f1f42984021d87ac43aaa16f38b706c2c965e02)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-12 15:09:57 +01:00
Robert Yang be72eaf86f runqemu: use self.rootfs to replace self.nfs_dir
We can use self.rootfs as self.nfs_dir when self.fstype is nfs, this can
reduce the code's complexity and we can re-use the code of checking
ROOTFS conflictions.

(From OE-Core rev: 1aafa13ae6faf620acac7338c42a8838e75da6b9)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-12 15:09:57 +01:00
Robert Yang d538cbb95e wic-image-minimal.wks: vda -> sda
Previously, runqemu grep root=/dev/sd or root=/dev/hd on the image, and
would use vda if no grep result, now we have set QB_DRIVE_TYPE to
"/dev/sd" by default, and the device will be /dev/sda, so use sda to
replace vda in the test case.

(From OE-Core rev: aa8df7e17bb4bb0a92938cbc36cd8b22bcf14cfa)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-12 15:09:57 +01:00
Robert Yang f57393b386 runqemu: do not rely on grepping images
Fixed when the image is large and not enough memory:
  grep: memory exhausted
  Aborted

[YOCTO #11073]

(From OE-Core rev: a99deb30a0138594147ae28aab016fe4b74b8959)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-12 15:09:57 +01:00
Robert Yang f2c09f7867 runqemu: run without arguments
Since we can get MACHINE and others from env vars and "bitbake -e",
"runqemu" can work without any arguments.

(From OE-Core rev: 9ebcb2b6f41420ae3686afad03bb26a68cfacf95)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-12 15:09:57 +01:00
Robert Yang f6aaba31cf runqemu: support env vars explicitly
Use self.env_vars to support get vars from environment explicity. The
MACHINE, ROOTFS and KERNEL was supported by shell based runqemu, and
the help text says support them from env vars, so add them back.

[YOCTO #11141]

(From OE-Core rev: 20008d0bfe2cacecba77e11b0a0faf3d959eaf1e)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-12 15:09:57 +01:00
Aníbal Limón b283b10d53 ptest-runner: Upgrade to minor version 2.0.2
To fix a problem when print ERROR after a ptest timeout, this
causes the user confusion about if a test ends or not.

[YOCTO #10842]

(From OE-Core rev: 210c518ba8f8d6ec6e9d34e0df8b963a3b2e0593)

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-11 18:10:18 +01:00
Carlos Rafael Giani 7252450159 gstreamer1.0-plugins-good: Backport patch for v4l2object videometa
This patch ensures videometa is added to mem2mem decoder output in case
the output frames have padding rows/columns

(From OE-Core rev: ef94ffee8f11db57ecea398af76dc22576c2068a)

Signed-off-by: Carlos Rafael Giani <dv@pseudoterminal.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-11 18:10:18 +01:00
Carlos Rafael Giani a9dfae4e26 gstreamer1.0-plugins-bad: Update packageconfigs and config flags
(From OE-Core rev: 5acad7f33aebfac4c5a3a68778f5860f954904fe)

Signed-off-by: Carlos Rafael Giani <dv@pseudoterminal.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-11 18:10:18 +01:00
Richard Purdie 283a9b3883 staging: Fix sysroot problem with populate_sysroot dependencies on do_fetch
Dependencies on svn:// urls were failing as the cleandirs on do_fetch was destroying
any sysroot setup by the extend_recipe_sysroot function.

Add code so that if the task do_fetch, we move the cleandirs to a separate function before
the extend_recipe_sysroot prefunc else we'd wipe out the sysroot we just created.

This allows fetcher do_populate_sysroot dependencies to work correctly again.

I did try various other approaches and a seperate function with cleandirs was the
cleanest way to add this without code duplication or too much complexity.

(From OE-Core rev: b32fbfc85b14059f0bfdeeffe2c52f8d3c20f276)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-11 18:10:18 +01:00
Yuanjie Huang ca22ef73d0 binutils: Fix CVE-2017-6965 and CVE-2017-6966
Backport upstream commit to address vulnerabilities:

CVE: CVE-2017-6965
[BZ 21137] -- https://sourceware.org/bugzilla/show_bug.cgi?id=21137

Fix readelf writing to illegal addresses whilst processing corrupt input
files containing symbol-difference relocations.

	PR binutils/21137
	* readelf.c (target_specific_reloc_handling): Add end parameter.
	Check for buffer overflow before writing relocated values.
	(apply_relocations): Pass end to target_specific_reloc_handling.

CVE: CVE-2017-6966
[BZ 21139] -- https://sourceware.org/bugzilla/show_bug.cgi?id=21139

Fix read-after-free error in readelf when processing multiple, relocated
sections in an MSP430 binary.

	PR binutils/21139
	* readelf.c (target_specific_reloc_handling): Add num_syms
	parameter.  Check for symbol table overflow before accessing
	symbol value.  If reloc pointer is NULL, discard all saved state.
	(apply_relocations): Pass num_syms to target_specific_reloc_handling.
	Call target_specific_reloc_handling with a NULL reloc pointer
	after processing all of the relocs.

(From OE-Core rev: 8c52a530ba2beb438aa47956bcec3777a1eafe5f)

Signed-off-by: Yuanjie Huang <yuanjie.huang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-11 18:10:18 +01:00
Alexander Kanavin 40bf913a72 libsolv: correctly attribute musl fixing patches
Also, they were previously squashed into a single patch; restore
the original two-patch arrangement.

As requested here:
http://lists.openembedded.org/pipermail/openembedded-core/2017-April/135460.html

(From OE-Core rev: 378b333fb09d106fb04901f5a4362fc0eb076e82)

Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-11 18:10:18 +01:00
Li Zhou a2d5c0b602 python2/3: Move config/Makefile from core package to dev package
Move config/Makefile in libdir from core package to dev package for
python, because it is only needed in development process.

(From OE-Core rev: 8b55d055f046677c18eeaefe3ca18869eedeb14d)

Signed-off-by: Li Zhou <li.zhou@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-11 18:10:18 +01:00
Alexander Kanavin 124b3ea99b openssl: fix the reference to native perl in ptests
This was causing a couple of ptest failures.

[YOCTO #10840]

(From OE-Core rev: 2e8e72790d3cc3236b6a785f3e04702e71e1ac3f)

Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-11 18:10:18 +01:00
Joshua Lock 1b4bd1b659 oeqa/sdk/cases/buildcpio: enable use of cached cpio from DL_DIR
All build project test cases will try to copy the requested source
artefacts from DL_DIR before fetching. This testcase is referencing bzipped
tarball whereas the recipe fetches a gzipped tarball.

Switch to fetching the gzipped tarball in the test case so that we're able
to use a cached tarball from DL_DIR

(From OE-Core rev: 27a5883e8ae01e69f5425efe8b035bea7087b2f9)

Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-11 18:10:18 +01:00
David Reyna ba1d8e3691 toaster: fix SDK artifact capture
Use the TaskArtifacts event to scan the SDK and ESDK manifests
to cleanly collect the respective artifact files.

The previous method was broken when the SDK file deployment moved
from the do_populate_sdk[_ext] tasks to their sstate tasks. That
method is disabled (but not yet removed) in preparation for the
rest of refactor work for the parent #10283 work.

[YOCTO #10850]

(From OE-Core rev: a6ec56d372f833be50f57bcb79b6ebe78bb93be0)

Signed-off-by: David Reyna <David.Reyna@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-11 18:10:18 +01:00
Andre McCurdy 31a293d0dc mtd-utils: refresh patches now merged upstream
Update Upstream-Status tags and apply 010-fix-rpmatch.patch
unconditionally, since it's merged unconditionally upstream.

(From OE-Core rev: 87dcaf2094baf9a7b7993c2ff1f60db73f4248f4)

Signed-off-by: Andre McCurdy <armccurdy@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-11 18:10:18 +01:00
Paul Eggleton 3df7ee4755 classes/buildhistory: save output file signatures for sstate tasks
Save a file per task listing sha256sums for each file staged, i.e.
the output of the task. Some caveats:

1) This only covers sstate tasks since it uses SSTATEPOSTUNPACKFUNCS,
   however those are generally the most interesting in terms of output
   anyway.
2) The signature is taken before applying any relocations, so any
   relocated files will actually have different signatures, but that's
   churn that you probably won't want to see here.
3) At the moment if you run the same build twice without sstate you will
   very likely see changes in the output for certain tasks due to things
   like timestamps being present in the binary output. Fixing that is
   a general Linux ecosystem problem - see this page for our efforts to
   resolve it on our side:
     https://wiki.yoctoproject.org/wiki/Reproducible_Builds

NOTE: you need to set your BUILDHISTORY_FEATURES value to include
"task" to enable collection of these signatures as it is is disabled by
default.

(From OE-Core rev: ca5d1273432e20059ab66d721a9eb314a54e81e7)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-11 18:10:18 +01:00
Paul Eggleton 54426c3a3b classes/uninative: set SSTATEPOSTUNPACKFUNCS[vardepvalueexclude] properly
Append to the value with appendVarFlag() instead of setting it outright,
so that we can also append to it in other places. Accordingly, this
varflag is pipe-separated (since we want to be able to exclude any
string fragment, in this case including the leading space), thus put a
leading pipe character to play nicely with any existing value.

(From OE-Core rev: a147838c3dfd4c53084a19b052b8d4e183293412)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-11 18:10:17 +01:00