Commit Graph

5322 Commits

Author SHA1 Message Date
Olaf Mandel 873a4eca11 (PRE)MIRRORS: fix pattern for npm:// without slash
For URIs with the npm:// transport but with no other slash in it, the
common MIRRORS and PREMIRRORS pattern of npm://.*/.* fails to match.
Make the last slash in the pattern optional in the mirros.bbclass and
own-mirrors.bbclass classes.

Many URIs with the npm:// transport have no slash after the host part:

  npm://registry.npmjs.org;name=foo;version=0.1.2

This means that MIRRORS and PREMIRRORS containing entries like the
first one will not match these URIs:

  npm://.*/.*   # fails to match
  npm://.*/?.*  # matches this and URIs with path components

For normal regular expressions, a pattern like 'npm://.*(/.*)?' would
probably be preferred, but that won't work here: the pattern gets split
into the substrings 'npm', '.*(' and '/.*)?', which are not valid
regular expressions individually.

(From OE-Core rev: 0d1e2b4507af28fc451b8fa94130a39ac342637d)

(From OE-Core rev: 1f5675a43e1781635fee15cc3674143c195da169)

Signed-off-by: Olaf Mandel <o.mandel@menlosystems.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-09-21 16:55:57 +01:00
Jose Alarcon 44c5efb453 rootfs-postcommands: add test for unsatisfied RRECOMMENDS
The do_rootfs log contains a number of unsatisfied package
recommendations. At the moment those are only visible when
reviewing the rootfs log.

This patch adds an extra check to surface any unsatisfied
recommendation  as WARNINGS to the build output.

Enable this check with:
ROOTFS_POSTPROCESS_COMMAND += "rootfs_log_check_recommends;"

(From OE-Core rev: 9d049bf7941f30e35c51775684559e95185fba96)

(From OE-Core rev: 514cadd28d05f5060b2538ef9b2920370c22bf73)

Signed-off-by: Jose Alarcon <jose.alarcon@ge.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-09-21 16:55:57 +01:00
Jose Alarcon 4d85aa71a0 rootfs-postcommands: remove empty line
(From OE-Core rev: 76a801da7fe5651d9e8310a2b32c275e1700daf3)

(From OE-Core rev: 189ea6f0232f1910736121fb15f80eea97932af8)

Signed-off-by: Jose Alarcon <jose.alarcon@ge.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-09-21 16:55:57 +01:00
Khem Raj 40ce8ca30c rootfs-postcommands.bbclass: Filter out dangling symlinks in ssh_allow_empty_password()
In images built with pam in DISTRO_FEATURES, we end up with dangling symlinks
if su is not packaged into image

$ ls /mnt/a/oe/build/tmp/work/raspberrypi3-bec-linux-gnueabi/core-image-minimal/1.0-r0/rootfs/etc/pam.d/su-l -l
lrwxrwxrwx 1 kraj users 2 Aug  9 07:56 /mnt/a/oe/build/tmp/work/raspberrypi3-bec-linux-gnueabi/core-image-minimal/1.0-r0/rootfs/etc/pam.d/su-l -> su

This causes image do_rootfs to fail

| sed: can't read /mnt/a/oe/build/tmp/work/raspberrypi3-bec-linux-gnueabi/core-image-minimal/1.0-r0/rootfs/etc/pam.d/s
u-l: No such file or directory
| WARNING: /mnt/a/oe/build/tmp/work/raspberrypi3-bec-linux-gnueabi/core-image-minimal/1.0-r0/temp/run.ssh_allow_empty_
password.19238:1 exit 2 from 'sed -i 's/nullok_secure/nullok/' /mnt/a/oe/build/tmp/work/raspberrypi3-bec-linux-gnueabi
/core-image-minimal/1.0-r0/rootfs/etc/pam.d/*'

Therefore we need to filter out dangling symlinks before sed'ing
things out

(From OE-Core rev: b92105e5a085c8cd3c650579644922ed97163e73)

(From OE-Core rev: ed1aaf24bfd4a09132c7e0247c460102ec85254d)

Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-09-21 16:55:57 +01:00
Alexander Kanavin 625b0a6702 package_rpm.bbclass: disable generation of .build-id links
As we're not using them and they're getting in the way of
reproducibility.

[YOCTO #11718]

(From OE-Core rev: 1bd3ed18379c330c1c733dc9f043dbbe8aa0d254)

(From OE-Core rev: 5c8f831c71932e2abebd72f8be1ec95e1b6c3ab7)

Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-09-21 16:55:57 +01:00
Alexander Kanavin 59f8925f60 package_rpm.bbclass: use multithreaded xz compression
RPM's default is single-threaded gz; the change greatly helps with
both buildtimes (when there is a small number of large-sized packages)
and disk space taken by resulting rpms.

(From OE-Core rev: f108c4d09926bd28e7a57b665fc8cb5373827780)

(From OE-Core rev: fd329fdc6374ebde3f5ab171dbe76bf5cec6771d)

Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-09-21 16:55:57 +01:00
Ng Wei Tee f6eff90737 rpm: allow arch-dependent binaries in noarch packages
This is needed for packages like linux-firmware which have a
legitimate reason for it. Oe-core has a separate package_qa
test for this situation, so any accidental inclusions of such
binaries will still be caught.

[YOCTO #11329]

(From OE-Core rev: 6aaff392d703183d19192e2d171e10a92f259c65)

(From OE-Core rev: 35059f271964da1f8112a45f19875b59fb9cd02f)

Signed-off-by: Ng Wei Tee <wei.tee.ng@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-09-21 16:55:57 +01:00
Richard Purdie 911146122c staging: Fix a logic error which caused dependency removal
There was a logic error in the dependency cleanup code which meant
it would remove dependencies which other tasks still depended upon.
Fix the path names so the comparisions work as intended.

This fixes dependencies accidentally disappearing from sysroots
under certain reconfiguration situations.

(From OE-Core rev: 1634fe5148b3501f2c1b75cf7fb704a2ef60424e)

(From OE-Core rev: 3969c068bc9557b405a5ad61c10c402ec93bd932)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit a6631eed6fc70b305e769998d6f22f345e37decc)
Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-09-11 22:15:59 +01:00
Richard Purdie 8329bfd156 staging: Ensure dependencies are removed before being added
Currently items are added to the sysroot, the obsolete items are removed. If
a change such as pkgconfig -> pkgconf is made, this leads to conflicts of
overlapping files in the sysroot.

In order to better support this, handle removing items before adding them.
This requires some minor refactoring to construct the installed list
before the main function loop, otherwise there are no changes in this
patch other than reordering the operations.

(From OE-Core rev: add4f107c151d32d9ea914bb0b93c3d3c17c776c)

(From OE-Core rev: 970d4df7336e5102ac61e66689e977fb2e41a1aa)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 5b45fe40db81292dd6bb57b210d1e4ba32e65e9e)
Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-09-11 22:15:59 +01:00
Richard Purdie 292cb6a32f staging: Avoid sysroot removal races
Currently a task could remove a dependency needed by another task leading
to build failures, often due to missing dependencies (e.g. dynamic libraries
not being found). This was often seen for all-arch recipes in package_write_rpm.

When removing a dependency, first check that no other task active for the
recipe has that same dependency.

(From OE-Core rev: ff3617cc2cd5618f48a25aa4e3b2014430fcbe23)

(From OE-Core rev: 67438ac91a40429affc952ad1d7e92c763f52bab)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 3de078642925e720c2fca5b89cd3abb8b29d2439)
Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-09-11 22:15:59 +01:00
Paul Eggleton 54efca180b classes/license: drop erroneous sha256 parameter in LIC_FILES_CHKSUM
In OE-Core commit a48fea275b, a check on
the value of a "sha256" parameter was added, however there was no
mention of this in the commit message and no corresponding code to
actually verify the checksum as sha256 was added along with it either,
so there's no point in getting the value. Additionally it was assuming
that a sha256 value would be present without checking first, with the
result that if you leave out the md5 value in a recipe intentionally in
order to get it to tell you the correct value on the next build, you got
a traceback instead of the appropriate error containing the information.
Drop this entirely - if we want to implement this we need to do it
properly.

(From OE-Core rev: e9eaa7d15fe7ab643ab19556dab84051f8f1974e)

(From OE-Core rev: fcedfce43adad8659a1376bb28b1ed41e621dd53)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-09-11 22:15:59 +01:00
Peter Kjellerstedt 7ea76684cb package.bbclass: Restore functionality to detect RPM dependencies
During the transition to dnf and rpm4, the functionality to
automatically make RPM determine dependencies was lost.

Before the transition, an OE specific tool called rpmdeps-oecore had
been added to the rpm suit. It was based on the rpmdeps tool that is
part of rpm. For each file specified on its command line, it would
output the provides and requires that RPM could determine.

During the transition to rpm4, rpmdeps-oecore was replaced with the
standard rpmdeps. However, what no one noticed was that unless rpmdeps
is given options, e.g., -P or -R, to tell it what it should output, it
will not output anything. Thus, it would do all the work to determine
the requirements, but would keep silent about it. And since no output
from rpmdeps is expected unless there are requirements, there were no
warnings indicating that everything was not working as expected.

Porting the old rpmdeps-oecore to work with rpm4 is not really
possible since it relied on being able to access internals of RPM that
are no longer available. However, it turned out that rpmdeps had a
debug option, --rpmfcdebug, that would output exactly the information
that we need, albeit in a different format and to stderr. To make this
usable, rpmdeps has now received a new option, --alldeps, which sends
the information we need to stdout.

Since enabling this may cause packages to break, it is required that
ENABLE_RPM_FILEDEPS_FOR_PYRO is set to "1" to activate it for Pyro.
The name of this variable has been chosen as to indicate that it only
affects Pyro (since releases before and after Pyro has it enabled by
default).

(From OE-Core rev: 1009498f23ad319825c00ba60a4693d15aada553)

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-08-31 17:57:12 +01:00
Peter Kjellerstedt 2d3908ae4c package_rpm.bbclass: Filter out unwanted file deps for nativesdk packages
Filter out any file dependencies on absolute paths and any
dependencies on Perl modules for nativesdk packages. It is assumed
that they will be provided by the native host if needed, and they mess
up the dependency handling if they are present.

(From OE-Core rev: 37f2d4df507c760ea4c12b67526db8277e5684eb)

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-08-31 17:57:12 +01:00
Peter Kjellerstedt 6c17b29db1 insane.bbclass: Ignore perl as dependency for nativesdk packages
(From OE-Core rev: 33d79cb5d35274e2925d8acb0f30530013e37545)

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-08-31 17:57:12 +01:00
Peter Kjellerstedt 14effcc3d6 toaster.bbclass: Ignore some dependencies in toaster_buildhistory_dump()
When using RPM, depends.dot may contain dependencies such as
"/bin/sh", which will confuse _toaster_load_pkgdatafile(). Ignore
them. While at it, also ignore dependencies that contain parentheses,
e.g., "libc.so.6(GLIBC_2.7)".

(From OE-Core rev: 80c117f46442ef442e34b7681ed3688789f505ac)

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-08-31 17:57:11 +01:00
Peter Kjellerstedt 986de4b7f0 toaster.bbclass: Simplify parsing of depends.dot
By using a single regular expression, the parsing of the depends.dot
file can be simplified a lot. This should also make it less
susceptible to formatting changes in that file.

(From OE-Core rev: 20684149bb659b34d3bcac8f202cb95d607567c1)

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-08-31 17:57:11 +01:00
Peter Kjellerstedt e56ff20931 buildhistory.bbclass: Improve the generated depends.dot file
* Convert incorrectly formatted dependencies such as:
    "bar -> "foo" ">=" "1.2.3"
  into dependencies with edge labels:
    "bar -> "foo" [label=">= 1.2.3"]
* Remove rpmlib() and config() dependencies such as:
    "foo" -> "rpmlib(CompressedFileNames)" [label="<= 3.0.4-1"]
  and:
    "base-files" -> "config(base-files)" [label="= 3.0.14-r89.49"]
* Remove the trailing semicolon that was added to each line. It fills
  no purpose.

(From OE-Core rev: 99ef2f26cf498e1693a947bb44e40c31c20ec525)

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-08-31 17:57:11 +01:00
Peter Kjellerstedt b539afa655 insane.bbclass: Improve the handling of runtime file dependencies
This makes the file-rdeps test support:
* versioned dependencies, e.g., "perl (>= 5.000)", and
* package dependencies among the file dependencies, e.g., "perl".

It also ignores all "perl(...)" dependencies since it is expected that
these are generated and handled by rpm itself and there is no reason
to second guess what it is doing.

(From OE-Core rev: 6d64b110ed0c22f2bcd10cde437a13e03c15b23e)

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-08-31 17:57:11 +01:00
Peter Kjellerstedt 9d33b246ca insane.bbclass: Report all file-rdeps errors, not just the first
(From OE-Core rev: ee6f208a780062365e53fb9546eef55749e1c654)

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-08-31 17:57:11 +01:00
Mike Looijmans 9842f64154 rootfs-postcommands.bbclass: Prevent linking testdata to itself
testdata and testdata_link may point to the same file, in particular
when IMAGE_LINK_NAME and IMAGE_NAME are equal.

Check if this is the case before creating a symlink that points to
itself and makes the next build fail.

(From OE-Core rev: f8ceec50af68650d6548738e5b5582e2b25424d7)

Signed-off-by: Mike Looijmans <mike.looijmans@topic.nl>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-08-29 12:22:06 +01:00
Andre McCurdy 85c41bfcf2 mirrors.bbclass: provide git repo fallbacks using the https protocol
Use MIRRORS to provide git repo fallbacks using the https protocol,
for cases where git native protocol fetches may fail due to local
firewall rules, etc.

These rules should cover all git native repos used by recipes within
oe-core, with the exception of mtd-utils, for which there's currently
no upstream alternative to the git native protocol for anonymous
access ( see http://git.infradead.org/mtd-utils.git ).

(From OE-Core rev: fe71cf3ded7109e0fa1812475479941c89873e4e)

Signed-off-by: Andre McCurdy <armccurdy@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit abb8895d5b42a5dc171360a261a2652acd14ee7e)
Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-08-29 11:57:29 +01:00
Tom Rini 43435bc2e4 image: Fix "metadata is not deterministic" when chaining 2+ CONVERSION_CMDs
When we have more than one CONVERSION_CMD being used, for example
ext4.gz.sha256sum we will see errors about "metadata is not
deterministic".  This is because we do not have a stable order of
intermediate files that will be removed in the generated shell command.
We fix this by calling sorted() on the set of rm_tmp_images so that we
will have a stable hash again.

Cc: Patrick Ohly <patrick.ohly@intel.com>
(From OE-Core rev: 26feb0580642ef8934206ea73cdce9e1f73d14ef)

Signed-off-by: Tom Rini <trini@konsulko.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 98a2afeb3a53bec7a72a4a9846e1dba636cc6f3d)
Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-08-29 11:57:29 +01:00
Tom Rini 88eec32150 image.bbclass: Correct chaining compression support
When chaining of compression/conversion types was added, we had a new
way to handle doing things like "ext4.bz2.sha256sum" or
"ext2.gz.u-boot".  However, because the U-Boot image class isn't
included normally, it wasn't properly converted at the time.  After the
support was added the "clean" argument that the .u-boot code uses no
longer functions.  The fix for this inadvertently broke chaining
compression/conversion.  First, correct the u-boot conversion code.

Fixes: 46bc438374de ("image.bbclass: do exact match for rootfs type")
Cc: Zhenhua Luo <zhenhua.luo@nxp.com>
Cc: Richard Purdie <richard.purdie@linuxfoundation.org>
Cc: Patrick Ohly <patrick.ohly@intel.com>
(From OE-Core rev: ad22f73519292f53d2ea6eaae99f8a919c4f2c26)

Signed-off-by: Tom Rini <trini@konsulko.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 0a7ce0b971a208956cb895ba5a869ec8c5d94703)
Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-08-29 11:57:29 +01:00
Gan Yau Wai b4ea7bfad6 insane.bbclass: fix override handling in RDEPENDS QA
The package_qa_check_rdepends() in insane.bbclass has
incorrectly replace its localdata OVERRIDES value with
the package name. Fixing it by appending the package name
to the existing OVERRIDES value. This resolves RDEPENDS QA
error when setting PACKAGECONFIG using a pn- override at
local.conf.

Cherry picked from master 60d28dd72daee235150ab6605cbf953f1ea691df

[YOCTO #11374]

(From OE-Core rev: ca84390b7894adb32a89ccfbd83fbb3fb7e808fc)

Signed-off-by: Gan Yau Wai <yau.wai.gan@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-08-29 11:57:29 +01:00
Martin Jansa 4ec0cbceec icecc.bbclass: prevent nativesdk builds depending on target specific KERNEL_CC
* without this we cause nativesdk-linux-libc-headers to depend
  on target specific KERNEL_CC (through icecc_get_tool ->
  icecc_is_kernel -> KERNEL_CC -> HOST_CC_KERNEL_ARCH ->
  TARGET_CC_KERNEL_ARCH -> TUNE_FEATURES(thumb) as shown by
  bitbake-diffsigs:

  OE qemux86@ ~/build/oe-core $ ls /OE/build/oe-core/tmp-glibc/sstate-diff/1499859497/qemu*/*sdk*/*/*do_configure.sigdata*
  /OE/build/oe-core/tmp-glibc/sstate-diff/1499859497/qemuarm/x86_64-nativesdk-oesdk-linux/nativesdk-linux-libc-headers/4.10-r0.do_configure.sigdata.3a9a423878d56524e0ee8e42eba1804f
  /OE/build/oe-core/tmp-glibc/sstate-diff/1499859497/qemux86/x86_64-nativesdk-oesdk-linux/nativesdk-linux-libc-headers/4.10-r0.do_configure.sigdata.401071dbaa88903ece37d35a47965ff2

  OE qemux86@ ~/build/oe-core $ bitbake-diffsigs /OE/build/oe-core/tmp-glibc/sstate-diff/1499859497/qemu*/*sdk*/*/*do_configure.sigdata*
  basehash changed from 39774238b66763c598153132e87a2c1a to aa2d66e770bf533e312536eb0a401c4c
  Variable TARGET_CC_KERNEL_ARCH value changed from '${@bb.utils.contains('TUNE_FEATURES', 'thumb', '-mno-thumb-interwork-marm', '', d)} TUNE_FEATURES{thumb} = Set' to ''

(From OE-Core rev: b07bcfa3e4400f7763e2276f9be3dc702bb58c2e)

Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-08-29 11:57:29 +01:00
Nicolas Dechesne 05f7969366 kernel.bbclass: fix KERNEL_IMAGETYPE(S) for Image.gz
KERNEL_IMAGETYPES lists all the kernel images that we want to build. in
cb17b6c2a7 (kernel.bbclass: support kernel image type of vmlinux.gz), some logic
was added to support vmlinux.gz which is not a target built by kernel
makefiles (only vmlinux). It is clear that the goal of this logic is only to
support vmlinux.gz and not others compressed format (such as Image.gz) which are
valid target for kernel makefiles.

For Image.gz we should rely on the kernel makefiles and not do the compression
in kernel class.

This patch updates the logic used to filter out non supported kernel target from
KERNEL_IMAGETYPES, and make vmlinux.gz a 'special case', instead of *.gz. If
more special cases are needed in the future, we could add them in a similar way.

This patch should be a no-op for anyone using vmlinux or vmlinux.gz, and on top
of that it is fixing the build for Image.gz which was not working until now.

(From OE-Core rev: d3a89450ae918f467a99ac1c33502fc9a5eb4162)

Signed-off-by: Nicolas Dechesne <nicolas.dechesne@linaro.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit cfc0c897656fe67e81a6a5dcd936dff785529f41)
Signed-off-by: Nicolas Dechesne <nicolas.dechesne@linaro.org>
Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-08-29 11:57:28 +01:00
Mikko Rapeli 279e22c16b kernel.bbclass: set CVE_PRODUCT to linux_kernel if not set by recipe
It is used by NVD database CVE's like:

https://nvd.nist.gov/vuln/detail/CVE-2017-7273

Setting this in kernel.bbclass fixes CVE reporting for all users of
the class.

(From OE-Core rev: 9b27be77980fdd51423576712dfd6f4a031c3372)

Signed-off-by: Mikko Rapeli <mikko.rapeli@bmw.de>
Signed-off-by: Ross Burton <ross.burton@intel.com>
(cherry picked from commit 2e3d325440a50265c73f7d2e782530a02458bc33)
Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-08-29 11:57:28 +01:00
Mikko Rapeli 39e40d2490 cve-check.bbclass: use weak assignment for default CVE_PRODUCT
This way also bbclasses can override it. For example kernel.bbclass
could set CVE_PRODUCT to linux_kernel for all users of the class
which compile Linux kernels.

(From OE-Core rev: 478b90369c48351193e2bc5e2eb2b5308d55bb2e)

Signed-off-by: Mikko Rapeli <mikko.rapeli@bmw.de>
Signed-off-by: Ross Burton <ross.burton@intel.com>
(cherry picked from commit 74672a7de5ada45ab8e25b89cbdea3ec33b63b7f)
Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-08-29 11:57:28 +01:00
Paul Eggleton a19c27b807 classes/staging: change fixme debug message from note to debug
These messages were added for debugging during the recipe specific
sysroot work. They may still be useful but they don't need to be notes -
if they are they show up in recipetool / devtool output when fetching
source.

(From OE-Core rev: a0e93d5c5dcf59d1898a3db727a5ab2d75e3d20e)

(From OE-Core rev: 81246ed59215b8f62800cfb20d8b71d8e528c0ec)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit bc1a4af950a2600028d89b7dcb4ff8f409b52739)
Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-07-27 22:36:45 +01:00
Bruce Ashfield 912d9f531e kernel-yocto: propagate configuration errors to bbclass
As pointed out by klapperichpaul@johndeere.com, missing configuration
fragments were being picked up twice, once by the tools and once by the
bbclass. Unfortunately, the tools error message was being detected as
configs, and hence no error was reported at all.

Rather than catching the output of the tools, we can instead check the
return code and propagate the error message from the tools directly to
the user.

[YOCTO #11649]

(From OE-Core rev: 3470a3839577b99322c10f830cdaa61128ef6b16)

Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
(cherry picked from commit 960652416e2390337df6d9734375d6829ceb6420)
Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-07-27 22:36:45 +01:00
Paul Eggleton 00a859afd0 classes/buildhistory: fix failures collecting output signatures
It's possible for tasks to stage symlinks that point to non-existent
files; an example is ncurses-native.do_populate_sysroot. There wasn't
any error checking here so this broke the build when "task" was included
in BUILDHISTORY_FEATURES. In any case we shouldn't be following symlinks
and getting the sha256sum of the link target - we need concern ourselves
only with the target path, so check if the file is a link and sha256 the
target path instead if it is. If it's neither a regular file nor a
symlink (perhaps a pipe or a device), just skip it.

(From OE-Core rev: f60520d97f53dafe783f61eb58fe249798a1e1be)

(From OE-Core rev: 66a0d184d8f55a8da03de9fedb18d166b80b198b)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-07-27 22:36:43 +01:00
California Sullivan 7c27bf2dd8 systemd-boot.bbclass: Add configuration data to secondary EFI partition
The secondary EFI partition is used when booting in EFI mode, and
without the configuration data we don't get any boot targets.

Partial fix to [YOCTO #11503].

(From OE-Core master rev: 84aa7a00810e135fdad3f77bdb1da7d1f5fb8627)

(From OE-Core rev: 915b01258ef426392bb9052c345f952670db4450)

Signed-off-by: California Sullivan <california.l.sullivan@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-07-07 09:19:08 +01:00
Richard Purdie f01b909a26 package_ipk: Clean up Source entry in ipk packages
There is the potential for sensitive information to leak through the urls
there and removing it brings this into the behavior of the other package
backends since filtering it is likely error prone.

Since ipks don't appear to be generated at all if we don't set this, set
the field to the recipe name used (basename only, no paths). This avoids
information leaking. We may want to drop the field if opkg can allow that
at a future point but the recipe name is a suitable identifier for now.

Reported-by: Andrej Valek <andrej.valek@siemens.com>
(From OE-Core rev: 0b5e0d072f93a958e4211a8aeb2fd8cc3c25cc21)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-06-16 10:21:02 +01:00
Jonathan Liu bebb3d36f8 image-vm: Avoid use of fold, tac and paste commands for DISK_SIGNATURE
These commands are not whitelisted by the HOSTTOOLS variable which
silently prevents the MBR disk signature from being written to the
image.

Reported-by: Michael Davis <michael.davis@essvote.com>
(From OE-Core rev: 5527af688f6ccaacd7ec24d29425d0c007d5341c)

Signed-off-by: Jonathan Liu <net147@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-06-12 17:02:36 +01:00
Joshua Lock 607bd9ec39 kernel: predefine KBUILD_BUILD_USER and KBUILD_BUILD_HOST
By exporting KBUILD_BUILD_USER with a pre-defined value we improve the
reproducibility of the kernel and remove the requirement for whoami in the
HOSTTOOLS.
KBUILD_BUILD_HOST also helps improve the reproducibility of the kernel.

For more kernel reproducibility options see:
https://lwn.net/Articles/437864/

(From OE-Core rev: 357801a491efc067c6d4bd9a2bfa6fff460357aa)

Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-06-11 23:00:58 +01:00
Chen Qi d2586b6fde cve-check.bbclass: make warning contain CVE IDs
When warning users about unpatched CVE, we'd better put CVE IDs into
the warning message, so that it would be more straight forward for the
user to know which CVEs are not patched.

So instead of:
  WARNING: gnutls-3.5.9-r0 do_cve_check: Found unpatched CVE, for more information check /path/to/workdir/cve/cve.log.
We should have:
  WARNING: gnutls-3.5.9-r0 do_cve_check: Found unpatched CVE (CVE-2017-7869), for more information check /path/to/workdir/cve/cve.log.

(From OE-Core rev: ad46069e7b58f2fba373131716f28407816fa1a6)

(From OE-Core rev: e0e1414a4574d4165a8dc5d0d9d0d5b5a660355f)

Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-06-05 23:30:21 +01:00
Richard Purdie abae42362a staging: Allow BB_LIMITEDDEPS to avoid BB_TASKDEPDATA
In the limited dependency case we don't use any of the data from
BB_TASKDEPDATA. Restructure the code so this variable doesn't have
to be set. This allows the function to be called from other contexts
without creating artificial constructs. There should be no functional
change, behaviour remains unchanged.

(From OE-Core rev: 71e5243e3ebadb90b45fe418dac3eaa2c1b896bd)

(From OE-Core rev: e962e257f4c124869953d1fbb3da7dbf564f818a)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-05-18 13:17:44 +01:00
Richard Purdie 78747cc912 sstate: Ensure native/cross recipes have relocation of HOSTTOOLS_DIR
The previous change to relocate HOSTTOOLS wasn't complete as some files,
particularly in gcc stashed build directories were not being correctly
relocated. This patch addresses the issue.

(From OE-Core rev: 21dd36cc12a033b012544c5d15a6f8afd84dabc9)

(From OE-Core rev: 64c2f8acd02e0e5dca234b36a2a7097c0c16f7c2)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-05-18 13:17:44 +01:00
Maxin B. John 0fcbb4d0ec GNU_MIRROR: switch from ftp to https
Based on the same reason behind DEBIAN's switch from ftp:
https://www.debian.org/News/2017/20170425

(From OE-Core rev: ba119d836c0f4b20a39c92fa2e64abb0d5a55ad4)

Signed-off-by: Maxin B. John <maxin.john@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-05-11 16:56:00 +01:00
Maxin B. John 97e194e555 DEBIAN_MIRROR: switch from ftp to http
All public-facing debian.org FTP services will be shut down on November 1, 2017
The mirrors should just be accessed using HTTP instead.

https://www.debian.org/News/2017/20170425

Fixes [YOCTO #11413]

(From OE-Core rev: c2cdc4d9155d7a3b9cba60fa9cbb448cf64c62bd)

Signed-off-by: Maxin B. John <maxin.john@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-05-11 16:56:00 +01:00
Maxin B. John eb30ca2b31 useradd: remove preinst script referring to recipe sysroot
Remove recipe-specific-sysroot details from the preinst scripts
generated by useradd.bbclass.

This was added to match the default from bitbake.conf. Unlike the default
case, the dependencies used by useradd mean that a default passwd/group
file is always present. This means we don't need the native sysroot fallback.

Fixes [YOCTO #11460]

(From OE-Core rev: dfc9323c1cd7814989766be5bd1861fbaa739d2d)

Signed-off-by: Maxin B. John <maxin.john@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-05-11 16:56:00 +01:00
Peter Kjellerstedt 7b56a4327b useradd.bbclass: Handle COMPONENTS_DIR when restoring state
The export of PSEUDO in useradd_sysroot() contains references to
${COMPONENTS_DIR}. These need to be handled when restoring
postinst-useradd-${PN} from the sstate cache.

(From OE-Core rev: 097875bc9ab9d60a452b01ac6825775983684d68)

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-05-11 16:56:00 +01:00
Peter Kjellerstedt 439bb8cc71 bitbake.conf: Add COMPONENTS_DIR for ${STAGING_DIR}-components
The path to where to install and find the sysroot components is used
in many places. This warrants it to get its own variable.

(From OE-Core rev: 70a84b525470f72339568409daf84845904e4cab)

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-05-11 16:55:59 +01:00
Peter Kjellerstedt 1d57ca352f sstate.bbclass, staging.bbclass: Handle HOSTTOOLS_DIR when restoring state
Paths to host tools that have been copied to ${HOSTTOOLS_DIR} may end
up in the sstate cache. They thus need to be corrected when restoring
from the sstate cache.

(From OE-Core rev: f8671aecf05a286dd2b34b07bb5fbbe0c31e26d0)

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-05-01 08:56:35 +01:00
Peter Kjellerstedt 160c4fd9ae bitbake.conf: Add HOSTTOOLS_DIR for ${TMPDIR}/hosttools
The path to where to install and find the tools copied from the host
environment is already used in a couple of places. This warrants it to
get its own variable.

(From OE-Core rev: 8164c466943ffedff399009bf5547dba4f06d6c8)

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-05-01 08:56:35 +01:00
Andreas Oberritter 470995c3c6 package_deb.bbclass: Avoid writing empty custom fields
Avoids parser errors if PACKAGE_ADD_METADATA_DEB is set to an
empty value.

(From OE-Core rev: f0959c0908dfb386d29f13fcd3e57b2b004c6c14)

Signed-off-by: Andreas Oberritter <obi@opendreambox.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-29 11:17:23 +01:00
Andreas Oberritter a428745d15 package_deb.bbclass: Fix multi-line package descriptions
In deb control files, each line of a long description starts with
a single space. Empty lines are represented by a single space
followed by a single full stop character.

(From OE-Core rev: f66278f471c0bf9421ce2c55a56a144a0f9332bf)

Signed-off-by: Andreas Oberritter <obi@opendreambox.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-29 11:17:23 +01:00
Andre McCurdy 33f378ee33 cmake.bbclass: use weakest ??= assignment for default OECMAKE_SOURCEPATH
Make it slightly easier to support situations where the default path
needs to be over-ridden more than once.

(From OE-Core rev: 07390e3d45cdf244079a6b91175512ebac789da0)

Signed-off-by: Andre McCurdy <armccurdy@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-29 11:17:23 +01:00
Peter Kjellerstedt edb7672c27 cmake.bbclass: Do not use bitbake variable syntax for shell variables
Using bitbake variable syntax (i.e., ${FOO}) for shell variables is
bad practice. First of all it is confusing, but more importantly it
can lead to weird problems if someone actually defines a bitbake
variable with the same name as the shell variable.

Also use lower case for local shell variables.

(From OE-Core rev: ea6befae799f45cf93771442f242cb023dd809d1)

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-29 11:17:22 +01:00
Bruce Ashfield 929cebfb97 kernel-yocto/kern-tools: fix do_validate_branches clean stage
It was reported that do_validate_branches was failing with the following
error:

  Log data follows:
  | DEBUG: Executing shell function do_validate_branches
  | HEAD is now at fe0fb8d Merge tag 'v4.10.9' into standard/base
  | mkdir: cannot create directory .: File exists
  |
  | [ERROR] Can't find patch dir at ./patches/standard/base
  | usage: kgit s2q
  | WARNING: exit code 1 from a shell command.
  | ERROR: Function failed: do_validate_branches

This was triggered by the execution of 'kgit-s2q --clean' after forcing
the SRCREV to something other than the tip of the branch. --clean is
being run to remove any sentinel files from previous kernel builds to
ensure that the tree is in a consistent state.

There were two bugs, --clean was being executed and not exiting the
script as it was supposed to. Hence validation for applying patches
was done, and threw the error that eventually makes it to the console.

And the second bug is that since do_validate_branches actually calls
kgit-s2q --clean, the dependency on kern-tools-native needs to be on
that function (versus do_kernel_metadata which runs later).

With the tweaked kern-tool + the dependency fix, we no longer see this
error.

(From OE-Core rev: 4d5890b54cbdac01ee748759578b7b22ed8e61a2)

Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-29 11:17:22 +01:00