In order to support workflows using devtool where a user might want to
modify tasks that exist in locked-sigs.inc, there must be a way to unlock
recipes.
This patch adds that support by allowing the user to add recipes to
SIGGEN_UNLOCKED_RECIPES. Recipes that exist in that variable will have
all their tasks unlocked, as well as any tasks that depend on that
recipe.
For example if foo->bar->baz, if you unlock baz, it will also unlock bar
so that foo can be rebuilt without explicitly specifying bar as being
unlocked.
[YOCTO #9195]
(From OE-Core rev: 8a8fc54d824767a6a94d12a4ace98b0bdbb1aa25)
Signed-off-by: Randy Witt <randy.e.witt@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The previous message when signatures didn't match between the metadata
and the locked signatures file, the message output was a bit confusing.
Now the message should be of the form:
The zlib-native:do_install sig is computed to be
53531910a2a7848432da89def942a91a, but the sig is locked to
d25ba9035f7ccb308e51bbe1066e8d27 in SIGGEN_LOCKEDSIGS_t-x86-64
which will hopefully be more useful in understanding the problem.
[YOCTO #9195]
(From OE-Core rev: 49eeabfff8bbea69401db41f7219e29acf47af73)
Signed-off-by: Randy Witt <randy.e.witt@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add the SIGGEN_LOCKEDSIGS_TASKSIG_CHECK and
SIGGEN_LOCKEDSIGS_SSTATE_EXISTS_CHECK variables to replace
SIGGEN_LOCKEDSIGS_CHECK_LEVEL.
SIGGEN_LOCKEDSIGS_TASKSIG_CHECK will no control whether there is a
warning or error if a task's hash in the locked signature file doesn't match
the computed hash from the current metadata.
SIGGEN_LOCKEDSIGS_SSTATE_EXISTS_CHECK will control whther there is a
warning or error if a task that supports sstate is in the locked
signature file, but no sstate exists for the task.
Previously you could only have warning/errors for both controlled by
SIGGEN_LOCKEDSIGS_CHECK_LEVEL. This was an issue in the extensible sdk,
because we know sstate won't exist for certain items in the reverse
dependencies list for tasks. However, we still want to error if task
signatures don't match.
[YOCTO #9195]
(From OE-Core rev: 0fe2a5e5ffd01e926d0f3d4c78ad9910296e2d1a)
Signed-off-by: Randy Witt <randy.e.witt@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
opkg does not return a non-zero exit code even if it found
errors. When that happens, parsing the output leads to strange
follow-up errors.
To avoid this we need to check explicitly for non-empty
stderr. Reporting only that on a failure also leads to shorter error
messages (stdout may be very large).
(From OE-Core rev: 7d9e915224a9bc451fddfbbfad533d9b06e9987d)
Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Show the actual command that failed when raising a CmdError. Makes
figuring out what actually failed much easier.
[YOCTO #9344]
(From OE-Core rev: 8e9c03df1810daab7171733f1713ef94d3a18ab2)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Using PF to calculate the rpm filename doesn't work when PR server is
enabled and an extra PR value can be injected. Add code to use packagedata
to obtain the full name, allowing the test to work when PR server is
in use.
(From OE-Core rev: 322904f62f11e794543362f04212242567c556a0)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
So that the packagedata module can be used externally to the core OE
environment, add a missing import.
(From OE-Core rev: da4df2313c8df92cf321a7631a9a389f895d4615)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Previously the list of packages that are considered unneeded for a
read-only rootfs was hardcoded. This made it impossible to, e.g., have
shadow installed on a system with a read-only rootfs, but where /etc
is mounted writable.
This also lists ${VIRTUAL-RUNTIME_update-alternatives} rather than
update-alternatives (as was previously the case) since this should
actually remove the intended package.
(From OE-Core rev: e3b881d4168e5b02ff00f5c470ba472ab8bbc747)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There appears to have been a lot of copy and pasting of the code
which prints tracebacks upon failure and limits the stack trace to
5 entries. This obscures the real error and is very confusing to the user
it look me an age to work out why some tracebacks weren't useful.
This patch removes the limit, making tracebacks much more useful for
debugging.
[YOCTO #9230]
(From OE-Core rev: 6069175e9bb97ace100bb5e99b6104d33163a3a2)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When systemd is enabled as init we need to notice when circular
dependencies in units happen because systemd try to solve this situation
removing the unit itself.
(From OE-Core rev: 04b8fcc95f339282edc9ab405d0ba0e51dbc1d91)
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
BitBake should append to recipe in a predictable order.
fix for [YOCTO #9145]
test for [YOCTO #9138]
(From OE-Core rev: 51bef86ce52fdc2455cd1879e3e19d5ccc5c1c9c)
Signed-off-by: Daniel Istrate <daniel.alexandrux.istrate@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Without this, do_package_write_rpm doesn't depend on rpm-native which
it really should since that is needed to build rpms.
[YOCTO #8047]
(From OE-Core rev: 3fab4f9920d004fe13fb01434d4c7f3b8bbd7895)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Most of wic functionality doesn't depend on .wicenv file,
so it's better to generate it only in test_image_env
test case where it's used.
(From OE-Core rev: caf9b41e1db7b565ef977200195d57b385127de9)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The algorithm was sub-optimal so replace it with something more elegant.
(From OE-Core rev: 6119a90173f9222efa6df25aacf873af85d64bcd)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We need the wic env files to be available and this no longer happens automatically
so ensure we have them by specifying a specific task dependency.
(From OE-Core rev: 15e10957a0c9f65eaa119f8cb4f817c2fe3d580f)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Now that make/remake needs a PREFERRED_PROVIDER, we need to set this
correctly during tests.
(From OE-Core rev: 1a41953331f42d69c0201dcfcbb7d8dc12422fde)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
It's useful to know if a binary is statically or dynamically linked, so add a
method to determine this.
(From OE-Core rev: 96813445e6618fd8442600d81e53c448310b6e8b)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(From OE-Core rev: ce14964d99741f1a4579bae18da5013498c365fd)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The python-expect is not installed on the distro such as Ubuntu by
default, and we can get rid of it.
Use RPM_GPG_PASSPHRASE to replace of RPM_GPG_PASSPHRASE_FILE which is
more straightforward.
(From OE-Core rev: 4a8a74c62836a20610daf029d4cec0b3087758b2)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When we add from a fetched URL we are supposed to turn the resulting
source tree into a git repository (if it isn't already one). However, we
were using the older deprecated option name here instead of the
positional argument, so "devtool add -f <url>" resulted in the repo
being created but "devtool add <url>" didn't, which was wrong.
Also update the oe-selftest tests to check that this worked.
(From OE-Core rev: a7b6b1f8cc1c096724f794ac9dee312b0f771f66)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Adds intel-corei7-64 with x86-common, this makes the x86-common
whitelist available for intel-corei7-64 bsp also.
[YOCTO #9179]
(From OE-Core rev: 34e7292fb40635cee1f1237ac3156530f8dfce37)
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This message appears on older hardware and is a benign warning
[YOCTO #9179]
(From OE-Core rev: e941853e3dd9e498dcf486686d30cd07d65d83fb)
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We're dropping RPM 4 as it's not well maintained, therefore remove
a selftest which tests RPM 4 availability.
(From OE-Core rev: ed005dd51c121c27a89a878bfc6abfca496bf51c)
Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Simplify the RPM code by removing support for RPM 4 now that we've
dropped the RPM 4 recipe.
(From OE-Core rev: 7db6f0a402948ce489bafadf2e389802f764f122)
Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a test to build core-image-sato with read-only-rootfs enabled.
[ YOCTO #9214 ]
(From OE-Core rev: c23dc788386a8d3636f7f656667dc87052cf73d9)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add DL_DIR and TIME to the hash tests. We can't add DATE for some reason.
(From OE-Core rev: 206a95065628a839c589452de7aa646c90e02f5d)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
'test_layer_git_revisions_are_displayed_and_do_not_fail_without_git_repo'
was renamed to 'test_layer_without_git_dir' which is shorter.
fix for [YOCTO #9243]
(From OE-Core rev: 7bd990e635e0b41f4ab9d71695a0309b5302178f)
Signed-off-by: Daniel Istrate <daniel.alexandrux.istrate@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This new file is encapsulating functionality for both
running tests with binaries support via TestNeedsBin() decorator
and exporting these binaries via testimage.bbclass file.
Addresses [YOCTO #7850], [YOCTO #8478], [YOCTO #8481],
[YOCTO #8536], [YOCTO #8694].
(From OE-Core rev: 14640f16b5ce09a14f88b3fa641d4cf2780f8b97)
Signed-off-by: Costin Constantin <costin.c.constantin@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This is not the correct way to handle this - it significantly increases
the time taken to run oe-selftest anywhere MACHINE is set to some other
value (for example "qemux86-64"), because all of the artifacts for
qemux86 need to then be built as well when running the test. If we need
to skip these tests on non-QEMU machines, the devtool test already
demonstrates how to do that.
This reverts commit 169e1eaa4fc5ed03e2307b68686a7f5b1db37a36.
(From OE-Core rev: f60da6d5be5469ed5b834759c3822471f81fcdd2)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The extra directory next to the recipe should only be created if there
are files to put into it; currently only the npm plugin does this. I
didn't notice the issue earlier because the test was actually able to
succeed under these circumstances if the recipe file came first in the
directory listing, which was a fault in my original oe-selftest test;
apparently on some YP autobuilder machines the order came out reversed.
With this change we can put the oe-selftest test that highlighted the
issue back to the way it was, with an extra check to reinforce that only
a single file should be created.
(From OE-Core rev: b8b778345eb0997c2cd952a1f61fdd2050b6b894)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Correctly handle the any other files/directories that may exist
during the test.
======================================================================
FAIL: test_recipetool_create_simple (oeqa.selftest.recipetool.RecipetoolTests)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-oe-selftest/build/meta/lib/oeqa/utils/decorators.py", line 106, in wrapped_f
return func(*args, **kwargs)
File "/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-oe-selftest/build/meta/lib/oeqa/selftest/recipetool.py", line 414, in test_recipetool_create_simple
self.fail('recipetool did not create recipe file; output:\n%s\ndirlist:\n%s' % (result.output, str(dirlist)))
AssertionError: recipetool did not create recipe file; output:
NOTE: Fetching http://www.dest-unreach.org/socat/download/socat-1.7.3.0.tar.bz2...
NOTE: Unpacking /srv/www/vhosts/autobuilder.yoctoproject.org/current_sources/socat-1.7.3.0.tar.bz2 to /tmp/recipetool-Uj7MIh/
NOTE: Recipe /tmp/recipetoolqaebTo9s/recipe/socat_1.7.3.0.bb has been created; further editing may be required to make it fully functional
dirlist:
['socat', 'socat_1.7.3.0.bb']
(From OE-Core rev: 4be0e15f74cff85edca9de55248939fb438f30ae)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When writing the index using ipk packages there could be a race condition
when populate the index. This happens because the architectures
are repeated (specially all) and the commands generated to write the index
run in parallel.
This change avoid the duplication of commands using a set instead of a list.
[YOCTO #8924]
(From OE-Core rev: 74adb14b0002e20099cc2c34e01862e8ddb8e013)
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signing package feeds will default to ascii armored signatures (ASC) the
other option being binary (BIN). This is for both rpm and ipk backends.
(From OE-Core rev: 862a3892feb2628282e1d6f2e4498a7a3bd60cbf)
Signed-off-by: Ioan-Adrian Ratiu <adrian.ratiu@ni.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Create gpg signed ipk package feeds using the gpg backend if configured
(From OE-Core rev: a2ee831cfb688bc64c071f75a1dff8a963abe287)
Signed-off-by: Ioan-Adrian Ratiu <adrian.ratiu@ni.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add support for multiple types of signatures (binary or ascii)
in export_pubkey(). There is no change in behaviour for the function,
the previous implicit default is the new parameter "armor" default.
(From OE-Core rev: 95ba4a982b887444908207e3180fe4bc46281d3b)
Signed-off-by: Ioan-Adrian Ratiu <adrian.ratiu@ni.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Starting from v2.1 passing passwords directly to gpg does not work
anymore [1], instead a loopback interface must be used otherwise
gpg >2.1 will error out with:
"gpg: signing failed: Inappropriate ioctl for device"
gpg <2.1 does not work with the new --pinentry-mode arg and gives an
invalid option error, so we detect what is the running version of gpg
and pass it accordingly.
[1] https://wiki.archlinux.org/index.php/GnuPG#Unattended_passphrase
(From OE-Core rev: 0413bd8e294ca8ac972ac68662b43a981952f5ae)
Signed-off-by: Ioan-Adrian Ratiu <adrian.ratiu@ni.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Implement ipk signing inside the sign_ipk bbclass using the gpg_sign
module and configure signing similar to how rpm does it. sign_ipk uses
gpg_sign's detach_sign because its functionality is identical to package
feed signing.
IPK signing process is a bit different from rpm:
- Signatures are stored outside ipk files; opkg connects to a feed
server and downloads them to verify a package.
- Signatures are of two types (both supported by opkg): binary or
ascii armoured. By default we sign using ascii armoured.
- Public keys are stored on targets to verify ipks using the
opkg-keyrings recipe.
(From OE-Core rev: a40f27aa7802e8a0bd87a5417e35adbface62d05)
Signed-off-by: Ioan-Adrian Ratiu <adrian.ratiu@ni.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently the recipe files are hardcoded and if the recipe
change the version, the test will fail.
This will change from using a harcoded file to look for the
file using bitbake-layers. Now, just the recipe name must
be specified.
(From OE-Core rev: 1ee24e435353d93374895eead81fb281e1338739)
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently when using a git version the check for the stamp, using regex,
will fail because of plus sign in the version.
With this change the version is escaped before adding it to the regex.
(From OE-Core rev: 1aefa6a4dec84a5581aab70451bb84801b3b3615)
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Rather than rolling all of an npm module's dependencies into the same
package, split them into one module per package, setting the SUMMARY and
PKGV values from the package.json file for each package. Additionally,
mark each package with the appropriate license using the license
scanning we already do, falling back to the license stated in the
package.json file for the module if unknown. All of this is mostly in
aid of ensuring all modules and their licenses now show up in the
manifests for the image.
Additionally we set the main LICENSE value more concretely once we've
calculated the per-package licenses, since we have more information at
that point.
(From OE-Core rev: 8226805f83d21e7c1d2ba21969f3e8ee4b137496)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
do_shared_workdir is not a proper sstate task, it always reruns if
needed, so special-case it in warnings when checking locked sigs.
(From OE-Core rev: 4b08f982a2b15bff9092f60f7957301bb2d2108b)
Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
meta/lib/oe/package_manager.py was also updated. This ensures that any
diagnostic messages are ignored from the output of rpmresolve.
The patches have been split into bug fixes (things that belong upstream)
and local changes that are OE specific.
The following patches are obsolete and have been removed:
rpm-remove-sykcparse-decl.patch
fstack-protector-configure-check.patch
rpm-disable-Wno-override-init.patch
rpm-lua-fix-print.patch
rpm-rpmpgp-fix.patch
verify-fix-broken-logic-for-ghost-avoidance-Mark-Hat.patch
(From OE-Core rev: ee97e53fcceabc6ef4ddc68f38c5fa0e05c5d9a8)
Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Test that layer git revisions are displayed and
do not fail without git repository.
fix for [YOCTO #8852]
(From OE-Core rev: 8adaad7f3a76d527f34d2caa4b032beba7e21840)
Signed-off-by: Daniel Istrate <daniel.alexandrux.istrate@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Prelink contains some hardcoded assumptions about the path layout of
the target system. Unfortunately if the system doesn't match, prelink
doesn't work. This breaks:
a) prelink of those images
b) the unsafe-references-in-binaries QA test (which uses prelink-rtld)
One way to work around this is to construct an ld.so.conf file which
lists the library paths in question. We do this in sanity QA check and
in the rootfs prelink code, being careful not to trample any existing
target ld.so.conf.
There is an additional problem that $LIB references in RPATHs won't be
handled correctly, I've not see any system use these in reality though
so this change at least improves things.
(From OE-Core rev: 7fd1d7e639c2ed7e0699937a5cb245c187b7c811)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a build-sdk command which is only available within the extensible
SDK that builds a derivative extensible SDK. The idea is recipes in the
workspace become a part of the new SDK - for example, this allows taking
a vendor provided SDK, adding a few libs and then producing a new SDK
with those included.
When normally building the extensible SDK, the workspace is excluded;
here we need to copy into the new SDK (renaming it in the process); the
recipes' task signatures become locked and thus the sources are no
longer needed, so they are removed along with the workspace bbappends
which would interfere with the locked signatures. Additionally we need
to just copy the configuration files (i.e. local.conf and auto.conf)
rather than filtering and appending to them since that work has already
been done when constructing the original SDK. The extra sstate artifacts
from workspace recipes are also determined and copied into the new SDK
in minimal mode (on the assumption that you won't set up a new sstate
mirror).
This reuses some code from build-image, so that needed to be
generalised to allow that.
Implements [YOCTO #8892].
(From OE-Core rev: 59e207ff6dd4b50a8905e14bc9292cf2794f4e7a)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Randy Witt
Patrick Ohly
Markus Lehtonen
Richard Purdie
Peter Kjellerstedt
Aníbal Limón
Daniel Istrate
Robert Yang
Ed Bartosh
Ross Burton
Paul Eggleton
Saul Wold
Joshua Lock
Costin Constantin
Mariano Lopez
Ioan-Adrian Ratiu
Mark Hatle