Unfortunately to implenent multiconfig support in bitbake some APIs
had to change. This updates code in OE to match the changes in bitbake.
Its mostly periperhal changes around devtool/recipetool
[Will need a bitbake version requirement bump which I'll make when merging]
(From OE-Core rev: 041212fa37bb83acac5ce4ceb9b7b77ad172c5c3)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This patch adds the notion of supporting multiple configurations within
a single build. To enable it, set a line in local.conf like:
BBMULTICONFIG = "configA configB configC"
This would tell bitbake that before it parses the base configuration,
it should load conf/configA.conf and so on for each different
configuration. These would contain lines like:
MACHINE = "A"
or other variables which can be set which can be built in the same
build directory (or change TMPDIR not to conflict).
One downside I've already discovered is that if we want to inherit this
file right at the start of parsing, the only place you can put the
configurations is in "cwd", since BBPATH isn't constructed until the
layers are parsed and therefore using it as a preconf file isn't
possible unless its located there.
Execution of these targets takes the form "bitbake
multiconfig:configA:core-image-minimal core-image-sato" so similar to
our virtclass approach for native/nativesdk/multilib using BBCLASSEXTEND.
Implementation wise, the implication is that instead of tasks being
uniquely referenced with "recipename/fn:task" it now needs to be
"configuration:recipename:task".
We already started using "virtual" filenames for recipes when we
implemented BBCLASSEXTEND and this patch adds a new prefix to
these, "multiconfig:<configname>:" and hence avoid changes to a large
part of the codebase thanks to this. databuilder has an internal array
of data stores and uses the right one depending on the supplied virtual
filename.
That trick allows us to use the existing parsing code including the
multithreading mostly unchanged as well as most of the cache code.
For recipecache, we end up with a dict of these accessed by
multiconfig (mc). taskdata and runqueue can only cope with one recipecache
so for taskdata, we pass in each recipecache and have it compute the result
and end up with an array of taskdatas. We can only have one runqueue so there
extensive changes there.
This initial implementation has some drawbacks:
a) There are no inter-multi-configuration dependencies as yet
b) There are no sstate optimisations. This means if the build uses the
same object twice in say two different TMPDIRs, it will either load from
an existing sstate cache at the start or build it twice. We can then in
due course look at ways in which it would only build it once and then
reuse it. This will likely need significant changes to the way sstate
currently works to make that possible.
(Bitbake rev: 5287991691578825c847bac2368e9b51c0ede3f0)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If you don't do this, with Python 3 you get a warning on exit under some
circumstances.
(Bitbake rev: 49502685df3e616023df352823156381b1f79cd3)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
unset VAR
will clear variable VAR
unset VAR[flag]
will clear flag "flag" from var VAR
(Bitbake rev: bedbd46ece8d1285b5cd2ea07dc64b4875b479aa)
Signed-off-by: Jérémy Rosen <jeremy.rosen@openwide.fr>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Rather than passing in a datastore to build on top of, use the data builder
object in the cache and base the parsed recipe from this. This turns
things into proper objects building from one another rather than messy
mixes of static and class functions.
This sets things up so we can support parsing and building multiple
configurations.
(Bitbake rev: fef18b445c0cb6b266cd939b9c78d7cbce38663f)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There are some cases we want to parse recipes without any cache
setup or involvement. Split out the standalone functions into
a NoCache variant which the Cache is based upon, setting the scene
for further cleanup and restructuring.
(Bitbake rev: 120b64ea6a0c0ecae7af0fd15d989934fa4f1c36)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Rather that the current mix of static and class methods, refactor
so that the cache has the databuilder object internally. This becomes
useful for the following patches for multi config support.
It effectively completes some of the object oriented work we've been
working towards in the bitbake core for a while.
(Bitbake rev: 7da062956bf40c1b9ac1aaee222a13f40bba9b19)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Needing to access these static methods through a class doesn't
make sense. Move these to become module level standalone functions.
(Bitbake rev: 6d06e93c6a2204af6d2cf747a4610bd0eeb9f202)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Simple refactoring to allow for multiconfig support.
(Bitbake rev: 266b848da40904446eb1d084bbdc5307a9b45197)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
With the introduction of multi-config and the possibility of distributed
builds we need arrays of workers rather than the existing two.
This refactors the code to have a dict() of workers and a dict of
fakeworkers, represented by objects. The code can iterate over these.
This is separated out from the multi-config changes since its separable
and clearer this way.
(Bitbake rev: 8181d96e0a4df0aa47287669681116fa65bcae16)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The current codepaths are rather confusing. Stop passing these
as parameters and use the ones from when the object is created.
(Bitbake rev: 8c992c148d9619b10eeae8bbd9376ecf408037a5)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
With some recent changes in the kern tools, we can drop some changes in
the yocto-bsp and yocto-kernel tools that ensured proper patching and
BSP inheritance.
In particular, we no longer need to signify the start of patching, and
we must instruct the tools that we only want configuration fragments
via inheritance .. no patches (since they are already applied).
(From meta-yocto rev: 34ed5eebd0b5baab98b6b2d7b3f06ca40932b37d)
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This error has occurred on the MinnowBoard Max and Turbot since its
inception. It supposedly indicates a non-working SD card reader, but
ours works fine. Whitelist the error.
(From OE-Core rev: d577028a1d756b70da056dee73df657cf8000baf)
Signed-off-by: California Sullivan <california.l.sullivan@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
These errors have been occuring since the introduction of the 4.4
kernel with no apparent functionality loss. Whitelist for now.
(From OE-Core rev: 47b9058994f15507fc18ce0b08ac82a4c052966e)
Signed-off-by: California Sullivan <california.l.sullivan@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
These errors can't be fixed without adding the firmware to the initramfs
and building it into the kernel, which we don't want to do for
genericx86-64. Since graphics still work acceptably without the firmware
blobs, just ignore the errors for that MACHINE.
(From OE-Core rev: d73a26a71b2b16be06cd9a80a6ba42ffae8412c4)
Signed-off-by: California Sullivan <california.l.sullivan@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Using kvm can provide significant speedups when running qemux86* machines
on an x86* host. Enabled by using the new QEMU_USE_KVM variable.
[YOCTO #9298]
(From OE-Core rev: ebac2c8d1fcd09ebce0659a4abb445e4f1c18571)
Signed-off-by: Bill Randle <william.c.randle@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
With the enhanced functionality, the term "compression" is no longer
accurate, because the mechanism also gets used for conversion
operations that do not actually compress data.
It is possible to remove this naming problem in a backward-compatible
manner by including COMPRESSIONTYPES in CONVERSIONTYPES and checking for
the old COMPRESS_CMD/DEPENDS as fallbacks.
[YOCTO #9346]
(From OE-Core rev: 9d68c024790850cab72ead1e3372a5fcec4ef7b0)
Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
grub-editenv edits the env block at runtime on a booted system. Other
tools can depend on it to configure a live system, for ex. to set next
boot mode upon reboot. By splitting grub-editenv, tools don't have to
depend on the entire grub package (grub-editenv just edits one file).
(From OE-Core rev: 24b832b6e31c4e358d0c7a0062b69f66469cdcee)
Signed-off-by: Alejandro del Castillo <alejandro.delcastillo@ni.com>
Signed-off-by: Ioan-Adrian Ratiu <adrian.ratiu@ni.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
It's simply been missing.
(From OE-Core rev: ebd1ea45e67211bd2ab0ec7affab409908126ef3)
Signed-off-by: André Draszik <git@andred.net>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We do that everywhere else, and otherwise anybody
extending SRC_URI through bbappend must know to
add a space at the end, which is an unusual
requirement.
(From OE-Core rev: 4e7c641b38296ff46ba56cc45e7b14c9e2aa4018)
Signed-off-by: André Draszik <git@andred.net>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We've been running with a set of kern-tools that were designed to work
with build systems that knew nothing about git, trees, commits, etc.
As such, there's been a set of shims/wrappers in place to work with
within bitbake/oe-core. These were the *me scripts: createme, updateme,
patchme and configme.
With this commit, we strip that legacy code and use the tools directly.
This means less complexity, fewer corner cases .. and no surprises
when the tools are arunning. As another benefit, the tools consume
much less time during a typical build and have no noticeable impact
on the overall build time.
Existing .scc files, features, and processing are not impacted as
these tools are compatible with existing feature descriptions and
kerne configuration fragments.
The audit of kernel configuration fragments is now detached
from the linux-yocto build structure and process. This means that
they can eventually be tweaked to offer kernel audit to any type of
kernel build and configuration process.
Additionally, the kernel symbol audit phase can now resolve symbol
dependencies and offer guidance when a symbol is missing:
WARNING: linux-yocto-4.4.15+gitAUTOINC+b030d96c7b_f5e2c49d58-r0 do_kernel_configcheck: [kernel config]: specified values did not make it into the kernel's final configuration:
---------- CONFIG_BT_6LOWPAN -----------------
Config: CONFIG_BT_6LOWPAN
From: /home/bruce/poky/build/tmp/work-shared/qemux86-64/kernel-source/.kernel-meta/configs/standard/features/bluetooth/bluetooth.cfg
Requested value: CONFIG_BT_6LOWPAN=y
Actual value:
Config 'BT_6LOWPAN' has the following conditionals:
BT_LE && 6LOWPAN (value: "n")
Dependency values are:
BT_LE [y] 6LOWPAN [n]
(From OE-Core rev: 0f698dfd1c8bbc0d53ae7977e26685a7a3df52a3)
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The existing kernel patching scripts don't like () in patch names, since they
are detected as function calls. Although the scripts will be updated to avoid
this error, it is worthwhile fixing the patch names in the meantime.
(From OE-Core rev: de7e4da0c7abf5dcd8b95ec993e70041475603c2)
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Integrating the following configuration changes:
features: usb-net: provide more coverage on USB network devices
features: broxton: enable iTCO watchdog support
features: broxton: enable iSMT support
features: broxton: enable LPC bridge function for Intel ICH and SCH
(From OE-Core rev: 02165c6bd9da6ac3a34eabe17d3a068afb6b1727)
Signed-off-by: Bruce Ashfield <bruce@zedd.org>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There was an issue with a netfilter backport in 4.1.28-stable. To
address it, we backport the -stable fix:
netfilter: x_tables: fix stable backport
Stable-4.1 backport of mainline commit 364723410175 ("netfilter:
x_tables: validate targets of jumps") doesn't handle correctly the fact
that 4.1 kernel is missing commit 482cfc318559 ("netfilter: xtables:
avoid percpu ruleset duplication") so that t->entries is still a per-cpu
array in find_jump_target().
Use the same fix as e.g. stable-3.14 backport.
Fixes: 8163327a3a92 ("netfilter: x_tables: validate targets of jumps")
Signed-off-by: Michal Kubecek <mkubecek@suse.cz>
(From OE-Core rev: c009297d44df98ba103ee267e40ffdbc837e411f)
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Upgrade dbus from 1.10.8 to 1.10.10.
(From OE-Core rev: e5581343303f2cf8724019c3cbfb92a87045a7f1)
Signed-off-by: Dai Caiyun <daicy.fnst@cn.fujitsu.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This at least partially addresses one of the build races we've seen
on the autobuilder in busybox. Its a straightforward backport from
upstream.
(From OE-Core rev: 8599059164ad0eb908fd1177044af8bc9a9881e4)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
gobject-introspection relies upon prelink-rtld. In order to function correctly,
we generate an ld.so.conf file which is generated before users of prelink-rtld
are called.
There is currently a race in gobject-introspection since the configuration file
may not have been created. This adds in code to ensure that regardless of codepath
(new build, existing build, from sstate), we trigger the creation of the configuration
file and avoid build failures.
(From OE-Core rev: 10e0c1a3a452baa05d160a92a54b2e33cf0fd061)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Changelog:
http://www.alsa-project.org/main/index.php/Changes_v1.1.1_v1.1.2
The FFT code in alsabat changed from double precision to single
precision floating point numbers, which is why the fftw dependency
changed to fftwf.
(From OE-Core rev: 2b44e468d20a0256fba896562e2e7d1ae593a4c8)
Signed-off-by: Tanu Kaskinen <tanuk@iki.fi>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Drop the self-baked force_rm() method.
(From OE-Core rev: c86bf80abd87acb0da5860806822c64ec9dee089)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Log failures of commands whose output is stored.
(From OE-Core rev: 240f6e7366c8a9ea830e531d307dd2e27a61a6bd)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Remove timestamps from the stderr log in order to make the console
output more readable, i.e. more in line with the output from unittest
runner.
(From OE-Core rev: d28eeeabde9b4b7160a273445023a44fd50e29ab)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
So that the log file would not miss any records.
(From OE-Core rev: 9ce6e20ce239067896dc65f09e3fef1173293065)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Prevent logger from writing to stderr when the tests are being run by
the TestRunner. During this time the logger output is only written to
the log file. This way the console output from the script is cleaner and
not mixed with possible logger records.
(From OE-Core rev: 36f58b5172d4e2e182aa447fb3ec4d1ac9f6820d)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Special runCmd() for build perf tests which doesn't raise an
AssertionError when the command fails. This causes command failures to
be detected as test errors instead of test failures. This way "failed"
state of tests is reserved for future making it possible to set e.g.
thresholds for certain measurement results.
(From OE-Core rev: 09590ac76a19ee1b1b4a9188f7fce5029f0de52a)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Convert scripts/oe-build-perf-test to be compatible with the new Python
unittest based buildperf test framework.
(From OE-Core rev: 249d99cd7ec00b3227c194eb4b9b21ea4dcb7315)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This commit converts the actual tests to be compatible with the new
Python unittest based framework.
(From OE-Core rev: 4e81967131863df7ee6c8356cb41be51f1b8c260)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The new class is derived from unittest.TextTestResult class. It is
actually implemented by modifying the old BuildPerfTestRunner class
which, in turn, is replaced by a totally new simple implementation
derived from unittest.TestRunner.
(From OE-Core rev: 89eb37ef1ef8d5deb87fd55c9ea7b2cfa2681b07)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Rename BuildPerfTest to BuildPerfTestCase and convert it to be derived
from TestCase class from the unittest framework of the Python standard
library. This doesn't work with our existing testcases or test runner
class and these need to be modified, too.
(From OE-Core rev: b0b434210a3dbd576f68344e29b8c20d18561099)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add logic for the cases when the time retrieved does
not have decimal part.
(From OE-Core rev: a6c9e515f8bc590612e3082ab1c4c254711c8e3b)
Signed-off-by: Jose Perez Carranza <jose.perez.carranza@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Another instance where expected failures need to be not reported to the error
reporting service.
(From OE-Core rev: bb1cbb8d5bd7639554edcddf1d2eac4abdbb48c7)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When an image uses systemd journald acts as a main syslog daemon using
/dev/log.
The test_syslog_logger try to log a predifined message into the syslog
using logger and then search using grep in /var/log/messages if this
fails for some reason (file rotated) now search the predifined message
into the journal.
(From OE-Core rev: 26d7e5060a35d20df6f2586b70ed8d2853cc0186)
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This patch adds the following packages: python3-enum (needed by python3-git),
python3-selectors (needed by python3-subprocess), python3-signal (needed by python3-subprocess),
and it also fixes the following ones with missing dependencies: python3-subprocess,
python3-compression, python3-datetime
[YOCTO #10127] [YOCTO #10124] [YOCTO #10122]
(From OE-Core rev: 0575e8c9fb52a7b594025fd20445a2edd06e3c69)
Signed-off-by: Alejandro Hernandez <alejandro.hernandez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie
Paul Eggleton
Jérémy Rosen
Bruce Ashfield
California Sullivan
Bill Randle
Patrick Ohly
Alejandro del Castillo
Dengke Du
André Draszik
Dai Caiyun
Tanu Kaskinen
Markus Lehtonen
Jose Perez Carranza
Ross Burton
Aníbal Limón
Alejandro Hernandez
Hongxu Jia