The recent change to connect through the shell logging functions caused
a regression - bb.error() and bb.fatal() cause a flag to be set
internally such that BitBake's UI will not print the full task log on
failure; unfortunately we have in a lot of places called die() or
bbfatal() within shell functions with a very terse message as a means of
exiting out, where we still want to see the full log (and we were
previously). We do still want to have fatal errors with proper messages
that don't result in the full log being printed, however we can't ignore
the typical usage of die(). Having added a mechanism to BitBake to log an
error and reset the flag, create a bbfatal_log() function that uses this
and call it from die() to restore the previous behaviour.
(From OE-Core rev: beec1cf3d22d7dbe85f332a055c72649f4bd3e92)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Some source archive are not standard format.
For example, sometimes, we still need decompress file once again after base_do_unpack,
in such case, the following warning will be caught.
WARNING: xxx : the directory ${WORKDIR}/xxxx (xxxxxx) pointed to by the S variable doesn't exist \
- please set S within the recipe to point to where the source has been unpacked to
So, we should do this QA check after all of unpack jobs been completed.
(From OE-Core rev: 81216da9fa90dc8a67a0f367742b1e73afec0aa1)
Signed-off-by: Bian Naimeng <biannm@cn.fujitsu.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The sstate checksums should be independent of whichever NATIVELSBSTRING is
detected. Add an automated QA test which tests this using bitbake -S.
To make this possible, we need to be able to override the value of
NATIVELSBSTRING so make a small change to allow this.
(From OE-Core rev: 2da156d491caf25dfa3efd567d6dbb451dd7e9dc)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Rather than just use d.getVar(X), use the more explict d.getVar(X, False)
since at some point in the future, having the default of expansion would
be nice. This is the first step towards that.
This patch was mostly made using the command:
sed -e 's:\(getVar([^,()]*\)\s*):\1, False):g' -i `grep -ril getVar *`
(From OE-Core rev: ab7c1d239b122c8e549e8112c88fd46c9e2b061b)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There may be typos or out of date values in PACKAGECONFIG, check and
warn them.
(From OE-Core rev: be085657bbab34bb8a822682897f96871bb2d8f4)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
"MACHINE=qemux86-64 bitbake init-ifupdown; MACHINE=genericx86-64 bitbake init-ifupdown"
shows a rebuild when it would be expected. The reason is a LIC_FILES_CHKSUM which
contains file://${WORKDIR}, an absolute path which doesn't exist in the first build
but does in the second, causing a signature change and a rebuild.
Fix the problem by ignoring any file:// url which resolves since TMPDIR for
license file dependency purposes.
(From OE-Core rev: f27ddf0de23871fc72cfc31f514f0e144aaa2082)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In a similar way to http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=aa1438b56f30515f9c31b306decef7f562dda81f
and http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=527b28c10955daf0387597020d69593ce24bcaa4
there is a find race in base.bbclass.
For recipes with PACKAGES_remove = "${PN}", the find which removes
.la files can race against deletion of other directories in WORKDIR
e.g.:
find: '/home/autobuilder/yocto-autobuilder/yocto-worker/nightly-oe-selftest/build/build/tmp/work/qemux86_64-poky-linux/init-ifupdown/1.0-r7/sstate-build-populate_lic': No such file or directory
| WARNING: /home/autobuilder/yocto-autobuilder/yocto-worker/nightly-oe-selftest/build/build/tmp/work/qemux86_64-poky-linux/init-ifupdown/1.0-r7/temp/run.do_configure.6558:1 exit 1 from
| find /home/autobuilder/yocto-autobuilder/yocto-worker/nightly-oe-selftest/build/build/tmp/work/qemux86_64-poky-linux/init-ifupdown/1.0-r7 -name \*.la -delete
Fix the race in the same way.
[YOCTO #7522]
(From OE-Core rev: 90861b8908d254154f4d1d613471070df8013da8)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
PRINC is no longer needed since we have the PR service instead
which people should be using. Drop the PRINC code.
This has been deprecated with a warning and then an error for quite some
time so we can remove it now.
(From OE-Core rev: a4d530bcf55f03258078c10a123e2717444e1060)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This means you can have one gcc version for some gcc recipes
(e.g. crosssdk/nativesdk) and another gcc version for target code.
Also remove the preferred version entry from the default toolchains
list since the version issue is now handled automatically.
We also need to specifically handle gcc-source in the license handling
code since expanding ${PV} in the base class isn't possible. Since
gcc-source doesn't generate any packages directly this shouldn't be
an issue and whitelisting in this way is easiest (and matches the
rest of the toolchain handling).
(From OE-Core rev: 67db7182faf6742b0d971d61d8c5ba34f69d2e12)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Hopefully make it a bit clearer to the user which recipe needs fixing
and what needs to be done to fix it.
(From OE-Core rev: 50e75d0cd7190a009776d72a4ae5c3e835895e6b)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(From OE-Core rev: ddac4f3c1350c3a4e0ea4ae732de0689325d883f)
Signed-off-by: Jonathan Liu <net147@gmail.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Previously, files with absolute paths in LIC_FILES_CHKSUM such as
"file://${COMMON_LICENSE_DIR}/foo" would not result in a qa failure
when the license file changed.
To fix this problem, add any files with absolute paths from
LIC_FILES_CHKSUM to the file-checksums varflag, so that changes in the
license file are detected and cause the qa task for licenses to run.
[Yocto #6450]
(From OE-Core rev: 5df681259ae078c9d04d426fb5971eaa79d35ba5)
Signed-off-by: Randy Witt <randy.e.witt@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently base.bbclass is creating S if it's not created by unpacking
an archive or fetching a repository. If we avoid creating S we can detect
when S hasn't been set correctly, since it will not exist. Then we can tell
the user that they should set S to a proper value, instead of just failing
with odd errors in later tasks.
Besides removing the auto-creation of S this change will introduce a warning
if S is set incorrectly. The reason for not display an error and return
is due to all external layers that might have recipes that will fail otherwise
and that might be a bit to hard to start with. So use a warning until people
have had a chance to cleanup affected recipes.
[YOCTO #5627]
(From OE-Core rev: 0d84b9a8ba408d168cb8a92e895d2f7338d6da1b)
Signed-off-by: Petter Mabäcker <petter@technux.se>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The do_configure may eventually call 'make clean' when the sstate
signature does not match. We should respect EXTRA_OEMAKE when doing
so, so use 'oe_runmake' for it.
(From OE-Core rev: 29cd284cab63fb1f6e82ad90bc8c92c1bbcafa88)
Signed-off-by: Otavio Salvador <otavio@ossystems.com.br>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add check_license_format function that shows warning if LICENSE don't have
valid operators and also if have space separated entries without operator,
add check_license_format validation into base class.
[YOCTO #6758]
(From OE-Core rev: 346a023a42f127881476e760e8fa4e04303849b9)
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The whitelist processing in code in base.bbclass does not play well with
wildcards in INCOMPATIBLE_LICENSES. The code expects bad_licenses to
contain actual license names, not wildcards.
Add incompatible_license_contains to replace bb.utils.contains(
"INCOMPATIBLE_LICENSE", **, **, **, d)
[YOCTO #5592]
(From OE-Core rev: 3587653a8d8abc7cfed6a5c6ecfa72bee283e451)
Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The evalation order was incorrect in some situations (CLEANBROKEN=1 and
GNUmakefile exists) the clean would be executed incorrectly.
Add brackets to correct the logic.
(From OE-Core rev: a9e95c3cd771b16bbf4c9eaa05c79d7b4ca7272a)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This field is now internal and won't be seen.
(From OE-Core rev: 9a715be3b2b4396495f2f8c10dc28b33f206efe4)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Apparently 1.5 years of warnings isn't enough to get anyone to take any
notice. We therefore make this an error so people can ignore it for
another 6 months whereafter we can finally give up and remove the
obsolete code.
(From OE-Core rev: 0c21d06dea96ca5f19ac2e3136c66da582036a11)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
It turns out we have quite a number of Makefiles out there without a
clean target. Rather than have all cases code an empty do_configure, add
a CLEANBROKEN variable which when set to "1" will disable the attempt to
"make clean".
This patch also adjusts various recipes which either have this problem
fixed, or have been reported to have make clean failures.
(From OE-Core rev: 45b7d3add14eafc25da62bab68d4ae133f8dcb57)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When something rebuilds say due to ${baselib} changing or some
other key variable, software is often not rebuilt due to the
fact that make detects no dependency change.
By running "make clean" when these changes occur, we can at least try
and ensure the correct rebuilds happen. We use the same checksum check as
autotools to decide if things have changed or not.
(From OE-Core rev: d06f2c089e01b7c67699800a7a54a4225233da93)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Without this changing just the SRCREV won't re-fetch unless you embed the SRCREV
into PV.
The downside here is that every hash changes, so this causes a full rebuild.
(From OE-Core rev: a414b17e1d783ad68a2d0f7d5922967449c05797)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The code in base.bbclass to spread PREFERRED_PROVIDERS values
to multilibs doesn't work for things which rely on TARGET_PREFIX,
such as virtual/${TARGET_PREFIX}gcc. This is because the expansion
of TARGET_PREFIX produces the wrong value if executed prior to
the assignment of TARGET_VENDOR_virtclass-multilib-libxx, which
will always happen since that assignment doesn't happen until recipe
parsing, but the PREFERRED_PROVIDERS expansion is happening
around ConfigParsed.
To solve this, we make a couple of changes. First, the creation
of the TARGET_VENDOR override values is moved into a new ConfigParsed
event handler in multilib_global. Second, the preferred_ml_updates()
function's code is moved into that function too. It seems safe to
assume that PREFERRED_PROVIDER values only need to be spread to
other multilibs when multilibs are in use.
I don't think this directly affects any use cases that don't involve
third-party or alternative toolchains.
(From OE-Core rev: 513f72274460e54fd35dda5ef70fa42ba2b284f8)
Signed-off-by: Peter Seebach <peter.seebach@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(From OE-Core rev: f31316b96a97e72d866f12a8e4bb33eaaa54b923)
Signed-off-by: Robert P. J. Day <rpjday@crashcourse.ca>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If INCOMPATIBLE_LICENSE=GPL-3.0 but the recipe sets LICENSE=GPLv3, the current
code won't trigger because they're different strings.
Fix this by attempting to canonicalise every license name to a SPDX name, so
both names in this example become GPL-3.0.
[ YOCTO #5622 ]
(From OE-Core rev: 8998e13fc95f11d15c34fb09d8451a9d4b69f2f1)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
With the renaming of the cross packages, its no longer possible to use
endswith("-cross") and similar to detect cross packages. Replace these
references with other techniques.
This resolves certain build from sstate failures which were due to the
system believing cross packages were target packages and therefore
dependency handling was altered.
(From OE-Core rev: 91edf4cac223298e50a4b8e59dd19f1b272e3418)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Now that gcc-cross, gcc-crosssdk and others have expanded variables in
their names, the code that previous just applied to cross-canadian needs
expanding to cover the other cases. Improve the conditional and also
fix a bug where the multilib prefix wasn't being added into the generated
versions.
(From OE-Core rev: d2b18d7692229d4b87b38becf173b1f8b6e90025)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Unfortunately expandKeys() notices overlap between the SDK and TARGET
PREFERRED_PROVIDERS entries when we set things up for the SDK recipes.
To avoid this, delete one of the sets of keys to avoid supurious warnings.
(From OE-Core rev: 9068a65baf4a28d04efad9297b16a2883ac42689)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add in the TARGET_ARCH and SDK_ARCH suffixes from the gcc-cross changes.
The -intermediate toolchain parts were canned a while ago so drop them.
(From OE-Core rev: 8a733097bddcf0868fb5f8ca08ea59b2cbc4a3a9)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There were two ConfigParsed event handlers in base.bbclass, this merges
them together for small efficiency wins.
(From OE-Core rev: ff919ed132b543f70e9635be7a31f799aafcf8d6)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Its possible for classes listed in INHERIT directives to use things like
the oe.utils functions. If that happens the user sees a traceback since
the modules don't become available until the ConfigParsed event.
This change to use immediate expansion means that the oe modules become available
much sooner and can be used in the core classes, including within base.bbclass.
(From OE-Core rev: a9ecad713f37f2703e99c6b856207abeb6c5ad1f)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* it's useful to know by how much we need to bump PR in original recipe to preserve upgrade path
(From OE-Core rev: 6e1633eb246fee3dee154513efa410d0930ea386)
Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The PRINC logic is now deprecated, the PR server should be used to handle
the automatic incrementing of the PR (package release) field. Add a warning
message to alert users to the change. The functionality will be removed
in the next release cycle.
(From OE-Core rev: 6f8d01e0097ded355a1eb38724f11ee65401392e)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
I was asked why we had these copy and update_data calls. I can
offer no good explanation other than them perhaps once being needed
for reasons long since forgotten and superceeded.
With modern bitbake these should not be needed and shouldn't have been
for a long time. Lets therefore remove them and simplify the code.
(From OE-Core rev: 33746924bea27e6f4d85898fe37d3e07d4317a3a)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This solves a regression introduced by OE-Core:9b75f6a (kernel:
use oldnoconfig before yes '' | make oldconfig).
The original oe_runmake explicitly calls 'die' command in case of
failure so the fallback code never runs. The fallback code needs
to handle the oe_runmake return code to call the backward
compatible callback so we introduced a new command called
oe_runmake_call which is used by oe_runmake.
This does not change the functional behaviour of oe_runmake so it
avoids any change except for the code which does need to handle
the oe_runmake exit code.
(From OE-Core rev: 51707282fc2c5a74588b2d79b2229513a883924f)
Signed-off-by: Otavio Salvador <otavio@ossystems.com.br>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Unpacking an src.rpm uses rpm2cpio.sh, which requires 'file'.
Without this, builds of rpm on a host without 'file' installed will fail with
very strange messages.
(From OE-Core rev: 97e1d84e2d1a74791ce6af88ddc27963bc0e1bec)
Signed-off-by: Christopher Larson <kergoth@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
gcc has cross and target components with a shared workdir. The unpack umask
settings need to match for all of these. We need to use strings in each
case to ensure the sstate code matches them correctly.
This patch tweaks various things to ensure the change adding the unpack umask
change doesn't break the compiler builds.
(From OE-Core rev: 67162438ee9c402b23c32853af9d313949eb6e4a)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* when git checkouts files from fetched clone it respects system umask
and creates files with different permissions, if such files are copied
to packages, resulting target images have also different permissions
on them.
* we need reproducible builds across different builders with different
system umask, so set 022 umask
[YOCTO #5590]
(From OE-Core rev: c9289c506633ffe5c482000d8d225e45454c064d)
Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
PREFERRED_PROVIDER_virtual_pkg has been incorrectly set with more
than one multilib prefixes. For example, if we have two alternative
multilibs lib64 and lib32, PREFERRED_PROVIDER_virtual_pkg will be
set to lib32-lib64-pkg or lib64-lib32-pkg, depending on which
multilib shows up first in the list.
(From OE-Core rev: 17a432dc059e24ba10d4baec988828c0025a5e46)
Signed-off-by: Lei Liu <lei.liu2@windriver.com>
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The PACKAGECONFIG flags were iterated over using dict.items(), but this
returns the items in an undefined order. As this order determines the
EXTRA_OECONF append order, we can get EXTRA_OECONF which are functionally
equivalent, but whose contents differ, resulting in not using shared state
archives we should be using.
(From OE-Core rev: 843a5dd8f8f0461e286d9fdb3ba55205b4275f88)
Signed-off-by: Christopher Larson <kergoth@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
These have been deprecated for a long time, convert the remaining
references to the correct modules and prepare for removal of the
compatibility support from bitbake.
(From OE-Core rev: 6a39835af2b2b3c7797fe05479341d71a3f3aaf6)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
base.bbclass had code which handled the PNBLACKLIST in case of
multilib use. This is better to be done in the blacklist.bbclass so it
has all logic in a single place.
(From OE-Core rev: 04f0fefeccc1e7e3af8a1f741350492ae3f171fc)
Signed-off-by: Otavio Salvador <otavio@ossystems.com.br>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The build banner displayed at the start of builds can be misleading since
the data store has not been finalised. As easy way to illustrate this is
to use something like:
DEFAULTTUNE = "i586"
DEFAULTTUNE_<machineoverride> = "core2"
and the banner will display the i586 tune yet the core2 tune will be
used. We can avoid this if we finalise a copy of the data before
displaying it.
[YOCTO #4225]
(From OE-Core rev: bdce39f22a0e8c8e1cf237322657220e4201077d)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Now that bitbake supports masking events for event handlers, lets use
this so event handlers are only called for events they care about. This
lets us simplify the code indentation a bit at least as well as mildly
improving the event handling performance.
(From OE-Core rev: bff73743280f9eafebe4591f7368ead91a4eb74d)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The do_package_setscene task needs to depend on fakeroot in order to correctly
install its files.
We can whitelist the dependency in the sstate handling code for some
performance improvements since we only need this if we're installing the
package from sstate.
Also use an append operator in base.bbclass for clarity.
(From OE-Core rev: 0810ea2a72bdea67a3d8002c4e12fb20f45cf1d5)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Update older code to use modern exception handling syntax which
is the form accepted by python 3.
(From OE-Core rev: b010501cd089e649a68f683be0cf4d0aac90fbe3)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The use of SOC_FAMILY here is old code and SOC_FAMILY is now implemented by
MACHINEOVERRIDES behind the scenes. It therefore makes more sense to use
the replacement value in this code. Just like SOC_FAMILY, this is a ":"
delimited variable so we should iterate over the components, not use
the value directly.
Finally, MACHINEOVERRIDES contains MACHINE so we don't need to check that
directly.
This makes the functionality match what most users would expect it to do
and is also compatible with the way things previously worked.
(From OE-Core rev: 8ceef74dd4f662b4c7e3c170ce486e966ebebeff)
Signed-off-by: Otavio Salvador <otavio@ossystems.com.br>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Paul Eggleton
Bian Naimeng
Richard Purdie
Robert Yang
Jonathan Liu
Randy Witt
Petter Mabäcker
Otavio Salvador
Aníbal Limón
Hongxu Jia
Ross Burton
Peter Seebach
Robert P. J. Day
Martin Jansa
Koen Kooi
Christopher Larson
Lei Liu