We do a hard link to speed up sdk creation but if your sstate-cache is
across a file system boundary, this tries and fails. This patch catches
that error and does a copy instead.
(From OE-Core rev: fb9fdd7a74917cdcab039aa3a9a9944b18246fea)
Signed-off-by: brian avery <brian.avery@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If you set up a local mirror in SSTATE_MIRRORS then you can end up with
symlinks in SSTATE_DIR rather than real files. We don't want these
symlinks in the sstate-cache prodcued by gen-lockedsig-cache, so
dereference any symlinks before copying.
(From OE-Core rev: d65a6ee9e7a9c63b9a16bdb5025af8a7c6433c4f)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Use the new oe-check-sstate to filter the sstate artifacts shipped with
the extensible SDK by effectively running bitbake within the produced
eSDK and and getting it to tell us which tasks it will restore from
sstate. This has several benefits:
1) We drop the *-initial artifacts from the minimal + toolchain eSDK.
This still leaves us with a reasonably large SDK for this
configuration, however it does pave the way for future reductions
since we are actually filtering by what will be expected to be there
on install rather than hoping that whatever cuts we make will match.
2) We verify bitbake's basic operation within the eSDK, i.e. that
we haven't messed up the configuration
3) We verify that the sstate artifacts we expect to be present are
present (at least in the sstate cache for the build producing the
eSDK). Outside deletion of sstate artifacts has been a problem up to
now, and this should at least catch that earlier i.e. during the
build rather than when someone tries to install the eSDK.
This does add a couple of minutes to the do_populate_sdk_ext time, but
it seems like the most appropriate way to handle this.
Should mostly address [YOCTO #9083] and [YOCTO #9626].
(From OE-Core rev: 4b7b48fcb9b39fccf8222650c2608325df2a4507)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When copying the sstate-cache into the extensible SDK, if the source
path had a trailing / and the destination path did not, there would be a
missing / between the path and the subdirectory name, and you'd end up
with subdirectories like "sstate-cacheCentOS-6.7". There are functions
in os.path for this sort of thing so let's just use them and avoid the
problem.
(From OE-Core rev: 5eb8f15c48b5f39a10eb2b63b026cf1ebfd05533)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When constructing the sstate-cache directory for the extensible SDK,
we were copying in any matching native sstate packages, and as the
signature doesn't actually change when the distro changes (since
NATIVELSBSTRING is just a path separator for the artifacts and is not
part of the signature) we ended up copying duplicated packages when the
distro changed e.g. upon host distro upgrade. Only search in the
NATIVELSBSTRING-named subdirectory for native packages and the issue
goes away.
Fixes [YOCTO #8885].
(From OE-Core rev: 6c6baf6aa1823b8b20123f505e45c2768a193ad5)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* Print some status when running
* When incorrect number of arguments specified, print usage text
(From OE-Core rev: ac38d245878b618ddf56f9a68834d344500e45a6)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
As part of populating the sstate-cache with an artifact (.tgz file) we
create a temp file and then atomically move it to the final name. Due to
the glob used in this script such temp files were being matched, and
between the time they were matched and the time the script started
copying files, the temp file may have vanished.
This fixes random "No such file or directory" failures building the
extensible SDK on build setups where the sstate-cache directory is shared
amongst multiple build machines.
(From OE-Core rev: 588de5198c641ea1cfc3e01a6d129296bd2f706b)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Since this previously always tried to use hardlinks you couldn't have
the source and destination be on different devices. This change allows
for that and also prevents failure in situations where the files already
existed.
(From OE-Core rev: cf675896340ebed7c4830b93d791ddb08999031f)
Signed-off-by: Randy Witt <randy.e.witt@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
I've been giving things some thought, specifically why sstate doesn't
get used more and why we have people requesting external toolchains. I'm
guessing the issue is that people don't like how often sstate can change
and the lack of an easy way to lock it down.
Locking it down is actually quite easy so patch implements some basics
of how you can do this (for example to a specific toolchain). With an
addition like this to local.conf (or wherever):
SIGGEN_LOCKEDSIGS = "\
gcc-cross:do_populate_sysroot:a8d91b35b98e1494957a2ddaf4598956 \
eglibc:do_populate_sysroot:13e8c68553dc61f9d67564f13b9b2d67 \
eglibc:do_packagedata:bfca0db1782c719d373f8636282596ee \
gcc-cross:do_packagedata:4b601ff4f67601395ee49c46701122f6 \
"
the code at the end of the email will force the hashes to those values
for the recipes mentioned. The system would then find and use those
specific objects from the sstate cache instead of trying to build
anything.
Obviously this is a little simplistic, you might need to put an override
against this to only apply those revisions for a specific architecture
for example. You'd also probably want to put code in the sstate hash
validation code to ensure it really did install these from sstate since
if it didn't you'd want to abort the build.
This patch also implements support to add to bitbake -S which dumps the
locked sstate checksums for each task into a ready prepared include file
locked-sigs.inc (currently placed into cwd). There is a function,
bb.parse.siggen.dump_lockedsigs() which can be called to trigger the
same functionality from task space.
A warning is added to sstate.bbclass through a call back into the siggen
class to warn if objects are not used from the locked cache. The
SIGGEN_ENFORCE_LOCKEDSIGS variable controls whether this is just a warning
or a fatal error.
A script is provided to generate sstate directory from a locked-sigs file.
(From OE-Core rev: 7e14784f2493a19c6bfe3ec3f05a5cf9797a2f22)
(From OE-Core rev: 884d4fa3e77cf32836f14a113c11489076f4a84d)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
brian avery
Paul Eggleton
Ed Bartosh
Randy Witt
Richard Purdie