Commit Graph

157 Commits

Author SHA1 Message Date
Alexandru DAMIAN da4c46fc78 sstate: list missing files for toaster
Toaster needs to record the attempts to restore
setscene tasks that don't have a sstate file.

We build a list of tasks for which we can't find an
sstate file, and if we're running under Toaster data
collection, we send it off with a MetadataEvent.

(From OE-Core rev: 109ae6c5c981610ab0d63d2c83dcd50b2e93276b)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-03-21 12:05:53 +00:00
Paul Eggleton a3f0852b14 classes/sstate: don't overwrite buildhistory vardepsexclude
This was stomping over the value added in buildhistory.bbclass, leading
to task signatures *still* changing when buildhistory was added to
INHERIT.

Fixes [YOCTO #5897].

(From OE-Core rev: f7a0aa412a921a41d301be072cedcda2662ffbd7)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-03-11 08:11:41 -07:00
Richard Purdie 2973c12838 sstate: Ensure SSTATEPOSTINSTFUNCS is accounted for in signatures
Bitbake can't interpret the python execution of these functions so
include the dependency manually.

[YOCTO #4507]

(From OE-Core rev: 39c53d26a1ea0859ca30456b6af0a28b2ea16769)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-03-05 15:50:20 +00:00
Richard Purdie e4ef6bbc21 sstate: Drop 'SafeDep' code from setscene validation function
I have a feeling this code exists from the time before we had proper
coverage of one sstate task by another task. At that time it was a
"poor" persons version of that idea, we now have much better
code internal to bitbake which handles this.

Worse, this code actually breaks certain rebuild scenarios,
e.g.:

bitbake libtool-cross
bitbake libtool-cross -c cleansstate
rm tmp -rf
bitbake libtool-cross

would fail as binutils-cross wasn't installed from sstate.

The easiest fix is to remove the obsolete/broken code.

[YOCTO #5773]

(From OE-Core rev: ccad07f35fb7f959e24fd50d04c7d10dd5cf20d0)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-02-25 08:01:09 +00:00
Paul Eggleton d7eb0f5731 classes/sstate: fix taints being undone on execution of sstate tasks
The code here that deletes stamps was also deleting the taint files; so
forcing an sstate task with -f would force it to execute and then
because the taint file was deleted in the process, the next execution
would simply restore the output from sstate again. We need to exclude
the taint files just like we did in bb.build.make_stamp().

Fixes [YOCTO #5805].

(From OE-Core rev: 4708859e5627488251dc4250d45cb5f4e9736b8a)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-02-25 08:01:09 +00:00
Richard Purdie 8d17c39cd6 sstate: Exclude SSTATE_EXTRAPATH from checksums
After the change to allow target recipes to depend on native recipes, the
native checksums becomes all the more critical. Add to this that we're now
accounting for pre/postfuncs and we have a cache reuse issue since the
distro LSB string is getting coded in when it shouldn't be.

This excludes that string and allows one set of native sstate to share
checksums with another set from a different host distro. They're separated
into different directories so this is fine for our use cases.

(From OE-Core rev: 1fa7d4331d994b9eeb6f973d1a1f04cb4df92c13)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-02-11 17:15:54 +00:00
Richard Purdie 4e9eb77108 sstate: Improve funciton checksums
Now that bitbake is recursing into pre/postfuncs, we need to ensure
the dependencies of these functions is correct. We don't want dependencies
on MACHINE or other related variables. This patch adds in appropriate
variable exclusions to achieve this.

(From OE-Core rev: 8461283a648d7c5affd51971ebd9b35a8a4c625f)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-02-04 22:51:43 +00:00
Robert Yang a6c6659b3f sstate.bbclass: remove previous version's stamp
There is a potential problem if we don't remove the previous version's
stamp, for example:

The depend chain is:
libtool-native -> autoconf-native -> m4-native
We have two m4-native: 1.4.9 and 1.4.7

1) Clean all of them to make a fresh build so that we can reproduce the
   problem
$ bitbake m4-native autoconf-native libtool-native -ccleansstate

2) Build libtool-native so that the m4-native_1.4.17 will be built
$ bitbake libtool-native

3) Set PREFERRED_VERSION_m4-native = "1.4.9" and build again
$ bitbake libtool-native

4) Set PREFERRED_VERSION_m4-native = "1.4.17" and build again
$ bitbake libtool-native -ccleansstate && bitbake libtool-native

Then the build will fail:
[snip]
| m4: unrecognized option '--gnu'
| Try `m4 --help' for more information.
| autom4te: m4 failed with exit status: 1
[snip]

The is because when we change m4-native to 1.4.17 and build
libtool-native again:
5) libtool-native depends on autoconf-native, and autoconf-native's
   version isn't change, so it can remove the current stamp and mirror
   the sstate (the one depends on m4-native_1.4.9) from the SSTATE_DIR
   correctly.

6) The mirrored autoconf-native depends on m4-native_1.4.17's
   do_populate_sysroot, and the stamp is already there (which is made
   by step 2), so it would do nothing, but this is incorrect, since
   the one that really in the sysroot is m4-native_1.4.9, then the
   error happens.

Remove previous version's stamp in sstate_clean() will fix the problem.

[YOCTO #5422]

(From OE-Core rev: 4659d29b1040349116549644e45035a5b37d9311)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-01-21 10:45:29 +00:00
Ming Liu e93a345fae sstate: add do_package to the noexec list in setscene_depvalid
do_package doesn't exist and are noexec anyway for native/cross/crosssdk
packages.

(From OE-Core rev: 1028ac813fa9803ebfff6bcfa7f8b67012609b27)

Signed-off-by: Ming Liu <ming.liu@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-01-07 13:58:32 +00:00
Ming Liu 22764d3f4d sstate: Add optimizing logic for crosssdk setscene dependencies
This patch mainly aims to add optimisation for crosssdk setscene dependency
validating which we haven't handled in current logic, and which I think we
could have as we've already implemented to native/cross, although there
are albeit not many crossdk tasks, we could still get some performance
enhancement.

(From OE-Core rev: 1094983ff87a8b745a5bc7bfe9514433ee3c4ad2)

Signed-off-by: Ming Liu <ming.liu@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-01-06 22:17:33 +00:00
Richard Purdie 325fb2bd24 sstate: Use datastore copies due to data changes persisting
The way the "all arch" PKGSPEC is enabled, it causes corruption
of the datastore of sstate operations against other tasks.

Data store copies are cheap and allow us to use that trick, resetting
to a clean copy of the data afterwards.

(From OE-Core rev: afaf16100efa79a275a2f4b9f2caa80decfdeb81)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-12-21 09:00:52 +00:00
Richard Purdie 3ada51f7aa sstate/sstatesig: Add populate_lic to list of arch invariant sstate tasks
Like fetch, unpack and patch, populate_lic doesn't vary between different
archs so we should mark it as such. This means better sstate cache reuse
with fewer duplicate files as well as less confusing sstate debugging.

sstatesig also needs to account for the fact BPN is used for sstate files
in these cases.

(From OE-Core rev: 3d59d0bed756f64d0092caa3892239c779c4a341)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-12-20 12:26:32 +00:00
Richard Purdie e5ee1b8c0d sstate: Convert to use ':' as a filename sperator and use SSTATE_SWSPEC globally
Currently the code has problems differentiating between "gcc-cross" and "gcc-cross-initial"
sstate files. We could add in a ton of special casing but tests show this isn't scaling
well. Using a more unique separator resolves the issue.

The choice of which separator to use is a hard one. We need something which isn't commonly
used in PN, PV, PR, *_OS and *_ARCH which rules out '-', '_' and it needs to work ok with
webservers/http which makes ';' and '%' harder.

The change also sets SSTATE_SWSPEC globally since writing out differently named siginfo
files for the fetch/unpack/patch tasks is a waste of diskspace, the hashes match for
all PN in the majority of cases and if they don't, its not a big issue as the hash is
different. This makes the results from sstate debugging more understandable.

(From OE-Core rev: 6f823a23c5f1d0ffa0a27db1c1bc1907de788505)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-12-20 12:26:31 +00:00
Richard Purdie 480bf037ae sstate: Get rid of crazy name mapping
When originally developed, it was thought a task may have more than one associated
sstate archive. The way the code has grown that idea is now not possible or needed.
We can therefore assume one sstate archive per task and drop the crazy name
mapping code. Simpler is better in this case.

The downside is that various sstate archives will change name so this forces a cache
rebuild. Given the other sstate changes going in at this time, this isn't really
a bad thing as things would rebuild anyway.

(From OE-Core rev: 5afe86a6854b21692fd97c5fc7fab50dbc068acb)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-12-18 17:08:05 +00:00
Richard Purdie 7935ae2e80 sstate/gcc: Fix shared workdir handling for siginfo files
For a shared workdir, any one of the fetch/unpack/patch tasks may run yet the
PN and architecture fields in SSTATE_PKGSPEC may differ. This makes looking up
the appropriate siginfo file near impossible.

I've tried several different ways of resolving this and this is the neatest
solution I could find, its still rather ugly. I believe the usefulness of
better sstate debugging outweighs the ugliness of the code.

This patch also changes the sstate_checkhashes() code to look for siginfo
files rather than the actual sstate packages themselves. This means the
function can be used in other contexts to find info files for tasks that
may not have sstate data. It is assumed that sstate mirrors will have both
files available. This is done to allow bitbake to query whether tasks have
matching signatures in sstate directories or not.

(From OE-Core rev: 068e4289b597699cbff2dfde44ba833af4535281)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-12-18 17:08:04 +00:00
Richard Purdie 88f0166770 sstate: Write out siginfo files for non-sstate tasks
Currently siginfo files are only written for sstate tasks. In order to be truly
debuggable, its helpful to have the siginfo for intermediate tasks. This
adds that functionality so the extra siginfo files are written out too.

This will be used to add better sstate debugging in future changes.

(From OE-Core rev: 04d108cd16f5ad8f92a62ea537d1330fee712470)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-12-18 17:08:04 +00:00
Roy Li 8c5f345497 sstate.bbclass: ignore the tar failure.
sstate_package creates hardlink from sysroot to SSTATE_BUILDDIR, and
sstate_create_package stores SSTATE_BUILDDIR into a archive file by
tar, these two task can be run simultaneously for different packages,
and make a hardlink for a file will lead to the change of the links
number of file, and if tar is reading this file, it will fail with exit
code 1, and report "file changed as we read it":

    DEBUG: Executing shell function sstate_create_package
    tar: x86_64-linux/usr/share/aclocal/xorg-macros.m4: file changed as we read it

4b3e353a5[sstate.bbclass: fix parallel building issue] tries to use the
tar parameter --ignore-failed-read to fix, but it does not work, and
tar parameter --warning=no-file-changed can close the warning, but can
not change the exit code. so close shell immediate exit, only fail
if tar returns not 1 and 0.

Exit codes of tar:
    http://www.gnu.org/software/tar/manual/html_section/Synopsis.html

(From OE-Core rev: fad604b719e00b03e09da5fdb485e72332275b4a)

Signed-off-by: Roy Li <rongqing.li@windriver.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-09-24 17:24:12 +01:00
Roy.Li 0acde33c75 sstate.bbclass: fix parallel building issue
sstate_package creates hardlink from sysroot to SSTATE_BUILDDIR, then
sstate_create_package will store SSTATE_BUILDDIR into a archive file by
tar, but once other packages install the same file into sysroot, the
creating the archive file will fail with below error:

    DEBUG: Executing shell function sstate_create_package
    tar: x86_64-linux/usr/share/aclocal/xorg-macros.m4: file changed as we read it

This kind of error is harmless, use --ignore-failed-read to ignore it.
The error in tar occurs when the timestamp of the file changes and this
can happen when the number of symlinks change. The file will be included
in the archive.

[YOCTO #5122]

(From OE-Core rev: 4b3e353a532c7b68b0bb86df4a2fcc44f8bb3ef2)

Signed-off-by: Roy.Li <rongqing.li@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-09-14 08:20:59 +01:00
Richard Purdie 554c892ccf meta: Don't use deprecated bitbake API
These have been deprecated for a long time, convert the remaining
references to the correct modules and prepare for removal of the
compatibility support from bitbake.

(From OE-Core rev: 6a39835af2b2b3c7797fe05479341d71a3f3aaf6)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-09-01 15:51:42 +01:00
Richard Purdie 4273aa4287 sstate: Fix the relative symlink replacement code
ant reported on irc that the sstate absolute to relative symlink creation
code wasn't working in klibc. He was correct although the level of breakage is
rather surprising since it only worked for one level of symlink (usr/include) with
everything else being broken.

The reason is probably that nothing really uses absolute paths, we use relative
paths where at all possible already. Nothing in the target sysroot should use
absolute paths for a start. In this regard, the klibc-dev package is broken and
needs fixing. It will currently break when building for one machine, then switching
to another of the same TUNE_PKGARCH and installing from sstate but that is a
separate issue.

This patch fixes the symlink creation code by firstly passing in the correct
value we need (where the symlink will end up) and seccondly, actually using it.

I've also tweaked the debug message to contain appropriate information and got
right of the double "//" value the existing code created in favour of the form
'./..' which looks neater.

(From OE-Core rev: 9b05c65450526522d7358d0c0901b594de546748)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-08-23 17:29:33 +01:00
Joe MacDonald 4d326e5edf sstate.bbclass: Add an extra check for sstate_mirrors
BB_NO_NETWORK disables any fetching, however if we're using an external
sstate cache, we may want to be able to fetch those objects even if we are
not fetching the upstream sources.  Denote this situation by setting
SSTATE_MIRROR_ALLOW_NETWORK in local.conf.  When it is found, for sstate
cache fetches, mask off BB_NO_NETWORK for the local function.

(From OE-Core rev: ed585cad2e1fdc323c05fa82055a071bcf98d1bc)

Signed-off-by: Joe MacDonald <joe.macdonald@windriver.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-07-05 15:34:26 +01:00
Christopher Larson 00c8834293 sstate.bbclass: fetch archive+siginfo in a single pstaging_fetch
This avoids unnecessary duplication of setup. The only visible change in
behavior will be the case if siginfo exists and the archive does not, in which
case it'll redownload both, but this doesn't seem unresasonable to me,
particularly since the archive is downloaded first, making this case
particularly unlikely.

(From OE-Core rev: aa4991c307d4bbdd06c3cbf8448240b74c5e01c4)

Signed-off-by: Christopher Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-06-11 16:16:02 +01:00
Christopher Larson fbf64380f4 sstate.bbclass: kill MIRRORS and FILESPATH to speed up fetching
The default FILESPATH isn't really of use, as we don't expect to find sstate
archives buried in layer recipe directories, and the default MIRRORS is
intended for use for fetching SRC_URI, not sstate.

(From OE-Core rev: 46402b2f5b69004751f6663d435bedae0ad9dab1)

Signed-off-by: Christopher Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-06-11 15:56:01 +01:00
Richard Purdie 7bf5c38e0f base/useradd/sstate: Ensure do_package setscene has correct fakeroot dependencies
The do_package_setscene task needs to depend on fakeroot in order to correctly
install its files.

We can whitelist the dependency in the sstate handling code for some
performance improvements since we only need this if we're installing the
package from sstate.

Also use an append operator in base.bbclass for clarity.

(From OE-Core rev: 0810ea2a72bdea67a3d8002c4e12fb20f45cf1d5)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-06-07 17:52:08 +01:00
Jackie Huang 50fae526de sstate.bbclass: fixup extra hardcoded paths
sstate has a logic to fixup hardcoded paths in scripts,
but it misses in some specific cases, so add
EXTRA_STAGING_FIXMES to the fixup hardcoded paths
mechanism, so that we can specify what hardcoded paths
need to be fixed in a recipe, e.g.
EXTRA_STAGING_FIXMES = "STAGING_BINDIR_TOOLCHAIN"

(From OE-Core rev: 2e840db56c45b4c63fded55f4ed763b7099284b9)

Signed-off-by: Jackie Huang <jackie.huang@windriver.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-06-04 11:43:35 +01:00
Richard Purdie 3512f797f9 classes/lib: Update to use print as a function call
In python 3 print is a function call. In some cases bb.note is a more
appropriate call to make.

(From OE-Core rev: 754874e68d1037c76fc4cc7d35d3fa12946d00f8)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-05-09 14:06:51 +01:00
Robert Yang c51e8b4532 sstate.bbclass: make hard links for staging files
Make hard links for staging files instead of copy to save the disk space
(3G will be saved for a core-image-sato build), and it doesn't affect
much on the build time.

The following directories are affected:
1) The sysroot
2) The DEPLOY_DIR
3) The pkgdata

[YOCTO #4372]

(From OE-Core rev: 5853e0f482b22258c909268fe71673a29e31989b)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-05-02 17:41:53 +01:00
Phil Blundell 2895bcf95a sstate: Set umask to 002 before packaging runs
Otherwise we might end up creating directories under sstate-cache with whatever
random umask has been selected for the task that we're trying to package.  This
would be a bad thing since it might result in losing group write access for
newly created dirs, and/or losing group read access for the sstate files
themselves.

(From OE-Core rev: d8c4f442c41bf3ac5e064630657cd3fa1b5c43b1)

Signed-off-by: Phil Blundell <philb@gnu.org>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-04-29 14:45:06 +01:00
Paul Eggleton c7f52855f9 classes/sstate: avoid traceback when no files have been staged
If no files have been staged we want to continue without error instead
of showing a traceback.

Fixes [YOCTO #4056].

(From OE-Core rev: ca36be708e54c0c86535bc8512295c76c48f6cf5)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-04-03 17:01:13 +01:00
Andreas Müller 13837e5a31 sstate.bbclass: remove reference to gtk+-native
(From OE-Core rev: 13bc0117a0a18165e83e2bcdd880e704a0df5e3f)

Signed-off-by: Andreas Müller <schnitzeltony@googlemail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-04-02 18:11:06 +01:00
Jeff Polk b50626c6f0 sstate: add -f to mv when moving sstate files into place
Under some conditions (ACLs enabled, NFS) mv can interactively prompt
before overwriting files.  Avoid hanging builds in that case by using
-f which should be harmless in other cases.

(From OE-Core rev: b1a085db9d8ad2a3117af6f50e510bc9c2f9407b)

Signed-off-by: Jeff Polk <jeff.polk@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-04-02 18:08:23 +01:00
Leonardo Sandoval 66ed85a0c9 sstate.bbclass: Enclose sstate search parameter with quotes
In case filenames have spaces, execution of the function
sstater_install will hang, so the print parameter %s must be
enclosed with quotes.

(From OE-Core rev: 545d7aa26dfefdc927e0f4e2cc37398ef2c63fa6)

Signed-off-by: Leonardo Sandoval <leonardo.sandoval@freescale.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-03-05 13:28:09 +00:00
Richard Purdie 7e2e2cf508 sstate: Add gtk-update-icon-cache-native to postinst recipes list
(From OE-Core rev: 59cd32a6661d76ee070810823614eb797d6cc153)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-03-05 13:14:54 +00:00
Laurentiu Palcu b1a7b15941 sstate.bbclass: consider postinstall dependencies too
When checking the dependencies in setscene_depvalid(), make sure we also
consider those dependencies needed when running the postinstalls on
host.

[YOCTO #3918]

(From OE-Core rev: 8de0616825ed1b238b3486077af6897834bcb62d)

Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-03-05 13:14:53 +00:00
Richard Purdie 6c6f6b7ffd sstate.bbclass: Ensure build directory is cleaned to start with
This directory is cleaned upon completion however if a previous build
crashes, it can lead to corrpution, hence ensure its clean at the start
too.

(From OE-Core rev: 8ef0e59d5a7da3671d1ad9a54fe068ed78f928d5)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-02-06 13:12:59 +00:00
Richard Purdie dd61d22584 sstate/path.py: Add copyhardlinktree() function and use for performance optimisation
Add a function which copys a tree as a set of hardlinks to the original
files, then use this in sstate to reduce some of the overhead of sstate
package creation since the file isn't actually copied.

(From OE-Core rev: 8e373e69acac853213a62afb8bbdf0adc0c5045a)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-02-06 13:12:59 +00:00
Richard Purdie b01b1a704b sstate: Move debug comment to more logical place
The same log message gets output multiple times in the log which look
confusing and is rather pointless. Move the log message to the correct
level.

(From OE-Core rev: 3917409004a830e7ad0646f05ad7421385cbd1de)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-01-30 14:09:09 +00:00
Richard Purdie 0d2a301bca sstate: Add optimisation for useradd injected dependencies
useradd.bbclass adds sstate dependencies on base-passwd, shadow-native and
shadow-sysroot. Due to the way these are injected, they interact badly with
the other dependency validation logic and end up pulling in dependencies we
don't actually need. This patch adds code to optimise those cases out.

(From OE-Core rev: 784ca68fcca4ffb34390d55d9343570cfdf0305f)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-01-25 12:42:48 +00:00
Richard Purdie 3247292683 Split do_packagedata task from do_package
Currently, do_rootfs has a dependency on all the do_package output being present
due to its usage of the pkgdata directories. This means that if you run:

bitbake xxxx-image -c rootfs

you end up having to fetch and unpack all the do_package data which is usually
large and inefficient. It also means rm_work has to leave all the do_package
data lying around so rootfs works.

This patch splits the actual creation of the pkgdata directory off into a separate
task, "packagedata" which happens immediately after do_package. We can then remap
the dependencies so this task is depended upon, not do_package. Sstate can then be
programmed not to require do_package at the appropriate times.

Whilst this patch doesn't do so, it opens the possibility of rm_work wiping
out the do_package output from WORKDIR as long as it also removed the do_package
stamp (both normal and setscene variants) and allowing more space savings
with rm_work which has been regularly requested.

(From OE-Core rev: 6107ee294afde395e39d084c33e8e94013c625a9)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-01-25 12:42:47 +00:00
Martin Ertsaas 2c7bba51a6 sstate: Do not add the --no-run-if-empty arguement to xargs when on Darwin, as it is not supported.
(From OE-Core rev: 73b34d43633717b99e4f2f669939939cfa59eecb)

Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-01-15 08:10:14 +00:00
Hongxu Jia be78e8190a sstate.bbclass:specify function dirs to avoid race
Specify dirs in which the shell function "sstate_create_package" and
"sstate_unpack_package" are executed and don't use ${B} as default dirs
to avoid possible race with task do_rootfs at deb image creation time.

[YOCTO #3674]

(From OE-Core rev: ccef1cf783669a4683eda9d4b44dbe6bcf426259)

Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-01-09 15:05:26 +00:00
Richard Purdie 1e548a7fc7 sstate.bbclass: Improve stamp-extra-info task handling to avoid warnings
If you change a machine to a different package architecture, you will see sstate
errors about overwriting files as the code stands today. Instead it should clean
out the files safely and correctly. This patch changes the naming of stamp-extra-info
manifest files to avoid this problem. It will potentially trigger warnings during
builds in existing TMPDIRs until the system adjusts to the new naming, these are
harmless.

[YOCTO #3521]

(From OE-Core rev: 2cc8ee57f8148844bb7bcd4aaf34f6891cf3d410)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-12-17 17:24:58 +00:00
Richard Purdie 2636cb1133 sstate.bbclass: Remove unused/uneeded variable
(From OE-Core rev: 33ddf788c04044c8fe7ffbadab8787a5b6603186)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-12-17 17:24:58 +00:00
Richard Purdie d46766ea5c sstate.bbclass: Simplify overcomplicated cleanall function
The code was written before SSTATETASKNAMES existed. Since it
does exist, lets simply the code.

(From OE-Core rev: 9817e2efdb94395655d711f5eadedcd249c8cffe)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-12-17 17:24:57 +00:00
Saul Wold d3154aa9f8 sstate: fix typo
(From OE-Core rev: 17d27fd6e2750e6455f656159da56c3350c4b3cd)

Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-12-06 22:59:10 +00:00
Richard Purdie dd3ad7f7f3 sstate: fix typo
(From OE-Core rev: 10cceb3854acd5c619d81011bac02da11ed580d2)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-12-06 16:36:08 +00:00
Saul Wold 97a91e81ea sstate: fetch .siginfo files from SSTATE_MIRROR
This would be useful for doing siginfo compares to understand why a build
is not reusing something when using SSTATE_MIRROR. No error will be reported
if it fails to find the .siginfo file

[YOCTO #2898]

[RP: Small tweaks]
(From OE-Core rev: 6d86690330f0d43839b904fced4b4b02cb27b8c6)

Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-12-06 12:31:19 +00:00
Martin Jansa 381c4b69c7 sstate.bbclass: fix detection of owners matching sstate files
* without this patch:
  Python 2.7.3
  >>> sstate_search_cmd = "grep -rl /OE/jansa-test/shr-core/tmp-eglibc/pkgdata/armv7a-vfp-neon-oe-linux-gnueabi/runtime-reverse/vim-common /OE/jansa-test/shr-core/tmp-eglibc/sstate-control --exclude=master.list | sed -e 's:^.*/::' -e 's:\.populate-sysroot::'"
  >>> cmd_array = sstate_search_cmd.split(' ')
  >>> search_output = subprocess.Popen(cmd_array, stdout=subprocess.PIPE).communicate()[0]
  grep: |: No such file or directory
  grep: sed: No such file or directory

* Adding shell=True and using cmd string instead of array makes it work:
  >>> search_output = subprocess.Popen(sstate_search_cmd, shell=True, stdout=subprocess.PIPE).communicate()[0]
  >>> print search_output
  manifest-armv7a-vfp-neon-gvim.package
  manifest-armv7a-vfp-neon-vim-tiny.package
  manifest-armv7a-vfp-neon-vim.package

  But still isn't 100% reliable, I guess it's caused by some other package
  being removed from sstate while grep is already running.
  So sometimes grep can show error on STDERR
  >>> search_output = subprocess.Popen(sstate_search_cmd, shell=True, stdout=subprocess.PIPE).communicate()[0]
  grep: /OE/jansa-test/shr-core/tmp-eglibc/sstate-control/manifest-armv7a-vfp-neon-systemtap.package: No such file or directory

(From OE-Core rev: d84f7d7a12b4271f7b2bfde9fb356d750abff15d)

Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-12-05 15:34:07 +00:00
Enrico Scholz 306761b026 sstate.bbclass: fixed file-conflict check
The value of subprocess.Popen().communicate()[0] is a string.
Checking for '!= None' will always match causing bogus warnings
regarding already staged files.

(From OE-Core rev: acdd76482efc3caf80e9e0b7359be7ca724ae09a)

Signed-off-by: Enrico Scholz <enrico.scholz@sigma-chemnitz.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-12-05 15:34:06 +00:00
Richard Purdie 13ecfdb331 packagegroup/allarch: Convert to use allarch class
Currently there is some odd behaviour of the packagegroup class in relation
to sstate since it sets PACKAGE_ARCH = "all" but does not use the allarch class
leading to it being undetected by sstate.

Previously it was not possible to use allarch as the recipe couldn't "undo"
settings made by the allarch class. Since this no longer happens when
PACKAGE_ARCH != all, we can use the allarch class.

This patch also fixes up one case we need to preserve TRANSLATED_TARGET_ARCH
and ensures sstate only assumes allarch when PACKAGE_ARCH is "all".

(From OE-Core rev: 591fa7c1ab9e9ff75fdce602c77ecdeda3a255d9)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-11-26 10:01:24 +00:00