Commit Graph

250 Commits

Author SHA1 Message Date
Richard Purdie 2f0c06a452 sstate: Add support for taking shared lockfiles
(From OE-Core rev: c411a10e06f479ff364c07766f7c77907b7b4a16)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-03-29 14:01:51 +01:00
Martin Jansa 25a6e5f98c sstate: use only unique set of SSTATETASK
* otherwise strange error like this:
  ERROR: Logfile of failure stored in: /OE/shr-core/tmp/work/armv7a-oe-linux-gnueabi/libtool-cross-2.4-r1/temp/log.do_package_write_ipk.25551
  Log data follows:
  | ERROR: Package already staged (/OE/shr-core/tmp/sstate-control/manifest-nokia900-libtool-cross.deploy-ipk)?!
  | ERROR: Function 'sstate_task_postfunc' failed
  NOTE: package libtool-cross-2.4-r1: task do_package_write_ipk: Failed
  ERROR: Task 11 (/OE/shr-core/openembedded-core/meta/recipes-devtools/libtool/libtool-cross_2.4.bb, do_package_write_ipk) failed with exit code '1'

  is shown in this case with package_ipk twice in INHERIT

* Thanks to Richard for fix

(From OE-Core rev: f2fe5e840b8aa0558b5462ef2c7517b2f14ec2ea)

Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-03-24 12:33:38 +00:00
Richard Purdie 273cfeccb7 sstate: Ensure a clean removes setscene stamps as well as the main task stamps
(From OE-Core rev: d07fe8aef537a8bcb96a802e18d7c980ff4c5ce2)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-03-19 01:16:32 +00:00
Richard Purdie 16e1f1051e sstate.bbclass: Turn absolute symbolic links into relative ones for sstate packages
(From OE-Core rev: 655139c2644d085331f4f6814119fbd904ff244b)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-03-18 23:23:29 +00:00
Paul Eggleton ed1cbc9fa8 sstate.bbclass: always delete stamp files in sstate_clean
For safety, always delete the stamp files in sstate_clean regardless of
whether the manifest file exists or not.

(From OE-Core rev: f781c35da9a11eefdb06bda72ca89753df863efa)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-03-15 13:20:48 +00:00
Paul Eggleton cbbf658041 sstate.bbclass: avoid deleting unrelated stamp files
Avoid deleting stamp files whose names contain the current task's name as a
substring. This will be especially important for example if do_package_write
is ever made an sstate task (as it would previously have deleted the stamps
here for do_package_write_ipk etc.)

(From OE-Core rev: ea743ea30e2289733d27979e8ec921648342da0e)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-03-15 13:20:47 +00:00
Richard Purdie fc1b893d1a sstate: Ensure the SRCURI fetcher cache is not used for sstate
(From OE-Core rev: 115b3b95e87320b4a6a678df45fece06469dfaeb)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-03-08 11:19:42 -08:00
Scott Garman fa6e786082 sstate.bbclass: add SSTATEPOSTINSTFUNCS
When doing builds using sstate cache, there was no way to run
the equivalent of a pkg_postinst function. This is needed by
the SGML-related documentation recipes to properly update the
catalog files when new DTDs and stylesheets are installed.

SSTATEPOSTINSTFUNCS is a new variable you can set to function(s)
in your recipe to run after install is completed from sstate.

Thanks to Richard Purdie for suggesting this solution.

Signed-off-by: Scott Garman <scott.a.garman@intel.com>
2011-02-23 15:52:51 +00:00
Zhai Edwin 49a18f1748 sstate: Fix bugs after new fetcher
Current sstate's fetch code doesn't reflect latest fetcher changes, so old
fetch style cause exception and fail silently.

[BUGID #708] got fixed.

Another issue is "import xxx" in python function from sstate.bbclass can only
sit in the head of the function, else have UnboundLocalError: "local variable
XXX referenced before assignment".

Signed-off-by: Zhai Edwin <edwin.zhai@intel.com>
2011-02-11 14:07:49 +00:00
Richard Purdie 984e90f4d7 meta/classes: Update classes to use new fetcher API
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-02-07 09:06:37 +00:00
Richard Purdie 9b2fd4b0fe license.bbclass: Add coverage by sstate code (also use cleandirs and dirs task attribures for directory handling)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-02-03 12:33:18 +00:00
Richard Purdie 8cae183a22 staging.bbclass: Only clean out the sysroot if we rerun configure, not at setscene time
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-01-28 10:28:19 +00:00
Richard Purdie c7ad2dada7 sstate: Ensure stale stamp files are removed as part of sstate_clean()
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-01-28 10:28:06 +00:00
Richard Purdie 754047b6ec sstate.bbclass: Fix stupid typo
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-01-27 23:26:10 +00:00
Richard Purdie 021cf7dabf sstate.bbclass: No need to dump the contents of the sstate package to stdout
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-01-27 22:38:11 +00:00
Richard Purdie bb7a42ee4e sstate.bbclass: Handle whitespace in the list of files to 'fixup' to resolve log warning
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-01-27 22:34:52 +00:00
Richard Purdie 8a6e8623b4 sstate.bbclass: Optimise to check if the manifest exists without taking locks
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-01-27 20:52:33 +00:00
Richard Purdie 4c55d2bd04 sstate.bbclass: Correctly handle empty sstate packages
Signed-off-by: Richard  Purdie <richard.purdie@linuxfoundation.org>
2011-01-26 12:02:30 +00:00
Richard Purdie 222796ed93 sanity/sstate: Due to the sysroot layout changes, the toolchain bootstrap process changes and the recent pseudo bug, bump the tmpdir layout version number and the sstate version numbers
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-01-25 12:32:50 +00:00
Yu Ke e2ac26e7b1 base/sstate.bbclass: Rename fetcher "go" method to "download" when using fetch v2
Signed-off-by: Yu Ke <ke.yu@intel.com>
2011-01-25 12:25:47 +00:00
Dongxiao Xu 7f41f1a888 bitbake: machine specific sysroots implementation
This commit changes the sysroots path to be machine specific.

Changes includes:

1) STAGING_DIR_TARGET and STRAGING_DIR_HOST points to machine specific
paths.

2) task stamp files. Adding ${MACHINE} info into stamp files for
do_populate_sysroots and do_package tasks. Add a BB_STAMPTASK_BLACKLIST
to keep native, nativesdk, crosssdk, and cross-canadian stamp unchanged.

3) siteconfig path. Separate the site config path for different machines
to avoid one machine adopting the cache file of another machine.

4) sstate. Add machine name to sstate manifest file.
Change relocation code for sstate paths since sysroot is machine.
Keep native, nativesdk, crosssdk, and cross-canadian unchanged.

5) toolchain scripts. Change the environment path to point to machine
specific sysroots in toolchain scripts bbclass.

6) Relocate la files when populating to a different machine of the same
architecture.

7) Exclude STAGING_DIR_TARGET and STAGING_DIR_HOST parameter from sstate
siginfo since they contain ${MACHINE} information.

Signed-off-by: Dongxiao Xu <dongxiao.xu@intel.com>
2011-01-25 12:25:46 +00:00
Dexuan Cui 46395d4856 sstate.bbclass: allow each step of toolchain bootstrap processes to do populate_sysroot independently
Some steps (like gcc-cross-initial, gcc-cross-intermediate and eglibc-initial)
will install to new locations to avoid file overwriting.

Signed-off-by: Dexuan Cui <dexuan.cui@intel.com>
2011-01-25 10:59:17 +00:00
Richard Purdie 542cebbc32 sstate.bbclass: Ensure clean/setscene sstate functions run under appropriate locking
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-01-19 11:48:55 +00:00
Koen Kooi ca148f4627 sstate.bbclass: fix multimachine builds when PACKAGE_ARCH = MACHINE_ARCH
previously you could only build one set of packages for multiple machines:

MACHINE=foo bitbake task-base
MACHINE=bar bitbake task-base

would only create task-base packages for foo, but not for both foo and bar.
Doing MACHINE=bar bitbake task-base -c cleanall would remove the packages for foo.

The solution is to use MULTIMACH_ARCH as suggested by Richard Purdie.

Signed-off-by: Koen Kooi <koen@dominion.thruhere.net>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
2011-01-04 09:46:26 -08:00
Paul Eggleton 3f814ff9c4 sstate.bbclass: delete sstate files when doing cleanall
This fixes the cleanall task to actually delete the sstage cache files. This
was not working correctly before due to attempting to use the current task's
hash in the package file name to be deleted, which of course is the hash of
the cleanall task when doing cleanall. The siginfo files are now deleted as
well.

Signed-off-by: Paul Eggleton <paul.eggleton@intel.com>
2010-12-20 17:54:16 +00:00
Dongxiao Xu 7e7bb24b85 sstate.bbclass: Remove the temp sstate-build-* directories in WORKDIR
Remove the temp sstate-build-* directoies.
Take poky-image-minimal as an example, this saves about 17% disk space.

Signed-off-by: Dongxiao Xu <dongxiao.xu@intel.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
2010-11-14 21:08:23 -08:00
Richard Purdie 89929e1f28 bitbake/fetch: Allow checking of a single url at a time (massive sstate speed improvement)
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-11-13 20:10:53 +08:00
Richard Purdie d6e8f7d8a6 sstate.bbclass: Only create symlinks to different localpath urls if the fetch succeeded
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-11-13 20:08:23 +08:00
Richard Purdie ecf2eb1efa sstate: Provide a temporary mechanism to stop collisions between toolchain components in builds from scratch using sstate
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-11-02 13:18:07 +00:00
Richard Purdie 3f599b3f6a sstate: Fix mirror handling for file:// urls
The fetcher has special handling for file:// mirror urls, being efficient and
just providing an updated path. Unfortunately the sstate fetching code wasn't
able to handle this. This patch detects this and injects a symlink to ensure
everything works. It also fixes some datastore references to be correct and
ensures the sstate download directory exists if it doesn't already.

Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-10-22 14:48:54 +01:00
Richard Purdie f806c499c0 sstate: Fix broken plaindirs support
When installing a sstate package, directories tracked by plaindirs were being installed
to the incorrect location. With the current implementation this was limited to
the do_package task.

This patch ensures plaindirs tracked files are created in the correct location, fixing
the bug where these files would go missing.

Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-10-22 12:51:35 +01:00
Richard Purdie fd4457199e base/sstate: Add cleanall task to remove downloads and sstate cached files
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-10-22 10:22:34 +01:00
Richard Purdie 240b6636f9 sstate: Fix SSTAGE_MIRRORS handling and improve example in local.conf.sample
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-10-07 15:42:56 +01:00
Richard Purdie 35d095c282 bitbake/sstate: Implement a lookup function to speed up setscene processing
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-10-05 22:26:33 +01:00
Richard Purdie ca5fff3806 sstate: Deal with a potential race when cleaning packages
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-10-01 14:07:18 +01:00
Joshua Lock 62fe6dc755 sstate: list directories in child first order in the manifest file
The directory list in the manifest file needs to be sorted such that child
directories are deleted first. Fortunately as the list is generated by walking
the directory tree achieving this is as simple as reversing the directory list.

Fixes [BUGID #269]

Signed-off-by: Joshua Lock <josh@linux.intel.com>
2010-09-29 17:10:41 +01:00
Richard Purdie 1e0e6f1185 sstate.bbclass: Add signature hash to sstate package names
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-09-29 12:12:53 +01:00
Richard Purdie 97dec8410b sstate.bbclass: Update to use SSTATE variables everywhere and remove the now unneeded complicated paths from the sstate files
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-09-29 12:12:26 +01:00
Richard Purdie dd28a9c516 sstate.bbclass: pstage -> sstate name changes (no global implications)
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-09-29 12:12:26 +01:00
Richard Purdie 7087fb788a sstate.bbclass: Rename PSTAGE2 -> SSTATE
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-09-29 12:12:26 +01:00
Richard Purdie e888596636 sstate: Trigger generation of siggen info when building sstate packages
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-09-13 15:41:07 +01:00
Jeff Polk 27dd7f82c9 sstate: Add sstate-interceptfuncs called between install and package
Adds the ability to impose intercept function call(s) after
sstate_install and before sstate_package in sstate_task_postfunc.

Signed-off-by: Jeff Polk <jeff.polk@windriver.com>
2010-09-10 12:55:34 +01:00
Richard Purdie e8fdc98779 sstate: Make sure the sstate function is the last to run, not the first
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-09-07 15:16:31 +01:00
Richard Purdie a6ce10e83e sstate: Handle symlink directory entries correctly
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-09-03 15:28:27 +01:00
Kevin Tian e055544ce1 sstate: fix dir name in manifest
dir name in manifest is incorrect:

/media/disk/builds/master-arm/tmp/sysroots/usr/
/media/disk/builds/master-arm/tmp/sysroots/lib/
/media/disk/builds/master-arm/tmp/sysroots/include/

Also add one debug note when removing manifest entries

Signed-off-by: Kevin Tian <kevin.tian@intel.com>
2010-08-31 16:38:20 +01:00
Kevin Tian 03983e9647 sstate: make do_cleanup really working
there's a bug in creating manifest_pattern, which means actually with
new sstate do_cleanup hasn't worked yet

We can use PSTAGE2_MANFILEPREFIX easily for this purpose.

Signed-off-by: Kevin Tian <kevin.tian@intel.com>
2010-08-31 16:37:49 +01:00
Richard Purdie 4b2a355a31 sstate/utility-tasks: Ensure do_clean functions correctly and removes shared state
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-08-27 12:33:58 +01:00
Richard Purdie 5c7531f78c sstate: Run sstate_cleanall() as part of do_setscene()
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-08-27 12:29:01 +01:00
Joshua Lock d46c362a46 sstate: fix architecture of sstate archives
Using TARGET_ARCH as the pacakge architecture for sstate archives lead to some
issues when using two machines with different BASE_PACKAGE_ARCH's but the same
TARGET_ARCH such as netbook and qemux86 machines.

Probable fix for [BUGID #223]

Signed-off-by: Joshua Lock <josh@linux.intel.com>
2010-08-25 18:01:23 +01:00
Richard Purdie f070ee959c Rename packaged-staging2 -> sstate and remove the old empty class
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-08-19 22:48:17 +01:00