Use "out of" rather than "from" in the output.
(From OE-Core rev: c712529030c71b45bb8ae647c17319c2647aedff)
Signed-off-by: Jan Sarenik <jajomojo@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* instead of hard coding list of sstate enabled task use
the same function as remove_duplicated to find them in
sstate-cache directory
(From OE-Core rev: dfdb397db5865c3287cd9ccb5ea9a336eb77ca90)
Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* it's possible that corresponding .tgz files were already removed
(e.g. with -d option and older version of this script) and this
won't find orphaned .siginfo or .done files to remove
* add sort -u to count files found multiple times only once
(From OE-Core rev: 5a58cd2474ed96d6f58576203bedf8211d309c99)
Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* there are .siginfo files in sstate-cache, but STAMPS_DIR calls them
.sigdata, make sure that such signatures are kept, because some tasks
like do_package, don't have _setscene or main task entry and are
removed:
stamps:
tmp-eglibc/stamps/i586-oe-linux/busybox/1.22.1-r0.do_package.sigdata.cd6f625471ef1b20a9379e90519db6f1
tmp-eglibc/stamps/i586-oe-linux/busybox/1.22.1-r0.do_package_write_ipk.f20fe66285219f23373fc64d5de1d412
tmp-eglibc/stamps/i586-oe-linux/busybox/1.22.1-r0.do_package_write_ipk.sigdata.f20fe66285219f23373fc64d5de1d412
tmp-eglibc/stamps/i586-oe-linux/busybox/1.22.1-r0.do_packagedata.sigdata.c55a3dbe90010c15aa3294753fbd402e
tmp-eglibc/stamps/i586-oe-linux/busybox/1.22.1-r0.do_packagedata_setscene.c55a3dbe90010c15aa3294753fbd402e.qemux86
sstate (we want to keep all in this case):
sstate-cache/c5/sstate:busybox:i586-oe-linux:1.22.1:r0:i586:3:c55a3dbe90010c15aa3294753fbd402e_packagedata.tgz
sstate-cache/c5/sstate:busybox:i586-oe-linux:1.22.1:r0:i586:3:c55a3dbe90010c15aa3294753fbd402e_packagedata.tgz.siginfo
sstate-cache/cd/sstate:busybox:i586-oe-linux:1.22.1:r0:i586:3:cd6f625471ef1b20a9379e90519db6f1_package.tgz
sstate-cache/cd/sstate:busybox:i586-oe-linux:1.22.1:r0:i586:3:cd6f625471ef1b20a9379e90519db6f1_package.tgz.siginfo
sstate-cache/f2/sstate:busybox:i586-oe-linux:1.22.1:r0:i586:3:f20fe66285219f23373fc64d5de1d412_package_write_ipk.tgz
sstate-cache/f2/sstate:busybox:i586-oe-linux:1.22.1:r0:i586:3:f20fe66285219f23373fc64d5de1d412_package_write_ipk.tgz.siginfo
(From OE-Core rev: c2e5c0b6bdc432449ad1792176aa28667c3d34b9)
Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* when debug is enabled it causes more confusion than what it's gaining
without debug enabled
(From OE-Core rev: 53899da20cd439ae53974e507f3dd86cd6879455)
Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* deleted files were counting .siginfo and .done files, but total_files were counting only .tgz archives
so it could show confusing output like:
Figuring out the sstate:xxx_packagedata.tgz ... (21098 from 15162 files for packagedata suffix will be removed)
* split the counts and show both
* fix .sigdata sometimes used instead of .siginfo
* this will also show new "suffixes" which have only .siginfo files but
no .tgz, e.g.:
Figuring out the sstate:xxx_rm_work.tgz ... (0 from 0 .tgz files for
rm_work suffix will be removed or 0 from 107630 when counting also
.siginfo and .done files)
because now we're storing siginfo files from all intermediate tasks,
but they are not being removed by this script
(From OE-Core rev: ec881997c748ed8bfb3fc75797367ce3599bd5b4)
Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* packagedata task was introduced in:
commit 6107ee294afde395e39d084c33e8e94013c625a9
Author: Richard Purdie <richard.purdie@linuxfoundation.org>
Date: Wed Jan 23 14:27:33 2013 +0000
Subject: Split do_packagedata task from do_package
* rm_by_stamps wasn't using do_packagedata or do_packagedata_setscene
stamp files to find which sstate archives to keep, so it was removing
all of them
(From OE-Core rev: 66ef2e62a7b5fe36c718f56a8ea9d7f6fd77c393)
Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* grep for AVAILTUNES isn't enough in cases where AVAILTUNE doesn't
match exactly with TUNE_PKGARCH, e.g. AVAILTUNE "cortexa8thf-neon"
and TUNE_PKGARCH "cortexa8t2hf-vfp-neon", instead of trying to find
dynamically every available TUNE_PKGARCH (we have _a lot_ of them
even with oe-core only), add parameter --extra-archs where user can
define extra architectures he supports in given build
* Don't replace '-' with '_' for extra-archs, it does apply to MACHINE
names and some AVAILTUNES, but e.g. cortexa8thf-neon shouldn't be
converted to cortexa8thf_neon
* Add empty architecture for populate_lic sstate archives
* Add ${build_arch}_${arch} combinations for toolchain recipes (e.g.
gcc-cross is using x86_64_i586
(From OE-Core rev: a27cc54fb2d0e59f3a800893c1848cb26a7c5335)
Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* it's good to see some the ratio of delted files until now it was
showing only when all or none files were to be removed
(From OE-Core rev: 54e6e25f1a369fa6c21ce0f9db3479b1a481825f)
Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* format of filenames for sstate archives was changed in:
commit 6f823a23c5f1d0ffa0a27db1c1bc1907de788505
Author: Richard Purdie <richard.purdie@linuxfoundation.org>
Date: Fri Dec 20 12:06:02 2013 +0000
Subject: sstate: Convert to use ':' as a filename sperator and use SSTATE_SWSPEC globally
* this one doesn't need special care for old sstate- names
they will be removed automatically as they don't match with
any checksum in rigth format from stamps directory
(From OE-Core rev: aa36f9c9b5abac58de899f98803d1c4375678044)
Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* format of filenames for sstate archives was changed in:
commit 6f823a23c5f1d0ffa0a27db1c1bc1907de788505
Author: Richard Purdie <richard.purdie@linuxfoundation.org>
Date: Fri Dec 20 12:06:02 2013 +0000
Subject: sstate: Convert to use ':' as a filename sperator and use SSTATE_SWSPEC globally
* remove_duplicated() wasn't able to find available architectures and
duplicate files since this change
* add extra step to remove old sstate archives starting with sstate-
(instead of sstate:)
(From OE-Core rev: ddb26341611c3dff41ea92a73d93ec01ae2865de)
Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* usefull for jenkins jobs, which will otherwise fail
because 1 was returned
(From OE-Core rev: a864de0f2a326f857125229fc986845044931196)
Signed-off-by: Henning Heinold <heinold@inf.fu-berlin.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The description of option `-d' is not correct in useage. It is used to
remove duplicate and debug at the same time. Use option `-D' to control
debug info output and the option `-d' to flag remove duplicate.
[YOCTO #3635]
(From OE-Core rev: fa0b40c233e757fe986aa45798b35b60b89c879f)
Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The implementation of `--remove-duplicated' has been modified by the commit:
http://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/?id=0740f82aea92da0195524e37c372f9981e7f5e6d
In above commit, sstate cache files with multiple archs are not considered
duplicate and don't need to be removed as duplicated any more.
Update the description of `--remove-duplicated' in usage to keep consistent
with implementation.
[YOCTO #3635]
(From OE-Core rev: c201fdc2f01f398060cd953a1640a685797d9e64)
Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* The problem is that you're looking for duplicates across all available archs, so
armv4t and armv7a populate_sysroot are considered duplicate and only last one is
kept, but to rebuild from sstate-cache you need both!
(From OE-Core rev: 46309e96300c35f6e8a5f33512a6a38720c629d2)
Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* find them dynamically in layers like AVAILTUNES
(From OE-Core rev: fb6d838b1b80f26e9261acfd3893a7ef7f9e1940)
Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* many layouts have other layers on the same level as oe-core checkout
(From OE-Core rev: 6faf27b771176ff691c17a2dbfa98ac7b3ed1fc7)
Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* it was also picking line
openembedded-core/meta/conf/machine/include/README:AVAILTUNES - This is a list of all of the tuning definitions currently
filling all_archs with wrong entries
(From OE-Core rev: ceeefd964026e11ce0bb3b0285dbf9917533ae5e)
Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* using checksum without recipe/task name doesn't seem like good idea IMHO
(From OE-Core rev: ef1044da4b30b9a1264a7c83727a722a4b0b60cc)
Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fix it for the downloaded sstate cache file
* The downloaded sstate cache file is in SSTATE_DIR, and there is a
symlink points to it, the sstate cache file should not be removed if
the corresponding symlink is kept, it would be removed when the
symlink is removed (with -L option).
* Fix a comment line.
[YOCTO #2897]
(From OE-Core rev: 12ed7e6bb215e70b119d5ce123e36e2fd380ff1e)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Several fixes:
* We have put the sstate file to SSTATE_DIR/??/ currently, but the
sstate file on the SSTATE_MIRRORS or the obsolete one is still in
SSTATE_DIR/ (no subdir), update the script to support manage them.
* Remove the related ".done" file in the SSTATE_DIR.
* Add a "-L, --follow-symlink" which will remove both the symbol link and
the destination file
* Change the "ls -u file_list" (access time) to "ls -t file_list"
(change tiem), since the "ls -u" and readlink will change the
symlink's access time, which would make the result inconsistent.
A solution is save the access time before every "ls -u" and "readlink",
save it back after the command, but this would cause performance lost
since it needs check each file and modify the symlink's status. Use
the "-t" doesn't cause much different.
[YOCTO #2897]
(From OE-Core rev: 209ec08787981345a7f62b10a8a5c2ace0887c8e)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Update it for the new layout of sstate-cache.
Note: It doesn't handle the old sstate-cache/sstate-xxx.tgz (or
siginfo), the user should remove it manually (rm -fr
sstate-cache/sstate-* would be OK).
[YOCTO #2897]
(From OE-Core rev: fcb61aa552a641faafdf2216fa57ba5ac6b9991f)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fix a few problems and enhance its functions, it shoud be more useful
than before.
* Search in meta and meta-* for archs, and grep AVAILTUNES for archs,
(only search meta, and grep DEFAULTTUNE before), add the host arch.
and also can search in extra layers with --extra-layer.
* Reduce the analyzing time when remove duplicated files. It would cost
more than 10 minutes to analyze 11,000 files before, now only needs
about 50 seconds.
* Check the access time rather than create time.
* Need the user's confirm before really remove the file, or use --yes to
assume yes.
* Add --stamps-dir to keep sstate files which are used by the build
directory, and remove others this can make the sstate cache dir clean,
it is faster and should be useful than the --remove-duplicated.
* Add --verbose to explain what is being done.
* Add "-d" which is short for --remove-duplicated
[YOCTO #2198]
(From OE-Core rev: 769a000428e4b2462a4e6d8f179b5816b8ec2417)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There would be many obsolete cache files in the SSTATE_DIR after several
builds, this script can remove the obsolete one for a pkg, only leave
the up to date one.
Here is the help text:
sstate-cache-management.sh <OPTION>
Options:
--help, -h
Display this help and exit.
--cache-dir=<sstate cache dir>
Specify sstate cache directory, will use the environment
variable SSTATE_CACHE_DIR if it is not specified.
--remove-duplicated
Remove the duplicated sstate cache files of one package, only
the newest one would be kept.
[YOCTO #1682]
(From OE-Core rev: 7c99ef6d2173b14e1109a540ee5ae47b56d707e7)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Jan Sarenik
Jacob Kroon
Martin Jansa
Henning Heinold
Hongxu Jia
Richard Purdie
Robert Yang