Commit Graph

103 Commits

Author SHA1 Message Date
Maxin B. John a205c4ce1d bbclass: fix spelling mistakes
Fix some spelling mistakes in bbclass files

(From OE-Core rev: ed484c06f436eea62c5d0b1a2964f219f3e5cb61)

Signed-off-by: Maxin B. John <maxin.john@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-11-24 15:50:27 +00:00
Paul Eggleton 7cebff6bdc classes/buildhistory: split package history values only once
We don't actually use values we read in here that are likely to
contain = characters but we might as well split the value properly in
case we do in future.

(From OE-Core rev: afe100436bdeefb024b924ee27ad68830f085ff4)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-11-16 11:39:39 +00:00
Paul Eggleton d14ffaf6a3 classes/buildhistory: add build result to commit message
We have the command in the commit message, we might as well have the
build result as well (succeeded/failed and whether or not it was
interrupted by the user). The interrupted part relies upon a change to
BitBake to extend the BuildCompleted event to include an attribute for
that, but will not fail if the attribute is not present.

(From OE-Core rev: 6ca7f5c0d0f024ae5d21368188e3428534cab2a4)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-30 12:36:12 +01:00
Paul Eggleton 734411a9e6 classes/buildhistory: include metadata revisions in commit message
We do already commit these into the repository itself, but have them in
the commit message as well as a reference. As part of this, refactor out
running "git commit" into a separate function so we don't have to
duplicate the code in the two places we call it.

Implements [YOCTO #7966].

(From OE-Core rev: c77a068e534b274dee7fb8a2399ecafd33da0587)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-30 12:36:12 +01:00
Paul Eggleton 2a08f4a0e5 classes/buildhistory: handle additional files at recipe level
Avoid an error when attempting to remove previous data if it's not a
subdirectory - we were assuming that anything that wasn't named "latest"
or "latest_srcrev" had to be a directory. This makes it possible to have
a buildhistory_emit_pkghistory_append which writes additional files at
the recipe level.

(From OE-Core rev: b018a046a578e41b105cf72b9fdeed0220ae3046)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-30 12:36:11 +01:00
Paul Eggleton 846cb34afa classes/buildhistory: fix permissions in package file listing
If we want the correct file permissions to show up here as they would on
the target, we need to run the command under pseudo. Normally we'd set
the fakeroot varflag on the function and that would be enough, but it
turns out that setting fakeroot on a non-task function that you execute
using bb.build.exec_func() isn't working at the moment. Work around this
by simply using FAKEROOTENV and FAKEROOTCMD. Unfortunately that means we
have to duplicate the command for the two cases but I couldn't find a
better means of doing that that actually works.

(From OE-Core rev: 1380aa333ed90559f4a24d52aefc52cadb60646c)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-30 12:36:11 +01:00
Paul Eggleton 1bfae0fd81 classes/buildhistory: tweak buildhistory_list_pkg_files
* Avoid using ${...} for shell variables (since they could be
  expanded as bitbake variables if present)
* Use files-in-package.txt rather than files-in-<packagename>.txt; the
  file is already in a subdirectory named with the package name and this
  naming is consistent with that of files-in-image.txt.

(From OE-Core rev: 6f3992728613c39403ef59bbcf1cb67d9e8c526b)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-30 12:36:11 +01:00
Paul Eggleton 5d34d3257a classes/buildhistory: indent recently added function consistently
Shell functions use tabs in this file.

(From OE-Core rev: 14eba06baacca25213e35afa7bfd126fc1f5586a)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-30 12:36:10 +01:00
Paul Eggleton ec412d5c76 classes/buildhistory: exclude . in file listings
We don't care about the permissions on the top-level directory in which
the files are contained, just everything under it; this also avoids
lists with just this entry in it for empty packages. Affects file
listings for both images and packages.

(From OE-Core rev: d503122646b1b148113a4be31b7e88723459dae6)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-30 12:36:10 +01:00
Paul Eggleton e6c330c7e5 classes/buildhistory: ensure we push when "no changes" commits are made
If there aren't any changes, we still make a commit to the buildhistory
repo, but this wasn't being pushed if BUILDHISTORY_PUSH_REPO is set.
Move the push to the end to make it unconditional.

(From OE-Core rev: af2fc914975d06fdd75a65d4b00cf9f5c1610528)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-30 12:36:09 +01:00
Leonardo Sandoval b250b5c7be buildhistory.bbclass: Check output folder is present before creating files-in-$pkg.txt
This is just a safe check to make sure the output folder is present, before
creating the files-in-$pkg.txt file.

(From OE-Core rev: d48233cb1fd84f1061a8ba3c15bacc180e93934c)

Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-16 15:09:16 +01:00
Leonardo Sandoval 01c9c7394d buildhistory.bbclass: Create a files-in-<pkg>.txt containg recipe's pkg files
A file similar (in format) to files-in-image.txt is created for each
recipe's build package. This would allow to easily see what the
recipe's package may install when including it into an image.

These files reside on the specific buildhistory recipe package (i.e.
buildhistory/packages/i586-poky-linux/busybox/busybox-dev/files-in-busybox-dev.txt
for busybox-dev package)

[YOCTO #5870]

(From OE-Core rev: e013fe79cdf67429ff986d8ee6af510b34b31c9d)

Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-27 22:42:54 +01:00
Paul Eggleton 1b2332d338 classes/buildhistory: ensure that git user email and name are set
The git user.email and user.name settings must be set or the commit to
the buildhistory repo (when BUILDHISTORY_COMMIT is set to "1") will
fail. If the user hasn't set these, set some dummy values since the
actual value is unlikely to be particularly useful and it's far worse
for the commit not to be done leaving the repository in a partially set
up state.

Prompted by [YOCTO #6973].

(From OE-Core rev: f62255bfa6c5a322c867b7c4ea5686ea7bfab3fe)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-24 07:19:18 +01:00
Paul Eggleton 9f03969994 oe-pkgdata-util: improve command-line usage
* Use argparse instead of optparse for standardised help output, options
  and a much cleaner code structure
* Look up pkgdata directory automatically so the user doesn't have to
  specify it
* Use standard logging

NOTE: this does mean a slight change in syntax - if you do want to
specify the pkgdata directory (usually only necessary if you're calling
it from within the build process) you need to use the parameter -p (or
 --pkgdata-dir) and specify this before the command, not after it.

Examples:

oe-pkgdata-util find-path /sbin/mke2fs
oe-pkgdata-util lookup-recipe libelf1
oe-pkgdata-util read-value PKGSIZE libc6
oe-pkgdata-util -p /home/user/oe/build/tmp/sysroots/qemux86-64/pkgdata read-value PKGSIZE libc6

(From OE-Core rev: 04dc571ac7c26f0dcf1a1fcd466482e22519998d)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-02-14 08:41:01 +00:00
Paul Eggleton 26e5b0f7b9 classes/image: ensure uninstalled packages do not appear in manifests
Since the rewrite of the image construction code in python a few
releases ago, we remove a couple of packages from the image as one of
the final steps when constructing the image (notably update-rc.d and
run-postinsts).  However, because of the order of operations, these
packages are still listed both in the buildhistory
installed_package*.txt files and in the manifest file created next to
the image, which is wrong.

There were two possible solutions to this: (1) change the order such
that the uninstallation occurs before calling ROOTFS_POSTPROCESS_COMMAND
or (2) add another hook variable in such that we can have the
package list collection code run at the right time. Because it's
currently possible (but very much not recommended) to install additional
packages within ROOTFS_POSTPROCESS_COMMAND, which may have postinstall
scripts and thus require the packages we would otherwise uninstall if we
were to take option 1, option 2 is really the least likely to cause
problems. Therefore, add ROOTFS_POSTUNINSTALL_COMMAND and make the image
and buildhistory classes use it.

Fixes [YOCTO #6479].

(From OE-Core rev: b198a189228648057c3be7d068598f50841b3bf9)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-02-03 14:53:50 +00:00
Alejandro Hernandez ed3e25379c image-buildinfo.bbclass: new class, writes build information to image
Writes build information to target filesystem on /etc/build such as enabled
layers, their current status and commit.

squashspaces was moved to oe/utils.py to make it available to different classes
and avoid code duplication.

[YOCTO #6770]

(From OE-Core rev: c9cc652e88ddedddf8a2f23fb9b8c024616526d7)

Signed-off-by: Alejandro Hernandez <alejandro.hernandez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-11-09 10:21:22 +00:00
Paul Eggleton a41d97a23b classes/buildhistory: fix for sstate class change
SSTATEPOSTINSTFUNCS is now set with = in sstate.bbclass, and because
the line here in buildhistory.bbclass ends up being parsed before that
we now need to use _append or the value we are adding will be wiped out.
This fixes buildhistory no longer recording package information since
OE-Core revision 9d659c6f20fa4a141b491c62a3ef0dfb1f896d9c.

(From OE-Core rev: b5c05b474554925524073023f3aac2532c87537f)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-09-10 13:52:31 +01:00
Martin Jansa e601c8b160 buildhistory: Restore parsing time optimalization
* it was added in:
  http://git.openembedded.org/openembedded-core/commit/meta/classes/buildhistory.bbclass?id=540a2a30be21c3eca4323efbe91e7dcfc31a4c97
* then accidentally removed in:
  http://git.openembedded.org/openembedded-core/commit/meta/classes/buildhistory.bbclass?id=6db6c74f2876df7a183990426bb4262972966441

(From OE-Core rev: cbd868c34718e8fd2a41f5382da71ee959ea4e71)

Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-08-25 10:26:00 +01:00
Martin Jansa 3223a704f9 buildhistory: rename build-id to build-id.txt and add more info
* use .txt extension like other reports (e.g. image-info.txt) it's
  also useful when looking at the file from web server which doesn't
  have text/plain as default content-type
* include whole BUILDCFG_HEADER/BUILDCFG_FUNCS instead of just layer
  info
* this makes it easier for distro to include more fields without the
  need to modify buildhistory.bbclass
* currently it adds following fields:
  Build Configuration:
  BB_VERSION        = "1.23.1"
  BUILD_SYS         = "x86_64-linux"
  NATIVELSBSTRING   = "Gentoo-2.2"
  TARGET_SYS        = "x86_64-oe-linux"
  MACHINE           = "qemux86-64"
  DISTRO            = "shr"
  DISTRO_VERSION    = "2013.07-next-20140725"
  TUNE_FEATURES     = "m64 core2"
  TARGET_FPU        = ""

(From OE-Core rev: 6db6c74f2876df7a183990426bb4262972966441)

Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-08-18 21:47:58 +01:00
Matthieu Crapet 77790c43b1 *.bbclass (shell): avoid pipe with sed
Replace:
cat <file> | sed -e xxx
By:
sed -e xxx <file>

(From OE-Core rev: e2026f5d32ac05396615224ac9ec927439e7e6b4)

Signed-off-by: Matthieu Crapet <Matthieu.Crapet@ingenico.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-05-08 13:00:33 +01:00
Otavio Salvador 4c14b09498 Globally replace 'base_contains' calls with 'bb.utils.contains'
The base_contains is kept as a compatibility method and we ought to
not use it in OE-Core so we can remove it from base metadata in
future.

(From OE-Core rev: d83b16dbf0862be387f84228710cb165c6d2b03b)

Signed-off-by: Otavio Salvador <otavio@ossystems.com.br>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-04-25 17:19:19 +01:00
Laurentiu Palcu 2cf83f49e3 buildhistory.bbclass: create proper dependency files for SDK
The old functions were calling the list_installed_packages() wrapper
function that only listed the packages in an image rootfs. Even for
target/host SDK. Also, a python crash was possible if 'bitbake -c
populate_sdk core-image-*' was called without calling 'bitbake
core-image-*' first. That's because the wrapper was always looking into
the image rootfs...

This commit fixes the problem and calls the right wrapper for image/sdk.

(From OE-Core rev: c1b1a6eb448aa1548e2ec669a9304b5a25bd8ba5)

Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-03-21 12:05:53 +00:00
Laurentiu Palcu 6029af2ec5 buildhistory.bbclass: Fix dependency files creation
Call the new python routines.

[YOCTO #5904]

(From OE-Core rev: 570baf4921a5b34cc97834e670b143073d136dea)

Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-03-11 20:28:56 -07:00
Paul Eggleton 229bcde8e4 classes/buildhistory: fix task signatures changing
Fix task signatures for recipes that add to SSTATEPOSTINSTFUNCS changing
when adding and removing INHERIT += "buildhistory" (really this time!)

This relies on the BitBake vardepvalueexclude feature, however it will
not fail without it - signatures will be changed in that case though.

Part of the fix for [YOCTO #5897].

(From OE-Core rev: 27c8a9a282358b9a8a330252ee2104b250777b38)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-03-07 14:49:32 +00:00
Laurentiu Palcu 7e595a5f94 buildhistory.bbclass: fix creation of installed packages list
Call the new python routines to create the packages list.

[YOCTO #5831]

(From OE-Core rev: e8bd5b85f144a3260e30f67c3d231b324967be3f)

Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-02-20 14:28:11 +00:00
Laurentiu Palcu 6a6cdadf31 buildhistory.bbclass: create wrapper functions around buildhistory_get_sdk_installed
bb.build.exec_func() does not allow passing arguments to the executed
scripts. Use wrappers instead.

(From OE-Core rev: c3dd44b751e351be52512e2aff885bf571c95fe5)

Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-02-13 17:53:30 +00:00
Paul Eggleton 4021080d8e classes/buildhistory: avoid buildhistory triggering rebuilds (again)
Postfuncs are now incorporated into task signatures as of BitBake
rev b84d010144de687667cf855ddcb41c9b863c236e, so we need to exclude
the one we're adding to do_fetch. The change to the SSTATEPOSTINSTFUNCS
variable value was also entering the signatures of do_populate_sysroot
and probably other tasks, so we need to use a slightly hacky trick to
avoid that as well.

Apart from the final do_rootfs / do_populate_sysroot, this now means
that adding and removing INHERIT += "buildhistory" will not cause tasks
to be re-run.

Also update the copyright date, properly this time.

(From OE-Core rev: 953df67eb877a6d0fc68d122964440a9a47de3c3)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-02-11 17:15:54 +00:00
Paul Eggleton 618931eb02 classes/buildhistory: write out files in UTF-8 format
If the package information contained unicode characters (for example, in
the package file listing) then writing them out as ASCII would fail, so
write them out using codecs.open() instead of open() using UTF-8. This
fixes ca-certificates failing in do_packagedata when buildhistory is
enabled.

(From OE-Core rev: dcf228fe69bfee4e22baad477ad407248c0f9cdb)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-02-06 15:30:44 +00:00
Paul Eggleton 220f4f5806 classes/buildhistory: fix expansion of escape sequences
OE-Core commit 259b8718a31b886f8a158aeb5de164840c9a28b2 fixed UTF-8
errors but broke decoding of escape sequences in strings (e.g.
pkg_postinst scripts had \n \t in them instead of newlines and tabs.)
We need a second call to decode() here as specifying 'string_escape' as
the second parameter won't do anything.

(From OE-Core rev: 15e0cdff08b8b9b826bbb9f00192a27318a3ee65)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-02-03 11:57:09 +00:00
Otavio Salvador f328a9d53b buildhistory.bbclass: Improve robustness in image file listing
The filenames sometimes may have strange names. With the 'awk' script
it handled a limited number of spaces in the filename and a package
installing a file named "test file with spaces" would have its name
truncated.

This patch uses the find's printf formating to simplify the code and
properly handle this case. From a testing image, the only diff produced
is:

,----[ files-in-image.txt diff ]
| --rwxr-xr-x root       root                0 ./usr/bin/test\ file\ with\
| +-rwxr-xr-x root       root                0 ./usr/bin/test file with spaces
`----

The options used are available since findutils 4.2.5, released in 19
Nov 2004, making it available in all supported host distributions.

(From OE-Core rev: b09e24449c1c9ae335732dd070eacf66777556a1)

Signed-off-by: Otavio Salvador <otavio@ossystems.com.br>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-01-28 11:32:50 +00:00
Ross Burton 7e17255fa8 buildhistory: be quiet when garbage collecting the git repo
(From OE-Core rev: b117403423f700eb4cc6967b9fb8ff3e2858e279)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-12-18 17:08:05 +00:00
Khem Raj b9e1f1118e buildhistory.bbclass: Specify lang in decoding strings
On systems where default locale is utf-8 we get errors like

File: 'buildhistory.bbclass', lineno: 38, function: write_pkghistory
0034: if pkginfo.rconflicts:
0035: f.write("RCONFLICTS = %s\n" % pkginfo.rconflicts)
0036: f.write("PKGSIZE = %d\n" % pkginfo.size)
0037: f.write("FILES = %s\n" % pkginfo.files)
*** 0038: f.write("FILELIST = %s\n" % pkginfo.filelist)
0039:
0040: for filevar in pkginfo.filevars:
0041: filevarpath = os.path.join(pkgpath, "latest.%s" % filevar)
0042: val = pkginfo.filevars[filevar]
Exception: UnicodeEncodeError: 'ascii' codec can't encode character
u'\xed' in position 337: ordinal not in range(128)

This patch specifies decode to use utf-8 so ascii and utf-8 based
locales both work

(From OE-Core rev: 259b8718a31b886f8a158aeb5de164840c9a28b2)

Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-12-10 17:42:44 +00:00
Roy Li 40b173c104 buildhistory.bbclass/image.bbclass: remove obsolete codes
After 1b8e4abd2d9c0 [bitbake.conf/package: Collapse PKGDATA_DIR into
a single machine specific directory], oe-pkgdata-util does not use
target_suffix parameter, so do not need to loop the vendor

(From OE-Core rev: 03c5f39b4d7dd8c81e0a130b7d5884e5af039a24)

Signed-off-by: Roy Li <rongqing.li@windriver.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-12-09 18:01:45 +00:00
Paul Eggleton 5c69ae26cd classes/buildhistory: fix reading of package-specific values from pkgdata
When writing out variable values to pkgdata, if the value has been set
in the datastore with an override for the package, we use the package
name override in the pkgdata key as well; however the recently added
code to read pkgdata in buildhistory.bbclass was just using the override
where we normally expect to have it. However, if a recipe overrides one
of the values that is normally set for the recipe on a per-package basis
(e.g. the external-sourcery-toolchain recipe sets PKGV this way) then
this led to KeyErrors. Re-write the pkgdata loading code to always strip
off the package name override if it is present.

(From OE-Core rev: e40e8e574b3688400a668d3ad76b6cef1920e3e0)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-12-05 14:25:30 +00:00
Paul Eggleton 1fa51bf949 classes/buildhistory: do git garbage collection after committing
We don't normally perform any operations (such as "git pull") that
trigger "git gc --auto", thus garbage collection never happens which
means performance of accessing the repository degrades noticeably over
time. Add an explicit "git gc --auto" to clean things up when needed.

Thanks to Elijah Newren and Ross Burton for suggesting this.

(From OE-Core rev: 4a45a999e0ad2e99581428a5a6d34f483c00544f)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-12-03 17:45:50 +00:00
Paul Eggleton d51b3f3758 classes/buildhistory: reduce parsing time
Disable several python functions if not parsing within the worker
context. This avoids executing expensive operations while parsing
recipes (which is unnecessary).

(Thanks to Richard Purdie for pointing out the issue and suggesting the
workaround.)

(From OE-Core rev: 540a2a30be21c3eca4323efbe91e7dcfc31a4c97)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-12-03 17:45:50 +00:00
Paul Eggleton cdfef971d3 classes/buildhistory: improve collection of package info
Use a function added to SSTATEPOSTINSTFUNCS and read the necessary
information out of pkgdata, instead of using a function executed
during do_package that reads the data directly. This has two benefits:

* The package info collection will now work when the package content is
  restored from shared state

* Adding/removing the inherit of buildhistory will no longer change the
  do_package signatures and force re-execution of that function for
  every recipe.

Fixes [YOCTO #5358]

(From OE-Core rev: cd7f7efcd5f297d876823b8f579ecefb9542b089)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-12-03 17:45:50 +00:00
Paul Eggleton 0cb38ee34a classes/buildhistory: add additional variables to image information
Add PACKAGE_EXCLUDE and NO_RECOMMENDATIONS to the info we track for
images, since these can change what ends up in the image.

(From OE-Core rev: a10189366f180b87f5be20b66834b7e7a9bb8c12)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-12-03 17:45:50 +00:00
Paul Eggleton 1c37266c2a classes/buildhistory: drop cruft from old SRCREV tracking implementation
This should have been removed when the implementation was rewritten in
OE-Core commit 2179db89436d719635f858c87d1e098696bead2a. The collected
values weren't being used anywhere since then.

(From OE-Core rev: cbc23a87c1897b7fda40f452dd36acb0bca3d197)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-12-03 17:45:49 +00:00
Richard Purdie e031a05b6d buildhistory/distrodata: Adapt to updated fetcher API
The buildhistory code has fallback paths for older bitbakes for now. The
distrodata class is much less used and it can be assumed a recent bitbake
is used in that case rather than adding fallback code.

(From OE-Core rev: 570cc145029fd9d5528aef5c27cb65164265c799)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-11-22 12:03:02 +00:00
Richard Purdie 8ebe7be3d9 bitbake.conf/package: Collapse PKGDATA_DIR into a single machine specific directory
Currently we have a hierarchy of pkgdata directories and the code has to put together
a search path and look through each in turn until it finds the data it needs.

This has lead to a number of hardcoded paths and file globing which
is unpredictable and undesirable. Worse, certain tricks that should be
easy like a GL specific package architecture become problematic with the
curretn search paths.

With the modern sstate code, we can do better and construct a single pkgdata
directory for each machine in just the same way as we do for the sysroot. This
is already tried and well tested. With such a single directory, all the code that
iterated through multiple pkgdata directories and simply be removed and give
a significant simplification of the code. Even existing build directories adapt
to the change well since the package contents doesn't change, just the location
they're installed to and the stamp for them.

The only complication is the we need a different shlibs directory for each
multilib. These are only used by package.bbclass and the simple fix is to
add MLPREFIX to the shlib directory name. This means the multilib packages will
repackage and the sstate checksum will change but an existing build directory
will adapt to the changes safely.

It is close to release however I believe the benefits this patch give us
are worth consideration for inclusion and give us more options for dealing
with problems like the GL one. It also sets the ground work well for
shlibs improvements in 1.6.

(From OE-Core rev: 1b8e4abd2d9c0901d38d89d0f944fe1ffd019379)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-09-14 08:21:00 +01:00
Paul Eggleton c5d2e9caca classes/buildhistory: avoid mangling names in dot graphs for images
We were mangling names here to avoid characters that are invalid for dot
node names, but if you just quote all names that isn't necessary and we
retain the original naming, allowing easy searching of the graph files.

(From OE-Core rev: 99efdad886aa5063a68912846c00a46ba9b15536)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-08-16 11:44:17 +01:00
Paul Eggleton 1e8e24358a classes/buildhistory: tag last 3 build revisions
This makes it easier to obtain the difference from the last three
builds particularly where a single build might account for more than one
commit in the buildhistory git repository (e.g. if package and image
changes occurred).

(From OE-Core rev: 9512f3787295d662678c2943da31197c8df39e99)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-08-16 11:44:17 +01:00
Paul Eggleton 742266d660 classes/buildhistory: show command line in commit message
Record the bitbake command line in the commit message as a further
context indication.

(From OE-Core rev: 34fb802b2170f0fe208e54e7a4d18f663db72f90)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-08-16 11:44:16 +01:00
Martin Jansa 855b3252d9 classes/buildhistory: record size of installed package not compressed archive
* usually it's more important to know how much space will each
  package take on target device then size of compressed package
* example for libewebkit0 with 4 different architectures, interesting
  that om_gta02 .ipk is bigger but it's smaller when installed

  before:
  MACHINE     DEFAULTTUNE       SIZE (.ipk file)
  om_gta04    cortexa8t-neon    15996 KiB libewebkit0
  qemux86_64  x86-64            16992 KiB libewebkit0
  spitz       xscale            16148 KiB libewebkit0
  om_gta02    arm920t           16260 KiB libewebkit0

  after:
  MACHINE     DEFAULTTUNE       SIZE (installed)
  om_gta04    cortexa8t-neon    60544 KiB libewebkit0
  qemux86_64  x86-64            63720 KiB libewebkit0
  spitz       xscale            60588 KiB libewebkit0
  om_gta02    arm920t           56268 KiB libewebkit0

(From OE-Core rev: 85e4a77138381a6086d5ebd3a28cb5a94bc26a19)

Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-08-16 11:44:16 +01:00
Paul Eggleton 23575b4fdb classes/buildhistory: remove default of BUILDHISTORY_CHECKVERBACKWARDS
Since this variable is no longer used, we don't need to set a default
for it.

(From OE-Core rev: a5230835c539781b5b035dc6d0be3cac5a5bd305)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-08-16 11:44:16 +01:00
Martin Jansa 5c7b8fc342 buildhistory: use bb.utils.mkdirhier instead of os.makedirs
* Multiple do_fetch[prefunc] can happen at the same time and
  if not os.path.exists(pkghistdir):
    os.makedirs(pkghistdir)
  isn't safe
* Use bb.utils.mkdirhier which doesn't raise error when directory exists

(From OE-Core rev: 9aac194d7db79129dcbed29cfb89c57dccf33729)

Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-08-12 13:09:50 +01:00
Martin Jansa 560dac6e7d buildhistory: use package_qa_handle_error for version-going-backwards
* this way it's easier for distro to select if such issue should be
  error, warning or ignored and also it gets recorded in qa.log

(From OE-Core rev: 44624853de8c88abdcae2ccd0902b390ff546d4c)

Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-08-12 13:09:50 +01:00
Ross Burton 60a8903ab2 buildhistory: add a note when committing to buildhistory
If there's been a lot of changes the commit to buildhistory can take a
significant amount of time, so print a note so there's some indication of
progress.

(From OE-Core rev: ccd911ba8887f5c9a72d5178617c2a3ee73264ff)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-08-06 12:39:54 +01:00
Paul Eggleton 827400fd09 classes/buildhistory: fix error when no packages are installed
"xargs -n1 basename" will execute basename even if there are no lines
piped into it, causing a "basename: missing operand" error if no
packages are installed, which will happen for the target portion of
buildtools-tarball.

(xargs' -r option could have been used here, but it is a GNU extension
and I thought it best to avoid that for the sake of future
interoperability).

(From OE-Core rev: 03d86123e9e804e62de38effd307f070d7863080)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-06-20 13:10:46 +01:00