Commit Graph

26 Commits

Author SHA1 Message Date
Richard Purdie 98bd952a5b Revert "buildhistory_analysis: fix error when comparing image contents"
This reverts commit 5b616aa7b6.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-04-04 16:16:50 +01:00
Paul Eggleton 5b616aa7b6 buildhistory_analysis: fix error when comparing image contents
OE-Core commit b7de1eaac9eed559b2d68058f5de67de74a6cb58 added an extra
argument to the compare_dict_blobs() function but missed adding the
argument to one call to compare two versions of the image-info.txt file.

(From OE-Core master rev: 24a45d752c3e3d0d8b59c040355e4fe7de22b041)

(From OE-Core rev: a51d96c44e6feac8322284c54bfc01ef598f8821)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-12-12 17:00:19 +00:00
Paul Eggleton e42f0acf46 classes/buildhistory: track contents of selected files in images
Add a BUILDHISTORY_IMAGE_FILES variable specifying a space-separated
list of files within an image to copy into buildhistory, so that changes
to them can be tracked. Typically this would be used for configuration
files, and by default this includes /etc/passwd and /etc/group, but the
user is free to extend this list by setting the variable in local.conf.

Implements [YOCTO #4154].

(From OE-Core rev: ed6bb6e3db518082d3a9c45d548bc1339be2c5ca)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-05-10 11:37:16 +01:00
Paul Eggleton 98755bd799 buildhistory_analysis: handle more R* variables
Report changes to RPROVIDES, RREPLACES, and RCONFLICTS. As RSUGGESTS
isn't widely used and isn't of huge concern if it changes, it is not
reported by default.

Implements [YOCTO #3391].

(From OE-Core rev: d20011571db96da79a8a0e056c6cef8e4c083608)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-02-11 14:46:10 +00:00
Paul Eggleton b660a10e47 buildhistory_analysis: fix broken list length checks
Fix erroneous use of .count instead of len(), which unfortunately is not
reported by Python as an error in a numeric comparison.

(From OE-Core rev: 63fd76190f503660119dcc8efdcfc6fbff406c26)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-12-06 13:51:38 +00:00
Paul Eggleton 3b664d0cfc buildhistory_analysis: update to use explode_dep_versions2()
Handle where multiple version specifications are present for the same
dependency.

(From OE-Core rev: 1600c916ae410c57a783a5aa35abe07a3f673311)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-10-03 17:18:29 +01:00
Ross Burton a71dfcc338 buildhistory: reduce the numeric delta to 10%
(From OE-Core rev: f6effbf62351761430438cfcce285c982271239c)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-08-06 16:09:15 +01:00
Ross Burton 36c7c64519 buildhistory: correctly handle reductions of numeric fields
(From OE-Core rev: 94fffb92e6a7a8021fb1acb06b9c4218703e23b2)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-08-06 16:09:15 +01:00
Paul Eggleton 4bf1c0a7dc buildhistory_analysis: ignore removal of self-dependencies
The recent removal of self-dependencies for dbg/dev packages produced
a fair amount of noise, so filter it out since we know this is a
reasonable change.

(From OE-Core rev: 3ec608462e4cb1eb77b6addbae1bda84fd76e4d2)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-08-06 15:29:46 +01:00
Paul Eggleton 507dba64e0 buildhistory_analysis: tidy up duplicate split_version function
This function is now provided by bb.utils and since we have now bumped
the minimum bitbake version, we can switch to that one instead.

(From OE-Core rev: 67a4d15f0a8032190e92026677150162a492a053)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-08-06 15:29:46 +01:00
Paul Eggleton 81485a47d5 classes/buildhistory: record PKG/PKGE/PKGV/PKGR
Save PKG (the actual output package name, which is often different due
to debian renaming), and PKGE/PKGV/PKGR (which may be manipulated in
certain special cases e.g. gitpkgv.bbclass in meta-oe, the
external-sourcery-toolchain recipe, etc.) Note that these are only
written when they are different from the normal package name in the
case of PKG, or PE/PV/PR for the other variables.

Also, use PKGE/PKGV/PKGR instead of PE/PV/PR when comparing package
versions since these actually represent the version that the package
manager sees.

Implements [YOCTO #2787].

(From OE-Core rev: 65d7e9b2d4d8115ac9fd513c04f39a2df9556a5a)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-08-06 15:29:46 +01:00
Paul Eggleton c0149ac6c4 classes/buildhistory: save preinst/postinst/prerm/postrm
Write the value of these package script variables into the packageinfo
so that any changes to them can be tracked (in separate files since they
are multi-line).

Inspired by an earlier patch from Andreas Müller <schnitzeltony@googlemail.com>

(From OE-Core rev: 988a417c857c01c87de6ba9602968059cf8d830f)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-08-06 15:29:45 +01:00
Paul Eggleton 1fe254f123 buildhistory_analysis: fix error when version specifier missing
Passing None to split_versions() will raise an exception, so check that
the version is specified before passing it in.

(From OE-Core rev: 90caa605bd24279e4a9204e279492754119a8a55)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-05-30 12:04:46 +01:00
Paul Eggleton 42c445bc04 buildhistory_analysis: avoid printing PE/PV/PR more than once
Don't print PE/PV/PR changes as related field changes of related field
changes (i.e. only print them once at the top level).

(From OE-Core rev: 33046605549501bed9ceff102614d86ba54a5891)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-03-21 15:33:07 +00:00
Paul Eggleton 4db604cd3a buildhistory_analysis: report if all items removed from a list
If all items have been removed from a list then state that explicitly.

(From OE-Core rev: 2434a751e9684058786d4595cef3678d397f3c39)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-03-21 15:33:07 +00:00
Paul Eggleton 275ef4900e buildhistory_analysis: skip FILELIST changes for dbg packages
Don't report when files are added or removed from dbg packages unless
it results in the package being empty.

(From OE-Core rev: ee57a989188519dd40310cb5576cbff4b247ad95)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-03-21 15:33:07 +00:00
Paul Eggleton 9d8e5ab47a buildhistory_analysis: hide version number increases in dependencies
If an item in RDEPENDS or RRECOMMENDS only increases in its version
number then don't report it as a change, since we don't care about
it. This significantly reduces the noise after upgrades.

(From OE-Core rev: f72b2a1bda35a99292063c1cc6ff563b397e190d)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-03-21 15:33:06 +00:00
Paul Eggleton 0b8a693fb4 buildhistory_analysis: use bb.utils.explode_dep_versions
Previously this had its own implementation of splitting a list of
packages with optional version e.g. "libncurses-dev (>= 5.9)"; switch to
using the already existing bitbake function which does this as it is
much better tested.

(From OE-Core rev: de21a483063d9803c4ce1d62b03913ccad2931bd)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-03-21 15:33:06 +00:00
Paul Eggleton 32483f0562 buildhistory_analysis: fix duplicate PE/PV/PR in related fields
Since PE, PV and PR appear in both the recipe history and package
history files these were showing up twice when they were added as
related fields to monitored changes. Only add them when the path is
exactly the same.

(From OE-Core rev: 406d025a15ff15a2edf39f00e0ea4e6b821b224a)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-03-19 14:37:41 +00:00
Paul Eggleton 5aec33be72 buildhistory_analysis: correctly split RDEPENDS/RRECOMMENDS
Split RDEPENDS and RRECOMMENDS correctly (which may contain version
number specifications after each item).

(From OE-Core rev: d974ed4330f33c4f8dfd635a83c61a0d44e688b6)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-02-21 17:59:38 +00:00
Paul Eggleton 79eb4b8fdb buildhistory_analysis: avoid noise due to reordering
Avoid noise in the output due to reordering of list variables (except
for PACKAGES where we just report that the order changed). Recent
changes to the buildhistory class itself will avoid this reordering
from occurring but this allows us to examine the results before and
after those changes.

(From OE-Core rev: e23c5b01766602c9c86b0a7ba170fb3b1aceb658)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-02-21 17:59:38 +00:00
Paul Eggleton 967de59f35 buildhistory_analysis: include related fields in output
Sometimes, when a value changes in the buildhistory it is useful to
know when a related (but not necessarily itself monitored) value
changes as it can help explain the change. For example, when the list
of installed packages for an image changes it could be caused by a
change to one of the image-related variables.

Related field changes are recorded as sub-items of each change.
Currently the only way to visualise these is via the buildhistory-diff
tool, so an example would be:

Changes to images/qemux86/eglibc/core-image-minimal (installed-package-names.txt):
  locale-base-de-de was added
  procps was added
  * IMAGE_LINGUAS: added "de-de"
  * IMAGE_INSTALL: added "procps"

Here we see that two additional packages have been added to the image,
and looking at the related changes to the two variables IMAGE_INSTALL
and IMAGE_LINGUAS we have the explanation as to why.

(From OE-Core rev: 2c5b90c6d1b1091779602ebe03e84674eb63ea83)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-01-19 14:59:23 +00:00
Paul Eggleton 5cf28333a7 buildhistory_analysis: improve field handling robustness
Avoid errors when comparing changes for KEY = value files (package info
files and image-info.txt):

* Handle keys appearing and disappearing - this will help to handle PE
  in package info files (which is only written when it is not blank) and
  when we add additional fields in future.
* Handle when old value is 0 for numeric field (avoid division by zero)
* Report when numeric field was empty or missing rather than 0 (but
  still treat it as 0 for comparison purposes)

(From OE-Core rev: 255d4bbf4d1e430d45f5fafb7d1c77d9ea67e174)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-01-19 14:34:33 +00:00
Paul Eggleton c1cebf6a2b buildhistory_analysis: correctly handle whitespace when splitting lists
Don't specify any argument to the split() function when handling changes
to list type variables (e.g. PACKAGES) so that the values are split by
any whitespace and only split once for a block of multiple whitespace
characters.

(From OE-Core rev: 15ad5d2c0e92fefdbb7c0cf064134b1cabfd84ac)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-01-19 14:34:33 +00:00
Paul Eggleton 2f4f8ef0fc buildhistory: record additional image info
Record some additional information about images - the uncompressed size
of the final image as well as the values of various variables that may
have influenced its contents. This is recorded in a machine-readable
"image-info.txt" file similar in structure to the package history files.

Also add some code to analyse changes to these values. (Most of the
variable values aren't monitored directly but will be used as contextual
information when they change at the same time as the content of the
image changing.)

(From OE-Core rev: 459ed6759a307b389f6ec1874136ec9aa0749120)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-01-19 14:34:32 +00:00
Paul Eggleton fcc5f6883c buildhistory: add script to check for significant changes
Adds a buildhistory-diff script which can be used to analyse changes in
the buildhistory git repository (as produced by buildhistory.bbclass),
and report significant ones that may need manual checking to ensure they
aren't regressions (e.g. package size changed by more than a certain
percentage, files added/removed/changed in the image, etc.)

The implementation is actually split into a small script and a Python
module, in order to make the logic re-usable in a future web-based
interface.

Implements the first part of [YOCTO #1566].

(From OE-Core rev: 5e5cbb9bd8cdce402b979680288ac8c51799a24d)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-01-06 12:11:30 +00:00