classes/buildhistory: save output file signatures for sstate tasks

Save a file per task listing sha256sums for each file staged, i.e.
the output of the task. Some caveats:

1) This only covers sstate tasks since it uses SSTATEPOSTUNPACKFUNCS,
   however those are generally the most interesting in terms of output
   anyway.
2) The signature is taken before applying any relocations, so any
   relocated files will actually have different signatures, but that's
   churn that you probably won't want to see here.
3) At the moment if you run the same build twice without sstate you will
   very likely see changes in the output for certain tasks due to things
   like timestamps being present in the binary output. Fixing that is
   a general Linux ecosystem problem - see this page for our efforts to
   resolve it on our side:
     https://wiki.yoctoproject.org/wiki/Reproducible_Builds

NOTE: you need to set your BUILDHISTORY_FEATURES value to include
"task" to enable collection of these signatures as it is is disabled by
default.

(From OE-Core rev: ca5d1273432e20059ab66d721a9eb314a54e81e7)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This commit is contained in:
Paul Eggleton 2017-04-07 16:57:23 +12:00 committed by Richard Purdie
parent 54426c3a3b
commit 3df7ee4755
1 changed files with 28 additions and 0 deletions

View File

@ -47,6 +47,11 @@ sstate_install[vardepsexclude] += "buildhistory_emit_pkghistory"
# then the value added to SSTATEPOSTINSTFUNCS:
SSTATEPOSTINSTFUNCS[vardepvalueexclude] .= "| buildhistory_emit_pkghistory"
# Similarly for our function that gets the output signatures
SSTATEPOSTUNPACKFUNCS_append = " buildhistory_emit_outputsigs"
sstate_installpkgdir[vardepsexclude] += "buildhistory_emit_outputsigs"
SSTATEPOSTUNPACKFUNCS[vardepvalueexclude] .= "| buildhistory_emit_outputsigs"
# All items excepts those listed here will be removed from a recipe's
# build history directory by buildhistory_emit_pkghistory(). This is
# necessary because some of these items (package directories, files that
@ -292,6 +297,29 @@ python buildhistory_emit_pkghistory() {
bb.build.exec_func("buildhistory_list_pkg_files", d)
}
python buildhistory_emit_outputsigs() {
if not "task" in (d.getVar('BUILDHISTORY_FEATURES') or "").split():
return
taskoutdir = os.path.join(d.getVar('BUILDHISTORY_DIR'), 'task', 'output')
bb.utils.mkdirhier(taskoutdir)
currenttask = d.getVar('BB_CURRENTTASK')
pn = d.getVar('PN')
taskfile = os.path.join(taskoutdir, '%s.%s' % (pn, currenttask))
cwd = os.getcwd()
filesigs = {}
for root, _, files in os.walk(cwd):
for fname in files:
if fname == 'fixmepath':
continue
fullpath = os.path.join(root, fname)
filesigs[os.path.relpath(fullpath, cwd)] = bb.utils.sha256_file(fullpath)
with open(taskfile, 'w') as f:
for fpath, fsig in sorted(filesigs.items(), key=lambda item: item[0]):
f.write('%s %s\n' % (fpath, fsig))
}
def write_recipehistory(rcpinfo, d):
bb.debug(2, "Writing recipe history")