sysmo-bts
/
generic-poky
9
0
Fork 0
Code Pull Requests Releases Activity

verify-bashisms: fix problems with tinfoil2 

tinfoil2 is based on a client/server architecture, which broke the
verify-bashisms script:

- The tinfoil instance and its data proxies can't be pickled, so
  all interaction with the bitbake server has to run in the main
  script process and only processing of the plain scripts can
  be done with multiprocessing:

  _pickle.PicklingError: Can't pickle <class 'bb.tinfoil.TinfoilCookerAdapter.TinfoilRecipeCacheAdapter'>: attribute lookup TinfoilRecipeCacheAdapter on bb.tinfoil failed

- The multiprocessing pool has to be created before initializing
  tinfoil, otherwise the pool workers end up trying to communicate
  with the bitbake server during shutdown:

  ERROR: UI received SIGTERM
  Process ForkPoolWorker-2:
  Traceback (most recent call last):
    File "/usr/lib/python3.4/multiprocessing/process.py", line 257, in _bootstrap
      util._exit_function()
    File "/usr/lib/python3.4/multiprocessing/util.py", line 286, in _exit_function
      _run_finalizers(0)
    ...
    File "/usr/lib/python3.4/multiprocessing/process.py", line 131, in is_alive
      assert self._parent_pid == os.getpid(), 'can only test a child process'
   AssertionError: can only test a child process

- func() needs to defined before creating the pool to avoid:

  AttributeError: Can't get attribute 'func' on <module '__main__' from '/work/openembedded-core/scripts/verify-bashisms'>

(From OE-Core rev: aa439f11c7f414774843720d68ebe0a6d3375ea6)

Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This commit is contained in:
Patrick Ohly 2017-01-31 13:50:29 +01:00 committed by Richard Purdie
parent 48c65ddaa4
commit 010f9fa86a
1 changed files with 25 additions and 19 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

44
scripts/verify-bashisms
View File

@ -71,6 +71,20 @@ if __name__=='__main__':
print("Cannot find checkbashisms.pl on $PATH, get it from https://anonscm.debian.org/cgit/collab-maint/devscripts.git/plain/scripts/checkbashisms.pl")
sys.exit(1)
# The order of defining the worker function,
# initializing the pool and connecting to the
# bitbake server is crucial, don't change it.
def func(item):
fn, scripts = item
result = []
for key, script in scripts:
r = process(fn, key, script)
if r: result.extend(r)
return fn, result
import multiprocessing
pool = multiprocessing.Pool()
tinfoil = get_tinfoil()
# This is only the default configuration and should iterate over
@ -83,32 +97,24 @@ if __name__=='__main__':
else:
initial_pns = sorted(pkg_pn)
pns = []
print("Generating file list...")
pns = {}
print("Generating scripts...")
for pn in initial_pns:
for fn in pkg_pn[pn]:
# There's no point checking multiple BBCLASSEXTENDed variants of the same recipe
realfn, _, _ = bb.cache.virtualfn2realfn(fn)
if realfn not in pns:
pns.append(realfn)
def func(fn):
result = []
data = tinfoil.parse_recipe_file(fn)
for key in data.keys():
if data.getVarFlag(key, "func") and not data.getVarFlag(key, "python"):
script = data.getVar(key, False)
if not script: continue
#print ("%s:%s" % (fn, key))
r = process(fn, key, script)
if r: result.extend(r)
return fn, result
data = tinfoil.parse_recipe_file(realfn)
scripts = []
for key in data.keys():
if data.getVarFlag(key, "func") and not data.getVarFlag(key, "python"):
script = data.getVar(key, False)
if script:
scripts.append((key, script))
pns[realfn] = scripts
print("Scanning scripts...\n")
import multiprocessing
pool = multiprocessing.Pool()
for pn,results in pool.imap(func, pns):
for pn, results in pool.imap(func, pns.items()):
if results:
print(pn)
for message,source in results: