Commit Graph

2097 Commits

Author SHA1 Message Date
Robert Yang d8af0f283a runqemu: improve finding of rootfs, kernel and dtb
* Search rootfs in the following order:
  - IMAGE_NAME*.FSTYPE
  - IMAGE_LINK_NAME*.FSTYPE

* Search kernel in the following order:
  - QB_DEFAULT_KERNEL
  - KERNEL_IMAGETYPE
  - KERNEL_IMAGETYPE*

* Search dtb in the following order:
   - QB_DTB
   - QB_DTB*
   - *.dtb

* Fix DTB, it should only work with "-kernel" option.

[YOCTO #10265]

(From OE-Core rev: 32ff0974ed06f797c6b7d9092a8dc9ae50e9a572)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-20 15:11:08 +01:00
Robert Yang fbb99b22f9 runqemu-gen-tapdevs: remove /etc/runqemu-nosudo when remove taps
It creats /etc/runqemu-nosudo when creats taps, so should remove it when
remove taps.

(From OE-Core rev: 3d4bf5b0ea581e3e9b388328b086d03f9174fd61)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-20 15:11:08 +01:00
Robert Yang 5637f8605f runqemu: use OECORE_NATIVE_SYSROOT from sdk
There is no STAGING_DIR_NATIVE or bitbake in a extracted sdk,
so check OECORE_NATIVE_SYSROOT and use it.

(From OE-Core rev: 93649edc034f2540ff55dc9b41638797209cfb9c)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-20 15:11:08 +01:00
Joshua Lock 5060e66c75 runqemu: work even if a *.qemuboot.conf isn't found
A qemuboot conf file is a convenience but it should still be
possible to invoke runqemu without them, especially for examples
such as using the SDK with an extracted rootfs via NFS.

As read_qemuboot() is always called we need to be sure that function
can return cleanly, without throwing Exceptions, even if a qemuboot
conf file isn't found.

(From OE-Core rev: 3541c21f1976b517b79a19882240a8f36b970292)

Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-20 15:11:08 +01:00
Joshua Lock 239d1706b0 runqemu: try symlinks when kernel or rootfs can't be found
If the kernel or rootfs names written to the qemuboot.conf can't
be found, try and find the symlinked variant of the filename.

This will help usability of runqemu, for example where a user
downloads an image and associated files as the symlinked names
yet the qemuboot.conf variables point to the full, non-linked,
file names.

(From OE-Core rev: ca5a686c6e165a51f95cb6a834cd53f6f66d42d4)

Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-20 15:11:08 +01:00
Joshua Lock 0fd72474e7 runqemu: clarify an INFO message
Make it clearer that we are looking for a file which ends with
qemuboot.conf

(From OE-Core rev: 2579e05269a14b53a54232a8bf4414ac2dfe6472)

Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-20 15:11:07 +01:00
Joshua Lock 2952affc63 runqemu: add guidance to resolve issues with missing files
When a required binary cannot be found print some guidance pointing
to using a sourced OE build environment or a qemuboot.conf file,
based on a similar message from the previous shell-based runqemu.

(From OE-Core rev: 87cfb5165490cd4e7a8c2570ef5a62898db8395e)

Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-20 15:11:07 +01:00
Paul Eggleton 383a4af0e9 devtool: add: drop superfluous validation for recipe name
Now that recipeutils.validate_pn() properly validates characters used in
the name, we can drop this bit checking for '/' since that's not
permitted by validate_pn(). (The FIXME comment here - that I myself
apparently wrote - is questionable since that function was clearly never
intended to allow '/', perhaps I was misled because it was broken and
did so).

(From OE-Core rev: e010d9be3709cf3c607ffc03c3188abe4e1e9eb4)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-20 15:11:07 +01:00
Paul Eggleton 90f925cd41 recipetool: create: support git short form URLs
In keeping with making recipetool create / devtool add as easy to use as
possible, users shouldn't have to know how to reformat git short form ssh
URLs for consumption by BitBake's fetcher (for example
user@git.example.com:repo.git should be expressed as
git://user@git.example.com/repo.git;protocol=ssh ) - instead we should
just take care of that automatically. Add some logic in the appropriate
places to do that.

(From OE-Core rev: 78c672a72f49c4b6cfd8c247efcc676b0ba1681a)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-20 15:11:07 +01:00
Paul Eggleton 147774fc1c recipetool: create: tweak license crunching
Filter out a plain "Licensed under the XXXX license" statement, as seen
in the capnproto project (and no doubt others).

(From OE-Core rev: ba4aa319fd49ee02ce2e30c2db0f3988c0e8833c)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-20 15:11:07 +01:00
Paul Eggleton 198c6d52c4 recipetool: create: pick up AC_PROG_SWIG
AX_PKG_SWIG is not the only commonly-used macro for detecting swig -
there's also AC_PROG_SWIG. As per AX_PKG_SWIG, add swig-native to
DEPENDS if AC_PROG_SWIG is found in configure.ac.

(From OE-Core rev: 847a1aa7153fc8a7b820353283a6f1e51d64f8de)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-20 15:11:07 +01:00
Paul Eggleton 1370edb449 recipetool: create: detect python autoconf macros
If python is required then we need to inherit pythonnative (or
python3native) otherwise do_configure will probably fail since it won't
be able to find python.

(From OE-Core rev: 63234cc45aee91b031657971f36997e1443f80ee)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-20 15:11:07 +01:00
Paul Eggleton 541c54e301 recipetool: create: fix error with git tree and no network
When creating a recipe for an existing local git clone, we attempt to
use the fetcher to determine if it supports the SRCREV variable.
Unfortunately running this code does a network check to get the latest
revision as a direct result of us using '${AUTOREV}' as a default value.
If you don't have a network connection this will of course fail. Rather
than have this block creating the recipe, catch the exception and just
guess from the URL.

Ultimately this should probably be fixed in the fetcher but for now this
will at least resolve the issue on this end.

(From OE-Core rev: f7e43f931d7d6019a3b2509b2b2635978fbbae36)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-20 15:11:07 +01:00
Paul Eggleton e30276cb18 recipetool: create: fix name/version extraction from filename
I ran into an example where recipetool was getting the name/version
completely wrong:

https://bitbucket.org/sortsmill/libunicodenames/downloads/libunicodenames-1.1.0_beta1.tar.xz

>From this it would create a libunicodenames-1.1.0-beta1_1.1.0-beta1.bb
file (likely because it couldn't split the file name and therefore took
all of it, then got the version from one of the files inside the
tarball). When this happens it's just irritating because you then have
to delete the recipe / run devtool reset and then run recipetool create
/ devtool add again and specify the version manually.

This patch is the result of systematically running the
determine_from_filename() function over the files on the Yocto Project
source mirror and my local downloads directory and fixing as many of the
generic issues as reasonably practical - it now gets the name and
version correct much more often. There are still cases where it won't,
but they are now in the minority.

(From OE-Core rev: 7b018b1d493a8d10fd02b8cc220990b191c87fe5)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-20 15:11:07 +01:00
Paul Eggleton 9aa1cf3a28 recipetool: create: improve python recipe license handling
Try to ensure that for Apache, GPL and LGPL where the values extracted
from the "Classifiers" field may not be version-specific, if there is a
versioned license in the free-form license field then use that instead.
Also insert the free-form license field as a comment in the recipe for
the user's reference.

(From OE-Core rev: 237f66042eedd906f654827b53bf9269738267ab)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-20 15:11:07 +01:00
Paul Eggleton 3a8a0bba9b recipetool: create: fix picking up name from local python source tree
Make use of the extravalues dict to send back other variable values from
the python handling plugin, and enable passing back PV and PN. This not
only places variable values in the final recipe a bit more consistently
with other types of source, it also allows the name and version to be
picked up fron a local source tree and not just when the recipe is
fetched from a remote URL that happens to have those in it.

(From OE-Core rev: 3e7029f28c6ea9bb1d283bcdc3fdfee11455af8e)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-20 15:11:07 +01:00
Robert Yang 6a5bd99bfc runqemu: acquire_lock() should fail when failed to open the file
The open(self.lock, 'w') may fail when the lock is created by other
users, return false for this case to let it try other devices.

Fixed:
runqemu - INFO - Running /sbin/ip link...
runqemu - INFO - Acquiring lockfile /tmp/qemu-tap-locks/tap0.lock...
Traceback (most recent call last):
  File "/buildarea/lyang1/poky/scripts/runqemu", line 972, in <module>
    ret = main()
  File "/buildarea/lyang1/poky/scripts/runqemu", line 963, in main
    config.setup_network()
  File "/buildarea/lyang1/poky/scripts/runqemu", line 810, in setup_network
    self.setup_tap()
  File "/buildarea/lyang1/poky/scripts/runqemu", line 761, in setup_tap
    if self.acquire_lock():
  File "/buildarea/lyang1/poky/scripts/runqemu", line 182, in acquire_lock
    lock_descriptor = open(self.lock, 'w')
PermissionError: [Errno 13] Permission denied: '/tmp/qemu-tap-locks/tap0.lock'

(From OE-Core rev: f364f773a0381a75b5992c8c8a1d63a81dbd4422)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-20 15:11:07 +01:00
Ross Burton 359feedfd5 scripts: add tool to scan for bashisms recipe shell scripts
Shell functions in bitbake are executed with /bin/sh so should be POSIX
compliant and not use Bash extensions, or at least only use extensions that are
implemented in both dash and ash (busybox).

This tool will extract all of the shell scripts from all recipes and run them
through checkbashisms (it assumes that checkbashisms is on $PATH).

There is a whitelist to filter out false-positives such as the use of $HOSTNAME
(a bashism) in functions where we have defined it, or using the 'type' builtin
which is supported by ash/dash.

[ YOCTO #8851 ]

(From OE-Core rev: d77fe838ab7631a19e90ff4226f0712e54aa4e22)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-16 15:24:03 +01:00
Markus Lehtonen 12cd9c01fc scripts: introduce buildstats-diff
New script for comparing buildstats from two separate builds. The script
has two modes: normally it prints the differences in task execution
(cpu) times but using --ver-diff option makes it just print the recipe
version differences without any cpu time data. Other command line
options are provided to alter the sort criteria of the data and to
filter out insignificant differences and/or short tasks.

(From OE-Core rev: e707718374ce1c95769a5f99aa3bfdfc0be685b2)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-16 15:24:03 +01:00
Humberto Ibarra b28e6d6039 oe-selftest: check for coverage version before starting tests
python coverage versions lower than 4.x have problems with some distros. Adding the 4.x version as requirement to continue with coverage tracking.

[YOCTO #10207]

(From OE-Core rev: a378b817504986173c4b0984a28aead247589b3f)

Signed-off-by: Humberto Ibarra <humberto.ibarra.lopez@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-15 12:15:07 +01:00
Ed Bartosh f7367de2ad create-pull-request: remove output directory
When 'git request-pull' fails it makes sense to remove output
directory. Otherwise create-pull-request will complain that
output directory already exists on the next run.

(From OE-Core rev: 8535784ba84060e0e7ca05e7771a58f9c70c69b9)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-15 12:15:07 +01:00
Robert Yang 26e46e6d90 runqemu: fix a race issue on lockdir
There might be a race issue when multi runqemu processess are
running at the same time:
| Traceback (most recent call last):
|   File "/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-ipk/build/scripts/runqemu", line 920, in <module>
|     ret = main()
|   File "/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-ipk/build/scripts/runqemu", line 911, in main
|     config.setup_network()
|   File "/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-ipk/build/scripts/runqemu", line 760, in setup_network
|     self.setup_tap()
|   File "/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-ipk/build/scripts/runqemu", line 697, in setup_tap
|     os.mkdir(lockdir)
| FileExistsError: [Errno 17] File exists: '/tmp/qemu-tap-locks'

(From OE-Core rev: ec33043477a0b915b0911f7d7eacb24361e4aaa8)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-14 22:22:13 +01:00
Richard Purdie 0d214406c4 scripts/runqemu: Add snapshot support
Allow access to the snapshot option of qemu to simplify some of our runtime
testing to avoid copying images.

(From OE-Core rev: 8fec4a5a004f0e99734f8c0820c66522d08f213e)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-09 12:07:32 +01:00
Richard Purdie b2dc9c7ee2 runqemu: Enable virtio RNG for all platforms
We have problems where systems simply stop booting and hang. This is due
to a lack of entropy which means ssh keys and networking can't be brought
up. Adding in the virtio-rng passthrough support allows host entropy to
pass into the guess and avoids these hangs.

This is particularly problematic after the gnutls upgrade which starts
using /dev/random instead of /dev/urandom but was an issue we'd occasionally
seem before that.

It particualrly affected x86 and ppc machines for some reason.

(From OE-Core rev: 51b001909f1856c45cf87091d6e4446c266d5786)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-09 12:07:32 +01:00
Richard Purdie eb1494f9da runqemu: Update to modern prefrerred net syntax
(From OE-Core rev: 5e61766d976b6d036946c1b4e4ac742a33a03815)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-09 12:07:32 +01:00
Richard Purdie d1cb381977 runqemu: Allow unique network interface MAC addresses
Current qemu instances all share the same MAC address. This shouldn't be an
issue as they are all on separate network interfaces, however on the slight
chance this is causing problems, its easy enough to ensure we use unique
MAC addresses based on the IP numbers we assign.

(From OE-Core rev: c01962bf88786dd84ad83cc1d315297607d29f7c)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-09 12:07:32 +01:00
Joshua Lock d1303c220e runqemu: fix run from testimage with non-standard DEPLOY_DIR_IMAGE
testimage.bbclass uses runqemu to execute runtime tests on a qemu
target, this means that bitbake is already running and `bitbake -e`
can't be called to obtain bitbake variables.

runqemu tries to work around being unable to read values for
bitbake variables by inferring the MACHINE from the
DEPLOY_DIR_IMAGE setting, however if a user sets that variable in
a manner which doesn't follow the systems expectations (i.e. if
running `bitbake -c testimage` against a directory of pre-generated
images in a user-specified path) the inferring of the MACHINE name
from the DEPLOY_DIR_IMAGE location will fail.

It's possible that check_arg_machine() shouldn't cause runqemu to
fail and that runqemu should proceed with the user-supplied value
even if it can't be verified. This patch simply ensures that a
workflow where the user sets DEPLOY_DIR_IMAGE continues to work
without changing too much of the runqemu code.

[YOCTO #10238]

(From OE-Core rev: f94ac02f459e2ea0fc471463966997814a67e0ca)

Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-09 12:07:32 +01:00
Joshua Lock 8a8948e0e1 runqemu: fixes for when invoked during a bitbake run
When runqemu is invoked from a running bitbake instance it will be
unable to call `bitbake -e` due to the lock held by the calling
bitbake instance.

Our test code sets an OE_TMPDIR environment variable from which we
can infer/guess paths. Add code to do so when self.bitbake_e can't
be set, much as the sh version of runqemu did.

[YOCTO #10240]

(From OE-Core rev: 1e8165ea2f19aecdc03ccd102ee44ef0544f0f39)

Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-09 12:07:32 +01:00
Joshua Lock d5d4869634 runqemu: better handle running on a host with different paths
If the STAGING_*_NATIVE directories from the config file don't exist
and we're in a sourced OE build directory try to extract the paths
from `bitbake -e`

(From OE-Core rev: 9326af1c20636320c70caecebd47aedafb3f2d25)

Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-09 12:07:32 +01:00
Joshua Lock e162303ecb runqemu: assume artefacts are relative to *.qemuboot.conf
When runqemu is started with a *.qemuboot.conf arg assume that image
artefacts are relative to that file, rather than in whatever
directory the DEPLOY_DIR_IMAGE variable in the conf file points to.

(From OE-Core rev: a6448371b87f754def669adfdc01b07d18003405)

Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-09 12:07:32 +01:00
Robert Yang b405712414 runqemu: refactor it and remove machine knowledge
Previously, runqemu had hard coded machine knowledge, which limited its
usage, for example, qemu can boot genericx86, but runqemu can't, we need
edit runqemu/runqemu-internal a lot if we want to boot genericx86.

Now bsp conf files can set vars to make it can be boot by runqemu, and
qemuboot.bbclass will save these info to DEPLOY_DIR_IMAGE/qemuboot.conf.
Please see qemuboot.bbclass' comments on how to set the vars.

* Re-write it in python3, which can reduce lines from 1239 to about 750
  lines
* All the machine knowledges are gone
* All of the TUN_ARCH knowledge are gone
* All the previous options are preserved, and there is a new way to run
  runqemu: (it doesn't need run "bitake -e" in such a case)
  $ runqemu tmp/deploy/images/qemux86
  or:
  $ runqemu tmp/deploy/images/qemuarm/<image>.ext4
  or:
  $ runqemu tmp/deploy/images/qemuarm/qemuboot.conf
* Fixed audio support, not limited on x86 or x86_64
* Fix SLIRP mode, add help message, avoid mixing with tap
* Fix NFS boot, it will extract <image>.tar.bz2 or tar.gz to
  DEPLOY_DIR_IMAGE/<image>-nfsroot when no NFS_DIR, and remove it after
  stop.
* More bsps can be boot, such as genericx86 and genericx86-64.
* The patch for qemuzynq, qemuzynqmp, qemumicroblaze has been sent to
  meta-xilinx' mailing list.
* I can't find any qemush4 bsp or how to build it, so it is not
  considered atm.

[YOCTO #1018]
[YOCTO #4827]
[YOCTO #7459]
[YOCTO #7887]

(From OE-Core rev: 60ca8a8d899b90a4693fd62b6ec97d0c76a9f6c5)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-09 12:07:32 +01:00
Paul Eggleton ce4ea7a730 recipetool: create: avoid extra blank lines in output recipe
If we output extra blank lines (because of some automated editing) then
it makes the output recipe look a bit untidy. You could argue that we
should simply have the editing code not do that, but sometimes we don't
have enough context there for that to be practical. It's simple enough
to just filter out the extra blank lines when writing the file, so just
do it that way.

(From OE-Core rev: cbebc9a2edf7d7a422ee5c71219e79e3b349de3b)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-08 00:36:49 +01:00
Paul Eggleton ff259b095d recipetool: create: support node.js code outside of npm
If you have your own node.js application you may not publish it (or at
least not immediately) in an npm registry - it might just be in a
repository on github or on your local machine. Add support to recipetool
create for creating recipes to build such applications - extract their
dependencies, fetch them, and add corresponding npm:// URLs to SRC_URI,
and ensure that LICENSE / LIC_FILES_CHKSUM are updated to match. For
example, you can now run:

  recipetool create https://github.com/diversario/node-ssdp

(I had to borrow some code from bitbake/lib/bb/fetch2/npm.py to
implement this functionality; this should be refactored out but now
isn't the time to do that refactoring.)

Part of the fix for [YOCTO #9537].

(From OE-Core rev: 4fb8b399c05a1b66986fc76e13525f6c5e0d9b58)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-08 00:36:49 +01:00
Paul Eggleton fa90c2f54d recipetool: create: allow license variable handling to be rerun
If you make adjustments to the source tree (as create_npm.py will be)
then you will need to re-run the license variable handling code at the
end so that we get all of the files that should go into
LIC_FILES_CHKSUM if nothing else. Split out the license variable
handling to a separate function in order to allow this.

(From OE-Core rev: f0d6f4b7e87ea781ac0dffcc8d0310570975811b)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-08 00:36:49 +01:00
Paul Eggleton b1c3e44dfb recipetool: create: add --keep-temp command line option
For debugging it's useful to be able to tell recipetool to keep the
temporary directory.

(From OE-Core rev: 480a6b745a85b2881e5cc1a0bbb572e3235ca008)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-08 00:36:48 +01:00
Paul Eggleton 17afc80320 recipetool: create: support git submodules
Ensure we fetch submodules and set SRC_URI correctly when pointing to a
git repository that contains submodules.

(From OE-Core rev: 65d5cc62d4ecfc78ce4b37b3886a7fe5aa05a75e)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-08 00:36:48 +01:00
Paul Eggleton 9885a9dd31 recipetool: create: fix mapping python dependencies to python-dbg package
When trying to map python module dependencies to the packages that
provide them, if we're looking for .so files that satisfy
dependencies then we need to exclude files found under the .debug
directory, otherwise the dependency will get mapped to the python-dbg
package which isn't correct.

For example, this fixes creating a recipe for pyserial and not getting
python-fcntl in RDEPENDS_${PN}, leading to errors when trying to use the
serial module on the target.

(From OE-Core rev: 46a068ca35975988a8e9c0310f71fdcee55937a4)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-08 00:36:48 +01:00
Paul Eggleton 4da96ce61f recipetool: create: AX_PKG_SWIG should add dependency on swig-native
If AX_PKG_SWIG is found in configure.ac, then what's being looked for is
the swig binary, not swig for the target - so fix the dependency
accordingly.

(From OE-Core rev: 2600cd6f6c63ecf79804e2bc6eb6f198a012d5d6)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-08 00:36:48 +01:00
Paul Eggleton 39d3aa2828 devtool: update-recipe: support files with subdir=
It's rare but there are recipes that have individual files (as opposed
to archives) in SRC_URI using subdir= to put them under the source tree,
the examples in OE-Core being bzip2 and openssl. This broke devtool
update-recipe (and devtool finish) because the file wasn't unpacked into
the oe-local-files directory and thus when it came time to update the
recipe, the file was assumed to have been deleted by the user and thus
the file was erroneously removed. Add logic to handle these properly so
that this doesn't happen.

(We still have another potential problem in that these files become part
of the initial commit from upstream, which could be confusing because
they didn't come from there - but that's a separate issue and not one
that is trivially solved.)

(From OE-Core rev: 9069fef5dad5a873c8a8f720f7bcbc7625556309)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-08 00:36:48 +01:00
Markus Lehtonen 1f706698cd oe-build-perf-test: fix log file path
The --log-file command line argument was slightly broken as {out_dir}
string replacement was not working as expected.

(From OE-Core rev: fc62f54e3d788cc79fd27664f05db7efccef23ab)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-06 10:24:04 +01:00
Markus Lehtonen 91ae03931e build-perf-test-wrapper.sh: fix handling of -C argument
Not specifying -C caused oe-build-perf-test to try to commit results to
the build directory.

(From OE-Core rev: 6f4786f5522c366a7fd92f630c3f32629a9f9471)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-05 11:56:02 +01:00
Paul Eggleton e616beba1c scripts: ensure tinfoil is shut down correctly
We should always shut down tinfoil when we're finished with it, either
by explicitly calling the shutdown() method or by using it as a
context manager ("with ...").

(From OE-Core rev: 5ec6d9ef309b841cdcbf1d14ac678d106d5d888a)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-03 23:45:54 +01:00
Paul Eggleton 4a5aa7ea4d scripts/contrib: update scripts for changes to internal API
The multiconfig changes altered some of the functions being called here,
so update the calls. Make use of the new Tinfoil.parse_recipe_file()
function to make parsing easier.

(From OE-Core rev: 95b6ceffd947271f315d8a7660797ab371adfbb9)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-03 23:45:52 +01:00
Paul Eggleton 4b4387455c lib/oe/patch: commit with a dummy user/email when PATCHTOOL=git
When using PATCHTOOL = "git", the user of the system is not really the
committer - it's the build system itself. Thus, specify "dummy" values
for username and email instead of using the user's configured values.
Various parts of the devtool code that need to make commits have also
been updated to use the same logic.

This allows PATCHTOOL = "git" and devtool to be used on systems where
git user.name / user.email has not been set (on versions of git where
it doesn't default a value under this circumstance).

If you want to return to the old behaviour where the externally
configured user name / email are used, set the following in your
local.conf:

PATCH_GIT_USER_NAME = ""
PATCH_GIT_USER_EMAIL = ""

Fixes [YOCTO #8703].

(From OE-Core rev: 765a9017eaf77ea3204fb10afb8181629680bd82)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-03 23:45:52 +01:00
Markus Lehtonen 35ae939e41 oe-build-perf-test: rename log file and implement --log-file
Rename the (main) log file of the oe-build-perf-test script from
'output.log' to 'oe-build-perf-test.log'. Also, add a new command line
option --log-file which makes it possible to use an alternative log file
name/path, if needed.  Note that the file name/path is relative to the
output directory.

(From OE-Core rev: 4909fae1a6d1d068b33252088b41b8d82d1a836c)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-03 09:58:41 +01:00
Markus Lehtonen e16f00862f oe-build-perf-test: update globalres and git even if tests failed
Write globalres log file and commit results to Git even if some tests
failed. Now that tests do not depend on each other there should be no
risk of bogus results caused by test failures.

(From OE-Core rev: 8036975b268fe209476e230555006facd3cbda71)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-03 09:58:41 +01:00
Markus Lehtonen 96e68f15f0 build-perf-test-wrapper.sh: make workdir configurable
New command line argument '-w' may be used to specify work dir other
than the default <GIT_DIR>/build-perf-test.

(From OE-Core rev: 824284895f25146520a624b7b97f7475d0135814)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-08-25 23:03:48 +01:00
Markus Lehtonen ee4c5f6171 build-perf-test-wrapper.sh: make archive dir configurable
Add new command line argument '-a' that can be used to define the
directory where results (tarballs) are archived. Giving an empty string
disables archiving which makes sense if you store results in Git.

(From OE-Core rev: d53cf92847aa80724be4412801c993948a09cd27)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-08-25 23:03:48 +01:00
Markus Lehtonen a34fd3cf27 build-perf-test-wrapper.sh: allow saving results in Git
Add new command line argument '-C' that allows saving results in a Git
repository.

(From OE-Core rev: 3d06795d8cd9017b042a7283c16ac71d4f6317a6)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-08-25 23:03:48 +01:00
Markus Lehtonen dc3025215b build-perf-test-wrapper.sh: parse args with getopts
Use getopts for parsing the command line. This changes the usage so that
if a commit (to-be-tested) is defined it must be given by using '-c',
instead of a positional argument.

(From OE-Core rev: b1f77ba41033397a2b25977963682b86f2f76471)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-08-25 23:03:48 +01:00