Commit Graph

2118 Commits

Author SHA1 Message Date
Markus Lehtonen c6047cd989 scripts/buildstats-diff: add read_ops and write_ops to --diff-attr
Two new options, making it possible to compare the number of filesystem
operations of tasks. Defaults for --min-val and --min-absdiff are set to
more or less arbitrary 500 and 50 operations, respectively.

(From OE-Core rev: 75292a1de1a59e19198d26b7c1291004a5ca92f3)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-10-01 21:45:56 +01:00
Markus Lehtonen 44bc3f47a3 scripts/buildstats-diff: add read_bytes and write_bytes to --diff-attr
These are I/O counter values from /proc/<pid>/io and represent the
number of bytes read from / written to the storage layer. Default values
for --min-val and --min-absdiff limits are set to 512kB and 128kB,
respectively.

(From OE-Core rev: 24a12e40caeb05dac13c417f35733761af219f03)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-10-01 21:45:56 +01:00
Markus Lehtonen 01a2b5823b scripts/buildstats-diff: introduce --diff-attr
A new command line option for choosing which "attribute" of the
buildstats to compare. At first, the already supported 'cputime' is the
only available option. But, refactoring done in this patch should make
it easy to add new attribute types.

(From OE-Core rev: 0782825138731b3f1e6a8e05d723c1d5cd60c90c)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-10-01 21:45:55 +01:00
Markus Lehtonen e05309194a scripts/buildstats-diff: do not hardcode field widths in output
Dynamically adjust the width of all fields in task diff output. Makes
it easier to print other units than cputime, too.

(From OE-Core rev: 559b858f2a3712ec21debb71681593bd7cf55041)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-10-01 21:45:55 +01:00
Markus Lehtonen 7113ac9439 scripts/buildstats-diff: implement BSTask class
New class representing buildstats data of a single task.

(From OE-Core rev: 472818a32f96699a6dc9c7c487f38d716678fd7a)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-10-01 21:45:55 +01:00
Markus Lehtonen 4cc1c430cd scripts/buildstats-diff: rename --min-time and --min-timediff args
Rename these arguments to --min-val and --min-absdiff in preparation for
supporting other "quantities" than just cputime.

(From OE-Core rev: 441336bc1750939c2da2d9e4dc5a6893b283bf68)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-10-01 21:45:55 +01:00
Markus Lehtonen 58bd0fd62d scripts/buildstats-diff: check that the given directory exists
(From OE-Core rev: 08082b96d8d09215f02e9251f354bb6e8bb3e712)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-10-01 21:45:55 +01:00
Jianxun Zhang d19645bcde wic: rename and amend systemd-boot wks file
Rename wks for systemd-boot per the suggestion from community.
Also amend description to distinguish it from others when
running "wic list images".

(From OE-Core rev: 6303dbbaa08214a37caf38e3b6b5a30a108bd3b7)

Signed-off-by: Jianxun Zhang <jianxun.zhang@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-10-01 21:45:55 +01:00
Ed Bartosh 5753147ea2 runqemu: explicitly set image format
QEMU produces a warning if drive format is not specified:
  WARNING: Image format was not specified for
  'tmp/deploy/images/qemux86-64/core-image-minimal-qemux86-64.wic'
   and probing guessed raw.
   Automatically detecting the format is dangerous for raw images,
   write operations on block 0 will be restricted.
   Specify the 'raw' format explicitly to remove the restrictions.

Set image format to 'vmdk', 'qcow2' or 'vdi' for correspondent image
types. Set it to 'raw' for the rest of image types.

(From OE-Core rev: 5100bb36502ef7c81220a3c4809eb1b3ac83801f)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-30 16:51:15 +01:00
Christopher Larson c8089ae233 recipetool: newappend: drop _provide_to_pn
This function was broken by the multi-config changes, and isn't needed anymore
now that recipeutils.pn_to_recipe can handle provides. Without this, the
newappend sub-command fails.

(From OE-Core rev: 4a5028dc3d1ab2f97465e63db5b05de73daebdfa)

Signed-off-by: Christopher Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-28 10:16:04 +01:00
Richard Purdie 04a3b6c0ca scripts/cleanup-workdir: Adapt to SDK_ARCH -> SDK_SYS chanages for crosssdk
With the change of crosssdk to use SDK_SYS, we need to update the script
to match.

(From OE-Core rev: 8ab1f6073f86d05493bb32a8135c9d912d72f5f0)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-28 10:16:03 +01:00
Joe MacDonald 527764b4c5 devtool: Add a line break to generated README
When devtool creates a new workspace, it produced a README with one very
long line and no space following 'bblayers.conf'.  Add a line break as was
intended.

(From OE-Core rev: 4ad1bcfc3c88ced5b7fc80c950613e31becb40f3)

Signed-off-by: Joe MacDonald <joe_macdonald@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-28 10:16:03 +01:00
Stephano Cetola 1d1f94b944 scripts/runqemu: provide better error message on runqemu ifup fail
If runqemu-ifup fails hen running testimage, a rather cryptic error
regarding "no tty present" is displayed. If this step fails, we
should at least point the user at runqemu-gen-tapdevs. A quick search
of this term in the manual will lead them to "Enabling Runtime Tests
on QEMU" which should give them all the info they need.

(From OE-Core rev: 3b6494fad2b8b65e0d52cda0cdf500e93c72823a)

Signed-off-by: Stephano Cetola <stephano.cetola@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-24 07:30:10 +01:00
Ed Bartosh 347316dc15 wic: remove partition images
Preserving images for every partition doubles disk space
consumed by an image build. As those images are not used,
so it's better to remove them after assembling final image.

(From OE-Core rev: 51171b4aa10f2218c5e27d785ca7bf4f3949a4b4)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-24 07:30:09 +01:00
Nathan Rossi 0156812271 scripts/runqemu: Using a cpio* rootfs has no special network
When booting a system with the rootfs being of cpio* type the networking
setup should still work the same as for all other root filesystem types.
This change removes the clearing of the NETWORK_CMD variable allowing
for the slirp/tap setup to be provided to QEMU.

(From OE-Core rev: 7d01a9c80de0cdbac3831301dd996c7b61754c74)

Signed-off-by: Nathan Rossi <nathan@nathanrossi.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-23 14:56:39 +01:00
Nathan Rossi deba7cac00 runqemu: Move virtio RNG to machine configuration
Not all QEMU machines (outside of those available in OE-Core) are
capable of using the virtio-rng-pci device due to various machine models
not having a pci/virtio bus. This makes it such that the use of the
'-device virtio-rng-pci' flag to QEMU is machine specific.

This patch removes the general addition of the flag to all runqemu
targets and adds the flag into the QB_OPT_APPEND for all the qemu*
machines in OE-Core that support its use (which is all of them).

(From OE-Core rev: e890c05e66a21702e9e8ccce794b74cb7f5518ed)

Signed-off-by: Nathan Rossi <nathan@nathanrossi.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-23 14:56:39 +01:00
Ed Bartosh d6616a421f directdisk*.wks: add serial console support
Added serial console to kernel command line to to make it
easier to boot wic images on devices without display.

Tested on MinnowBoard MAX.

(From OE-Core rev: 9a774e3bcd5dc4c85d642acc7bf26095b8c620e4)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-22 11:13:24 +01:00
Joshua Lock 9294261c03 runqemu: don't fail during check_arg_machine()
If DEPLOY_DIR_IMAGE doesn't exist during check_arg_machine() we
will attempt to guess a suitable value later when check_and_set()
calls validate_paths(), therefore this shouldn't raise an exception

(From OE-Core rev: ed8d6f391c567048bd50dc3234804915f8212cef)

Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-21 22:02:16 +01:00
Joshua Lock 5d3c56f2a5 runqemu: don't try and invoke bitbake when running in a toolchain env
If a MACHINE value is passed we can't validate it by running bitbake
as the toolchain environment doesn't include the build system, we
must assume that the passed value for MACHINE is correct.

(From OE-Core rev: 2c569678566c49b3ea237ef2de0fbae782263449)

Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-21 22:02:16 +01:00
Joshua Lock c97912a17d runqemu: try and guess qemu-system binary when MACHINE isn't set
Emulate some logic from the prior, shell based, version of runqemu
to try and infer the correct setting for MACHINE from the kernel
and rootfs filenames.

(From OE-Core rev: a5adabe1414061d6864c5913dd5e66a4527838f1)

Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-21 22:02:16 +01:00
Joshua Lock cd47b648af runqemu: validate paths and attempt to infer unset paths
We need to validate and ensure all paths are set regardless of
whether runqemu was invoked with a .qemuboot.conf file or
otherwise. Split this logic out into a separate method called
during check_and_set()

(From OE-Core rev: e843b2d49a151c1fe0d2a7ba00c41d2a35775736)

Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-21 22:02:16 +01:00
Robert Yang d8af0f283a runqemu: improve finding of rootfs, kernel and dtb
* Search rootfs in the following order:
  - IMAGE_NAME*.FSTYPE
  - IMAGE_LINK_NAME*.FSTYPE

* Search kernel in the following order:
  - QB_DEFAULT_KERNEL
  - KERNEL_IMAGETYPE
  - KERNEL_IMAGETYPE*

* Search dtb in the following order:
   - QB_DTB
   - QB_DTB*
   - *.dtb

* Fix DTB, it should only work with "-kernel" option.

[YOCTO #10265]

(From OE-Core rev: 32ff0974ed06f797c6b7d9092a8dc9ae50e9a572)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-20 15:11:08 +01:00
Robert Yang fbb99b22f9 runqemu-gen-tapdevs: remove /etc/runqemu-nosudo when remove taps
It creats /etc/runqemu-nosudo when creats taps, so should remove it when
remove taps.

(From OE-Core rev: 3d4bf5b0ea581e3e9b388328b086d03f9174fd61)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-20 15:11:08 +01:00
Robert Yang 5637f8605f runqemu: use OECORE_NATIVE_SYSROOT from sdk
There is no STAGING_DIR_NATIVE or bitbake in a extracted sdk,
so check OECORE_NATIVE_SYSROOT and use it.

(From OE-Core rev: 93649edc034f2540ff55dc9b41638797209cfb9c)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-20 15:11:08 +01:00
Joshua Lock 5060e66c75 runqemu: work even if a *.qemuboot.conf isn't found
A qemuboot conf file is a convenience but it should still be
possible to invoke runqemu without them, especially for examples
such as using the SDK with an extracted rootfs via NFS.

As read_qemuboot() is always called we need to be sure that function
can return cleanly, without throwing Exceptions, even if a qemuboot
conf file isn't found.

(From OE-Core rev: 3541c21f1976b517b79a19882240a8f36b970292)

Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-20 15:11:08 +01:00
Joshua Lock 239d1706b0 runqemu: try symlinks when kernel or rootfs can't be found
If the kernel or rootfs names written to the qemuboot.conf can't
be found, try and find the symlinked variant of the filename.

This will help usability of runqemu, for example where a user
downloads an image and associated files as the symlinked names
yet the qemuboot.conf variables point to the full, non-linked,
file names.

(From OE-Core rev: ca5a686c6e165a51f95cb6a834cd53f6f66d42d4)

Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-20 15:11:08 +01:00
Joshua Lock 0fd72474e7 runqemu: clarify an INFO message
Make it clearer that we are looking for a file which ends with
qemuboot.conf

(From OE-Core rev: 2579e05269a14b53a54232a8bf4414ac2dfe6472)

Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-20 15:11:07 +01:00
Joshua Lock 2952affc63 runqemu: add guidance to resolve issues with missing files
When a required binary cannot be found print some guidance pointing
to using a sourced OE build environment or a qemuboot.conf file,
based on a similar message from the previous shell-based runqemu.

(From OE-Core rev: 87cfb5165490cd4e7a8c2570ef5a62898db8395e)

Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-20 15:11:07 +01:00
Paul Eggleton 383a4af0e9 devtool: add: drop superfluous validation for recipe name
Now that recipeutils.validate_pn() properly validates characters used in
the name, we can drop this bit checking for '/' since that's not
permitted by validate_pn(). (The FIXME comment here - that I myself
apparently wrote - is questionable since that function was clearly never
intended to allow '/', perhaps I was misled because it was broken and
did so).

(From OE-Core rev: e010d9be3709cf3c607ffc03c3188abe4e1e9eb4)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-20 15:11:07 +01:00
Paul Eggleton 90f925cd41 recipetool: create: support git short form URLs
In keeping with making recipetool create / devtool add as easy to use as
possible, users shouldn't have to know how to reformat git short form ssh
URLs for consumption by BitBake's fetcher (for example
user@git.example.com:repo.git should be expressed as
git://user@git.example.com/repo.git;protocol=ssh ) - instead we should
just take care of that automatically. Add some logic in the appropriate
places to do that.

(From OE-Core rev: 78c672a72f49c4b6cfd8c247efcc676b0ba1681a)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-20 15:11:07 +01:00
Paul Eggleton 147774fc1c recipetool: create: tweak license crunching
Filter out a plain "Licensed under the XXXX license" statement, as seen
in the capnproto project (and no doubt others).

(From OE-Core rev: ba4aa319fd49ee02ce2e30c2db0f3988c0e8833c)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-20 15:11:07 +01:00
Paul Eggleton 198c6d52c4 recipetool: create: pick up AC_PROG_SWIG
AX_PKG_SWIG is not the only commonly-used macro for detecting swig -
there's also AC_PROG_SWIG. As per AX_PKG_SWIG, add swig-native to
DEPENDS if AC_PROG_SWIG is found in configure.ac.

(From OE-Core rev: 847a1aa7153fc8a7b820353283a6f1e51d64f8de)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-20 15:11:07 +01:00
Paul Eggleton 1370edb449 recipetool: create: detect python autoconf macros
If python is required then we need to inherit pythonnative (or
python3native) otherwise do_configure will probably fail since it won't
be able to find python.

(From OE-Core rev: 63234cc45aee91b031657971f36997e1443f80ee)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-20 15:11:07 +01:00
Paul Eggleton 541c54e301 recipetool: create: fix error with git tree and no network
When creating a recipe for an existing local git clone, we attempt to
use the fetcher to determine if it supports the SRCREV variable.
Unfortunately running this code does a network check to get the latest
revision as a direct result of us using '${AUTOREV}' as a default value.
If you don't have a network connection this will of course fail. Rather
than have this block creating the recipe, catch the exception and just
guess from the URL.

Ultimately this should probably be fixed in the fetcher but for now this
will at least resolve the issue on this end.

(From OE-Core rev: f7e43f931d7d6019a3b2509b2b2635978fbbae36)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-20 15:11:07 +01:00
Paul Eggleton e30276cb18 recipetool: create: fix name/version extraction from filename
I ran into an example where recipetool was getting the name/version
completely wrong:

https://bitbucket.org/sortsmill/libunicodenames/downloads/libunicodenames-1.1.0_beta1.tar.xz

>From this it would create a libunicodenames-1.1.0-beta1_1.1.0-beta1.bb
file (likely because it couldn't split the file name and therefore took
all of it, then got the version from one of the files inside the
tarball). When this happens it's just irritating because you then have
to delete the recipe / run devtool reset and then run recipetool create
/ devtool add again and specify the version manually.

This patch is the result of systematically running the
determine_from_filename() function over the files on the Yocto Project
source mirror and my local downloads directory and fixing as many of the
generic issues as reasonably practical - it now gets the name and
version correct much more often. There are still cases where it won't,
but they are now in the minority.

(From OE-Core rev: 7b018b1d493a8d10fd02b8cc220990b191c87fe5)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-20 15:11:07 +01:00
Paul Eggleton 9aa1cf3a28 recipetool: create: improve python recipe license handling
Try to ensure that for Apache, GPL and LGPL where the values extracted
from the "Classifiers" field may not be version-specific, if there is a
versioned license in the free-form license field then use that instead.
Also insert the free-form license field as a comment in the recipe for
the user's reference.

(From OE-Core rev: 237f66042eedd906f654827b53bf9269738267ab)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-20 15:11:07 +01:00
Paul Eggleton 3a8a0bba9b recipetool: create: fix picking up name from local python source tree
Make use of the extravalues dict to send back other variable values from
the python handling plugin, and enable passing back PV and PN. This not
only places variable values in the final recipe a bit more consistently
with other types of source, it also allows the name and version to be
picked up fron a local source tree and not just when the recipe is
fetched from a remote URL that happens to have those in it.

(From OE-Core rev: 3e7029f28c6ea9bb1d283bcdc3fdfee11455af8e)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-20 15:11:07 +01:00
Robert Yang 6a5bd99bfc runqemu: acquire_lock() should fail when failed to open the file
The open(self.lock, 'w') may fail when the lock is created by other
users, return false for this case to let it try other devices.

Fixed:
runqemu - INFO - Running /sbin/ip link...
runqemu - INFO - Acquiring lockfile /tmp/qemu-tap-locks/tap0.lock...
Traceback (most recent call last):
  File "/buildarea/lyang1/poky/scripts/runqemu", line 972, in <module>
    ret = main()
  File "/buildarea/lyang1/poky/scripts/runqemu", line 963, in main
    config.setup_network()
  File "/buildarea/lyang1/poky/scripts/runqemu", line 810, in setup_network
    self.setup_tap()
  File "/buildarea/lyang1/poky/scripts/runqemu", line 761, in setup_tap
    if self.acquire_lock():
  File "/buildarea/lyang1/poky/scripts/runqemu", line 182, in acquire_lock
    lock_descriptor = open(self.lock, 'w')
PermissionError: [Errno 13] Permission denied: '/tmp/qemu-tap-locks/tap0.lock'

(From OE-Core rev: f364f773a0381a75b5992c8c8a1d63a81dbd4422)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-20 15:11:07 +01:00
Ross Burton 359feedfd5 scripts: add tool to scan for bashisms recipe shell scripts
Shell functions in bitbake are executed with /bin/sh so should be POSIX
compliant and not use Bash extensions, or at least only use extensions that are
implemented in both dash and ash (busybox).

This tool will extract all of the shell scripts from all recipes and run them
through checkbashisms (it assumes that checkbashisms is on $PATH).

There is a whitelist to filter out false-positives such as the use of $HOSTNAME
(a bashism) in functions where we have defined it, or using the 'type' builtin
which is supported by ash/dash.

[ YOCTO #8851 ]

(From OE-Core rev: d77fe838ab7631a19e90ff4226f0712e54aa4e22)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-16 15:24:03 +01:00
Markus Lehtonen 12cd9c01fc scripts: introduce buildstats-diff
New script for comparing buildstats from two separate builds. The script
has two modes: normally it prints the differences in task execution
(cpu) times but using --ver-diff option makes it just print the recipe
version differences without any cpu time data. Other command line
options are provided to alter the sort criteria of the data and to
filter out insignificant differences and/or short tasks.

(From OE-Core rev: e707718374ce1c95769a5f99aa3bfdfc0be685b2)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-16 15:24:03 +01:00
Humberto Ibarra b28e6d6039 oe-selftest: check for coverage version before starting tests
python coverage versions lower than 4.x have problems with some distros. Adding the 4.x version as requirement to continue with coverage tracking.

[YOCTO #10207]

(From OE-Core rev: a378b817504986173c4b0984a28aead247589b3f)

Signed-off-by: Humberto Ibarra <humberto.ibarra.lopez@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-15 12:15:07 +01:00
Ed Bartosh f7367de2ad create-pull-request: remove output directory
When 'git request-pull' fails it makes sense to remove output
directory. Otherwise create-pull-request will complain that
output directory already exists on the next run.

(From OE-Core rev: 8535784ba84060e0e7ca05e7771a58f9c70c69b9)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-15 12:15:07 +01:00
Robert Yang 26e46e6d90 runqemu: fix a race issue on lockdir
There might be a race issue when multi runqemu processess are
running at the same time:
| Traceback (most recent call last):
|   File "/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-ipk/build/scripts/runqemu", line 920, in <module>
|     ret = main()
|   File "/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-ipk/build/scripts/runqemu", line 911, in main
|     config.setup_network()
|   File "/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-ipk/build/scripts/runqemu", line 760, in setup_network
|     self.setup_tap()
|   File "/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-ipk/build/scripts/runqemu", line 697, in setup_tap
|     os.mkdir(lockdir)
| FileExistsError: [Errno 17] File exists: '/tmp/qemu-tap-locks'

(From OE-Core rev: ec33043477a0b915b0911f7d7eacb24361e4aaa8)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-14 22:22:13 +01:00
Richard Purdie 0d214406c4 scripts/runqemu: Add snapshot support
Allow access to the snapshot option of qemu to simplify some of our runtime
testing to avoid copying images.

(From OE-Core rev: 8fec4a5a004f0e99734f8c0820c66522d08f213e)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-09 12:07:32 +01:00
Richard Purdie b2dc9c7ee2 runqemu: Enable virtio RNG for all platforms
We have problems where systems simply stop booting and hang. This is due
to a lack of entropy which means ssh keys and networking can't be brought
up. Adding in the virtio-rng passthrough support allows host entropy to
pass into the guess and avoids these hangs.

This is particularly problematic after the gnutls upgrade which starts
using /dev/random instead of /dev/urandom but was an issue we'd occasionally
seem before that.

It particualrly affected x86 and ppc machines for some reason.

(From OE-Core rev: 51b001909f1856c45cf87091d6e4446c266d5786)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-09 12:07:32 +01:00
Richard Purdie eb1494f9da runqemu: Update to modern prefrerred net syntax
(From OE-Core rev: 5e61766d976b6d036946c1b4e4ac742a33a03815)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-09 12:07:32 +01:00
Richard Purdie d1cb381977 runqemu: Allow unique network interface MAC addresses
Current qemu instances all share the same MAC address. This shouldn't be an
issue as they are all on separate network interfaces, however on the slight
chance this is causing problems, its easy enough to ensure we use unique
MAC addresses based on the IP numbers we assign.

(From OE-Core rev: c01962bf88786dd84ad83cc1d315297607d29f7c)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-09 12:07:32 +01:00
Joshua Lock d1303c220e runqemu: fix run from testimage with non-standard DEPLOY_DIR_IMAGE
testimage.bbclass uses runqemu to execute runtime tests on a qemu
target, this means that bitbake is already running and `bitbake -e`
can't be called to obtain bitbake variables.

runqemu tries to work around being unable to read values for
bitbake variables by inferring the MACHINE from the
DEPLOY_DIR_IMAGE setting, however if a user sets that variable in
a manner which doesn't follow the systems expectations (i.e. if
running `bitbake -c testimage` against a directory of pre-generated
images in a user-specified path) the inferring of the MACHINE name
from the DEPLOY_DIR_IMAGE location will fail.

It's possible that check_arg_machine() shouldn't cause runqemu to
fail and that runqemu should proceed with the user-supplied value
even if it can't be verified. This patch simply ensures that a
workflow where the user sets DEPLOY_DIR_IMAGE continues to work
without changing too much of the runqemu code.

[YOCTO #10238]

(From OE-Core rev: f94ac02f459e2ea0fc471463966997814a67e0ca)

Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-09 12:07:32 +01:00
Joshua Lock 8a8948e0e1 runqemu: fixes for when invoked during a bitbake run
When runqemu is invoked from a running bitbake instance it will be
unable to call `bitbake -e` due to the lock held by the calling
bitbake instance.

Our test code sets an OE_TMPDIR environment variable from which we
can infer/guess paths. Add code to do so when self.bitbake_e can't
be set, much as the sh version of runqemu did.

[YOCTO #10240]

(From OE-Core rev: 1e8165ea2f19aecdc03ccd102ee44ef0544f0f39)

Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-09 12:07:32 +01:00
Joshua Lock d5d4869634 runqemu: better handle running on a host with different paths
If the STAGING_*_NATIVE directories from the config file don't exist
and we're in a sourced OE build directory try to extract the paths
from `bitbake -e`

(From OE-Core rev: 9326af1c20636320c70caecebd47aedafb3f2d25)

Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-09 12:07:32 +01:00