Commit Graph

2555 Commits

Author SHA1 Message Date
Ed Bartosh 01d37e7537 wic: direct: set bootloader.source in the __init__
Moved setting of bootloader source from do_create method
to __init__ as it doesn't have anything to do with image
creation.

(From OE-Core rev: 361b890da1c7b24de0a62516545e4c164830081d)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-02-15 20:06:45 -08:00
Ed Bartosh 58528cd5a7 wic: direct: move UUID generation to PartitionedImage
Moved code that generates partition UUIDs from DirectPlugin to
PartitionedImage class as it's more logical to have it there.
This allows the code to be reused by other imager plugins.

Got rid of having yet another list of partitions in PartitionedImage.
Reused the list passed from DirectPlugin.

(From OE-Core rev: 20c70dd617d4abfff507e210610a58a9989f7c9b)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-02-15 20:06:45 -08:00
Ed Bartosh d5f42cad0c wic: direct: add 'realnum' attribute to partition
Replaced call of _get_part_num method with an attribute.
This eliminates the need to call the method and loop over
partitions every time we need to know realnum for partition.

(From OE-Core rev: 96a26b6ddf22346dafa06b00816579439e98445f)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-02-15 20:06:45 -08:00
Ed Bartosh 77dc8e6ef3 wic: direct: remove useless code
Removed catching CreatorError and raising it again.

(From OE-Core rev: c17353c8ec9ecb52368e08dad999aafc9ac1b734)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-02-15 20:06:45 -08:00
Ed Bartosh 870171a205 wic: direct: don't catch ImagerError
Don't transform ImagerError exception into warning.
Let wic to catch it on the upper level.

(From OE-Core rev: 0cc84c81dd4a776aa37c8d1bcdcc3fa3af78d1e4)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-02-15 20:06:45 -08:00
Ed Bartosh 417e5fbe2b wic: direct: remove set_bootimg_dir setter
Removed java-like setter set_bootimg_dir. It's more pythonic
to access public attributes directly.

(From OE-Core rev: c2a6ca4883ea59e6492ad3b4aa0e9bc358b87fed)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-02-15 20:06:45 -08:00
Ed Bartosh 6d91c5f6f4 wic: ksparser: set default disk to 'sda'
Set default value of --ondisk to 'sda' to ensure
we always have disk name for the partition.

This is a first step of replacing --ondisk with
disk <name> attribute of .wks. This is better as
all partitions share the same disk.

(From OE-Core rev: caa243a86ba50c676f8eb0a71440885a49f10cc4)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-02-15 20:06:45 -08:00
Ed Bartosh 93906fb830 wic: remove utils/oe/__init__.py
This file and utils/oe folder are not needed anymore as
all modules were removed or moved out of this directory.

(From OE-Core rev: e3b73b1c07620cde423cc7db7e2f7d8b1ad25e25)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-02-15 20:06:45 -08:00
Ed Bartosh cdcc6e2753 wic: move PartitionedImage class to direct.py
As PartitionedImage is only used in direct.py it makes sense
to move it there. It's easier to maintain (and refactor) it
this way.

(From OE-Core rev: 2550622371f5c50857e5d58eabab01a1823c6fc3)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-02-15 20:06:44 -08:00
Ed Bartosh adf5e2956c wic: move disk operations to PartitionImage class
Disk operations were spread over DirectPlugin, DiskImage and Image
code making the code hard to understand.

Renamed Image class to PartitionedImage.
Removed DiskImage class.
Moved disk operations to PartitionedImage.

There was an implicit support for multiple disks: if different devices
were specified in .wks file (e.g. --ondisk sda and --ondisk sdb), wic
would theoretically generate multiple images. This is quite confusing
option and the code supporting it was broken for a long time. The same
effect (multiple output images) can be achieved in obvious and clear
way - by using multiple .wks files.

This functionality was removed. PartitionedImage works only with
one image. This makes the code less complex and easier to maintain.

(From OE-Core rev: 4dc9dbfc7fbc16d349a019e8973d50905cd28244)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-02-15 20:06:44 -08:00
Kristian Amlie f6a064d969 wic: Add --exclude-path option to rootfs source plugin.
It will omit the given path from the resulting partition, and if the
given path ends in a slash, it will only delete the content, and keep
the directory.

Since mkfs only accepts whole directories as input, we need to copy
the rootfs directory to the workdir so that we can selectively delete
files from it.

Since we want to use the copyhardlinktree() function, we need to put
the generic oe lib in the module search path.

(From OE-Core rev: 6602392db3d391d926dead49fcc54326015cfe35)

Signed-off-by: Kristian Amlie <kristian.amlie@mender.io>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-02-15 20:06:44 -08:00
Ed Bartosh 7cb17e3e9e wic: remove unused argument scripts_path
There is no need to pass scripts_path from main wic module
down the stack as it's not used there.

Removed scripts_path argument from DirectPlugin class
and wic_create function.

(From OE-Core rev: f9f72c506befdff13260f37ded0beaea3aa30fad)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-02-15 20:06:44 -08:00
Ed Bartosh 74aee7a1a7 wic: direct: remove unused import
Removed unused import of wic.errors module.

(From OE-Core rev: 318ba1edfcf0cacf6adae7e1af625dcb43ec3881)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-02-15 20:06:44 -08:00
Ed Bartosh 348c9c50c1 wic: direct: remove unused plugin attributes
Removed unused _disks, _disk_format and _disk_names
attributes from DirectPlugin class.

(From OE-Core rev: 178df49cf1674bfcf1cb7295d0494c3b23929d22)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-02-15 20:06:44 -08:00
Ed Bartosh a9be5de4a1 wic: make sure layout_partitions is called once
Removed artificial _partitions_layed_out attribute and
unneeded call of layout_partitions method.

(From OE-Core rev: 702772edc839c220140ac0572bb14b4e44c81c1c)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-02-15 20:06:44 -08:00
Ed Bartosh fd1f6fb201 wic: use the same partition object in direct and partitionedfs
Partition attributes were copied to the dictionary in partitionedfs
code, which makes the code hard to follow.

Used partition object passed from direct.py module as is in
partitionedfs.

(From OE-Core rev: 97db24d34847a641868f9ee83aae56f9dd5e0a8a)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-02-15 20:06:44 -08:00
Ed Bartosh 4edcd63ba1 wic: direct: get rid of _get_parts getter
Replaced _get_parts getter with direct attribute
access to self.parts

Removed code that implicitly created partition
if there are no partitions mentioned in .wks file

(From OE-Core rev: 7ece57a80f4002d0d83dc322092e9178380ab509)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-02-15 20:06:44 -08:00
Ed Bartosh 6355150674 wic: partitionedfs: rename __create_partition and __add_disk
Renamed private methods with leading double underscores:
__create_partition -> _create_partition
__add_disk -> _add_disk

There is no point to have those names mangled, one underscore
is enough.

(From OE-Core rev: 26f3218070d34ccd4e81fa3b8e1a15c03583d070)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-02-15 20:06:44 -08:00
Ed Bartosh 86e190a614 wic: partitionedfs: get rid of __add_partition
3 lines long private method __add_partition is called only
from add_partition method. Merged them together to increase
readability.

(From OE-Core rev: b7fb20fffada61211dda6d41f99407618428f23c)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-02-15 20:06:44 -08:00
Ed Bartosh b8354df789 wic: partitionedfs: merged __format_disks and create
Private method __format_disks is called only from create
method making the code less readable. Merged the code
into one method.

(From OE-Core rev: b76b1bd404487df38fd99bc0d0e6a59acb10c9d3)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-02-15 20:06:44 -08:00
Markus Lehtonen f4ba140c59 build-perf-test-wrapper.sh: fix issues with non-existing path arguments
Without the '-s' option realpath will error out if the given path does
not exist.

(From OE-Core rev: b80aba08ba56c7e8f847966b3593f6cedd1b1ee5)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-02-15 20:06:41 -08:00
Markus Lehtonen 1440b3e9aa oe-build-perf-test: drop --commit-results
Drop support for committing results into Git repository. The
functionality was not directly related to testing and feels unnecessary
complication of the script. The functionality has been moved into a
separate oe-git-archive script.

[YOCTO #10582]

(From OE-Core rev: 4de387c0cfcb6b58760c6b6e150474abe82bfe4c)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-02-15 20:06:41 -08:00
Markus Lehtonen 37c9f3f180 build-perf-test-wrapper.sh: use oe-git-archive
Start to use the new helper script for archiving results data in Git.

[YOCTO #10582]

(From OE-Core rev: 3d8a1df37407686ed699485c37c8517f1d79755d)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-02-15 20:06:41 -08:00
Markus Lehtonen 3c862c647c scripts/oe-git-archive: support creating bare repositories
[YOCTO #10582]

(From OE-Core rev: 995cb3bf2b3303067828a1358051e5e11caad843)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-02-15 20:06:41 -08:00
Markus Lehtonen 69c7907bd9 scripts: Implement oe-git-archive
A helper script for committing data to git and pushing it upstream. The
motivation for the script stems from the need to archiving QA test
results in git, but it could be used elsewhere, of course. The script
needs to be run under an initialized build environment because it
utilizes bitbake configuration information.

[YOCTO #10582]

(From OE-Core rev: 801e612b137b9d5366639d5b1635151347da5983)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-02-15 20:06:41 -08:00
Richard Purdie bfd0a39bdf classes: Drop now unneeded update_data calls
Now that the datastore works dynamically we don't need the update_data calls
so we can just remove them. They're not actually done anything at all for
a while.

(From OE-Core rev: 8de0c5d3bd01919e2bf0394f9c485936d6098cec)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-02-15 09:29:55 -08:00
Paul Eggleton e8f4e06e76 devtool: improve parse failure handling
With the move to tinfoil2, the behaviour when parsing failed has changed
a bit - exceptions are now raised, so handle these appropriately.
Specifically when if parsing the recipe created when running devtool add
fails, rename it to .bb.parsefailed so that the user can run bitbake
afterwards without parsing being interrupted.

(From OE-Core rev: b9592bd3c1ab2155b9f62d18ce593b9c88eff279)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-02-07 14:50:09 +00:00
Paul Eggleton e0a94f2e1f recipetool: create: do not treat numbers in SCM URLs as versions
Numbers within SCM (e.g. git) URLs are extremely unlikely to be valid
version numbers - more likely they are just part of the name, thus don't
try to extract them and use them as the version - doing so causes pretty
bad behaviour within devtool:

--------- snip ---------
$ devtool add https://github.com/inhedron/libtr50
NOTE: Fetching git://github.com/inhedron/libtr50;protocol=https...
...
NOTE: Using default source tree path .../build/workspace/sources/libtr
...
RecursionError: maximum recursion depth exceeded while calling a Python object
--------- snip ---------

(This was because ${PV} was being substituted into the URL, but PV's
value was being set to include ${SRCPV}, so there was a circular
reference.)

(From OE-Core rev: 3427508b6ce865654f8bf01a6fc04b83c70315d3)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-02-07 14:50:09 +00:00
Paul Eggleton 6ac9c605e6 recipetool: create: properly handle npm optional dependencies
npm's package.json supports two types of dependencies -
optionalDependencies and dependencies; in the code for creating a recipe
from a non-npm source (e.g. a git repository) we were not handling
optionalDependencies and thus when pointed at a node.js application
outside of npm we weren't taking care of all dependencies.

(From OE-Core rev: 2b66cb9982d10ce1744d430858eaef3e5a72c8c0)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-02-07 14:50:09 +00:00
Ed Bartosh 07f32d7af8 wic: fix call of serial_console_form_kargs
As syslinux module has been recently removed and
serial_consloe_form_kargs became local API in rootfs_pcbios_ext plugin
it should be called without syslinux. prefix.

(From OE-Core rev: d701673b658d879726d6cf846a6d5f4173c3b0e5)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-02-05 09:22:18 +00:00
Ed Bartosh f9b58d0d11 wic: isoimage-isohybrid: use wic-tools to get syslinux path
wic-tools recipe specific sysroot contains syslinux as wic-tools
depends on it. Used wic-tools target to get syslinux path should
guarantee that syslinux is installed there and can be used.

(From OE-Core rev: be8ebac3b055070f690d42836bb0fdad32d29204)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-02-05 09:22:18 +00:00
Ed Bartosh 71ce8d09e0 wic: flatten imager class hierarchy
wic code is hard to follow due to deep and twiggy class
inheritance tree.

Flatten imager tree:
 wic -> wic_create -> Creator -> DirectPlugin -> DirectImageCreator
to
 wic -> wic_create -> DirectPlugin
by
 removing Creator class and creator module
 merging DirectImageCreator into DirectPlugin

Changed APIs to use the same parameters names.

Passed parsed command line options as an object down the stack.

(From OE-Core rev: 1e28d512341ce470c7afb256a01e597ab87170ca)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-02-05 09:22:17 +00:00
Anders Darander 8243ebdbb7 python-3-manifest: split out typing
This allows us to use typing.py without having to add the whole
python3-misc package.

(From OE-Core rev: 66c282541a13f2d1224d3ba933a953c0f613fb2a)

Signed-off-by: Anders Darander <anders@chargestorm.se>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-02-05 09:22:16 +00:00
Anders Darander 5776eea84d python-3-manifest: add ipaddress to python3-io
This allows us to use ipaddress without requiring the add the whole
python3-misc.

(From OE-Core rev: d0e9013f6c68104369de940f5dd8a6ef76cd0210)

Signed-off-by: Anders Darander <anders@chargestorm.se>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-02-05 09:22:16 +00:00
Anders Darander db6977f72c python-3-manifest: add _compat_pickle to python3-pickle
This allows us to depend on _compat_pickle.* wihtout having to add the whole
python3-misc.

(From OE-Core rev: 6c435672b27d1c97ce3776c98d073a92fde887cd)

Signed-off-by: Anders Darander <anders@chargestorm.se>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-02-05 09:22:16 +00:00
Anders Darander 64ab82509d python-3-manifest: add socketserver to netserver
socketserver.* should be part of python3-netserver.

(From OE-Core rev: 8aa71c7133f996f6ed09a3c094e553bed7f33c7a)

Signed-off-by: Anders Darander <anders@chargestorm.se>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-02-05 09:22:16 +00:00
Anders Darander ce5661ddc1 python-3-manifest: add argparse to RDEPENDS for netclient
http/server.py requires argparse.

(From OE-Core rev: 04d3e78b31c247ee521a2d4c15286a8c030637cb)

Signed-off-by: Anders Darander <anders@chargestorm.se>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-02-05 09:22:16 +00:00
Anders Darander 67127bec02 python3-manifest: move htlm.py to python3-html
This allows us to use html.py without importing misc.

(From OE-Core rev: d1fefcaa10b85837d18458e291a3df9093b3a6ea)

Signed-off-by: Anders Darander <anders@chargestorm.se>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-02-05 09:22:16 +00:00
Anders Darander cf610aa69f python-3-manifest: fix adding imp to importlib
Commit: 512334f102a33833d39af53467894315f0715d07
	"python-3.5-manifest: Add imp to importlib"

added imp to importlib in the generated manifest, but not in the generator script.

(From OE-Core rev: bfe1d948ab034017622110c8365be7026e139fbf)

Signed-off-by: Anders Darander <anders@chargestorm.se>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-02-05 09:22:16 +00:00
Ed Bartosh 3571e3e77f wic: get rid of image_output_dir variable
Used options.outdir instead of image_output_dir.
There is no sense to use extra variable for this.

(From OE-Core rev: c6013dcb158a84d48cc2677f1509681cf9e0a3cb)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-02-02 17:37:45 +00:00
Ed Bartosh 14652a42d8 wic: direct: fix creation of work directory
It was a typo in current code: mktemp was used instead of
mkdtemp to create work directory. This is fixed by using
mkdtemp.

Create work directory as a subdirectory of output directory
to make sure both are on the same partition to make moving
of result image faster.

This also fixes possible disk space issues as mkdtemp uses
TMPDIR, TEMP or TMP environment variables to get default value
of its 'dir' parameter. Those variables are usually pointing
to /tmp, which is not the best location to create huge images.

(From OE-Core rev: e6579b4e007ce89000c46c09917fbd3c72f18a73)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-02-02 17:37:45 +00:00
Ed Bartosh 81ef850314 wic: engine: create output dir
Make sure output directory exists before creating an image.
Create it if it doesn't exist.

(From OE-Core rev: e52bfb0e81d3fb2a474f08b2e2b8b89aadc61d14)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-02-02 17:37:45 +00:00
Ming Liu 6469647f48 scripts/runqemu: fix a typo
(From OE-Core rev: c72d5acb9c2f4a7d4dfe0e78aae832b10aec4429)

Signed-off-by: Ming Liu <peter.x.liu@external.atlascopco.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-02-02 17:37:45 +00:00
Markus Lehtonen 42eace0770 build-perf-test-wrapper.sh: implement locking
In order to prevent multiple instances of the script running at the same
time.

(From OE-Core rev: 96a194de890f7ef1e6e5e036b32848e0f9d1bcf5)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-02-02 17:37:45 +00:00
Ed Bartosh f1957bf59e wic: remove syslinux.py
This module contains singe function serial_console_form_kargs, which
is used only by rootfs_pcbios_ext plugin. Moved it there and removed
syslinux module to make it easy to find and mainain plugin code.

[YOCTO #10619]

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-02-02 17:37:45 +00:00
Ed Bartosh 653aaea3cc wic: code cleanup
Fixed indentation, unused imports, trailing lines etc.

[YOCTO #10619]

(From OE-Core rev: 5fa7768bfb4b6d464c6a812822b0665f52e7bea4)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-02-02 17:37:44 +00:00
Ed Bartosh 1f7ce90af6 wic: msger.py: remove unused APIs
Removed unused enable_logstderr and disable_logstderr APIs.

[YOCTO #10619]

(From OE-Core rev: 835f7a5c4c51f3d0d31d0193258b50596ceacb5a)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-02-02 17:37:44 +00:00
Ed Bartosh 9048f52f3a wic: removed code from __init__.py
The code deals with non-existing directory
and can be removed.

[YOCTO #10619]

(From OE-Core rev: e148884b8145fad212c4085f2ead5c3e5e3e42a2)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-02-02 17:37:44 +00:00
Ed Bartosh 614851f1db wic: move oe/misc.py one level up
Flattened directory structure:
   moved wic/utils/oe/misc.py -> wic/utils/misc.py

[YOCTO #10619]

(From OE-Core rev: 392532a2748ff2e6412eeb79163662b5168611ce)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-02-02 17:37:44 +00:00
Ed Bartosh 2d6f96048e wic: move 2 APIs to wic.engine
Moved find_canned and get_custom_config APIs to engine module.
Removed empty wic.utils.misc module.

[YOCTO #10619]

(From OE-Core rev: 10e9afac46575d3f557b7cb505daa31ce9ce85fa)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-02-02 17:37:44 +00:00
Ed Bartosh 759c24c404 wci: misc: removed build_name API
This API is not used in wic code.

[YOCTO #10619]

(From OE-Core rev: 9d14cb8bb8cf9559b7ff0efa03427624c18e2506)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-02-02 17:37:44 +00:00
Ed Bartosh 08217a4c80 wic: partition: simlify calling plugin methods
Replaced parse_sourceparams function with list comprehension.
Used local variables instead of attributes.
Moved global variable to the local scope.

[YOCTO #10619]

(From OE-Core rev: 4adbac84046ff744f1452b5ff4d017d17d2d45e2)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-02-02 17:37:44 +00:00
Ed Bartosh d8a89baffb wic: removed test file
This file is not used anywhere in the wic code.

[YOCTO #10619]

(From OE-Core rev: e3c43ce3261663225aeba50b8c6229577574f9b7)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-02-02 17:37:44 +00:00
Ed Bartosh 53bd29d65e wic: renamd direct_plugin.py -> direct.py
As this files is located in plugins/imager subdirectory it's
obvious that it's an imager plugin. Renamed to direct.py to
be consistent with plugin naming scheme.

[YOCTO #10619]

(From OE-Core rev: d5db8c2ee91bdd51bfbb2ebf61aea8ff0378d512)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-02-02 17:37:44 +00:00
Ed Bartosh 8fda677a08 wic: simplified code of direct_plugin
Removed unused methods.
Got rid of get_default_source_plugin and _full_name methods.

[YOCTO #10619]

(From OE-Core rev: c5706f71dee531557a6b1290665283b8637fff6e)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-02-02 17:37:44 +00:00
Ed Bartosh 59496d9cdc wic: pylinted direct_plugin
Fixed wrong continued indentation, unused import and
trailing new line pyling warnings.

[YOCTO #10619]

(From OE-Core rev: e78762a4b52c31357ef29a5b93c53db7190fa4f2)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-02-02 17:37:44 +00:00
Ed Bartosh f36c70b067 wic: improve naming in direct_plugin classes
Synchronized attribure names in DirectImageCreator and
DirectPlugin for better readability. Simplified code,
removed unneeded global variable disk_methods.

(From OE-Core rev: b87b9ef84791615636424a224f74386a4aa0c2fa)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-02-02 17:37:44 +00:00
Ed Bartosh 2db161f1c0 wic: get rid of __rootfs_dir_to_dict method
Replaced class method __rootfs_dir_to_dict with a list
comprehension.

[YOCTO #10619]

(From OE-Core rev: 266fd31410771db4c06539f7368c196a6d03000a)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-02-02 17:37:44 +00:00
Ed Bartosh ee10c2f1bb wic: moved content of direct.py to direct_plugin
This move simplifies directory structure and makes
further refactoring easier. The code from direct.py was used
only in direct_plugin, so it's safe to move it there.

[YOCTO #10619]

(From OE-Core rev: a8f5ebb26183faa9af6eb72f4dabfcf83aa1e8d4)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-02-02 17:37:44 +00:00
Ed Bartosh dba8cf377c wic: removed conf.py and empty config file.
Removed as they're not used anymore in wic code.

(From OE-Core rev: ffa2f3d7bf883d5add911b7c5d0be2b347733524)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-02-02 17:37:44 +00:00
Ed Bartosh 0ff04e1e95 wic: direct_plugin: stop using config manager
This is a preparation to removing conf.py and config/wic.conf
from the codebase.

Got rid of using configmgr global object in direct_plugin and direct
modules. It was used to implicitly parse kickstart file and set
couple of variables.

Replaced usage of configmgr by passing parameters directly to the
DirectImageCreator.

[YOCTO #10619]

(From OE-Core rev: 79191119de010acb107f9392a991108728858441)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-02-02 17:37:44 +00:00
Ed Bartosh 65a99448a4 wic: creator: stop using config manager
This is a preparation to removing conf.py and config/wic.conf
from the codebase.

confmgr object is complicated for no reason and almost
useless as all configuration info comes from command line and
bitbake variables. It's used it creator.py to store information
about output directory, logs and some never used functionality
like tmpfs for future use, which doesn't actually happen.

[YOCTO #10619]

(From OE-Core rev: 702ee7a1fe30d87d55ba9528ae89abff4e294fd9)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-02-02 17:37:44 +00:00
Patrick Ohly d99b29838d verify-bashisms: support warnings with more than one line of source code
All warnings start with "possible bashism in", followed by one or more
(in the case of line continuation) lines of source code. To support
more than one line, we now split by matching against the known intro
text.

Example:

 $ verify-bashisms guile
 ...
 /.../openembedded-core/meta/recipes-devtools/guile/guile_2.0.13.bb
  possible bashism in guile_cross_config line 94 ($'...' should be "$(printf '...')"):
         	echo '#!'`which ${BUILD_SYS}-guile`$' \\\n--no-auto-compile -e main -s\n!#\n(define %guile-build-info '\'\( \
 			> ${B}/guile-config.cross

(From OE-Core rev: e2dd3621c45e854b4eb054b4d4537487462cdd39)

Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-31 15:28:41 +00:00
Patrick Ohly 0b95e47180 verify-bashisms: check scripts only once, include original file and line
Several scripts that are defined in .bbclass files end up in multiple
different recipes. It's better (faster, less repetitive error reports)
to check them only once.

In addition, the real information for the developer is where he can
find the script, not which recipe file uses it. verify-bashisms now
prints the original file instead of the recipe whenever possible
(i.e. 'filename' is set) and also bumps the line number so that it is
relative to the file and not the script.

Example with one real error and one added just for testing:

  $ verify-bashisms core-image-minimal core-image-sato
  Loading cache: 100% |#################################################################################| Time: 0:00:00
  Loaded 2935 entries from dependency cache.
  Parsing recipes: 100% |###############################################################################| Time: 0:00:01
  Parsing of 2137 .bb files complete (2101 cached, 36 parsed). 2935 targets, 412 skipped, 0 masked, 0 errors.
  Generating scripts...
  Scanning scripts...

  /.../openembedded-core/meta/classes/populate_sdk_ext.bbclass
   possible bashism in install_tools line 515 (should be 'b = a'):
  	if [ "${SDK_INCLUDE_TOOLCHAIN}" == "1" -a ! -e $unfsd_path ] ; then
   possible bashism in install_tools line 521 (type):
            type fixme

(From OE-Core rev: ca4932b60f464430266cc43e34122b2973e8a200)

Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-31 15:28:41 +00:00
Patrick Ohly 3108bff175 verify-bashisms: revise update-rc.d whitelist entry
The actual code recently changed to:
   if ${@use_updatercd(d)} && type update-rc.d >/dev/null 2>/dev/null; then

(From OE-Core rev: 32ae3e686db067a2a63932782970db79eb1703e8)

Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-31 15:28:41 +00:00
Patrick Ohly 010f9fa86a verify-bashisms: fix problems with tinfoil2
tinfoil2 is based on a client/server architecture, which broke the
verify-bashisms script:

- The tinfoil instance and its data proxies can't be pickled, so
  all interaction with the bitbake server has to run in the main
  script process and only processing of the plain scripts can
  be done with multiprocessing:

  _pickle.PicklingError: Can't pickle <class 'bb.tinfoil.TinfoilCookerAdapter.TinfoilRecipeCacheAdapter'>: attribute lookup TinfoilRecipeCacheAdapter on bb.tinfoil failed

- The multiprocessing pool has to be created before initializing
  tinfoil, otherwise the pool workers end up trying to communicate
  with the bitbake server during shutdown:

  ERROR: UI received SIGTERM
  Process ForkPoolWorker-2:
  Traceback (most recent call last):
    File "/usr/lib/python3.4/multiprocessing/process.py", line 257, in _bootstrap
      util._exit_function()
    File "/usr/lib/python3.4/multiprocessing/util.py", line 286, in _exit_function
      _run_finalizers(0)
    ...
    File "/usr/lib/python3.4/multiprocessing/process.py", line 131, in is_alive
      assert self._parent_pid == os.getpid(), 'can only test a child process'
   AssertionError: can only test a child process

- func() needs to defined before creating the pool to avoid:

  AttributeError: Can't get attribute 'func' on <module '__main__' from '/work/openembedded-core/scripts/verify-bashisms'>

(From OE-Core rev: aa439f11c7f414774843720d68ebe0a6d3375ea6)

Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-31 15:28:41 +00:00
Patrick Ohly 48c65ddaa4 verify-bashisms: explicitly shut down server
Current tinfoil2 requires manually shutting down the server.
Without that, the script hangs during exit. This might change
in the future.

(From OE-Core rev: 62f42ce063115f4a02c219e323252223818a150f)

Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-31 15:28:41 +00:00
Patrick Ohly 83469f229d verify-bashisms: point out where to get checkbashisms.pl
The current SourceForge project seems to be unmaintained (last release
2.0.0.2 from 2015) while the copy used by Debian is quite active (last
commit 2016-09-30).

Ideally, checkbashisms.pl should get installed automatically via a
recipe, but for now at least provide the link for manual installation.

(From OE-Core rev: 65e74348b4ed40b24671776410d2a579dcc7abab)

Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-31 15:28:40 +00:00
Patrick Ohly 15a97dea65 verify-bashisms: fix typo
Variable was renamed, it's now called "output".

(From OE-Core rev: c3d125b4132e5a706f6265a5bcaedafbe8e5355f)

Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-31 15:28:40 +00:00
Juro Bystricky 8b448ab012 runqemu: allow bypassing of network setup
At present it is silently assumed all QEMU machines support networking.
As a consequence, one cannot run QEMUs without network emulation
using "runqemu".
This patch allows bypassing any network setup providing the qemuboot.conf
file contains:

    qb_net = none

[YOCTO#10661]

(From OE-Core rev: 6a9454027ced4efbb401a23df94f711b8253c8fa)

Signed-off-by: Juro Bystricky <juro.bystricky@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-31 14:43:02 +00:00
Andrea Galbusera 6476182c33 oe-publish-sdk: add pyshtables.py to .gitignore
pyshtables.py should be ignored by git as it is generated. If kept in
the repo, causes subsequent runs of sdk-update to fail.

[ YOCTO #10963 ]

(From OE-Core rev: 0ac1387bd12b5d023dea06ffe65d1fdcb050bcb4)

Signed-off-by: Andrea Galbusera <gizero@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-31 14:43:00 +00:00
Ed Bartosh d5b2539213 isoimage-isohybrid: renamed variable hdd_dir
Renamed variable hdd_dir to deploy_dir as this variable is
assigned to the value of DEPLOY_DIR_IMAGE.

(From OE-Core rev: fc84893c319f3e8c72f13c58f25e0b54095c3a49)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-31 14:38:32 +00:00
Ed Bartosh 6740b8fdbb isoimage-isohybrid: use TRANSLATED_TARGET_ARCH instead of MACHINE_ARCH
isoimage-sihybrid plugin uses MACHINE_ARCH to get the name of initrd image.
It doesn't work for all machines, for example for quemux86-64 machine
MACHINE_ARCH is quemux86_64 and initrd name is
core-image-minimal-initramfs-qemux86-64.cpio.gz

Used TRANSLATED_TARGET_ARCH variable to get the initrd image name.
Replaced MACHINE_ARCH->TRANSLATED_TARGET_ARCH in WICVARS variable to
make it available from <image>.env file.

(From OE-Core rev: edf0830412de66eada9ef3f3947ca42e301c5377)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-31 14:38:32 +00:00
Tom Zanussi 66d08e0fda wic: Look for image artifacts in a common location
Rather than have each image type look for artifacts in image-specific
locations, move towards having them look for artifacts in a common
location, in this case DEPLOY_DIR_IMAGE

Use the existing deploy.bbclass to have the bootloaders put their binaries
in DEPLOY_DIR_IMAGE and then wic will find them and place them in the image

(From OE-Core rev: 1c8acea91e775b1b6f8b25e774aaba3e790fc59b)

Signed-off-by: Alejandro Hernandez <alejandro.hernandez@linux.intel.com>
Signed-off-by: Tom Zanussi <tom.zanussi@linux.intel.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-31 14:38:32 +00:00
Ed Bartosh 8a04795db6 wic: isoimage-isohybrid: stop using HDDDIR
Stop using HDDDIR in isoimage-isohybrid wic plugin.
This variable is set by hddimg code, which is going to be
removed soon.

All required artifacts should be available from ISODIR.
wic-image.bbclass has been modified to build iso artifacts,
so it should be safe to remove usage of HDDDIR.

[YOCTO #10835]

(From OE-Core rev: 4ceb0df10db7254b290784cabc5279ffa2b946a4)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-31 14:38:32 +00:00
Ed Bartosh a508cb384c wic: use INITRD_LIVE in isoimage-isohybrid
INITRD variable is not set if hddimg is disabled.
isoimage-isohybrid can't get correct name for initrd if INITRD
variable is not set.

Added INITRD_LIVE to WICVARS and used it in isoimage-isohybrid
code to get initrd artifact name. Used INITRD if INITRD_LIVE is not set.

(From OE-Core rev: c672753be60550b15d5d593c251b61776d5b104d)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-31 14:38:32 +00:00
Jukka Laitinen cebf67086f wic: partitionedfs: set partition name for gpt partitions
Set proper gpt partition name for the partitions in case given
in the configuration

(From OE-Core rev: bc6e6a34f35bd081d828160bab8ee12e770c7e1e)

Signed-off-by: Jukka Laitinen <jukka.laitinen@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-31 14:38:32 +00:00
Ed Bartosh 3407f25216 wic: direct.py: get rid of names with two underscores
Attributes with two leading underscores are mangled in Python
and used mainly for avoiding name clashes with names from
subclasses. They're not needed in most of wic classes.

(From OE-Core rev: 2f92c0490f1acf5a6926fc6654ce3b6588ddcc24)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-31 14:38:32 +00:00
Ed Bartosh d59196c3a8 wic: remove unused API DirectImageCreator.get_disk_names
(From OE-Core rev: f77efbe81a5d3d16ecb1872cf237bb3e56c63c39)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-31 14:38:31 +00:00
Ed Bartosh e9bd1ed673 wic: make workdir a temporary directory
Wic used hardcoded path /var/tmp/wic/ as a work directory,
which caused conflicts if two wic instances run in parallel.

Made work directory unique and temporary. Moved results from
work directory to output directory when they're ready.

[YOCTO #10783]

(From OE-Core rev: 5d948f04823dedfa26a879aa6a5dc23de2bb9354)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-31 14:38:31 +00:00
Ed Bartosh 75318c7b0c wic: get rid of baseimager inheritance
Simplified DirectImageCreator code by removing inheritance
from BaseImageCreator. This inheritance doesn't make much sense
as DirectImageCreator is the only class that was inherited from
BaseImageCreator.

(From OE-Core rev: 4e9952514211ef4b9a3731ce915090385f335a31)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-31 14:38:31 +00:00
Ed Bartosh 88dce2cf41 wic: change default output directory
Set default output directory to current dir.

[YOCTO #10783]

(From OE-Core rev: ad116c4d02ccf36e22fbf3e45e45bc508849a833)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-31 14:38:31 +00:00
Markus Lehtonen 48f79758ea scripts/buildstats-diff: simplify timestamp handling
Simply use floats instead of datetime and timedelta objects for handling
timestamps.

(From OE-Core rev: d97c844f388bd4c52248fe597d5985ef20d5a96d)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-28 23:32:16 +00:00
Patrick Ohly 95cdd91a80 runqemu: more verbose error message about missing qemuboot.conf
When invoking "runqemu" with a mistyped image or architecture name,
the resulting error message is about the missing qemuboot.conf,
without any indication about the root cause:

 $ runqemu core-image-mimimal ext4 intel-corei7-64
 runqemu - INFO - Assuming MACHINE = intel-corei7-64
 runqemu - INFO - Running MACHINE=intel-corei7-64 bitbake -e...
 runqemu - INFO - MACHINE: intel-corei7-64
 runqemu - INFO - DEPLOY_DIR_IMAGE: /fast/build/refkit/intel-corei7-64/tmp-glibc/deploy/images/intel-corei7-64
 Traceback (most recent call last):
   File "/work/openembedded-core/scripts/runqemu", line 1095, in <module>
     ret = main()
   File "/work/openembedded-core/scripts/runqemu", line 1082, in main
     config.read_qemuboot()
   File "/work/openembedded-core/scripts/runqemu", line 643, in read_qemuboot
     raise Exception("Failed to find <image>.qemuboot.conf!")
 Exception: Failed to find <image>.qemuboot.conf!

Including the name of the actual file the scripts expects to find plus
adding some hints what to check for might help. The error now is:

 $ runqemu core-image-mimimal ext4 intel-corei7-64
 ...
 Exception: Failed to find <image>.qemuboot.conf = .../tmp-glibc/deploy/images/intel-corei7-64/core-image-mimimal-intel-corei7-64.qemuboot.conf (wrong image name or BSP does not support running under qemu?).

The comment about the BSP is included because that would be the real
reason why the file might be missing.

(From OE-Core rev: 946c4558f6c2726d0f12e48974568188a4ffef0d)

Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-26 10:44:27 +00:00
Ross Burton 53d29591b3 wipe-sysroot: remove
This script was useful to delete the sysroot so that future builds happen from a
clean sysroot.  Now that recipes have their own sysroot this script doesn't need
to exist anymore, and if in fact if used will break your TMPDIR.

(From OE-Core rev: e8bc8dc1b2a54a00792c61e134f48a6c06ad861e)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-26 10:44:27 +00:00
Benjamin Esquivel 44ac2995b8 selftest: automatically add the meta-selftest layer
adding the meta-selftest layer is a small bump you always encounter
when you want to run selftest. Adding an automatic procedure to include
the layer if it is not yet present.

(From OE-Core rev: b5000dd7b1d74d5dfede869ffa75ed87f5a8553c)

Signed-off-by: Benjamin Esquivel <benjamin.esquivel@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-23 12:05:22 +00:00
Paul Eggleton dd83bdb2f7 create-pull-request: fix for OE cgit URL change
The /cgit.cgi/ part is no longer valid in cgit URLs on
git.openembedded.org as of recent infrastructure changes.

(From OE-Core rev: a17ac420290c7e0debddec78b9540ae8726720dc)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-23 12:05:22 +00:00
Paul Eggleton 61f206195d devtool: check locale and refuse to start if it isn't UTF-8
We need to ensure the locale is UTF-8 or otherwise strange errors will
occur later on during execution - the same reason we check this in
BitBake itself. Unfortunately this check has to be before command line
parsing and therefore showing the help text in response to --help, since
that relies upon parsing bitbake's configuration (as we need to load
plugins in other layers).

Fixes [YOCTO #10908].

(From OE-Core rev: 370c6ba16c72bb52e80da098a5812ed1e09ac659)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-23 12:05:22 +00:00
Robert Yang 9dd223bf18 runqemu: fixes for slirp, network device and hostfwd
Fixed:
- Add QB_NETWORK_DEVICE to set network device, it will be used by both
  slirp and tap.
- Set QB_NETWORK_DEVICE to "-device virtio-net-pci" in qemuboot.bbclass
  but runqemu will default to "-device e1000" when QB_NETWORK_DEVICE is
  not set, this is because oe-core's qemu targets support
  virtio-net-pci, but the one outside of oe-core may not,
  "-device e1000" is more common.
- Set hostfwd by default: 2222 -> 22, 2323 -> 23, and it will choose a
  usable port when the one like 222 is being used. This can avoid
  conflicts when multilib slirp qemus are running. We can forward more
  ports by default if needed, and bsp.conf can custom it.
- Use different mac sections for slirp and tap to fix conflicts when
  running both of them on the same host.

[YOCTO #7887]

CC: Nathan Rossi <nathan@nathanrossi.com>
CC: Randy Witt <randy.e.witt@linux.intel.com>
(From OE-Core rev: 7dddd090806914a62d977730440d803e48f44763)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-23 12:05:22 +00:00
Robert Yang 5ea3627dbb runqemu: support multiple qemus running when nfs
Fixed:
* In build1:
  $ runqemu nfs qemux86-64
  In build2:
  $ runqemu nfs qemux86-64

  It would fail before since the port numerbs and conf files are
  conflicted, now make runqemu-export-rootfs work together with runqemu to
  fix the problem.

* And we don't need export PSEUDO_LOCALSTATEDIR in runqemu, the
  runqemu-export-rootfs can handle it well based on NFS_EXPORT_DIR.

* Remove "async" option from unfsd to fix warning in syslog:
  Warning: unknown exports option `async' ignored

* Fixed typos

Both slirp and tap can work.

(From OE-Core rev: 84b2281595bbdb497daa42640e3ee4658bf0bed8)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-23 12:05:22 +00:00
Markus Lehtonen 8e01802c9f oe-build-perf-test: remove unused imports and fix indent
[YOCTO #10590]

(From OE-Core rev: 0b1892fa9165407a156609ff7cb3708e21bacd8c)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-23 12:05:22 +00:00
Markus Lehtonen 0981bcb098 oe-build-perf-test: save test metadata in a separate file
The patch introduces a new metadata (.json or .xml) file in the output
directory. All test meta data, e.g. git revision information and tester
host information is now stored there. The JSON report format is slightly
changed as the metadata is not present in results.json anymore.

[YOCTO #10590]

(From OE-Core rev: 2036c646019660e32f1bc277fdec0cdbff0afdd4)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-23 12:05:22 +00:00
Markus Lehtonen 07c245792e oe-build-perf-test: enable xml reporting
Add --xml command line option to oe-build-perf-test script for producing
a test report in JUnit XML format instead of JSON.

[YOCTO #10590]

(From OE-Core rev: 21ae1c491b93675254b7733640662b566ed76f98)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-23 12:05:21 +00:00
Mariano Lopez 46ce0b5ef3 testexport.bbclass: Migrate testexport to use new framework
This migrates current testexport implmentation to use the
new OEQA framework.

[YOCTO #10686]

(From OE-Core rev: 92cb884c989460563a063b29d2be8b7acd20577e)

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-23 12:05:21 +00:00
Aníbal Limón 90f4325dd5 scripts/oe-test: Add new oe-test script
The new oe-test script will be use to run test components with
one single script.

The oe-test script search for test components inside meta/lib/oeqa,
the test components needs to implement OETestContextExecutor inside
context module in order to be supported by oe-test.

[YOCTO #10230]

(From OE-Core rev: 04b69cff3957731fa1ed2f7d23f2f616978ed0b7)

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-23 12:05:18 +00:00
Ed Bartosh 7998501f47 wic: bootimg-pcbios: use wic-tools target
Used wic-tools target to get location of syslinux.

(From OE-Core rev: 5afd7def3b101bb3d650db4d005cb8d36aaf074a)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-23 12:05:18 +00:00
Ed Bartosh e0193182c5 wic: rebuild wic-tools if its sysroot doesn't exist
Rebuild wic-tools if its native sysroot doesn't exist to ensure
that all required tools are available.

(From OE-Core rev: 0d005d099a2b8ee1303b98710cdc78e06e14ab39)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-23 12:05:18 +00:00
Ed Bartosh 7d5f44b455 wic: misc.py: add parameter 'cache' to get_bitbake_vars
This parameter will be used to allow or prevent variable caching.

For example, we don't want to cache value of RECIPE_SYSROOT_NATIVE
wic-tools variable as we may decide to rebuild wic-tools.

(From OE-Core rev: e4269fdb4c3ef06b97df063f8586f74986215c83)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-23 12:05:18 +00:00
Ed Bartosh 657f1e44b1 wic: fixed test_iso_image test case
Fixed isoimage-isohybrid plulgin and correspondent wic tet case:
- used wic-tools target when getting varlue of STAGING_LIBDIR variable
- ensured that image is built with efi and hddimg enabled

(From OE-Core rev: 5878484da64c38c7fde45bb06d76e22e608eb022)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-23 12:05:18 +00:00
Ed Bartosh 349dbad11d wic: fixed recipe name
Renamed grub-efi-native -> grub-efi in NATIVE_RECIPES
dictionary as '-native' suffix is added to the recipe name in
the code.

(From OE-Core rev: cd0e7b20de2de67793d0454f4e5cf6bb6f4a16f7)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-23 12:05:18 +00:00
Ed Bartosh 4ab4e155f8 partition.py: use FAKEROOTCMD variable to get path to pseudo
wic used native sysroot to get path to pseudo utility.
This approach doesn't work with recipe specific sysroots.

Using FAKEROOTCMD should fix the issue.

(From OE-Core rev: 03e051d594f285ea3c014d45e9b30028e683c602)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-23 12:05:17 +00:00
Ed Bartosh 168f62e427 wic: fix getting path of native sysroot
wic used STAGING_DIR_NATIVE variable as a path to native sysroot.
This doesn't work with recipe specific sysroots as STAGING_DIR_NATIVE
points to the native sysroot of the current recipe.

Used RECIPE_SYSROOT_NATIVE variable of wic-tools recipe
to fix the issue.

(From OE-Core rev: de9d7d14cd03e4dfc5812890a53c79b706b56537)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-23 12:05:17 +00:00
Richard Purdie 41fb482ee1 oe-selftest: Error if the user has buildhistory enabled
buildhistory interfers with various tests so error if the user has it enabled,
hence avoiding time spent running the tests only for them to fail.

For example, if there is an sstate cache and something comes from that cache,
you can get a "version went backwards" warning from buildhistory which would
then change the outcome of a test. Its safer/easier to disable it.

(From OE-Core rev: 941ec7c50c30052e346e72fef2920135dba89a21)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-20 11:53:49 +00:00
Paul Eggleton 9b3e808624 devtool: fix source extraction to work with recipe specific sysroots
When extracting source for a recipe within devtool (for extract, modify
or upgrade) We need to redirect WORKDIR, STAMPS_DIR etc. under a
temporary directory so that:
 (a) we pick up all files that get unpacked to the WORKDIR, and
 (b) we don't disturb the existing build
However, with recipe-specific sysroots the sysroots for the recipe will
be prepared under WORKDIR, and if we used the system temporary directory
i.e. usually /tmp) as used by mkdtemp by default, then our attempts to
hardlink files into the recipe-specific sysroots will fail on systems
where /tmp is a different filesystem, and we'd have to fall back to
copying the files which is a waste of time. Put the temp directory under
the WORKDIR to prevent that from being a problem.

(From OE-Core rev: e10a973cd9390eacb13bdb99693a0622bd3695f5)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
[RP: Add needed mkdirhier call]
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-20 11:53:49 +00:00
Markus Lehtonen 56a7f41711 oeqa.utils.metadata: allow storing any bitbake config variables
Make it possible to store any bitbake config variables in the metadata.
Config values will be stored under a new config element in the xml report:
    <config>
        <variable name="MACHINE">qemux86</variable>
    </config>

The value of MACHINE is moved there instead of having a dedicated
<machine> element.

[YOCTO #10590]

(From OE-Core rev: 6e7e6e37664b0a86111272f5f6f4a4e1d0f23302)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-19 22:47:22 +00:00
Markus Lehtonen ceed2c7eb5 oeqa.utils.metadata: rename 'revision' to 'commit'
Revision is a bit vague and could point to a tag, for example. Git
commit objects are unambiguous and persistent so be explicit that the
element should contain git commit hash.

[YOCTO #10590]

(From OE-Core rev: a6521b89e86ddba6bd646dd3974d7b0390323b56)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-19 22:47:22 +00:00
Markus Lehtonen 4e67232f34 oeqa.utils.metadata: drop 'unknown' git data elements
It's better just to not have the xml elements than to have elements with
faux data. One could have git branch named 'unknown', for example.

[YOCTO #10590]

(From OE-Core rev: fce531c21f5e56d0f416b3405a0b0fc5ba567679)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-19 22:47:22 +00:00
Markus Lehtonen df8c0540e4 oeqa.utils.metadata: re-organise distro information
Use the same format, based on /etc/os-release, as for host distro
information.

[YOCTO #10590]

(From OE-Core rev: 0156ef46ccf5334ee72f0202f1089249c62af37b)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-19 22:47:22 +00:00
Robert Yang dcaeaf2df1 runqemu-export-rootfs: fix inconsistent var names
Fixed:
$ runqemu nfs qemux86-64
[snip]
On your target please remember to add the following options for NFS
nfsroot=IP_ADDRESS:/path/to/nfsroot,nfsvers=3,port=,mountprog=,nfsprog=,udp,mountport=
[snip]

Note that the values are null, this is because their var names are
inconsistent.

[YOCTO #10519]

(From OE-Core rev: 10c136a382006c0ec2679a70f47ff2446c10372c)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-19 22:47:20 +00:00
Robert Yang 65a8d4d185 scripts/runqemu: fix checking for <file>.cpio.gz
When "runqemu /path/to/<file>.cpio.gz", it used the last suffix "gz" as
the fstype which was wrong. Check filename against self.fstypes firstly
can fix the problem.

(From OE-Core rev: 68c7589b67a83977331a04356b53aa51680a1d9d)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-19 22:47:20 +00:00
Maciej Borzecki cb10cfd64f wic: partitionedfs: account for non-table partitions when checking if logical parititon is needed
Commit 8c1c43b790 `wic: Create a logical partition
only when it is really mandatory` did not account for partitions that are not
present in partition table.

(From OE-Core rev: 57b05e924bba7b2fff07a34690474c0fa3046865)

Signed-off-by: Maciej Borzecki <maciej.borzecki@rndity.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-19 22:47:20 +00:00
Tom Zanussi 1cae998af9 scripts: python3 fixes and new tool ksum
'ksum.py' generates a combined summary of vmlinux and module sizes for
a built kernel, as a quick tool for comparing the overall effects of
systemic tinification changes.  Execute from the base directory of the
kernel build you want to summarize.  Setting the 'verbose' flag will
display the sizes for each file included in the summary.

(From OE-Core rev: 016b19c2589582d7ec3c8cac9cfa75a1edc716fe)

Signed-off-by: Tom Zanussi <tom.zanussi@linux.intel.com>
Signed-off-by: Alejandro Hernandez <alejandro.hernandez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-16 18:05:14 +00:00
Chen Qi 6bb64c8fa4 scripts/oe-selftest: fix typo
Change 'agains' to 'against'.

(From OE-Core rev: b72dba9cadb1ae0ee3feb184af1c5bb4a99f70e3)

Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-16 18:05:14 +00:00
Alistair Francis 7f013f2a54 runqemu: Allow the user to specity no kernel or rootFS
In some cirsumstances the user doesn't want to supply a kernel, rootFS
or DTB to QEMU. This will occur more now that QEMU supports loading
images using a '-device loader' method.

Allow users to specify 'none' for QB_DEFAULT_FSTYPE or QB_DEFAULT_KERNEL
to avoid supplying these options to QEMU.

(From OE-Core rev: 2cc01c4e46b05b7ffcc8a11e7ebde6c43256c3c3)

Signed-off-by: Alistair Francis <alistair.francis@xilinx.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-16 18:05:13 +00:00
Ed Bartosh fae29ac083 wic: _exec_cmd: produce error if exit code is not 0
Current code doesn't always show error output of the
external command and even ignores non-zero exit code.

Moved checking of exit code value to the lowest level
possible: to _exec_cmd. This should make wic to always
check exit code of the external command and issue
an error if it's not 0.

[YOCTO #10816]

(From OE-Core rev: 7f68001579c08509332d633b27b5c2ea9386b6c9)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-16 18:05:13 +00:00
Ed Bartosh c1bb8c9b14 direct.py: fix getting image name
part.rootfs_dir was used as an image name in the code.
However, when multi-rootfs feature is used this attribute
points to the name of the rootfs, e.g. if --rootfs command line
is rootfs1=core-image-minimal partf.rootfs_dir is 'rootfs1'.

The code also fails when image name is not provided in wic
commandline. For example, when wic is called with
--rootfs-dir=<path> part.rootfs_dir will contain path and
wic will crash trying to call bitbake -e <path> to get
value of ROOTFS_SIZE variable.

Fixed the code by getting image name properly and checking
if it's not a path.

[YOCTO #10815]

(From OE-Core rev: 2e05d9709f1308fc95d3406b8a409ea982c5b474)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-16 18:05:13 +00:00
Ed Bartosh e652fa4420 canned-wks: remove mpc8315e-rdb.wks
This file has been moved to meta-yocto-bsp/wic/

(From OE-Core rev: efa7639b31c51e2874ba61fd68f9e2cb51145eaf)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-16 18:05:13 +00:00
Ioan-Adrian Ratiu 436df4de37 wic/isoimage-isohybrid: remove do_stage_partition()
The purpouse of this function was to check dependencies for building a
hybrid iso and build them using bitbake if not found. Calling bitbake in
this context means this wic plugin itself cannot be instrumented inside
bitbake recipes which is undesirable, the benefits of this are clear:
there is no need to maintain outside scripts to generate an iso using wic
and the isohybrid building logic can be further abstracted away into an
isohybrid.bbclass in the future which can be easily inherited or something
similar.

So remove the function and add all dependencies to NATIVE_RECIPES so that
wic can print useful errors when they're not built.

To automate building the isohybrid image dependencies, add the following
somewhere in your image build inheritence hierarcy (or maybe create a
bbclass in the future to do these sort of things automatically):

DEPENDS += "syslinux syslinux-native cdrtools-native e2fsprogs-native \
            parted-native dosfstools-native mtools-native grub-efi-native"

(From OE-Core rev: ba4346069ab87f1cf942d1928f911eca6a9d65cd)

Signed-off-by: Ioan-Adrian Ratiu <adrian.ratiu@ni.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-16 18:05:12 +00:00
Derek Straka 413e11ba98 python-3.5-manifest: Add http module to the netclient package
Adding http module from Python's standard library. This allow use
of the http module without installing all python-misc modules.

(From OE-Core rev: 3d0060f9703ba39fbdaafcbdf91e0c319b56f7b3)

Signed-off-by: Derek Straka <derek@asterius.io>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-16 18:05:12 +00:00
Ed Bartosh 5651e8da60 wic: fix parsing of 'bitbake -e' output
Current parsing code can wrongly interpret arbitrary lines
that are of 'key=value' format as legitimate bitbake variables.

Implemented more strict parsing of key=value pairs using
regular expressions.

(From OE-Core rev: f0ec387ad40fb9c098ac8d761993bc2bacc76e65)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-16 18:05:12 +00:00
Alejandro Hernandez a250452f56 gummiboot: Remove old gummiboot recipe, related class and wks file
Since the gummiboot project is no longer being maintained
and we are using systemd-boot as a replacement instead,
we can now clean up all remaining gummiboot files.

[YOCTO #10332]

(From OE-Core rev: 65eb3f51b70baaf24de871301a7247d5baed00ed)

Signed-off-by: Alejandro Hernandez <alejandro.hernandez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-16 18:05:11 +00:00
Paul Eggleton 0f315faff8 oe-selftest: fix behaviour if oe-selftest.log is a dangling symlink
If you delete the log file that the oe-selftest.log symlink points to
but not the symlink itself, because we were using os.path.exists() here
the code assumed that the symlink didn't exist when in fact it still
did. Use os.path.lexists() instead.

(From OE-Core rev: 263af91a0efd21e041ecdb0c40f9b2d4e735f67d)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-09 13:39:11 +00:00
Alejandro Hernandez 2dce2648e3 gummiboot: Remove/change gummiboot references with systemd-boot
After systemd-boot was introduced, its been tested for a while with no major
issues being found until now, this patch completely replaces all gummiboot
instances with systemd-boot ones, taking the next step into cleaning
up systemd-boot/gummiboot.

[YOCTO #10332]

(From OE-Core rev: f9a61d3400ad9068a6d83b8eb6aefe3098c58e68)

Signed-off-by: Alejandro Hernandez <alejandro.hernandez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-09 13:39:11 +00:00
Richard Purdie 022a8b58c8 meta/scripts: Various getVar/getVarFlag expansion parameter fixes
There were a few straggling expansion parameter removals left for
getVar/getVarFlag where the odd whitespace meant they were missed
on previous passes. There were also some plain broken ussages such
as:

d.getVar('ALTERNATIVE_TARGET', old_name, True)
path = d.getVar('PATH', d, True)
d.getVar('IMAGE_ROOTFS', 'True')

which I've corrected (they happend to work by luck).

(From OE-Core rev: 688f7a64917a5ce5cbe12f8e5da4d47e265d240f)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-09 13:39:11 +00:00
Richard Purdie f2b847a290 oe-selftest: Improve BUILDDIR environment handling
Its possible something (like bitbake/tinfoil2) may mess around with the
environment and using the enviroment as a global variable store isn't
particularly nice anyway.

This patch changes the BUILDDIR usages so that the environment isn't used
as a global store and a global variable is used instead. Whilst that
is still not perfect, it does avoid the current double and triple backtraces
we're seeing where tinfoil2/bitbake has trampled the enviroment leading
to failures of failures making debugging even harder.

(From OE-Core rev: 689b676bbf2f1a5fadb04aeb41d5e68e35356545)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-05 13:54:06 +00:00
Patrick Ohly b359e926e2 runqemu: let command line parameters override defaults
It may be necessary to override the parameters gathered for the qemu
invocation. For example, the qemux86 machine configuration sets "-vga
vmware", but when using OVMF as BIOS, only "-vga std" is supported.

By putting the parameters derived from custom runqemu parameters like
"qemuparams" after the parameters derived from the machine
configuration the user gets the possibility to override those.

(From OE-Core rev: b6feb7578d60289c8b6e376cfaac8a3ee45e72f9)

Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-05 13:54:06 +00:00
Alejandro Hernandez 9d4a69fd0d example-recipe: Fix LDFLAGS compilation issue on newly created recipes
(From meta-yocto rev: aa3e99e24c3234b6f3f19169939d4cd498d0ce36)

Signed-off-by: Alejandro Hernandez <alejandro.hernandez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-22 08:53:03 +00:00
Ed Bartosh b05b1add7a wic: add kickstart file for MPC8315
Added kickstart file to produce partitioned image for
MPC8315 reference hardware.

[YOCTO #8719]

(From OE-Core rev: 8c873b4fd92b72adfc49d20bbfab4779857fb6c9)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-22 08:50:17 +00:00
Maciej Borzecki 1988bae5bf wic: add --fixed-size wks option
Added new option --fixed-size to wks. The option can be used to indicate
the exact size of a partition. The option cannot be added together with
--size, in which case an error will be raised. Other options that
influence automatic partition size (--extra-space, --overhead-factor),
if specifiec along with --fixed-size, will raise an error.

If it partition data is larger than the amount of space specified with
--fixed-size option wic will raise an error.

(From OE-Core rev: fdd217ba874bd480e0180830fe2e6bd54dde19d9)

Signed-off-by: Maciej Borzecki <maciej.borzecki@rndity.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-22 08:50:16 +00:00
Paul Eggleton 358befb76e devtool: modify: fix usage on the kernel
When using devtool modify on the kernel, we have to do a bit of a dance
with tinfoil instances because we only find out that we're working on a
kernel recipe after tinfoil is initialised, but then we need to build
kern-tools-native which we're doing just by running bitbake directly.
With the tinfoil2 changes, a datastore for the recipe that we were
keeping around across the opening and closing of tinfoil is no longer
able to be used. Re-parse the recipe to avoid this problem.

(In future this whole thing will be able to be done in the same tinfoil
instance thanks to tinfoil2, but that refactoring is yet to be done.)

(From OE-Core rev: 06127d0115ba449bf04e2579cd1010065e0ed6e3)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-22 08:50:16 +00:00
Ed Bartosh 28c79d8aeb wic: look for wks files in <layer>/wic
Currently wic looks for wks files in
<layer dir>/scripts/lib/wic/canned-wks/ directories.
This path is too nested and doesn't look consistent with the
naming scheme of layer directories.

Added <layer>/wic directory to the list of paths
to look for wks files.

(From OE-Core rev: 803b5fa798cf7580c7c10401eb04a1cccf51b8ab)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-22 08:50:16 +00:00
Ed Bartosh 2f4008d694 oe-selftest: import git module only when needed
git module is not included into standard Python
library and therefore causes import errors on the systems
where PythonGit is not installed.

As git module only used in the code implementing --repository
functionality it's better to import git only in the scope
that requires it.

[YOCTO #10821]

(From OE-Core rev: 66be32c1a075201d6ee0e9b9e10b84e6a2ace745)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-20 15:22:17 +00:00
Christopher Larson db08ffee0a wic: obey the rootfs size from the metadata
When no --size is specified for the rootfs in the .wks, we want to obey the
rootfs size from the metadata, otherwise the defined IMAGE_ROOTFS_EXTRA_SPACE
and IMAGE_OVERHEAD_FACTOR will not be obeyed. In some cases, this can result
in image construction failure, if the size determined by du was insufficient
to hold the files without the aforementioned extra space.

This fallback from --size to ROOTFS_SIZE was already implemented when
--rootfs-dir is specified in the .wks, but it did not occur otherwise, neither
when --rootfs-dir= was passed to `wic create` nor when IMAGE_ROOTFS was used.
This made a certain amount of sense, as this fallback logic happened at such
a level that it wasn't able to identify which partitions were rootfs
partitions otherwise. Rather than doing it at that level, we can do it in
prepare_rootfs(), which is run by the rootfs source plugins.

Note that IMAGE_OVERHEAD_FACTOR and a --overhead-factor in the .wks will now
both be applied when --size isn't specified in the .wks. A warning is added
about this, though a user won't see it unless wic fails or they examine the
do_image_wic log.

Fixes [YOCTO #10815]

(From OE-Core rev: 1d50e11286722c4114c1ae0bc285f846cd85fc4c)

Signed-off-by: Christopher Larson <chris_larson@mentor.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-17 09:57:04 +00:00
Joshua Lock 6bba41832b scripts: remove True option to getVarFlag calls
getVarFlag() now defaults to expanding by default, thus remove the
True option from getVarFlag() calls with a regex search and
replace.

Search made with the following regex:
getVarFlag ?\(( ?[^,()]*, ?[^,()]*), True\)

(From OE-Core rev: 3e4806063fe11092b2307f113a6c0b0f04104091)

Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-16 10:23:23 +00:00
Joshua Lock c0f2890c01 scripts: remove True option to getVar calls
getVar() now defaults to expanding by default, thus remove the True
option from getVar() calls with a regex search and replace.

Search made with the following regex: getVar ?\(( ?[^,()]*), True\)

(From OE-Core rev: 0a36bd96e6b29fd99a296efc358ca3e9fb5af735)

Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-16 10:23:23 +00:00
Mariano Lopez 3f6e6f4e69 scripts/runqemu: Allow to use qemu from host.
This will add support to use qemu from the running host,
with this is possible to put qemu-native in ASSUME_PROVIDED
variable.

By default it will try to get qemu from the build sysroot,
and only if it fails will try to use the host's qemu.

(From OE-Core rev: fe7fd2cd3a9c4fb5b31bd3cab81c96a3b81cb540)

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-16 10:23:23 +00:00
Paul Eggleton 229091babb devtool: prevent BBHandledException from showing traceback
If we don't catch this then attempting to run devtool in non-memres mode
when bitbake is already running will produce a traceback instead of just
an error message.

(From OE-Core rev: e01b75dff599ffa2b66e6608b28bbb3564365eee)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-14 12:30:50 +00:00
Paul Eggleton 4448b2a122 devtool: extract: disable basehash mismatch errors
Using the setVariable commands here followed by buildFile will result in
"basehash mismatch" errors, and that's expected since we are deviating
*at runtime* from what was previously seen by changing these variable
values. Set BB_HASH_IGNORE_MISMATCH to turn off the errors.

(From OE-Core rev: b0169796f294bbec0397b7eae86454a46b68cdc5)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-14 12:30:50 +00:00
Paul Eggleton 85de43fccd devtool: fix extraction of source to work in memres mode
Extracting the source for a recipe (as used by devtool's extract, modify
and upgrade subcommands) requires us to run do_fetch, do_unpack,
do_patch and any tasks that the recipe has inserted inbetween, and do so
with a modified datastore primarily so that we can redirect WORKDIR and
STAMPS_DIR in order to have the files written out to a place of our
choosing and avoid stamping the tasks as having executed in a real build
context respectively. However, this all gets much more difficult when in
memres mode since we can't call internal functions such as
bb.build.exec_func() directly - instead we need to execute the tasks on
the server. To do this we use the buildFile command which already exists
for the purpose of supporting bitbake -b, and setVariable commands to
set up the appropriate datastore.

(I did look at passing the modified datastore to the buildFile command
instead of using setVar() on the main datastore, however its use of
databuilder makes that very difficult, and we'd also need a different
method of getting the changes in the datastore over to the worker as
well.)

(From OE-Core rev: eb63b5339014fc72ba4829714e0a96a98e135ee2)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-14 12:30:50 +00:00
Paul Eggleton 86b55e6026 recipetool: add OE lib path
The autotools code imports oe.package; we weren't experiencing a problem
with this probably due to OE itself adding that path previously.

(From OE-Core rev: a61d7bf8447b2d2c65eb34315c86086ff35c8bc9)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-14 12:30:50 +00:00
Paul Eggleton 45adbe3709 classes/patch: move in logic to commit for additional tasks
If PATCHTOOL is "git", and PATCH_COMMIT_FUNCTIONS is set to "1", for
additional tasks between do_unpack and do_patch, make a git commit. This
logic was previously implemented in devtool itself, but it makes more
sense for it to be implemented in the patch class since that's where the
rest of the logic is for this (or in lib/oe/patch.py). It also makes
it possible for this to work with tinfoil2.

(From OE-Core rev: f24f59ea1d8bc335ea8576f6a346d0935f4a3548)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-14 12:30:49 +00:00
Paul Eggleton 70ce595237 devtool / recipetool: use tinfoil parsing API
Use Tinfoil.parse_recipe_file() and Tinfoil.parse_recipe() instead of
the recipeutils equivalents, and replace any local duplicate
implementations. This not only tidies up the code but also allows these
calls to work in memres mode.

(From OE-Core rev: f13b56266ee96dfab65a3a7db50e8051aa9f071a)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-14 12:30:49 +00:00
Paul Eggleton 1e29b57e92 devtool: package: don't try to initialise tinfoil twice
setup_tinfoil() already calls prepare(), we don't need to call it again
ourselves and doing so with tinfoil2 results in "ERROR: Only one copy of
bitbake should be run against a build directory". Calling prepare()
twice should probably still be allowed, so that ought to be fixed
separately, but in the mean time this code is still wrong so fix it
here.

(From OE-Core rev: 38b8a7d4aff096ea0a62f2ddf3fe2de1df591bf5)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-14 12:30:49 +00:00
Ola x Nilsson d65886bb69 oe-pkgdata-util: Make read-value handle override variables
Some variables in pkgdata files have a package-name override.  When
the bare variable can not be found, try with the override-variant.

PKGSIZE is one such variable, and already had special code to handle this.

Test included.

(From OE-Core rev: 6df99cda894033cba68bc6ab91e47f67e0d788a5)

Signed-off-by: Ola x Nilsson <ola.x.nilsson@axis.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-14 12:30:49 +00:00
Ed Bartosh a5438f8f74 oe-buildenv-internal: show usage output
Show usage text if script is not sourced.
Tested in bash, zsh and dash.

[YOCTO #10751]

(From OE-Core rev: ac7a905b18acb8bd9b2412b6682afbe1d7e18d7b)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-13 22:55:21 +00:00
Ed Bartosh 556bdb6642 oe-find-native-sysroot: create usage output
Created usage output for oe-find-native-sysroot script.

[YOCTO #10751]

(From OE-Core rev: 8ddfc48c7f3e2ca45c035cec492fdc31c6ad484f)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-13 22:55:21 +00:00
Ed Bartosh 0ebe2e64ba oe-git-proxy: create usage output
Created usage output for oe-git-proxy script.

[YOCTO #10751]

(From OE-Core rev: ec0fdb5e896fc20dbafcc8ae507b17c011dc56fd)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-13 22:55:21 +00:00
Ed Bartosh ca916f279f oepydevshell-internal.py: standardize usage output
Made usage output of oepydevshell-internal.py to look
similar to the output of other oe scripts.

[YOCTO #10751]

(From OE-Core rev: e6480af22a7a12c655efed14f8f1aea658f26b1c)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-13 22:55:21 +00:00
Ed Bartosh 8351dc41a4 oe-setup-builddir: create usage output
Created usage output for oe-setup-builddir script.

[YOCTO #10751]

(From OE-Core rev: 77606455df7d45fd014c3603e1cf1b24efd37695)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-13 22:55:20 +00:00
Ed Bartosh 78087eba73 oe-setup-rpmrepo: standardize usage output
Made usage output of oe-setup-rpmrepo to look similar to the
output of other oe scripts.

[YOCTO #10751]

(From OE-Core rev: 5423c9a412c680b781417a64b412838845b5d075)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-13 22:55:20 +00:00
Ed Bartosh 3c3f5c5160 oe-trim-schemas: create usage output
Created usage output for oe-trim-schemas script.

[YOCTO #10751]

(From OE-Core rev: a9fac12157ed434a6711de1af77eda1f4a8e9e8a)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-13 22:55:20 +00:00
Ed Bartosh e8fbd7158b oe-run-native: standardize usage output
Made usage output of oe-run-native to look similar to the
output of other oe scripts.

[YOCTO #10751]

(From OE-Core rev: e1c96125ea674509fbc9b36dc671b7a53bd848ac)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-13 22:55:20 +00:00
Ola x Nilsson 8dfc417780 recipetool: Load plugins in a well defined order
To allow recipetool plugins in one layer to shadow another in a well
defined way, first search BBPATH/lib/recipetool directories and then
scripts/lib/recipetool and load only the first found.

The previous search and load loop would load all found plugins with the
ones found later replacing any found before.

(From OE-Core rev: 7a8726a3662a3909dc6ef6e8d1029d0b1aa938c3)

Signed-off-by: Ola x Nilsson <ola.x.nilsson@axis.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-13 22:55:20 +00:00
Ola x Nilsson 14abe7982c devtool: Load plugins in a well defined order
To allow devtool plugins in one layer to shadow another in a well
defined way, first search BBPATH/lib/devtool directories and then
scripts/lib/devool and load only the first found.

The previous search and load loop would load all found plugins with the
ones found later replacing any found before.

(From OE-Core rev: 1b2b8a0a80de17ea053002fdd124055d2798029a)

Signed-off-by: Ola x Nilsson <ola.x.nilsson@axis.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-13 22:55:20 +00:00
Ed Bartosh 07dcfef1ea wic: rename command line option -p -> -s
Short variant of wic command line option --skip-build-check
is incorretly named -p. It's named -s in wic help and Yocto
documentation.

(From OE-Core rev: 8d4121459334349f10bfb0f4cda00a8461aab245)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-13 22:55:19 +00:00
Alessio Igor Bogani 8c1c43b790 wic: Create a logical partition only when it is really mandatory
Don't worth bother with logical partition on MBR partition type (aka
msdos) if disk image generated by wic should have 4 partitions.

(From OE-Core rev: 36a558fbdc96094626e7de1a3510691e30885368)

Signed-off-by: Alessio Igor Bogani <alessio.bogani@elettra.eu>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-13 22:55:19 +00:00
Mariano Lopez dc721f7590 oe-selftest: Add option to submit test result to a git repository.
This new option allows to commit the result to a git repository,
along with the results it will add a metadata file for information
of the current selftest run, such as: hostname, machine, distro,
distro version, host version, and layers.

This implementation will have a branch per different hostname,
testing branch, and machine.

To use this feature use:

oe-selftest <options> --repository <repository_link>

[YOCTO #9954]

(From OE-Core rev: 758e4b7e619f9aaa79e3910b5539ff8fb2d1064a)

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-13 22:55:19 +00:00
Paul Eggleton 8fe1f25ea0 recipetool: fix encoding-related errors creating python recipes
Yet another instance of us expecting a string back from subprocess when
in Python 3 what you get back is bytes. Just decode the output within
run_command() so we avoid this everywhere.

(From OE-Core rev: 103faae78cdff5280c7b7cdb7ca01e0868d02ec9)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-08 10:31:29 +00:00
Patrick Ohly b8a4eb5265 pybootchartgui: support reading reduced /proc logs
Pre-processing /proc data during the build considerably reduces the
amount of data written to disk: 176KB instead of 4.7MB for a 20
minuted build. Parsing also becomes faster.

buildstats.bbclass only writes the reduced logs now, but support for
the full /proc files is kept around as reference.

(From OE-Core rev: b5e47df9af1ebbb477074587fdeae17eb2f55582)

Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-07 10:38:00 +00:00
Patrick Ohly 820c042b36 pybootchartgui: simplify drawing of memory usage
The internal representation after parsing now matches exactly
what the drawing code needs, thus speeding up drawing a bit.
However, the main motivation is to store exactly that required
information in a more compact file.

(From OE-Core rev: ca06e67a0bb5820b38fda4c8dfee20764c1e59ae)

Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-07 10:37:59 +00:00
Patrick Ohly 6b5037bf2c pybootchartgui: render disk space usage
This adds a new, separate chart showing the amount of disk space used
over time for each volume monitored during the build. The hight of the
graph entries represents the delta between current usage and minimal
usage during the build.

That's more useful than showing just the current usage, because then a
graph showing changes in the order of MBs in a volume that is several
GB large would be just flat.

The legend shows the maximum of those deltas, i.e. maximum amount of
space needed for the build. Minor caveat: sampling of disk space usage
starts a bit later than the initial task, so the displayed value may
be slightly lower than the actual amount of space needed because
sampling does not record the actual initial state.

(From OE-Core rev: 263d189d066b578debf08b2bd07494a69b70f70d)

Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-07 10:37:59 +00:00
Patrick Ohly 0cd48fcef4 pybootchartgui/parsing.py: fix error handling in meminfo parser
When matching fails, m.group(0) is invalid and can't be used in the
error message.

(From OE-Core rev: ddfea21e06a2e6b1a1e766969f8c134a6de7388a)

Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-07 10:37:59 +00:00
Patrick Ohly 1416bb3244 pybootchartgui/draw.py: skip empty CPU and disk usage charts
The only real change is the addition of two if checks that skips the
corresponding drawing code when there is no data.

(From OE-Core rev: 1658fd5e9ca1ba793cae604c2a395d54e3ec9056)

Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-07 10:37:59 +00:00
Patrick Ohly 521887ea61 pybootchartgui: show system utilization
This enables rendering of the original bootchart charts for CPU, disk
and memory usage. It depends on the /proc samples recorded by the
updated buildstats.bbclass. Currently, empty charts CPU and disk usage
charts are drawn if that data is not present; the memory chart already
gets skipped when there's no data, which will also have to be added
for the other two.

(From OE-Core rev: 233d3e50b361feea07803a9c0f2a691e687c6cd5)

Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-07 10:37:59 +00:00
Patrick Ohly a2c2092195 pybootchartgui/draw.py: fix drawing of samples not starting at zero
The code did not handle x scaling correctly when drawing starts at
some time larger than zero, i.e. it worked for normal bootchart data,
but not for the system statistics recorded by buildstats.bbclass.

(From OE-Core rev: 166f8f9aaa1f01fc6d6a5451f8f06b815c51ffae)

Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-07 10:37:59 +00:00
Patrick Ohly 7b8ffa18fd pybootchartgui/draw.py: allow moving process chart up and down
Substracting curr_y when determining the hight of the process chart is
wrong because the height is independent of the position where the
chart is about to be drawn. It happens to work at the moment because
curr_y is always 10 when render_processes_chart() gets called. But it
leads to a negative height when other charts are drawn above it, and
then the grid gets drawn on top of those other charts.

Substracting some constant is relevant because otherwise the box is
slightly larger than the process bars. Not sure exactly where that
comes from (text height?); leg_s seems a suitable constant and happens
to be 10, so everything still gets rendered exactly as before.

(From OE-Core rev: b6bb690728c329ae448f89a1b68298c6dd8a573a)

Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-07 10:37:59 +00:00
Patrick Ohly 6d373fdf7b scripts/send-pull-request: Avoid multiple chain headers
When creating a patch set with cover letter using the
send-pull-request script, both the "In-Reply-To" and "References"
headers are appended twice in patch 2 and subsequent.

That's because git-format-patch already inserted them and then
git-send-email repeats that. Suppressing mail threading in
git-send-email with --no-thread avoids the problem and is the
right solution because it works regardless whether git-send-email is
called once or twicee.

Repeating these headers is a violation of RFC 2822 and can confuse
mail programs. For example, Patchwork does not detect a patch series
problem when there are these extra headers.

[YOCTO #10718]

(From OE-Core rev: 303a1aa3df43eb0b693d8602062fa33c4a08fdd6)

Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-07 10:37:59 +00:00
Leonardo Sandoval 70f99b427b bb-perf: plot histograms base on buildstats data
Scripts that produces script data to be consumed by gnuplot.
There are two possible plots depending if either the
-S parameter is present or not:

    * without -S: Produces a histogram listing top N recipes/tasks versus
      stats. The first stat defined in the -s parameter is the one taken
      into account for ranking
    * -S: Produces a histogram listing tasks versus stats.  In this case,
      the value of each stat is the sum for that particular stat in all recipes found.
      Stats values  are in descending order defined by the first stat defined on -s

EXAMPLES

1. Top recipes' tasks taking into account utime

    $ buildstats-plot.sh -s utime | gnuplot -p

2. Tasks versus utime:stime

    $ buildstats-plot.sh -s utime:stime -S | gnuplot -p

3. Tasks versus IO write_bytes:IO read_bytes

    $ buildstats-plot.sh -s 'IO write_bytes:IO read_bytes' -S | gnuplot -p

(From OE-Core rev: 74408c19fba89de54c093fccf65b3a072d6a197b)

Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-11-30 15:48:05 +00:00
Dominic Sacré 3f2ab3f7cc python3: Build and package precompiled modules
Remove the patch that was applied in the python3 and python3-native
recipes to skip compilation of python modules.

Modify generate-manifest-3.5.py to match '__pycache__' directories in
FILES_*.
This is necessary because Python3 puts .pyc files in '__pycache__'
subdirectories one level below the corresponding .py files, whereas in
Python2 they used to be right next to the sources.

This change significantly reduces the startup overhead of Python3
scripts. For example, on a Cortex-A9, "python3 -c pass" took 0.40s
before, and 0.19s after.

(From OE-Core rev: bb4d689769703177dbb0df0935e15016b879f42b)

Signed-off-by: Dominic Sacré <dominic.sacre@gmx.de>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-11-30 15:48:05 +00:00
Jiajie Hu bfd8c35c3f devtool: fix handling of unicode characters from subprocess stdout
In previous implementation, a UnicodeDecodeError exception will be
raised if multi-byte encoded characters are printed by the subprocess.
As an example, the following command will fail in an en_US.UTF-8
environment because wget quotes its saving destination with '‘'(0xE2
0x80 0x98), while just the first byte is provided for decoding:

    devtool add recipe http://example.com/source.tar.xz

The patch fixes the issue by avoiding such kind of incomplete decoding.

(From OE-Core rev: 1875ea92546d23abcab1b40b562477a0016f712d)

Signed-off-by: Jiajie Hu <jiajie.hu@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-11-23 11:10:15 +00:00
Leonardo Sandoval 4bc5353c92 scripts: Specify the stats to take into account
There are many more stats on buildstats that 'Elapsed time', so make the script
more flexible to support all stats. Some cmd line examples:

$ buildstats.sh -s 'utime'

Buildstats' data covers proc's stats in different areas, including CPU times,
IO, program system resources and child program system resources. In order
to print values on each of these sets from command line, one can use the
following:

$ buildstats.sh -H -s 'TIME' | less

$ buildstats.sh -H -s 'IO' | less

and 'RUSAGE' and 'CHILD_RUSAGE' for program and program's child system
resources.

(From OE-Core rev: 81479b191287ccbf4cf94fa2d0ad46813091bca1)

Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-11-23 11:10:14 +00:00
Paul Eggleton 6e182aaa76 devtool: modify: support recipes with only local files as source
The hello-mod recipe is unusual in that it has only local files in
SRC_URI and builds these out of ${WORKDIR}. When you use devtool modify
on it, devtool puts all of those files in an "oe-local-files"
subdirectory of the source tree, which is not ${S} (or ${B}) any more
and thus building the recipe afterwards fails. It's a bit of a hack, but
symlink the files in oe-local-files into the source tree (and commit the
symlinks with an ignored commit so that the repo is clean) to work
around the problem. We only do this at time of extraction, so any files
added to or removed from oe-local-files after that won't be handled, but
I think there's a limit to how far we should go to support these kinds
of recipes - ultimately they are anomalies.

I initially tried a hacky workaround where I set effectively set B =
"${WORKDIR}" and that allowed it to build, but other things such as the
LIC_FILES_CHKSUM checks still broke because they expected to find files
in ${S}. Another hack where I set the sourcetree to point to the
oe-local-files subdirectory works for hello-mod but not for makedevs
since whilst that is similar, unlike hello-mod it does in fact have
files in the source tree (since it has a patch that adds COPYING) and
thus the same issue occurred.

Also tweak one of the tests that tries devtool modify / update-recipe on
the makedevs recipe to try building it since that would have caught this
issue.

Fixes [YOCTO #10616].

(From OE-Core rev: 857c06d6a1d161bf5a01311d07758bd4241929a3)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-11-23 11:10:14 +00:00
Stephano Cetola 8b686f281c recipetool: add postinst to .deb import
The .deb import feature did not import postinst, postrm, preinst, or
prerm functions. This change checks to see if those files exist, and
if so, adds the appropriate functions.

[ YOCTO #10421 ]

(From OE-Core rev: ebb73aa6ad920bfd6a23f8c20105d6bcf07dd3d5)

Signed-off-by: Stephano Cetola <stephano.cetola@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-11-23 11:10:14 +00:00
Maciej Borzecki 40c07e1041 oe-selftest: enforce en_US.UTF-8 locale
Replicate bitbake and eforce en_US.UTF-8 locale so that ouptut of locale-aware
tools remains stable.

(From OE-Core rev: 17cd2cb99d3610fd77595ff116b2168188c250cd)

Signed-off-by: Maciej Borzecki <maciej.borzecki@rndity.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-11-23 11:10:14 +00:00
Paul Eggleton 13c3a4bfa2 devtool: update-recipe: support replacing remote patches
If you have a patch remotely fetched in a recipe (e.g. from an http
server) that needs updating then add a local version and substitute the
entry in SRC_URI to point to it.

One can argue about how desirable it is to be modifying patches fetched
in this way, but then one can argue about how desirable it is to have
such patches in the recipe in the first place - and in any case if
devtool update-recipe is to correctly transfer changes to such patches
made in the git repository within the source tree to the recipe then
there isn't much choice but to do it this way.

(From OE-Core rev: a19c26cc78a181f9dd2706dd42e7e450d7ad4082)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-11-23 11:10:13 +00:00
Paul Eggleton 55a157f4e6 devtool: update-recipe: fix handling of compressed local patches
It is possible to use gzip or bzip2 to compress patches and still refer
to them in compressed form in the SRC_URI value within a recipe. If you
run "devtool modify" on such a recipe, make changes to the commit for
the patch and then run devtool update-recipe, we need to correctly
associate the commit back to the compressed patch file and re-compress
the patch, neither of which we were doing previously.

Additionally, add an oe-selftest test to ensure this doesn't regress in
future.

Fixes [YOCTO #8278].

(From OE-Core rev: e47d21624dfec6f71742b837e91da553f18a28c5)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-11-23 11:10:13 +00:00
Paul Eggleton 0ffd92707b devtool: update-recipe: check output before treating it as a string
As of the move to Python 3 and the fixes we applied at that time,
bb.process.run() will return a byte array of length 0 rather than an
empty string if the output is empty. That may be a bug that we should
fix, but for now it's easiest to just check the result here before
treating it as a string. This fixes running "devtool update-recipe" or
"devtool finish" on a recipe which has no source tree, for example
initramfs-framework.

Fixes [YOCTO #10563].

(From OE-Core rev: 66bf6978fc807ecc422fb6b6328f68bc3406cf15)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-11-23 11:10:12 +00:00
Jussi Kukkonen 0cdbd2ad94 conf: Remove legacy X input drivers
xf86-input-libinput is RRECOMMENDed by xserver-xorg, these
legacy drivers are not needed.

(From meta-yocto rev: 715f72d7842b4a789a78e7889b2f01cd41f02df8)

Signed-off-by: Jussi Kukkonen <jussi.kukkonen@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-11-15 15:19:55 +00:00
Maciej Borzecki 59c8ae1742 oe-selftest: fix handling of test cases without ID in --list-tests-by
Running `oe-selftest --list-tests-by module wic` will produce the
following backtrace:

Traceback (most recent call last):
  File "<snip>/poky/scripts/oe-selftest", line 668, in <module>
    ret = main()
  File "<snip>/poky/scripts/oe-selftest", line 486, in main
    list_testsuite_by(criteria, keyword)
  File "<snip>/poky/scripts/oe-selftest", line 340, in list_testsuite_by
    ts = sorted([ (tc.tcid, tc.tctag, tc.tcname, tc.tcclass, tc.tcmodule) for tc in get_testsuite_by(criteria, keyword) ])
TypeError: unorderable types: int() < NoneType()

The root cause is that a test case does not necessarily have an ID
assigned, hence its value is None. Since Python 3 does not allow
comparison of heterogeneous types, TypeError is raised.

(From OE-Core rev: 71c6790689e2cbd3c4e882335b3b03e635ad46ed)

Signed-off-by: Maciej Borzecki <maciej.borzecki@rndity.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-11-15 15:19:55 +00:00
Maciej Borzecki 2eeed4838b wic: fix function comment typos
Fix typos in documentation of Image.add_partition() and
Image.__format_disks().

(From OE-Core rev: f5bf7bf253224912c66bab89f48ff63a73e0d698)

Signed-off-by: Maciej Borzecki <maciej.borzecki@rndity.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-11-15 15:19:55 +00:00
Maciej Borzecki 294410cce3 wic: check that filesystem is specified for a rootfs partition
We explicitly check for --fstype if no source was provided for a
partition. However, this was not the case for rootfs partitions. Make
sure to raise an error if filesystem was left unspecified when preparing
a rootfs partition image.

(From OE-Core rev: b8c35fcad57810a87aa25ebeb533adf286eed565)

Signed-off-by: Maciej Borzecki <maciej.borzecki@rndity.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-11-15 15:19:55 +00:00
Maciej Borzecki b92de9a6db wic: use partition size when creating empty partition files
It seems that prepare_empty_partition_ext() and
prepare_empty_partition_btrfs() got broken in commit
c8669749e3, thus one could observe the
following backtrace:

Backtrace:
  File "<snip>/poky/scripts/lib/wic/plugins/imager/direct_plugin.py", line 93, in do_create
    creator.create()
  File "<snip>/poky/scripts/lib/wic/imager/baseimager.py", line 159, in create
    self._create()
  File "<snip>/poky/scripts/lib/wic/imager/direct.py", line 290, in _create
    self.bootimg_dir, self.kernel_dir, self.native_sysroot)
  File "<snip>/poky/scripts/lib/wic/partition.py", line 146, in prepare
    method(rootfs, oe_builddir, native_sysroot)
  File "<snip>/poky/scripts/lib/wic/partition.py", line 325, in prepare_empty_partition_ext
    os.ftruncate(sparse.fileno(), rootfs_size * 1024)
NameError: name 'rootfs_size' is not defined

(From OE-Core rev: 0bf686739a5e8a1193f5be5aa60afbf2ea1e5074)

Signed-off-by: Maciej Borzecki <maciej.borzecki@rndity.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-11-15 15:19:55 +00:00
Maciej Borzecki f0181e79ed wic: make sure that partition size is always an integer in internal processing
The size field of Partition class is expected to be an integer and ought
to be set inside prepare_*() method. Make sure that this is always the
case.

(From OE-Core rev: a37838f995ae642b0b8bdd47a605a264fb3bf200)

Signed-off-by: Maciej Borzecki <maciej.borzecki@rndity.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-11-15 15:19:55 +00:00
Ed Bartosh 2ef30ef624 systemd-bootdisk.wks: use PARTUUID
Root device name in systemd-bootdisk.wks is 'sda'. This can cause
images, produced using this wks to refuse booting if real device
name is not 'sda'. For example, when booting MinnowBoard MAX from
MicroSD card the boot process stucks with this message on the boot
console output: Waiting for root device /dev/sda2...
This happens because real device name of MicroSD card on this device
is mmcblk1.

Used --use-uuid option for root partition. This should make
wic to put partiion UUID instead of device name into kernel command
line.

[YOCTO #10485]

(From OE-Core rev: 5b73d5f484cc844affe91ec19d881d42e187f30c)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-11-15 15:19:54 +00:00
Alistair Francis 1cb4fdef89 runqemu: Split out the base name of QB_DEFAULT_KERNEL
The function write_qemuboot_conf() in qemuboot.bbclass always inserts
the full path into QB_DEFAULT_KERNEL. Remove this path before using the
variable.

(From OE-Core rev: 7c0fdfa1316011b856a795d8e42c36ac8b5638b2)

Signed-off-by: Alistair Francis <alistair.francis@xilinx.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-11-15 15:19:53 +00:00
Paul Eggleton 43e652f3d1 devtool: add "rename" subcommand
When you run devtool add on a source tree we attempt to figure out the
correct name and version for the recipe. However, despite our best
efforts, sometimes the name and/or version we come up with isn't
correct, and the only way to remedy that up until now was to reset the
recipe, delete the source tree and start again, specifying the name this
time. To avoid this slightly painful procedure, add a "rename"
subcommand that lets you rename the recipe and/or change the version.

(From OE-Core rev: 9303d8055c45a0f6af295d70a6f6a8b9d8d8a7c9)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-11-07 11:04:22 +00:00
Paul Eggleton e0b62c38ac recipetool: create: separate LICENSE items with & by default
recipetool sets the LICENSE value based on licenses detected from the
source tree. If there are multiple licenses then they were being
separated by spaces, but this isn't actually legal formatting and if
you're using "devtool add" you get a warning printed when devtool
parses the recipe internally.

Earlier I had made a conscious decision to do it this way since it's up
to the user to figure out whether the multiple licenses should all apply
(in which case they'd be separated with &) or if there is a choice of
license (in which case | is the correct separator). However, I've come
to the conclusion that we can just default to & and then the ugly
warning goes away, and it's the safest alternative of the two (and most
likely to be correct, since it's more common to have a codebase which is
made up of code with different licenses, i.e. all of them apply to the
combined work).

I've tweaked the comment that we add to the recipe to explicitly state
that we've used & and that the user needs to change that if that's not
accurate.

Fixes [YOCTO #10413].

(From OE-Core rev: ecac6aee8cf3313350b58c21012bcd67cfb915e4)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-11-07 11:04:22 +00:00
Paul Eggleton e0944011e6 devtool: finish: warn if moving recipe to unconfigured destination layer
If you run devtool finish to move a recipe created in the workspace by
devtool add or devtool upgrade to a layer, and that layer is not
currently included in bblayers.conf (perhaps unintentionally), then the
recipe will no longer be visible to bitbake. In this scenario, show a
warning so that the user isn't surprised by the recipe "going missing".

(From OE-Core rev: 4da8a58e2997db4f24ae0cac0ba27259d7857a05)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-11-07 11:04:22 +00:00
Paul Eggleton 09754b4119 devtool: finish: fix error if destination layer is not in bblayers.conf
If devtool finish is run on a recipe where the recipe file itself is in
the workspace (e.g. where devtool add / devtool upgrade has been used)
and the specified destination layer is not in bblayers.conf, then we
need to avoid running bitbake -c clean at the end because the recipe has
been moved, but the bbappend is still present in the workspace layer at
that point and so if we do it will fail due to the dangling bbappend.
It's difficult to do the clean at the point we'd want to because tinfoil
is holding bitbake.lock for most of the time, but in any case cleaning
the recipe is less important than it used to be since we started
managing the sysroot contents more strictly, so just disable cleaning
under these circumstances to avoid the problem.

Fixes [YOCTO #10484].

(From OE-Core rev: c6980307d43632f4172e79d9607004203af4e9c8)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-11-07 11:04:22 +00:00
Paul Eggleton 452a1133dd devtool: add: show recipetool create output
When running devtool add, instead of hiding the recipetool create
output, change it so that it's appropriate to show in the devtool
context and show it in real-time. This means that you get status output
such as when a URL is being fetched (though currently no progress
information.) recipetool create now has a hidden --devtool option to
enable this display mode.

(From OE-Core rev: 219aec8803de4ef04c514c87ecfb15359c9424a6)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-11-07 11:04:22 +00:00
Ed Bartosh c8669749e3 wic: call os.ftruncate instead of running truncate
Replaced running of truncate utility with the standard library
call os.ftruncate

(From OE-Core rev: 1ba6101ceaee354816e690d44bc9a5dd8dcf4011)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-11-06 23:35:36 +00:00
Ulf Magnusson 4d2658eeee task-time: Add simple buildstats analysis script
The 'task-time' Python script is used for simple manual analysis of
buildstats. It displays task timing information in the same format (and
using the same calculation) as the Bash 'time' builtin, and can
optionally sort tasks by real (wall-clock), user (user space CPU), or
sys (kernel CPU) time used.

The timing information comes from the getrusage(2) fields added by
commit adfdca4df1 ("buildstats: Improve to add getrusage data and
corrected IO stats"). That commit is required for the script to work.

Example 1: Running 'task-time' on a specific task buildstat:

  $ task-time ./20161005235448/gettext-0.16.1-r6/do_compile
  ./20161005235448/gettext-0.16.1-r6/do_compile:
  real  0m54.560s
  user  0m46.028s
  sys   0m2.772s

Example 2: Running 'task-time' on a directory, sorting on wall-clock
time:

  $ task-time tmp/buildstats/20161018083535 --sort real
  tmp/buildstats/20161018083535/bash-4.3.30-r0/do_fetch:
  real    10m59.140s
  user    0m1.152s
  sys     0m0.320s

  tmp/buildstats/20161018083535/readline-native-6.3-r0/do_fetch:
  real    8m57.310s
  user    0m0.860s
  sys     0m0.288s

  tmp/buildstats/20161018083535/perl-5.22.1-r0/do_compile:
  real    4m28.840s
  user    4m1.348s
  sys     0m15.816s

  ...

Example 3: Running 'task-time' on all do_compile buildstats for a
particular build by using shell globbing, sorting on user space CPU
time:

  $ task-time tmp/buildstats/20161018083535/*/do_compile --sort user
  tmp/buildstats/20161018083535/qemu-native-2.7.0-r1/do_compile:
  real    0m49.570s
  user    21m45.236s
  sys     1m44.380s

  tmp/buildstats/20161018083535/linux-yocto-4.8+gitAUTOINC+03bf3dd731_67813e7efa-r0/do_compile:
  real    0m49.530s
  user    21m39.588s
  sys     1m59.576s

  tmp/buildstats/20161018083535/gcc-cross-i586-6.2.0-r0/do_compile:
  real    1m8.130s
  user    15m54.256s
  sys     1m28.776s

  ...

Example 4: Comparing a task between two builds:

  $ task-time 201610052{25856,35448}/gettext-0*/do_compile --sort real
  20161005235448/gettext-0.16.1-r6/do_compile:
  real	0m54.560s
  user	0m46.028s
  sys	0m2.772s

  20161005225856/gettext-0.19.8.1-r0/do_compile:
  real	0m41.520s
  user	2m17.312s
  sys	0m7.536s

(From OE-Core rev: 76dfad5b598e2937554bddeecf47482b14a854cd)

Signed-off-by: Ulf Magnusson <ulfalizer@gmail.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-11-06 23:35:36 +00:00
Paul Eggleton 46aae1066e oe-setup-builddir: fix TEMPLATECONF error message
This directory shouldn't contain local.conf and bblayers.conf - just
templates for them; except it doesn't have to contain those, it just has
to exist to pass this test. Change the error message accordingly, and
mention TEMPLATECONF so that the user has at least some context.

(From OE-Core rev: 61adaaa4348c670769f8750223977dbefe369ffb)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-11-06 23:35:35 +00:00
Todor Minchev 33ceab7979 runqemu: add user mode (SLIRP) support to x86 QEMU targets
Using 'slirp' as a command line option to runqemu will start QEMU
with user mode networking instead of creating tun/tap devices.
SLIRP does not require root access. By default port 2222 on the
host will be mapped to port 22 in the guest. The default port
mapping can be overwritten with the QB_SLIRP_OPT variable e.g.

QB_SLIRP_OPT = "-net nic,model=e1000 -net user,hostfwd=tcp::2222-:22"

(From OE-Core rev: 80e6fc678f3dcd774d9376cdf2a6afcba2cd0b09)

Signed-off-by: Todor Minchev <todor.minchev@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-11-06 23:35:35 +00:00
Robert P. J. Day 3d00fa90b7 oe-pkgdata-util: Use standard verb form in help info.
"Shows" -> "Show", to be consistent with standard form of help output.

(From OE-Core rev: 5a7994df6cdb5af8d240e2802e6bb3d9671f17e3)

Signed-off-by: Robert P. J. Day <rpjday@crashcourse.ca>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-11-06 23:35:35 +00:00
Ed Bartosh 2d4778b1c8 mkefidisk.sh: add deprecation warning to the output
mkefidisk.sh will soon be deprecated in favor of .wic images.

Added deprecation warning to the script to inform users that
this script will soon be removed from the codebase.

(From OE-Core rev: ccef84fac7b20d483df87aac8c620459fe31b6af)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-11-06 23:35:32 +00:00
Olaf Mandel 9af59d07a3 combo-layer: handle file_exclude matching dirs
If file_exclude matches a directory, os.unlink() got called with this
directory as an argument.

Filter out paths that end in a directory separator.

This still leaves the (then empty) directories, but this does not affect
the git commit.

(From OE-Core rev: e84b9185cc8f8e9f9df0e050543bb3a2c59426c3)

Signed-off-by: Olaf Mandel <o.mandel@menlosystems.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-10-28 16:15:21 +01:00
Martin Jansa 98e9845e31 sstate-sysroot-cruft: Add /etc/ld.so.conf to whitelist
* it reports at least 2 issues in every build (this file in
  native and target sysroot) add it to whitelist

(From OE-Core rev: 798107887294072cacb23b668f446e151fe3c35f)

Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-10-28 16:15:19 +01:00
Martin Jansa d7affc5dfa test-dependencies.sh: Strip also '\.bb: .*' before adding failed recipe to list of failed
* format of bitbake tasks changed in:
  2c88afb   taskdata/runqueue: Rewrite without use of ID indirection

-ERROR: Task 4 (/OE/build/oe-core/openembedded-core/meta/recipes-devtools/rpm/sftp.bb, do_fetch) failed with exit code '1'
+ERROR: Task /OE/build/oe-core/openembedded-core/meta/recipes-devtools/rpm/sftp.bb:do_fetch (/OE/build/oe-core/openembedded-core/meta/recipes-devtools/rpm/sftp.bb:do_fetch) failed with exit code '1'

  so strip not only '\.bb, .*' used before, but also '\.bb:.*' to drop
  the task name to get recipe name.

* for more details see:
  http://lists.openembedded.org/pipermail/openembedded-core/2016-June/123132.html

* without this change you can see test-dependencies.sh trying to rebuild packages
  like:
  Building recipe: fbprogress (6/21)
  Building recipe: fbprogress.bb:do (7/21)
  where the later of course doesn't exist as a recipe

(From OE-Core rev: 26ed215c30c183a3814889c145670220fd3b8bfa)

Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-10-28 16:15:19 +01:00
Olaf Mandel e02f0f1cb1 combo-layer: handle ambiguous git arguments
If a branch/src-repository has the same name as a file/directory, git
since 1.4.0(?) gives an error like the one below:
ambiguous argument 'bitbake': both revision and filename
Use '--' to separate paths from revisions, like this:
'git <command> [<revision>...] -- [<file>...]'

Add two dashes to make the intent clear.

(From OE-Core rev: a84ec3ac15a59f72fcb46d97942009c8e459b5d0)

Signed-off-by: Olaf Mandel <o.mandel@menlosystems.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-10-28 16:15:18 +01:00
Olaf Mandel 050b8969ab combo-layer: fix crashes on wrong tempfile usage
When calling tempfile.NamedTemporaryFile().write(str()), at least on
Python 3.4.2 this fails with this error:
TypeError: 'str' does not support the buffer interface

Change the file-mode for all such files from binary to text mode.

(From OE-Core rev: d08f3882a35eec8a042d2501715684444e353605)

Signed-off-by: Olaf Mandel <o.mandel@menlosystems.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-10-28 16:15:18 +01:00
Tobias Hagelborn 61e7ad5141 buildhistory-collect-srcrevs: Fix multiple SRCREV definitions
Fixed copy & paste error causing error when extracting SRCREV
for packages containing multiple SRCREV definitons.

(From OE-Core rev: abcb6efb6af2b2f31a12c373f8fcabce71d305c0)

Signed-off-by: Tobias Hagelborn <tobias.hagelborn@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-10-28 16:15:17 +01:00
Paul Eggleton a8169807e8 devtool: runqemu: work around runqemu script path assumption
The new runqemu script assumes that if OECORE_NATIVE_SYSROOT is set then
it shouldn't try to run bitbake to find out the values of various
variables such as DEPLOY_DIR_IMAGE; this assumption is incorrect for the
extensible SDK. To work around this, clear OECORE_NATIVE_SYSROOT in the
environment when running runqemu.

Fixes [YOCTO #10447].

(From OE-Core rev: abff69a48bf3076ce8e21356accdc8d85d2c8dbf)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-10-25 22:40:09 +01:00
Ed Bartosh a8efeb7df3 canned-wks: use GPT partition table
According to UEFI specification all EFI platforms must support
GUID Partition Table(GPT) disk layout. Here is a list of advantages
of using GPT disk layout over the legacy MBR partitioning:

 - Logical Block Addresses (LBAs) are 64 bits (rather than 32 bits).
 - Supports many partitions (rather than just four primary partitions).
 - Provides both a primary and backup partition table for redundancy.
 - Uses version number and size fields for future expansion.
 - Uses CRC32 fields for improved data integrity.
 - Defines a GUID for uniquely identifying each partition.
 - Uses a GUID and attributes to define partition content type.
 - Each partition contains a 36 character human readable name.

Used GPT partitioning in all EFI kickstart files.
Tested result images on NUC, MinnowBoard MAX and MinnowBoard Turbot.

(From OE-Core rev: 66d4efeb0face5fc8a1755dcd2ed3a611997a04d)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-10-11 08:27:27 +01:00
Ed Bartosh a7648af6e9 systemd-bootdisk.wks: update kernel command line
Used ttyS0 console.
Removed usage of ttyPCH0 (FRI2 leftover)
Decreased bootloader timeout to 5 seconds
Removed 'vmalloc=256MB snd-hda-intel.enable_msi=0' as it's not
needed for any of reference BSPs.

(From OE-Core rev: ee16a283d695112236906338887c6eba109c9fb5)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-10-11 08:27:27 +01:00
Ed Bartosh b5e8924068 mkgummidisk.wks: update kernel command line
Used ttyS0 console.
Removed usage of ttyPCH0 (FRI2 leftover)
Decreased bootloader timeout to 5 seconds
Removed 'vmalloc=256MB snd-hda-intel.enable_msi=0' as it's not
needed for any of reference BSPs.

(From OE-Core rev: 7b2443364a42c731bc9eefa6bb460262464e26b6)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-10-09 12:31:40 +01:00
Leonardo Sandoval 65107a9abe machine.cfg: Unset CONFIG_64BIT for qemu i386 architecture
In modern kernels, ARCH=x86 defaults to 64 bits, so explicitly
unset this word length for qemu i386 architectures.

(From meta-yocto rev: 9750779bc78de3ea33d7f88e4972a5b21c3f811b)

Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-10-08 07:48:03 +01:00
Ed Bartosh 926a1e6551 mkefidisk.wks: update kernel command line
Used ttyS0 console.
Removed usage of ttyPCH0 (FRI2 leftover)
Decreased bootloader timeout to 5 seconds
Removed 'vmalloc=256MB snd-hda-intel.enable_msi=0' as it's not needed
for any of reference BSPs.

(From OE-Core rev: b46825a6ce5cecd24fed814bae48f9cfd0b97568)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-10-07 16:43:58 +01:00
Ed Bartosh dc67037b78 mkefidisk.wks: use MSDOS partition table
Stopped using GPT partition table in mkefidisk.wks as it's not
supported by all reference hardware.

(From OE-Core rev: 07bb11b097a67ff89ae633fa1992db5494d75c0c)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-10-07 16:43:58 +01:00
Ed Bartosh 12ea11e4f8 wic: rewrite MBR disk identifier
Disk identifier created by parted doesn't match the one we generated
and used in bootloader config. We need to rewrite it to make our image
bootable.

Modified involved API and data structures to access previously
generated disk identifiers after MBR is initialized. Written disk
identifiers to MBR.

(From OE-Core rev: 221d3bdd6e0ab8a4d25e2c96bd976cbec4e76681)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-10-07 16:43:58 +01:00
Ed Bartosh 5763d8f9cd wic: generate PARTUUID for MDOS partitions
Added generation of partition UUIDs for MSDOS partitions.
UUID for MSDOS partitions is <disk identifier>-<partition number>,
where disk identifier is a random 4 bytes long number. It's usually
generated when MBR/partition table is initialized.

As UUID is used to point to the root partition in bootloader config
we need to generate it before the MBR is initialized.
After MBR is created we need to rewrite system identifier to match
it with what is used in bootloader config. This will be implemented
in the next commit.

(From OE-Core rev: 6ecc6addf4080eda75a15af077816c81c6bf70a5)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-10-07 16:43:58 +01:00
Ed Bartosh e8ce5083c5 wic: set PARTUUID only for gpt partition table
sgdisk fails to set PARTUUID for msdos partitions as it's only
supported for GPT partitions.

Checked partition table format to run sgdisk --partition-guid
only for GPT partitions.

(From OE-Core rev: 0c1804821917d6d3a062ca19b63667d030138f21)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-10-07 16:43:58 +01:00
Ulf Magnusson f12825d4bf scripts: Rename 'native' to 'oe-run-native'
Makes it a bit more descriptive and potentially more discoverable. Most
people seemed to prefer an oe- prefix, so let's go with that.

(From OE-Core rev: 97e526ca10a00010987ffa3b90ec48337503a573)

Signed-off-by: Ulf Magnusson <ulfalizer@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-10-06 07:53:51 +01:00
Stephano Cetola 3c51b742c9 devtool: modify command fails to ignore source files
With recent changes to recipeutils, the list of local files returned
by get_recipe_local_files could possibly include source files. This
only happens when the recipe contains a SRC_URI using subdir= to put
files in the source tree. These files should be ignored when
populating the list of local files for oe-local-files directory.

[YOCTO #10326]

introduced in
OE-Core revision 9069fef5dad5a873c8a8f720f7bcbc7625556309

(From OE-Core rev: 31f1bbad248c36a8c86dde4ff57ce42efc664082)

Signed-off-by: Stephano Cetola <stephano.cetola@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-10-06 07:51:01 +01:00
Leonardo Sandoval ff3c3b8996 yocto-bsp: linux-yocto recipes update (4.4 to 4.8) for x86-64 arch
[YOCTO #10301]

(From meta-yocto rev: 088662bd13fa7366cb471be4be746aa195defa3f)

Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-10-05 10:10:12 +01:00
Leonardo Sandoval 8cdef44537 yocto-bsp: linux-yocto recipes update (4.4 to 4.8) for PowerPC arch
[YOCTO #10301]

(From meta-yocto rev: f9d795172ae18489eaf40af7d32f8402299b805c)

Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-10-05 10:10:12 +01:00
Leonardo Sandoval 338c775499 yocto-bsp: linux-yocto recipes update (4.4 to 4.8) for qemu arch
[YOCTO #10301]

(From meta-yocto rev: b0e3518ffae3e72d7aad20860b3da79b70d74383)

Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-10-05 10:10:11 +01:00
Leonardo Sandoval fc34ac070a yocto-bsp: linux-yocto recipes update (4.4 to 4.8) for i386 arch
[YOCTO #10301]

(From meta-yocto rev: 0b58b90844c2898c604e7310c3fa5577044c0f08)

Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-10-05 10:10:11 +01:00
Leonardo Sandoval f43da797b9 yocto-bsp: linux-yocto recipes update (4.4 to 4.8) for MIPS64 arch
[YOCTO #10301]

(From meta-yocto rev: 586ff58a6a803539fd1cf6709dbcedf3ec88924e)

Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-10-05 10:10:11 +01:00
Leonardo Sandoval eb629657aa machine.conf: Remove duplicate xserver choices
All kernel choices today (linux-yocto_4.* and custom) have the same xserver options,
so remove the duplicate lines.

(From meta-yocto rev: c456b5cf172e5ee1fca078383cad189325ea05f5)

Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-10-05 10:10:11 +01:00
Leonardo Sandoval fe18ded6c3 yocto-bsp: linux-yocto recipes update (4.4 to 4.8) for MIPS arch
[YOCTO #10301]

(From meta-yocto rev: 01e16ff4d1df17daed184279868e151cbf35a1a8)

Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-10-05 10:10:11 +01:00
Leonardo Sandoval 6020616e7e yocto-bsp: linux-yocto recipes update (4.4 to 4.8) for arm arch
[YOCTO #10301]

(From meta-yocto rev: ad6e937db32721cfec8d4d85818d028878c5bc23)

Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-10-05 10:10:11 +01:00
Peter Kjellerstedt 79236477a1 devtool: deploy-target: Avoid unnecessary dependency on awk on the target
Relying on that awk is installed on the target just to extract the
fourth column (i.e., the free volume size) from `df -P` is an
unnecessary dependency for devtool deploy-target. As it is already
using sed to mangle the output from `df -P`, this can easily be
modified to only extract the free volume size.

(From OE-Core rev: 7bab454b0bf0075fbb2a5de06286a9da1df2adc6)

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-10-05 10:10:11 +01:00
Paul Eggleton 18caacae06 devtool: add: build nodejs-native if npm is needed and not available
If the user runs devtool add on an npm:// URL (or source tree that uses
node.js), and npm is not available, just build nodejs-native instead of
telling the user they need to do it; if that fails because there isn't
any such recipe (which would be the default, since it's not in OE-Core)
then produce a slightly more readable error message hinting at what the
user needs to do.

Note that this forces the use of nodejs-native rather than npm on the
host - this makes sense for two reasons: (1) we need it to be compatible
with nodejs for the target, and (2) we have to have a recipe for that
anyway, so allowing you to avoid having a recipe for the native version
isn't really beneficial.

There's a bit of a hack in here in order to allow this - for node.js
sources that aren't fetched via npm we don't know that they are that
until we've fetched and unpacked them, by which time we're inside
recipetool and have an active tinfoil instance that will prevent bitbake
being run. To avoid this being an issue, we allow recipetool to get to
the point where we know we need npm and then exit with a specific exit
code, at which point devtool can try to build it and then if that
succeeds, it will re-execute recipetool. This is definitely not ideal,
but it can't really be refactored and done properly until we do the
tinfoil2 refactoring; in the mean time though we still want to be
helpful to the user.

Fixes [YOCTO #10337].

(From OE-Core rev: f40662bde5aab158c4e4c3c3ff5e68665a4194a5)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-10-05 10:10:11 +01:00
Paul Eggleton ee697d84ba devtool: add: display a warning for deprecated -f/--fetch option
We want to remove the -f/--fetch option at some point (as you can now
specify a URL as a positional argument instead) so display a warning
that it's deprecated if it is used.

(From OE-Core rev: 43476d77a91d50454ca26e016a3413b24e9f3aec)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-10-05 10:10:10 +01:00
Paul Eggleton 0c5fd7ad16 devtool: add: fix error message when only specifying a recipe name
We were supposed to be printing out the specified recipe name here but I
forgot to specify a parameter for the string.

(From OE-Core rev: 87f844e533adfc229a5d26857a82cc6b125216c8)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-10-05 10:10:10 +01:00
Ed Bartosh 3159738032 mkefidisk.wks: use partition UUID and GPT partition table
This is a preparation to use mkefidisk as a default wks for
genericx86* BSPs. This change enables usage of partition UUID
instead of device name to specify root partition in kernel
command line. It should make images to boot on devices with
boot device names that differ from what's mentioned in wks file.

(From OE-Core rev: 23cca700870230b46d251086441136e99659ef12)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-10-01 21:45:56 +01:00
Ed Bartosh 7b31253f3c scripts: add new script 'native'
Added 'native' convenience shell script to run native tools.
Example of usage:
  > bitbake bmap-tools-native
  > native bmaptool --version

(From OE-Core rev: 84274b35945c6b1b732b052bcbf13935923db803)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-10-01 21:45:56 +01:00
Markus Lehtonen a17ce357fe build-perf-test-wrapper.sh: accept test case failures
Utilize the new return value (2) from oe-build-perf-test. Do not exit
with an error in case some individual tests fail. Even if some tests
fail we still want to complete successfully, that is, display and
archive the results and do cleanup. The individual tests do not depend
on each other anymore so test failures shouldn't affect the results of
successful tests.

(From OE-Core rev: e3c7d8a98a261a6a8c913e7fcd19264df501636d)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-10-01 21:45:56 +01:00
Markus Lehtonen e4e6ff3be3 oe-build-perf-test: return 2 if some tests failed
Add a new return value '2' that indicates that some tests failed but
there were no fatal errors (i.e. configuration mistakes or bugs in the
tests themselves).

(From OE-Core rev: 194e95f3f068456f30c0e971eb8e6e775279427c)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-10-01 21:45:56 +01:00
Markus Lehtonen 4df40248b3 build-perf-test-wrapper.sh: show defaults for '-a' and '-w'
Display default values for '-a' and '-w' command line arguments in the
usage help text.

(From OE-Core rev: 580708398f22333bc4b5899e4129a8939fb7ce12)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-10-01 21:45:56 +01:00
Markus Lehtonen eb1c8cd1d9 build-perf-test-wrapper.sh: check for positional arguments
Stricter checking of command line arguments. The script doesn't use any
positional arguments so don't accept any and error out if those are
found.

(From OE-Core rev: 4725ee8e4e4837446dfa3a319eb68cc9572c55eb)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-10-01 21:45:56 +01:00
Zubair Lutfullah Kakakhel a8002cb367 runqemu: Add little endian variations for MIPS
Add mipsel and mips64el as an option.

(From OE-Core rev: 072dd5b3b164ca7a5fd9dc969c991c15adeb0cbe)

Signed-off-by: Zubair Lutfullah Kakakhel <Zubair.Kakakhel@imgtec.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-10-01 21:45:56 +01:00
Markus Lehtonen 4f5801303e scripts/buildstats-diff: implement --multi option
Makes it possible to average over multiple buildstats. If --multi is
specified (and the given path is a directory) the script will read all
buildstats from the given directory and use averaged values calculated
from them.

All of the buildstats must be from a "similar" build, meaning that no
differences in package versions or tasks are allowed. Otherwise, the
script will exit with an error.

(From OE-Core rev: 315f44ba39e9b13facacd0fd3796fa87329d9d69)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-10-01 21:45:56 +01:00
Markus Lehtonen a235c1b7c2 scripts/buildstats-diff: make logger msg format a bit more readable
(From OE-Core rev: 49ae4382dd0a71f45989af3679bd35ce2bfa82f8)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-10-01 21:45:56 +01:00
Markus Lehtonen cbbe51f55f scripts/buildstats-diff: use exception for internal error handling
(From OE-Core rev: 17b27b7a8bfc8b1c9ee274d1ed2d5b57bea13bf5)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-10-01 21:45:56 +01:00
Markus Lehtonen 4cdf47a569 scripts/buildstats-diff: add walltime to --diff-attr
For comparing the elapsed wall clock time of tests. Default values for
--min-val and --min-absdiff are 5 seconds and 2 seconds.

(From OE-Core rev: 8e7a5beb2ce116bcd87111d190a4ac5d771e8884)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-10-01 21:45:56 +01:00
Markus Lehtonen c6047cd989 scripts/buildstats-diff: add read_ops and write_ops to --diff-attr
Two new options, making it possible to compare the number of filesystem
operations of tasks. Defaults for --min-val and --min-absdiff are set to
more or less arbitrary 500 and 50 operations, respectively.

(From OE-Core rev: 75292a1de1a59e19198d26b7c1291004a5ca92f3)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-10-01 21:45:56 +01:00
Markus Lehtonen 44bc3f47a3 scripts/buildstats-diff: add read_bytes and write_bytes to --diff-attr
These are I/O counter values from /proc/<pid>/io and represent the
number of bytes read from / written to the storage layer. Default values
for --min-val and --min-absdiff limits are set to 512kB and 128kB,
respectively.

(From OE-Core rev: 24a12e40caeb05dac13c417f35733761af219f03)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-10-01 21:45:56 +01:00
Markus Lehtonen 01a2b5823b scripts/buildstats-diff: introduce --diff-attr
A new command line option for choosing which "attribute" of the
buildstats to compare. At first, the already supported 'cputime' is the
only available option. But, refactoring done in this patch should make
it easy to add new attribute types.

(From OE-Core rev: 0782825138731b3f1e6a8e05d723c1d5cd60c90c)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-10-01 21:45:55 +01:00
Markus Lehtonen e05309194a scripts/buildstats-diff: do not hardcode field widths in output
Dynamically adjust the width of all fields in task diff output. Makes
it easier to print other units than cputime, too.

(From OE-Core rev: 559b858f2a3712ec21debb71681593bd7cf55041)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-10-01 21:45:55 +01:00
Markus Lehtonen 7113ac9439 scripts/buildstats-diff: implement BSTask class
New class representing buildstats data of a single task.

(From OE-Core rev: 472818a32f96699a6dc9c7c487f38d716678fd7a)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-10-01 21:45:55 +01:00
Markus Lehtonen 4cc1c430cd scripts/buildstats-diff: rename --min-time and --min-timediff args
Rename these arguments to --min-val and --min-absdiff in preparation for
supporting other "quantities" than just cputime.

(From OE-Core rev: 441336bc1750939c2da2d9e4dc5a6893b283bf68)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-10-01 21:45:55 +01:00
Markus Lehtonen 58bd0fd62d scripts/buildstats-diff: check that the given directory exists
(From OE-Core rev: 08082b96d8d09215f02e9251f354bb6e8bb3e712)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-10-01 21:45:55 +01:00
Jianxun Zhang d19645bcde wic: rename and amend systemd-boot wks file
Rename wks for systemd-boot per the suggestion from community.
Also amend description to distinguish it from others when
running "wic list images".

(From OE-Core rev: 6303dbbaa08214a37caf38e3b6b5a30a108bd3b7)

Signed-off-by: Jianxun Zhang <jianxun.zhang@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-10-01 21:45:55 +01:00
Ed Bartosh 5753147ea2 runqemu: explicitly set image format
QEMU produces a warning if drive format is not specified:
  WARNING: Image format was not specified for
  'tmp/deploy/images/qemux86-64/core-image-minimal-qemux86-64.wic'
   and probing guessed raw.
   Automatically detecting the format is dangerous for raw images,
   write operations on block 0 will be restricted.
   Specify the 'raw' format explicitly to remove the restrictions.

Set image format to 'vmdk', 'qcow2' or 'vdi' for correspondent image
types. Set it to 'raw' for the rest of image types.

(From OE-Core rev: 5100bb36502ef7c81220a3c4809eb1b3ac83801f)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-30 16:51:15 +01:00
Christopher Larson c8089ae233 recipetool: newappend: drop _provide_to_pn
This function was broken by the multi-config changes, and isn't needed anymore
now that recipeutils.pn_to_recipe can handle provides. Without this, the
newappend sub-command fails.

(From OE-Core rev: 4a5028dc3d1ab2f97465e63db5b05de73daebdfa)

Signed-off-by: Christopher Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-28 10:16:04 +01:00
Richard Purdie 04a3b6c0ca scripts/cleanup-workdir: Adapt to SDK_ARCH -> SDK_SYS chanages for crosssdk
With the change of crosssdk to use SDK_SYS, we need to update the script
to match.

(From OE-Core rev: 8ab1f6073f86d05493bb32a8135c9d912d72f5f0)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-28 10:16:03 +01:00
Joe MacDonald 527764b4c5 devtool: Add a line break to generated README
When devtool creates a new workspace, it produced a README with one very
long line and no space following 'bblayers.conf'.  Add a line break as was
intended.

(From OE-Core rev: 4ad1bcfc3c88ced5b7fc80c950613e31becb40f3)

Signed-off-by: Joe MacDonald <joe_macdonald@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-28 10:16:03 +01:00
Stephano Cetola 1d1f94b944 scripts/runqemu: provide better error message on runqemu ifup fail
If runqemu-ifup fails hen running testimage, a rather cryptic error
regarding "no tty present" is displayed. If this step fails, we
should at least point the user at runqemu-gen-tapdevs. A quick search
of this term in the manual will lead them to "Enabling Runtime Tests
on QEMU" which should give them all the info they need.

(From OE-Core rev: 3b6494fad2b8b65e0d52cda0cdf500e93c72823a)

Signed-off-by: Stephano Cetola <stephano.cetola@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-24 07:30:10 +01:00