Commit Graph

2012 Commits

Author SHA1 Message Date
Paul Eggleton 3ec9a621d0 recipetool: record unknown license files
Add a comment to the recipe listing license files that were found but
not able to be identified, so that the user can find and examine them
by hand fairly easily.

Fixes [YOCTO #9882].

(From OE-Core rev: 4b7d1bf8172533e9ac91a49ade152a05e2ee4146)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-26 08:56:32 +01:00
Paul Eggleton 0b2ca66874 classes/populate_sdk_ext: filter sstate within the extensible SDK
Use the new oe-check-sstate to filter the sstate artifacts shipped with
the extensible SDK by effectively running bitbake within the produced
eSDK and and getting it to tell us which tasks it will restore from
sstate. This has several benefits:

1) We drop the *-initial artifacts from the minimal + toolchain eSDK.
   This still leaves us with a reasonably large SDK for this
   configuration, however it does pave the way for future reductions
   since we are actually filtering by what will be expected to be there
   on install rather than hoping that whatever cuts we make will match.

2) We verify bitbake's basic operation within the eSDK, i.e. that
   we haven't messed up the configuration

3) We verify that the sstate artifacts we expect to be present are
   present (at least in the sstate cache for the build producing the
   eSDK). Outside deletion of sstate artifacts has been a problem up to
   now, and this should at least catch that earlier i.e. during the
   build rather than when someone tries to install the eSDK.

This does add a couple of minutes to the do_populate_sdk_ext time, but
it seems like the most appropriate way to handle this.

Should mostly address [YOCTO #9083] and [YOCTO #9626].

(From OE-Core rev: 4b7b48fcb9b39fccf8222650c2608325df2a4507)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-26 08:56:31 +01:00
Paul Eggleton 2ac1fdf537 scripts: add oe-check-sstate script
Add a script to check which sstate artifacts would be installed by
building a given target - by default this is done with a separate
TMPDIR to ensure we get the "from scratch" result. The script produces a
list of tasks that will be restored from the sstate cache. This can also
be combined with BB_SETSCENE_ENFORCE* to check if sstate artifacts are
available.

The implementation is a little crude - we're running bitbake -n and
looking at the output. In future when we have the ability to execute
tasks from tinfoil-based scripts we can look at rewriting that part of
it to use that instead.

(From OE-Core rev: 4d059e02099e6244765027f2771192434764c606)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-26 08:56:31 +01:00
Ed Bartosh 2d5b282957 wic: rawcopy: make source filenames unique
Rawcopy plugin copies source files to build folder before using them
to assemble result image. After assembling the image wic renames
source files to <image>.p<partition number>. If the same source file
is used in multiple partitions wic breaks trying to rename file that
doesn't exist.

Added <line number> suffix to the files when copying them to the
build dir. This should make filename unique even if the same source
file is used for multiple partitions.

[YOCTO #9826]

(From OE-Core rev: 43a809bfe99024083b4ab4eb9895b084c9c4fa80)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-21 07:47:52 +01:00
Robert Yang 736eab6e73 oe-selftest: print errors when failed to find test
For example:
$ oe-selftest --run-tests-by name hello world
2016-07-12 00:33:28,678 - selftest - ERROR - Failed to find test: hello
2016-07-12 00:33:28,679 - selftest - ERROR - Failed to find test: world

(From OE-Core rev: 665a0f93bde0d61e0c7ceab072ca3f1f22b2f700)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-21 07:47:52 +01:00
André Draszik bffb0fbfcc oe-git-proxy: don't depend on syslinux
gethostip comes from syslinux. It seems odd to depend on a bootloader
to clone a git repository.

Switch to using getent from the c-library, which should be available
on every system.

We now also support the case where a hostname resolves to more than
one IP address.

(From OE-Core rev: c91dbf3ca2faec95195c85b65aa6cab7de9bca2c)

Signed-off-by: André Draszik <git@andred.net>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-20 10:28:51 +01:00
Paul Eggleton d62fd7711d devtool: add finish subcommand
Add a subcommand which will "finish" the work on a recipe. This is
effectively the same as update-recipe followed by reset, except that the
destination layer is required and it will do the right thing depending
on the situation - if the recipe file itself is in the workspace (e.g.
as a result of devtool add), the recipe file and any associated files
will be moved to the destination layer; or if the destination layer is
the one containing the original recipe, the recipe will be overwritten;
otherwise a bbappend will be created to apply the changes. In all cases
the layer path can be loosely specified - it could be a layer name, or
a partial path into a recipe. In the case of upgrades, devtool finish
will also take care of deleting the old recipe.

This avoids the user having to figure out the correct actions when
they're done - they just do "devtool finish recipename layername" and
it saves their work and then removes the recipe from the workspace.

Addresses [YOCTO #8594].

(From OE-Core rev: fa550fcb9333d59b28fc0e4aebde888831410f5c)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-20 10:28:49 +01:00
Paul Eggleton 10a5af5eb4 devtool: upgrade: record original recipe files
This provides us with the information we need to remove the original
version recipe and associated files when running "devtool finish" after
"devtool upgrade".

(From OE-Core rev: 92eb42c347af919cd9f8739515fdf806c12b5ba8)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-20 10:28:49 +01:00
Paul Eggleton 7af3295b0b devtool: update-recipe: refactor to allow calling separately
This will be called by "devtool finish" to allow it to update the recipe
or create the bbappend depending on the destination.

(From OE-Core rev: 5067cdc73483b53d46d9bf584723e41957c7ec54)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-20 10:28:49 +01:00
Paul Eggleton 5d78525a2d devtool: reset: refactor to allow calling separately
This will be called by "devtool finish" to allow it to reset the recipe
at the end.

(From OE-Core rev: b8d398516556eaf97679e28ad58448f570984b52)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-20 10:28:49 +01:00
Paul Eggleton c4297f3e22 devtool: update-recipe: don't copy local files that haven't changed
If there are files in the oe-local-files directory which are identical
to the original version, then we shouldn't be copying them to the
destination layer. This is particularly important when using the -a
option to create a bbappend.

(From OE-Core rev: 9230bfcc839eb35630949f0a8ed058ca1fa944b1)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-20 10:28:49 +01:00
Paul Eggleton 8be2077a38 devtool: update-recipe: find and use existing files directories
devtool update-recipe was defaulting to the ${BPN} named directory when
adding patches next to a recipe, but that meant if you already had files
in a ${BP} named directory (i.e. name and version) or "files" then you'd
end up with two directories next to the recipe, which is usually not
what you want. To avoid this, look through FILESPATH and take the first
one that's the same level or one level down from the recipe and already
exists, if any.

(From OE-Core rev: c7a8190cf8bdf86ba850b6780b8e951e90232c06)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-20 10:28:49 +01:00
Derek Straka 89eab25990 python3: update manifest RDEPENDS for importlib and compression packages
zipfile.py has dependencies on importlib, threading, and shell
importlib has a dependency on lang
operator and contextlib added to the lang package instead of falling into misc

(From OE-Core rev: 769ad8e114fda1fe112d3747408edbeb7b066a85)

Signed-off-by: Derek Straka <derek@asterius.io>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-20 10:28:47 +01:00
Maciej Borzecki 45743114c8 wic: mountpoint is an optional argument
According to wic documentation partition mount point is an optional
argument. Skipping mount point also makes sense in certain
configurations when one needs to specify a partition that is not mounted
by the running system, such as a recovery or a mirror partition (in dual
rootfs setups).

(From OE-Core rev: 5e063a4c6bb0e0623a4d25bb2bf6eecd9ad6b9f1)

Signed-off-by: Maciej Borzecki <maciej.borzecki@rndity.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-20 10:28:47 +01:00
Jianxun Zhang 4c9b1a472f wic: clone gummiboot wks for systemd-boot
This wks is just a copy of gummiboot wks with bootloader
changed to systemd-boot. A very basic boot test on a X86
target is done with a direct-boot image generated by:

wic create mksystemd-bootdisk -e core-image-minimal

Because we plan to replace gummiboot with systemd-boot at any
time in the future, we summarize history (as much as I can) of
the current gummiboot wks before it's gone:

-----------------------------------------------------------------
commit 7d4bb40905
Author: Ed Bartosh <ed.bartosh@linux.intel.com>
Date:   Wed Sep 2 13:58:02 2015 +0300

    wic: get rid of scripts/lib/image

    Moved content of scripts/lib/image/ to scripts/lib/wic as
    one directory with the same name as a tool is self-explanatory
    and less confusing than two.

    (From OE-Core rev: 5dc02d572794298b3362378cea3d7da654456c44)

    Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
    Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

commit 77561e7191
Author: Ed Bartosh <ed.bartosh@linux.intel.com>
Date:   Wed Sep 2 13:58:01 2015 +0300

    wic: use ext4 in canned .wks files

    Latest kernel doesn't have ext3 compiled in. Wic images produced
    from canned .wks can't boot because of that. Switching to ext4
    fixes this issue.

    (From OE-Core rev: d281a65a81f369fc8d75023b8f911ce4106969c1)

    Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
    Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

commit 072657ce98
Author: Tom Zanussi <tom.zanussi@linux.intel.com>
Date:   Mon Aug 11 20:35:41 2014 -0500

    wic: Add mkgummidisk kickstart file

    This is the same as mkefidisk but uses gummiboot instead of grub-efi.

    (From OE-Core rev: 5979409ebfab0bb07b3c2b2fcf14a722c441f07b)

    Signed-off-by: Tom Zanussi <tom.zanussi@linux.intel.com>
    Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
-----------------------------------------------------------------

(From OE-Core rev: 546cd8352b8adce074831ec31cfa3bb2bf2f0084)

Signed-off-by: Jianxun Zhang <jianxun.zhang@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-20 10:28:46 +01:00
Jianxun Zhang 3091a8c58a wic: add systemd-boot option in EFI boot image plugin
The new systemd-boot enabled in OE is the old gummiboot
merged into systemd project. Our intention is to replace
gummiboot with systemd-boot in OE once every feature based
on gummiboot is supported with systemd-boot.

Before we can purge gummiboot, we temporarily keep both of
the two bootloaders supported.

Patch doesn't do replacement for every "gummi" occurrence.
We think cleaning can be done in background after we disable
people to use gummiboot, so we just keep change small and
safe this time.

(From OE-Core rev: daa5f8b886408eb6a17898b18ac97d5a0d76d2cc)

Signed-off-by: Jianxun Zhang <jianxun.zhang@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-20 10:28:46 +01:00
Jussi Kukkonen fd08714c26 python3: Add compression to python3-shell dependencies
python3-shell needs python3-compression for tarfile.

(From OE-Core rev: fe5979534bd4fc1f3e5401c9a86e4aff571aec24)

Signed-off-by: Jussi Kukkonen <jussi.kukkonen@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-12 23:10:16 +01:00
Paul Eggleton f465039737 recipetool: create: support specifying a file as the local source
It is currently possible to specify a file (e.g. a tarball) on the local
disk as the source, but you have to know to put file:// in front of it.
There's really no need to force users to jump through that hoop if they
really want to do this so check if the specified source is a file and
prefix it with file:// if that's the case.

Also ensure the same works for "devtool add" at the same time.

(From OE-Core rev: 71350003790c38e84b0e525a71a2fe5d24e3d083)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-12 23:10:15 +01:00
Paul Eggleton 54486a1ac3 recipetool: create: fix handling of github URLs
For a while now, Github hasn't been advertising a specific repository
URL since cloning the web URL with git works. Armed with this knowledge
and fully expecting people to just paste the github URL, we need to
handle this situation specially. If it looks like a github URL to the
root of a repository then treat it as a git repository instead of a
normal https URL to be fetched by the wget fetcher.

(From OE-Core rev: 7998dc3597657229507e5c140fceef1e485ac402)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-12 23:10:15 +01:00
Maxin B. John 2c9291dfb9 graph-tool: convert to python3
move graph-tool to python3

(From OE-Core rev: 0d0864ae0ff9e53623ad1c7146b071f2a046f21f)

Signed-off-by: Maxin B. John <maxin.john@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-12 23:10:15 +01:00
Paul Eggleton baed9bd848 scripts/contrib/devtool-stress: exclude more recipes by default
These recipes can't be used with devtool because they can't be unpacked
in the normal way.

(From OE-Core rev: b2cf098969b8b800a78d650cf60c0b5ad31c85b5)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-12 23:10:15 +01:00
Paul Eggleton 89f8348dc5 scripts/contrib/devtool-stress: skip incompatible recipes
If devtool returns exit code 4 then record the recipes as "skipped"
rather than "failed" - these are recipes we know cannot work (usually
because they don't provide any source).

(From OE-Core rev: 8fc109f1cb6eb437c12d2d11a6937de6f035e296)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-12 23:10:15 +01:00
Paul Eggleton 5c91537ab2 devtool: return specific exit code for incompatible recipes
Certain recipes cannot be used with devtool extract / modify / upgrade -
usually because they don't provide any source. Return a specific exit
code (4) so that scripts such as scripts/contrib/devtool-stress.py know
the difference between this and a genuine failure.

(From OE-Core rev: ffd295fed4ab81fc0bd00bb145ef4d72c49584bf)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-12 23:10:15 +01:00
Paul Eggleton d56caf3408 devtool: reset: allow reset to work if the recipe file has been deleted
We were attempting to open the recipe file unconditionally here - we
need to account for the possibility that the recipe file has been
deleted or moved away by the user.

(From OE-Core rev: 47822a2aff56fd338c16b5ad756feda9f395a8a1)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-12 23:10:15 +01:00
Paul Eggleton 117c80f9dc devtool: update-recipe: fix --initial-rev option
In OE-Core revision 7baf57ad896112cf2258b3e2c2a1f8b756fb39bc I changed
the default update-recipe behaviour to only update patches for commits
that were changed; unfortunately I failed to handle the --initial-rev
option which was broken after that point. Rework how the initial
revision is passed in so that it now operates correctly.

(From OE-Core rev: b2ca2523cc9e51a4759b4420b07b0b67b3f5ac43)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-12 23:10:15 +01:00
Richard Purdie c706bfbabb terminal: Fix gnome-terminal to work with recent versions
Currently gnome-terminal just returns straight away, opening a terminal in a new
separate process we have no insight into. For patch resolution, this leads to
spawning many different terminal windows, for pydevshell, it just flashes a window
up and then closes.

We need to block until the command completes but gnome-terminal gives us no way
to do this. We therefore write the pid to a file using a "phonehome" wrapper
script, then monitor the pid until it exits.

[YOCTO #7254]
(also fixing do_devpyshell)

(From OE-Core rev: 76e8ab47c936674b8bb9bf1c48de53b30f5bf74a)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-08 10:50:18 +01:00
Humberto Ibarra 6a2753bc35 scripts: Fixed typo in parameter that was causing exception
There is a typo in the logging parameters, "filname" is being used instead of "filename" for yocto-kernel, yocto-layer and wic scripts. This didn't cause issues before since python 2 didn't validate unused parameters but with python >= 3.4.3 an exception is thrown. This patch fixes this parameter name.

[YOCTO #9834]

(From OE-Core rev: 5d123a6ec0e97652b2ec4295428797a336ef357a)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-08 09:57:25 +01:00
Otavio Salvador dc626e3bdc runqemu: Avoid duplicating custom QEMU options for AArch64
When detecting the second serial options we shouldn't append the
custom QEMU options otherwise we will end duplicating those.

(From OE-Core rev: 79798f20b2c0b98d84c3c4b14600635ff8ddfdad)

Signed-off-by: Otavio Salvador <otavio@ossystems.com.br>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-08 09:57:25 +01:00
Paul Eggleton 1063622cc6 recipetool: create: drop unused convert_pkginfo() function
Code cleanup, no functional changes - this code was never used.

(From OE-Core rev: 397b76c7f26e38e761b94b1f7987aafd55048e10)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-08 09:57:24 +01:00
Paul Eggleton 75f1a0ed42 recipetool: create: avoid decoding errors with Python 3
We're opening source files with the default encoding (utf-8) but we
can't necessarily be sure that they are UTF-8 clean - for example,
recipetool create ftp://mama.indstate.edu/linux/tree/tree-1.7.0.tgz
prior to this patch resulted in a UnicodeDecodeError. Use the
"surrogateescape" mode to avoid this.

Fixes [YOCTO #9822].

(From OE-Core rev: 50fcd9d1b9a20d49bc873467a82a071f2f2f8b5a)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-08 09:57:24 +01:00
Ed Bartosh 94fb1a57de oepydevshell-internal: python3: encode/decode pty content
As /dev/pty opened in binary mode its content has to
be decoded when reading from it and encoded when writing to it.

(From OE-Core rev: 211870ddbce5c966b2882e97cb2efe29b72a62a4)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-08 09:57:24 +01:00
Ed Bartosh e8feb1976a devpyshell: python3: flush stdout explicitly
Opening text stream in unbuffered mode raises the following
exception In Python 3:
    ValueError: can't have unbuffered text I/O

Fixed by leaving std* streams in text mode and flushing
stdout explicitly.

(From OE-Core rev: 732001cb268683f5b56e251e2964ec5b694a2147)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-08 09:57:23 +01:00
Paul Eggleton 6677dd37ad classes/sstate: add a mode to error if sstate package unavailable
If BB_SETSCENE_ENFORCE is set to "1" and an sstate package fails to
download outside of the whitelist specified by
BB_SETSCENE_ENFORCE_WHITELIST, then fail immediately so you can tell
that the problem was caused by failing to restore the task from sstate.

Part of the implementation of [YOCTO #9367].

(From OE-Core rev: 9e711b54487c3141d7264b8cf0d74f9465020190)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-08 09:57:23 +01:00
Humberto Ibarra 4e889c04ba scripts/lib/bsp/help.py: Changed help of yocto-bsp to match latest syntax
With the changes to migrate from optparse to argparse there was a syntax change that needs an update in the help. This is basically just the change of 'properties' and 'property' from positional arguments to options. This patch makes the required changes.

[YOCTO #8321]

(From meta-yocto rev: b171379b5ca54d55ea763421794a651e71bbda91)

Signed-off-by: Humberto Ibarra <humberto.ibarra.lopez@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-07 13:38:13 +01:00
Humberto Ibarra 479d15b326 yocto-bsp: Refactor script to use argparse instead of optparse
Optparse is deprecated and should be avoided. The arparse library is better suited and has more tools to handling the parsing of arguments. This patch makes necessary changes to migrate to the better library and uses arparse subcommand feature to improve organization of this script.

[YOCTO #8321]

(From meta-yocto rev: 3f45993b96d4d960da0efe8672dc323c9db091a2)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-07 13:38:13 +01:00
Humberto Ibarra 4ab8650eeb scripts/yocto-bsp: Removing version from yocto-bsp
The version option is not maintained and is useless inside this script.
There is no reason for this script to have an independent version value.

[YOCTO #8321]

(From meta-yocto rev: 8f8790e56d00f2eaaf6508fb1909335f1fbef5ff)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-07 13:38:13 +01:00
Humberto Ibarra 4252ec78a4 scripts/lib/bsp/engine.py: Add error message for wrong input file
Format of properties file is expected to be a simple json detailing properties, if this format fails an exception is thrown. This patch adds a graceful error message to the case when the properties file has a wrong format.

[YOCTO #9750]

(From meta-yocto rev: 7e543aa19d0d4b2112e6316783fb31b76df3493e)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-07 13:38:13 +01:00
Humberto Ibarra 71bc7fd8ce scripts/lib/bsp/help.py: Fixed pager for yocto-bsp help
Python3 requires strings to be encoded as bytes before sending them through a subprocess pipe. The help.py file is not considering this and fails when issuing paged help commands. This patch adds this encoding to solve the problem.

[YOCTO #9868]

(From meta-yocto rev: 35b487a47f0cbb99fdee2ec9cc8b56b814c8860e)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-07 13:38:13 +01:00
Humberto Ibarra 21282724d8 scripts: Fixed typo in parameter that was causing exception
There is a typo in the logging parameters, "filname" is being used instead of "filename" for yocto-kernel, yocto-layer and wic scripts. This didn't cause issues before since python 2 didn't validate unused parameters but with python >= 3.4.3 an exception is thrown. This patch fixes this parameter name.

[YOCTO #9834]

(From meta-yocto rev: 844fbba63b146a2ded3fced0d62bf047bf844af3)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-07 13:38:13 +01:00
Markus Lehtonen 678a02e112 scripts/contrib: introduce build-perf-test-wrapper.sh
A shell script wrapper around oe-build-perf-test script. The purpose of
this wrapper is to act as a executor script, making it possible to run
the tests with a single command. The wrapper script initializes the
build environment, runs oe-build-perf-test and archives the results.

(From OE-Core rev: 946a076c2ce20dd8f7cfa1acbdab1268d406d3e1)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-01 16:22:48 +01:00
Markus Lehtonen 0c92770ed2 oe-build-perf-test: add --out-dir command line argument
The new option defines the output directory for the test result data.

(From OE-Core rev: bc865b5cb5a9a76048ee9c55a29f5e1a926bb543)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-01 16:22:48 +01:00
Markus Lehtonen c3ee14ef38 oe-build-perf-test: enable locking
Makes it possible to guard that multiple tests are not run in parallel.

(From OE-Core rev: 181e92e7a1bccf678b3eb1bf547608a142784f97)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-01 16:22:47 +01:00
Markus Lehtonen eb36f00002 oe-build-perf-test: implement --globalres-file option
Using this option the script appends test results into a 'global results
file'. A CSV-formatted output of the results. This option is to provide
compatibility with the old build-perf-test.sh.

(From OE-Core rev: e9f18e63220e452f2b0c878998e57d944ae83980)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-01 16:22:47 +01:00
Markus Lehtonen 8329d56cd4 oe-build-perf-test: enable logging into file
(From OE-Core rev: d5bbcdaf332b2a15382672d6a40b2d2b807b9dde)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-01 16:22:47 +01:00
Markus Lehtonen 1d88659ef6 oeqa.buildperf: implement BuildPerfTestRunner class
The new class is responsible for actually running the tests and
processing their results. This commit also adds a decorator function for
adding new tests. No automatic test discovery, at least yet.

(From OE-Core rev: bf90aecb7e150d6bfac7240286c797b79d26528b)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-01 16:22:46 +01:00
Markus Lehtonen cee685ca0d oeqa.buildperf: functionality to drop kernel caches
Add a new utility class for dropping Linux kernel caches.  It uses sudo
and tee to write to the drop_caches file. Checking if the user has the
permissions to drop caches (without a password) is done by trying to
writing an invalid value to the drop_caches file. This way, we will find
if writing (with tee) is possible but not really dropping caches, yet.

User can avoid giving the password by adding something like:
<user> ALL = NOPASSWD: /usr/bin/tee /proc/sys/vm/drop_caches
to the system sudoers file.

(From OE-Core rev: c9cb248429ced50c96d11ba5361c272d4c9b9323)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-01 16:22:46 +01:00
Markus Lehtonen c855cb807d oe-build-perf-test: add pre-run sanity check
The script will be required to be run in an initialized bitbake build
environment.

(From OE-Core rev: 1bce7b10283255a4498d11ead920c1f3b1dec4de)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-01 16:22:46 +01:00
Markus Lehtonen 14108a88ab scripts: introduce oe-build-perf-test
Initial wireframe for re-writing build-perf-test.sh in Python.

(From OE-Core rev: 764eb2d011305b84501cc183531a2a5353b0b5ab)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-01 16:22:46 +01:00
Nathan Lynch 98121ad2a5 recipetool: recognize less common makefile names
GNU make looks for "makefile" and "GNUmakefile" in addition to
"Makefile", so add these other names to the heuristic for detecting a
make-based project.

(From OE-Core rev: 204d19b02265e5b2241888e4c92c0a730f3d3472)

Signed-off-by: Nathan Lynch <nathan_lynch@mentor.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-01 16:22:43 +01:00
Martin Jansa 52adfed684 test-dependencies.sh: Strip also '\.bb; .*' before adding failed recipe to list of failed
* format of bitbake tasks changed in:
  2c88afb   taskdata/runqueue: Rewrite without use of ID indirection

-ERROR: Task 4 (/OE/build/oe-core/openembedded-core/meta/recipes-devtools/rpm/sftp.bb, do_fetch) failed with exit code '1'
+ERROR: Task /OE/build/oe-core/openembedded-core/meta/recipes-devtools/rpm/sftp.bb:do_fetch (/OE/build/oe-core/openembedded-core/meta/recipes-devtools/rpm/sftp.bb:do_fetch) failed with exit code '1'

  so strip not only '\.bb, .*' used before, but also '\.bb;.*' to drop
  the task name to get recipe name.

* for more details see:
  http://lists.openembedded.org/pipermail/openembedded-core/2016-June/123132.html

* without this change you can see test-dependencies.sh trying to rebuild packages
  like:
  Building recipe: fbprogress (6/21)
  Building recipe: fbprogress.bb:do (7/21)
  where the later of course doesn't exist as a recipe

(From OE-Core rev: b7d6d4203cf2021ee2b9b84c8faf15198bfb536d)

Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-01 16:22:42 +01:00