Commit Graph

2555 Commits

Author SHA1 Message Date
Robert Yang f6aaba31cf runqemu: support env vars explicitly
Use self.env_vars to support get vars from environment explicity. The
MACHINE, ROOTFS and KERNEL was supported by shell based runqemu, and
the help text says support them from env vars, so add them back.

[YOCTO #11141]

(From OE-Core rev: 20008d0bfe2cacecba77e11b0a0faf3d959eaf1e)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-12 15:09:57 +01:00
Li Zhou a2d5c0b602 python2/3: Move config/Makefile from core package to dev package
Move config/Makefile in libdir from core package to dev package for
python, because it is only needed in development process.

(From OE-Core rev: 8b55d055f046677c18eeaefe3ca18869eedeb14d)

Signed-off-by: Li Zhou <li.zhou@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-11 18:10:18 +01:00
Paul Eggleton 1f19d9dfe5 buildhistory-diff: add option to compare actual signature differences
Use the code underpinning bitbake-diffsigs to add an option to
buildhistory-diff to determine and display the differences between the
actual signature inputs, with a twist - we collapse identical
changes across different tasks, showing only the most recent task to
have that difference, meaning that there's less noise to wade through
when you just want to know what changed in order to cause some
rebuilding you're seeing.

(From OE-Core rev: 86cb4b01f2020553902554e512c02147eb4e0f51)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-11 18:10:17 +01:00
Paul Eggleton 9049c09793 buildhistory-diff: add option to compare task signature list
Having added writing out of the task signature list to buildhistory
(when BUILDHISTORY_FEATURES includes "task"), we now need a way to
compare the list. This just shows which tasks have been added / changed
signature / removed.

(From OE-Core rev: 63bd7e9f780a98dda458d612877495756bcc5463)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-11 18:10:17 +01:00
Paul Eggleton 313e551b55 buildhistory-diff: operate from buildhistory directory
If the cwd is named "buildhistory" and the user hasn't specified an
alternative path on the command line, then assume that the current
directory is the buildhistory directory. This makes it easier to run
buildhistory-diff and also interact with the buildhistory git repository
as you no longer have to jump into the buildhistory directory and up to
the parent again when doing so.

(From OE-Core rev: e4ccec2e4c2f521a2bb473083b42aefd494eea23)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-11 18:10:17 +01:00
Richard Purdie 684eefd7a3 scripts: Drop cleanup-workdir
This script appears broken and is actively breaking build directories.
For example, binutils-cross-i586 gets run do_fetch to do_populate_sysroot by:

$ oe-selftest -r devtool.DevtoolTests.test_devtool_virtual_kernel_modify

then:

$ oe-selftest -r oescripts.BuildhistoryDiffTests.test_buildhistory_diff

wipes out the contents of tmp/work/x86_64-linux/binutils-cross-i586/2.28-r0/
but does not wipe out the corresponding stamps, then:

$ oe-selftest -r runtime-test.Postinst.test_postinst_rootfs_and_boot

needs binutils-cross-i586:do_populate_lic and if and only if this is
unavailable from sstate, it fails since it thinks the source is already
unpacked when it isn't resulting in:

WARNING: binutils-cross-i586-2.28-r0 do_populate_lic: Could not copy license file /home/pokybuild/yocto-autobuilder/yocto-worker/nigh
tly-oe-selftest/build/build/tmp/work/x86_64-linux/binutils-cross-i586/2.28-r0/git/COPYING3.LIB to /home/pokybuild/yocto-autobuilder/yocto
-worker/nightly-oe-selftest/build/build/tmp/work/x86_64-linux/binutils-cross-i586/2.28-r0/license-destdir/binutils-cross-i586/COPYING3.LI
B: [Errno 2] No such file or directory: '/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-oe-selftest/build/build/tmp/work/x86_64-l
inux/binutils-cross-i586/2.28-r0/git/COPYING3.LIB'
ERROR: binutils-cross-i586-2.28-r0 do_populate_lic: QA Issue: binutils-cross-i586: LIC_FILES_CHKSUM points to an invalid file: /home/
pokybuild/yocto-autobuilder/yocto-worker/nightly-oe-selftest/build/build/tmp/work/x86_64-linux/binutils-cross-i586/2.28-r0/git/COPYING3 [license-checksum]

and similar errors.

Its safer for users to wipe tmp than try and maintain scripts which try
to remove pieces of tmp and get it wrong so remove the script.

(From OE-Core rev: 809b23c829f352c0eae455ea89f53e2a9ee87f06)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-10 23:00:44 +01:00
Markus Lehtonen dff701ffd4 scripts/oe-build-perf-report: improve guessing of args
Search remote branches, too, when finding the latest commit.

(From OE-Core rev: 7b502702be5d484c6f1f903426ba63cf04f8ca4a)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-10 23:00:44 +01:00
Markus Lehtonen 182dd45f1f oe-build-perf-report-email.py: use proper fallback email address
Use properly formatted fallback email address instead of just the
username.

(From OE-Core rev: 5e07504d7a74f0641e2a374b9d12590ce9c9cc89)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-10 23:00:44 +01:00
Markus Lehtonen 60658499c2 build-perf-test-wrapper.sh: support extra args for email script
Make it possible to provide (extra) command line arguments to the
oe-build-perf-test-email script via a new environment variable
OE_BUILD_PERF_REPORT_EMAIL_EXTRA_ARGS.

(From OE-Core rev: 9cdd4ea5e006fe326bdf39ea437b9ba61a66b778)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-10 23:00:44 +01:00
Dmitry Rozhkov e9cf698d09 python3: fix run-time deps for core python3 libraries
The http.server module from python3-netclient imports the html module
which is in python3-html. Also xmlrpc.server imports pydoc which is a
part of python3-pydoc. But those run-time dependencies are missing
from python3-netclient and python3-xmlrpc respectively.

Add the missing run-time dependencies.

(From OE-Core rev: 8e30b726c44f873e5fd9d3f36c3464a29b97abd8)

Signed-off-by: Dmitry Rozhkov <dmitry.rozhkov@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-10 23:00:43 +01:00
Patrick Ohly 7d4fa58c5c yocto-compat-layer: limit report of signature changes
Typically a single change cascades through the entire task dependency
chain. Developers had to figure that out themselves, based on hard to
read and interpret output (not sorted, no indention, no explanations):

   $ yocto-compat-layer.py -n meta-xxxx
   ...
   AssertionError: True is not false : Layer meta-xxxx changed signatures.
   webkitgtk:do_install changed fe2edc9082bc0da98f9cb1391c52f565 -> b3a44684c5cd9aacd3f7c6ed88eefab5
   gstreamer1.0-plugins-good:do_configure changed 3b2f8211be3fe08422bf6087f3af16d1 -> 7d80e42fa1f4f01ff4dfe2ea4477d382
   pulseaudio:do_package_qa changed 5d0a58ada66ff17f5576555302ac319a -> 0e13bcb96143d1ae54c451bc3de0aa30
   epiphany:do_prepare_recipe_sysroot changed 29e1b277dbcb005bd54950594c50d91b -> d3c45527b37677a0668ce483c6db3052
   ...
   gst-player:do_packagedata changed 9ce6efdd357dd74919bc4957458b1e95 -> d0c083ce629f37adfc9c4ba9eff81f83
   gstreamer1.0-plugins-base:do_install changed 1161cd867d15bea63e5dd5d9abf0519c -> 5bf2b652a2d77fee3eedb35af2f201a0
   gstreamer1.0-rtsp-server:do_packagedata changed 6781dc3070f80b843ed1970d74dd323e -> 454620c2e3b9fea87e525d14b6ed0344
   alsa-plugins:do_packagedata changed 1808c3f737cb805b169d004e948ea19c -> 480124b7fa5eab1f73bf96440d725231

Now the tool automates the problem analysis: it retrieves the depgraph
using the tinfoil API and only reports those tasks with modified
signatures whose dependencies have not changed, i.e. those tasks which
definitely introduce a change.

>From the previous example, that just leaves two tasks that need to be
checked:

   AssertionError: False is not true : Layer meta-xxxx changed 120 signatures, initial differences (first hash without, second with layer):
      gstreamer1.0-plugins-base:do_fetch: 76973f19f2e30d282152bdd7e4efe5bb -> e6e7c6fa9f2bd59d7d8d107f7c6ca1ac
      pulseaudio:do_install: 668eb1e30af129df9806b0aa0d7c10cd -> 1196bdb88eef56eeee4613bb06b9387e

This pruning might be a bit too aggressive in the sense that tasks
which inherit a change and then add more changes themselves won't be
reported initially. They will be found when fixing the reported tasks
and re-running the check.

For a developer it seems better to have something listed which
definitely is a problem and needs fixing instead of everything,
including the tasks which don't need fixes.

(From OE-Core rev: 7ab0e09de75bfd7e7498bfa72d1f2f5d02a96747)

Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-10 23:00:42 +01:00
Patrick Ohly efd3b0ee30 yocto-compat-layer: fix also other command invocations
In commit 5b9ac62ab535d, one place was fixed where a command was
invoked such that failures caused double stack traces and stderr was
lost. The same problem also occurs elsewhere, triggered for example by
a layer with parsing problems.

Now a new utility method is used instead of repeating the code.

(From OE-Core rev: b6c72c0d169473e2626938be2ee59f850624612e)

Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-10 23:00:42 +01:00
Markus Lehtonen f81a532dd6 oe-build-perf-report-email.py: fix one file path
Sending report email was not working correctly if the script was given
an html report path that contained directory components.

(From OE-Core rev: 1da641661bb5963fcbd7ac2c20bc997c3eae6f18)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-10 23:00:42 +01:00
Robert Yang 74d7d12b37 runqemu: use realpath for imgdir
The DEPLOY_DIR_IMAGE maybe relative or absolute path since it can be
read from env vars, so use realpath for both imgdir and
DEPLOY_DIR_IMAGE when compare.

(From OE-Core rev: dad9f27278850d0d3818344fea877835632576cb)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-10 23:00:42 +01:00
Robert Yang c70a74402f runqemu: fix 2 typos
* "is it" -> "it is"
* Remove "<image>.qemuboot.conf =" in the error message which looked strange.

(From OE-Core rev: a6152dd9f6f4e17855548ceffa8d864855a67f5c)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-10 23:00:42 +01:00
Chen Qi 9cacf8488c wic: improve error message
When using `wic create mkefidisk -e core-image-minimal', the following
error message appeared.

  Please bake it with 'bitbake parted-native' and try again.

However, following this command doesn't do any help. The same problem
still appeared.

The problem is that when we 'bitbake parted-native', it doesn't have
anything to do with core-image-minimal. And the required tool 'parted'
is not under core-image-minimal's recipe-sysroot-native directory.

Improve the error message so that following it could get things done.

(From OE-Core rev: f0425c0a0f1c98f65bf61fd9aa7e023ed41a35fa)

Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-05 23:22:13 +01:00
Ed Bartosh b5bc885ba7 Revert "filemap: remove FilemapSeek class"
FIEMAP API is not supported by tmpfs file system, but
SEEK_HOLE/SEEK_DATA is supported.

Returned back FilemapSeek class that implements support
of SEEK_HOLE/SEEK_DATA API to make sparse_copy API working
on tmpfs again.

This reverts commit 6b80c13f7a.

(From OE-Core rev: e75bd6a7dd5c1b4bad039c35cf4a2ffc2f77c60a)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-05 23:22:13 +01:00
Markus Lehtonen 86eaa6d83f build-perf-test-wrapper.sh: support sending email reports
Add new '-E' command line option for sending an email report to
specified recipient.

(From OE-Core rev: 46e76ffd460933ab35da4cfd3509f7c5de5ecd93)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-05 23:22:13 +01:00
Markus Lehtonen 7e7c31fd00 build-perf-test-wrapper.sh: support pushing to remote Git
Implement new '-P' option for spefifying a Git remote where to push
results after committing to a local Git repository.

(From OE-Core rev: d8e14df29d28bfe805dc746f43c9f3a7726e57ce)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-05 23:22:13 +01:00
Markus Lehtonen 5576ce2de3 build-perf-test-wrapper.sh: correctly check test script exit status
Test of the exit code was accidentally moved to wrong place when
oe-git-archive was taken into use.

(From OE-Core rev: ed43b2dfe019f35086967a0c8dc605bc6629c75b)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-05 23:22:13 +01:00
Markus Lehtonen d3d24ef2f2 build-perf-test-wrapper.sh: make it possible to specify Git branch name
Support <branch>:<commit> format for the -c argument. This makes it
possible to test older commits of a certain branch (not just the tip of
it) so that the branch name will still be correctly recorded in the test
report data.

(From OE-Core rev: be3d1718a99e59e636f349586e0a64eb8e2824a4)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-05 23:22:13 +01:00
Markus Lehtonen 04e31f33c8 scripts/oe-build-perf-report: fix typo
(From OE-Core rev: 0a05ccfeba2f185ef9ea78a23bc376d4b97ec547)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-05 23:22:13 +01:00
Markus Lehtonen df20260016 scripts/oe-build-perf-report: fix 'charts ready' console message
The javascript console log messages are used in scraping, when
converting an html test report to html email. Before this patch a
console message indicating that all charts have been drawn was not
correctly sent if the last test failed (or didn't have chart data for
some other reason) which, in turn, caused oe-build-perf-report-email.py
script to fail with a timeout.

(From OE-Core rev: 79b90ae02257002ea831a48f6798794b7711c1f8)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-05 23:22:13 +01:00
Markus Lehtonen 728c668473 scripts/oe-build-perf-report: better guessing of args
When getting info from the latest commit, don't search all refs but only
branches. We don't get correct data from refs/tags/* or refs/notest/*,
for example.

(From OE-Core rev: f84d0bd7deb4c19fdb1e821b3a50e8c8f54a731b)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-05 23:22:13 +01:00
Mark Hatle 6afe0e07ec yocto-compat-layer.py: Fix the signature validation
The initial signatures need to be collected -after- the dependency layers have
been added to the system.  Otherwise changes that happen due to dependencies,
outside of the layer being scanned, will show up as signature problems.

The add_layer function was split into two pieces so that we can process
the dependencies first, and then add the layer itself for the comparison.

(From OE-Core rev: 4eb0932e755b7cb582a8db811aeed1397ecb92cc)

Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-05 23:22:13 +01:00
Mark Hatle d2d019a11b compatlayer/__init__.py: Allow add_layer to process recursive deps
When processing a layer for dependencies, you have to process the layer
itself, it's dependencies, the dependencies dependencies and so forth until
all items have been processed.

i.e.:  LayerA requires LayerB requires LayerC requires layerD

The end result should be LayerB, LayerC and LayerD are all dependencies of
LayerA.

(From OE-Core rev: 6e0a268b750fb6701604dd936cd2cf3b47a6e804)

Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-05 23:22:13 +01:00
Mark Hatle efd07494c5 yocto-compat-layer.py: Add --dependency argument
When processing a large number of items, there are times that it would be
nice to be able to pass in a series of layers that can be used as dependencies
for the layer that is being scanned.  This avoids the significant overhead
of processing all of the layers to compatibility.

(From OE-Core rev: 57fc8a9771174b7d0533a42c045053adefa537a8)

Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-05 23:22:13 +01:00
Mark Hatle 2a04cb8d22 yocto-compat-layer.py: Add status for skipped items
If items were skipped because the dependencies could not be found, we
want to record this was skipped so we can display it later.

(From OE-Core rev: 402ef1c8bd1ea994581a39672f31ab1203a0899d)

Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-05 23:22:12 +01:00
Aníbal Limón 704613b2ec scripts: Add yocto-compat-layer-wrapper
This script will be used to create it's own build directory to make
runs of yocto-compat-layer.py againts layers isolated.

Example:

$ source oe-init-build-env
$ yocto-compat-layer-wrapper LAYER_DIR LAYER_DIR_N

[YOCTO #11164]

(From OE-Core rev: 9414382f96d4a5d81cca440c75140950ca515aab)

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-05 23:22:12 +01:00
Peter Kjellerstedt 4bd647af83 update_gio_module_cache: Do not chown a non-existing file
Only change the ownership of ${libdir}/gio/modules/giomodule.cache if
it exists.

(From OE-Core rev: df2e1a8fbadffac0f1781a0d07e050356a007327)

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-05 23:22:12 +01:00
brian avery 0ffc7b8d2e devtool: point runqemu to correct native bindir
devtool/runqemu.py was relying on STAGING_BINDIR_NATIVE to find the host
tools it needed like qemu-system-<arch>.  In the post RSS world, this no
longer exists. This patch points it to
{STAGING_DIR}/{BUILD_ARCH}/{bindir_native}.

[YOCTO #11223]

(From OE-Core rev: 1910f9e9336bfedc8278a3bc02e7e7f934a4fc86)

Signed-off-by: brian avery <brian.avery@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-05 23:22:12 +01:00
brian avery f200f37699 scripts: change way we find native tools (pseudo)
oe-find-native-sysroot: Recipe Specific Sysroots have eliminated the
large STAGING_DIR_NATIVE. Now, we will rely on the meta-ide-support
sysroot that is what was populating the large STAGING_DIR_NATIVE in
previous versions anyway. We now look for RECIPE_SYSROOT_NATIVE.

[YOCTO #11119]

(From OE-Core rev: d8b4c58676705e9749347be0e8f191ccccc37f05)

Signed-off-by: brian avery <brian.avery@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-05 23:22:12 +01:00
Markus Lehtonen 034702f520 scripts/contrib: add oe-build-perf-report-email
Script for sending build perf test reports as an email. Mangles an html
report, generated by oe-build-perf-report, into a format suitable for
html emails. Supports multipart emails where a plaintext alternative can
be included in the same email.

Dependencies required to be installed on the host:
- phantomjs
- optipng

[YOCTO #10931]

(From OE-Core rev: 9e97ff174458f7245fc27a4c407f21a9d2e317ab)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-01 23:28:20 +01:00
Markus Lehtonen 9f299876f7 scripts: add oe-build-perf-report script
A new tool for pretty-printing build perf test results stored in a Git
repository. The scripts is able to produce either simple plaintext
report showing the difference between two commits, or, an html report
that also displays trendcharts of the test results. The script uses
Jinja2 templates for generating HTML reports so it requires
python3-jinja2 to be installed on the system.

[YOCTO #10931]

(From OE-Core rev: 3b25404f0f99b72f222bdca815929be1cf1cee35)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-01 23:28:20 +01:00
Bob Cochran 1771bfd1a7 python: remove stale link to "Python for Embedded Systems Site"
Reference url is a stale, non existent site that returns a 404, so get rid of it

Change impacts both the manifest files and the scripts that generate the manifests

Run the following from within recipes-devtools/python

../../../scripts/contrib/python/generate-manifest-2.7.py > python-2.7-manifest.inc
../../../scripts/contrib/python/generate-manifest-2.7.py -n > python-native-2.7-manifest.inc

../../../scripts/contrib/python/generate-manifest-3.5.py > python-3.5-manifest.inc
../../../scripts/contrib/python/generate-manifest-3.5.py -n > python-native-3.5-manifest.inc

(From OE-Core rev: ae13f580b759211c1a6b59a276f75d589f1db11c)

Signed-off-by: Bob Cochran <openembedded@mindchasers.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-01 23:28:19 +01:00
Saul Wold 19e99786be yocto-bsp/i386 machine.cfg: Explicitly disable 64BIT
Since we do not set the 64 bit flags, newer kernels seem to build 64bit
config files by default. This is due to a hard-coded uname -m check that
selects the KBUILD_DEFCONFIG based on the host, not the cross target.

Similar to e9ec769926b2378e63380bd7762ce7ce201af151 in the yocto-kernel-cache repo

(From meta-yocto rev: e35017cc67f6d3c5cc00488d3460de0dcec773b3)

Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-01 08:15:32 +01:00
Robert Yang f57a53d5ab oe-run-native: OLDPATH -> OLD_PATH
It's a typo.

(From OE-Core rev: 1271d50e622cb3a0eef662de7112da04f05ef5bb)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-01 08:14:58 +01:00
Daniel Schultz 28997864e2 wic: partition: Update fsck parameters
These parameters are copied from the ext image class.

-D will let fsck perform further directory optimizations
-v might be helpful for debugging purpose

(From OE-Core rev: be93e378506a85772503005294503cfc348a552c)

Signed-off-by: Daniel Schultz <d.schultz@phytec.de>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-01 08:14:58 +01:00
Ed Bartosh 67ec097bc3 wic: remove prepare_empty_partition_squashfs
There is not much sense in creation of empty squashfs
partition. It's also not possible to create empty squashfs
partition of specified size.

Even more, prepare_empty_partition_squashfs method is
absolutely broken. It raises exception when called and
even its signature differs from the rest of of similar
methods. It means that nobody uses it and it's safe
to remove it.

Removed prepare_empty_partition_squashfs method and
testing of empty squashfs partition.

(From OE-Core rev: 9152960f250cb4df1e559d747fb09005675a0d75)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-01 08:14:58 +01:00
Ed Bartosh d62a97ec54 wic: fix list of supported fstypes in help content
Added vfat and msdos to the list of supported fstypes in
'wic help kickstart' output.

[YOCTO #11137]

(From OE-Core rev: ab7f2a77e124d8859002619e7ba3117e8a165df7)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-01 08:14:58 +01:00
Ed Bartosh 1586663fb3 wic: set correct system id for msdos partitions
Explicitly set system id 0x6(FAT16) for msdos partitions.

Removed old code that attempts to achieve the same result
using 'parted ... lba off'.

(From OE-Core rev: 230452faf151e277bfb2b49526923f8097755b35)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-01 08:14:58 +01:00
Ed Bartosh e5959eb480 wic: set FAT 16 for msdos partitions
Used '-F 16' parameter for mkdosfs to create FAT16 partitions for
'msdos' partition type.

[YOCTO #11137]

(From OE-Core rev: b6243a03ced9a719a5801afcee014b03313cc43c)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-01 08:14:58 +01:00
Ed Bartosh 88e1d55de2 wic: support 'msdos' fstype
Added prepare_empty_partition_msdos and prepare_rootfs_msdos
methods to support 'msdos' filesystem type.

Created aliases prepare_empty_partition_vfat and prepare_rootfs_vfat
to continue supporting creation of vfat patitiions.

(From OE-Core rev: f06c507078da72f616f45effe5005cc01615a17c)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-01 08:14:57 +01:00
Ed Bartosh b4d15e0713 wic: don't silently skip unknown fstypes
Fixed wic code that loops through hard-coded list of known fstypes
to find prepare_rootfs_<fstype> or prepare_empty_partition_<fstype>
methods and silently skipping unknown fstypes.

(From OE-Core rev: ebb8fb5f81f473156c9aa4bf1965e538492a851b)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-01 08:14:57 +01:00
Ed Bartosh 54cd064c66 wic: allow only supported fstypes
Restricted possible values of --fstype to the list of
supported types. This should catch incorrect values
when .wks file is being parsed.

Removed checks for empty fstype and mentioning of
unsupported fstype 'ontrackdm6aux3'.

(From OE-Core rev: 21af89a6d44ccea6aef975ffd2483a8fad1231de)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-01 08:14:57 +01:00
Ed Bartosh ad1bce56c4 wic: remove unused code from runner module
Removed unused APIs 'outs' and 'quiet'.
Removed 'catch' parameter from runner.runtool API as wic
uses only one value of it. Removed the code that handles
unused values of 'catch' parameter.

[YOCTO #10618]

(From OE-Core rev: 1e45a4f72b16c7ab64f46907d2d2ee9cd749dc23)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-01 08:14:57 +01:00
Ed Bartosh d39a158855 wic: remove runner.show API
Replaced runner.show call to exec_cmd call in bootimg-pcbios
plugin. Removed runner.show API as it's not used anywhere else.

[YOCTO #10618]

(From OE-Core rev: 9749336c37249af99c92478c3e4dc8821cb9a816)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-01 08:14:57 +01:00
Ed Bartosh 2122dd7718 wic: use wic-tools STAGING_DATADIR as bootimg_dir
If bootloader artifacts are not found in default bootimg_dir
use wic-tools sysroot for the same purpose. This should
prevent wic from failing if bootloader artifacts can't be
found in image native sysroot.

(From OE-Core rev: 9674bbd0585fc25ccd362f233b83d07ff8f6ff53)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-01 08:14:57 +01:00
Ed Bartosh 3a53361084 wic: remove fsimage plugin
Removed fsimage plugin and prepare_rootfs_from_fs_image API as
they duplicate functionality of rawcopy plugin. Fsimage plugin makes
wic to remove the image artifact from deployment directory, which
can cause problems too.

[YOCTO #10618]

(From OE-Core rev: 9a470752f5698f791f8f78e28d163a0b9c695186)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-01 08:14:57 +01:00
Ed Bartosh 6b80c13f7a filemap: remove FilemapSeek class
FIEMAP API was added to Linux kernel 2.6.28 back in 2008
SEEK_HOLE and SEEK_DATA API was added much letter.
As FIEMAP is used by filemap module as a default API it's
safe to remove FileMpSeek class as it's never used.

[YOCTO #10618]

(From OE-Core rev: 44e9406ea6e3263d2fb95e9d534a21f74f318480)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-01 08:14:57 +01:00
Richard Purdie 8b41b5ddfe multilib_header: Update wrapper to handle arm 32/64 bit
Having arm 32/64 bit headers coexisting turns out to be tricky. Unfortunately
our wrapper works using wordsize.h and this differs on arm so we can't use it.

Therefore replicate the logic here for arm. I did look into writing our
own wordsize.h but we also need to remap kernel headers on arm and
since wordsize.h comes from libc, that doesn't work for kernel headers.

(From OE-Core rev: 141dc7136c9c62da1d30132df4b3244fe6d8898d)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-31 10:10:29 +01:00
Aníbal Limón 0dae983296 scripts/lib/compatlayer: detect_layers always use realpath's
If you are using relative paths and change to other folder for
execution it will fail, so use realpaths always.

[YOCTO #11164]

(From OE-Core rev: 14283700f8ec9dcb29cbc00c92d76173f1601bf5)

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-31 10:10:29 +01:00
Christopher Larson 87ec35603d sysroot-relativelinks: also consider links to dirs on the host
Dead symlinks, or symlinks to existing files will show up in 'files' of an
os.walk, but symlinks to existing directories show up in 'dirs', so we need to
consider both.

As one example where this is an issue, the symlink from /usr/lib/ssl/certs was
left pointing to /etc/ssl/certs rather than the relative path when the sdk was
built on hosts where the latter exists.

(From OE-Core rev: c5b522378fff13962a5187d9d09979866f805cb5)

Signed-off-by: Christopher Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-29 16:37:27 +01:00
Abdur Rehman 07171aba2b update_gio_module_cache: fix host user contamination
update_gio_module_cache intercept creates file:
$D${libdir}/gio/modules/giomodule.cache

Change ownership of this file to root:root to avoid user contamination
by host.

(From OE-Core rev: 9a23af37ad11a7176248ade88511f34fe6dd97bb)

Signed-off-by: Abdur Rehman <abdur_rehman@mentor.com>
Signed-off-by: Christopher Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-29 16:37:27 +01:00
Aníbal Limón 7effe18700 scripts/yocto-compat-layer.py: Add dependency validation in add_layer
Some layers don't have dependencies so add a validation to avoid
exception when trying to None.split().

(From OE-Core rev: 39103285029a0bb7b64dc5a305c484988b4c651a)

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-29 16:37:27 +01:00
Leonardo Sandoval e6adeac7aa recipes-kernel: Skip kernel version check on kernel templates
Currently, SRCREV is set to AUTOREV, causing mismatch between PV and the kernel
version shown in the makefile (see below to see the bitbake log). The solution
is to skip this check, suggesting to remove it once SRCREV is locked.

ERROR: linux-yocto-4.10+gitAUTOINC+01f18cba44_53be19cad6-r0.1 do_kernel_version_sanity_check: Package Version (4.10+gitAUTOINC+01f18cba44_53be19cad6) does not match of kernel being built (4.10.5). Please update the PV variable to match the kernel source or set KERNEL_VERSION_SANITY_SKIP="1" in your recipe.
ERROR: linux-yocto-4.10+gitAUTOINC+01f18cba44_53be19cad6-r0.1 do_kernel_version_sanity_check: Function failed: do_kernel_version_sanity_check (log file is located at /home/lsandov1/poky2/build/tmp/work/myqemu-poky-linux/linux-yocto/4.10+gitAUTOINC+01f18cba44_53be19cad6-r0.1/temp/log.do_kernel_version_sanity_check.26904)
ERROR: Logfile of failure stored in: /home/lsandov1/poky2/build/tmp/work/myqemu-poky-linux/linux-yocto/4.10+gitAUTOINC+01f18cba44_53be19cad6-r0.1/temp/log.do_kernel_version_sanity_check.26904
ERROR: Task (/home/lsandov1/poky2/meta/recipes-kernel/linux/linux-yocto_4.10.bb:do_kernel_version_sanity_check) failed with exit code '1

[YOCTO #11064]

(From meta-yocto rev: 79bfd911f826c6c7e26827bb200dd69e6f567b17)

Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-28 08:52:14 +01:00
Stephano Cetola 243693be54 devtool: open kconfig fragment in binary mode
When devtool writes to the kconfig fragment, it writes the output of
the diff command returned from pipe.communicate(). This function
returns binary objects. We should open the kconfig fragment file in
binary mode if we expect to write binary objects to it.

[YOCTO #11171]

(From OE-Core rev: 72bec63ab0e78753fb6ed1794d11beef9485c014)

Signed-off-by: Stephano Cetola <stephano.cetola@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-28 08:43:12 +01:00
Ed Bartosh 871363251a wic: fix generation of partition UUID
Partition UUID for msdos partitioning is based on a 4 bytes long
system identifier. Wic uses random number to generate system
identifier. For the numbers starting with 0 partition uuid is
shorter, which makes wic images non-bootable as kernel expects
8 charactest in PARTUUID parameter.

Padded system identifier with '0' when generating partition UUID
to make it always 8 characters long. This should fix the boot
issue.

(From OE-Core rev: bdaba95af2b2c9174311374436f184d2a927f6f1)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-27 11:08:34 +01:00
Leonardo Sandoval 7ddcbc5eca scripts/contrib: scripts that updates kernel templates
Updating the kernel templates (those use by the yocto-bsp script) is a mechanical
process (and prone to errors) which consists of copying latest kernel template then
applying string replacements from old to new kernel version. This script collects
these commands allowing quick updates in the future.

(From meta-yocto rev: 450313d9d8bb1e728ed5a7208decd30f50633be1)

Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-27 08:15:24 +01:00
Leonardo Sandoval 6ce9cc464a scripts/lib/bsp/substrate: kernel templates for 4.10
New set of templates that target the 4.10 kernel version.

[YOCTO #11174]

(From meta-yocto rev: 8a11b0cc2a1bfaab25200c251b301c812141ba58)

Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-27 08:15:24 +01:00
Robert Yang 4ca286d45c scripts/wic: fix typo
roofs -> rootfs

(From OE-Core rev: 131629ca6238ea0596f5dd11d659ce2ca63067a4)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-27 08:15:07 +01:00
Markus Lehtonen 5e3eaf7c57 build-perf-test-wrapper.sh: store buildstats under git-notes
In order to dramatically reduce the data footprint of the result
branches. This makes cloning/pulling result repositories a lot faster.
Buildstats can still be accessed, when needed by doing
git fetch refs/notes/buildstats/*:refs/notes/buildstats/*

[YOCTO #10582]

(From OE-Core rev: 5b52bbf09bb2fbec9d2d54f3c3c46c74a09fbfd0)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-27 08:15:06 +01:00
Markus Lehtonen 746b6fbe68 scripts/oe-git-archive: implement --notes
Option for adding git-notes to the commit.

[YOCTO #10582]

(From OE-Core rev: 0ef7c143262a441c38235ea71832ca7714ce4a35)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-27 08:15:06 +01:00
Markus Lehtonen e10d1ea082 scripts/oe-git-archive: implement --exclude
May be used for excluding certain files from the commit.

[YOCTO #10582]

(From OE-Core rev: fd125cf694bebefbe9a98fd1bb199d6ca472dad5)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-27 08:15:06 +01:00
Markus Lehtonen 60f32a00dc scripts/oe-git-archive: fix pushing
Git arguments were badly laid out.

(From OE-Core rev: cea9c1380891a85f686cb3b5cb42b6166f38e8d5)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-27 08:15:06 +01:00
Markus Lehtonen 7132f54041 oe-build-perf-test: pack all buildstat in one file
Write out all buildstats into one big json file, instead of using
multiple per-measurement files. Individual buildstats will be indexed
using "<test_name>.<measurement_name>" as the key. Also, changes the
per-testcase working directories into temporary directories that will be
removed after test execution as there are no more per-testcase data files
to store permanently.

[YOCTO #10582]

(From OE-Core rev: a7f2e8915db379021f3409ca640de5d3b054a830)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-27 08:15:06 +01:00
Ed Bartosh cc970c23a7 wic: use kernel_dir to find systemd-efi bootloader
Fixed test failure caused by using bootimg_dir path
instead of kernel_dir to find systemd-efi bootloader:
FAIL: test_systemd_bootdisk (oeqa.selftest.wic.Wic)
...
FileNotFoundError: [Errno 2] No such file or directory:
'tmp/work/qemux86_64-poky-linux/core-image-minimal/1.0-r0/recipe-sysroot/usr/share'

(From OE-Core rev: 552bd782d9ffe275b8fc7d7cb0b9aaceb51888c3)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-24 23:43:33 +00:00
Chen Qi 11b81e3fba runqemu-gen-tapdevs: fix file path in example
Fix the path in example so that '/usr/bin/tunctl' could be found under
this path.

(From OE-Core rev: d1b8e1460b25018ac8f65d3647bc736f3393ed3e)

Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-24 23:43:32 +00:00
Ed Bartosh fa10b24950 wic: improve getting syslinux path
Used wic-tools STAGING_DATADIR if syslinux can't be found
in default bootimg_dir.

(From OE-Core rev: 79a935cfc86ffce6f4b4f328b90337de36ba6dbb)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-23 13:19:49 +00:00
Ed Bartosh 5c04e37140 wic: remove empty plugin methods
Removed do_install_disk and do_configure_partition methods from
bootimage-partition, fsimage and rawcopy plugins as they're empy
and not used.

[YOCTO #10618]

(From OE-Core rev: d58c8ef2836418056f776a9586e0bb0d33afb788)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-23 13:19:49 +00:00
Ed Bartosh 3ea735d390 wic: use STAGING_DATADIR as a default for bootimg_dir
Default value for bootimg_dir was not set in main wic script
unlike the rest of artifacts: kernel_dir, roofts_dir and
native_sysroot. Set it to the value of STAGING_DATADIR for
consistency and to avoid confusion of wic plugin developers.

(From OE-Core rev: afc486b6316f1118c8dcc74a5e217bb217dd1f85)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-23 13:19:49 +00:00
Ed Bartosh b78c564cca wic: fix bug in sparse_copy
sparse_copy creates output file that is smaller than input
file when input file ends with unmapped blocks.

Used truncate(<input file size>) when output file is created
to ensure the size of output file will always be equal to
the size of input file.

(From OE-Core rev: 567186f995302a095a771baede4ff5034d1d1862)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-23 13:19:49 +00:00
Ed Bartosh 627952a234 wic: use kernel_dir instead of bootimg_dir
bootimg_dir is usually set to the value of STAGING_DATADIR and
kernel_dir - to the value of DEPLOY_DIR_IMAGE, so usage of
kernel_dir is more logical in bootimg-efi, bootimg-partition and
rawcopy plugins.

Replaced usage of bootimg_dir to kernel_dir in 3 above mentioned
plugins that use DEPLOY_DIR_IMAGE as a default artifact location.

(From OE-Core rev: 48a5d67d7cafdeac821e6f35bea380521ef017d5)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-23 13:19:49 +00:00
Ed Bartosh ce627abf86 wic: don't set creator.bootimg_dir in plugins
Changing bootimg_dir in plugins is a dangerous hack as this
value comes from wic command line and can be used by multiple
plugins.

(From OE-Core rev: 2457ea5338f7309316b474562b4723e8cb09286d)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-23 13:19:49 +00:00
Ed Bartosh d38d526991 wic: remove rootfs_pcbios_ext plugin
Removed broken unused plugin.

(From OE-Core rev: 6d162118a2567ccbcdbc2a9ca14a8c358fa1a055)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-23 13:19:49 +00:00
Aníbal Limón f57eac5dc9 scripts/yocto-compat-layer.py: Handle layer dependencies when test
If some layer depends on other tries to find layer dependency, if the
layer dependency isn't found avoid to test the layer and notice the
user.

(From OE-Core rev: 1e7cf9bb71521f1632dd2e6b01fe7fcc95732983)

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-22 11:35:22 +00:00
Aníbal Limón 4703aa2b3b scripts/yocto-compat-layer.py: Add option to disable layer autodiscovery
Sometimes there is a need to only analyze the layer specified by the
command line, the new option -n will disable autodiscovery of layers
and only will try to test specified layers.

(From OE-Core rev: f2f6f0c938226802163698ef14a8a9103da362a0)

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-22 11:35:22 +00:00
Aníbal Limón 995cb88233 scripts/lib/compatlayer: Remove require of meta- in layer dir name
The layers isn't required to have a dirctory name start with meta-
so remove the validation.

(From OE-Core rev: 576c6486f547b1d7422cdd12f688aef74ee632ae)

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-22 11:35:22 +00:00
Aníbal Limón 6786f662cd scripts/compatlayer: Add exclude of meta-world-pkgdata on get_signatures
The meta-world-pkgdata recipe can be modified when a layer is added
may be can add recipes to world target, so exclude by default.

[YOCTO #11162]

(From OE-Core rev: cde30d70b04eb66bff66a351ddf733022e98ddab)

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-22 11:35:22 +00:00
Aníbal Limón 07dd071bf6 scripts/yocto-compat-layer.py: Make output log argument optional
Only create a log file when --output-log option is specified, since
logger is dumping to stdout by default is better to let the user
decide if a log needs to be created.

[YOCTO #11160]

(From OE-Core rev: f91ccdeb8b0b3e4063ed2bf22215a25f8902cbd9)

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-22 11:35:22 +00:00
Aníbal Limón 1e3ef54408 scripts/yocto-compat-layer.py: Dump log to stdout instead of stderr
The common unix tools uses stdout as standard for log output, by default
python logging uses stderr if not stream is specified.

[YOCTO #11160]

(From OE-Core rev: 95ad04f7daed17bde5be5fc264f6c731fecfdfa9)

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-22 11:35:21 +00:00
Aníbal Limón 15e65f5f07 scriptutils: Add support for specify stream on logger_create
It is a good idea to let the script to choose what stream wants
to dump the logging output.

[YOCTO #11160]

(From OE-Core rev: 60c9b06f4085f6bddb6aa397a38bfc2ceb10f964)

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-22 11:35:21 +00:00
Daniel Schultz 898de507d3 wic: partition.py: Add fsck to avoid corrupt EXT file systems
This patch avoids the creation of a corrupt EXT file system.

Since there are no checks if a EXT file system was successfully created,
this should add to prevent possible system failures.

(From OE-Core rev: 95ac3d0637cad7e12f246b8bd64ab64348df0483)

Signed-off-by: Daniel Schultz <d.schultz@phytec.de>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-22 11:35:21 +00:00
brian avery ab34d08772 gen-lockedsig-cache: catch os.link error
We do a hard link to speed up sdk creation but if your sstate-cache is
across a file system boundary, this tries and fails. This patch catches
that error and does a copy instead.

(From OE-Core rev: fb9fdd7a74917cdcab039aa3a9a9944b18246fea)

Signed-off-by: brian avery <brian.avery@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-22 11:35:21 +00:00
Paul Eggleton 995b02b00c devtool: tidy up handling of parse failures
Since the tinfoil2 refactoring, if an error occurred during parsing, we
were showing a traceback and not correctly exiting (since we weren't
calling shutdown()). Fix both of these issues.

(From OE-Core rev: 18304036e1b513fd12c049dbf549ba75c503ed84)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-22 11:35:21 +00:00
Kristian Amlie 76d182c757 wic/partionedfs: Avoid reserving space for non-existing ext. partition
We don't need the gap that the extended partition occupies if we
already know that we have less than five partitions. Saves up to one
full alignment of space.

(From OE-Core rev: 485315dc170e29962a8848db38db73abafd0586e)

Signed-off-by: Kristian Amlie <kristian.amlie@mender.io>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-22 11:35:20 +00:00
Anders Darander 6fa0fda116 devtool/recipetill: npm install of devDependencies
Web applications built using e.g. angular2, usually requires that the
packages in devDependencies are available.

Thus, add an option '--fetch-dev' to both devtool add and recipetool, to
add npm packages in devDependencies to DEPENDS.

(From OE-Core rev: f246f820d53b459596fde6758a09f7a0d7db7c4c)

Signed-off-by: Anders Darander <anders@chargestorm.se>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-22 11:35:20 +00:00
Ed Bartosh 4a57a7bc8d runqemu: output network configuration
runqemu adds network configuration parameters to the kernel
command line to configure guest networking. This works only
for the images that run with external kernel using qemu -kernel
parameter. It doesn't work for the images that use bootloader
to boot kernel as -kernel parameter is not used and network
configuration is not possible to get.

Added host and guest ip addresses and netmask of tap link
to the runqemu output. This should allow external programs
that execute runqemu to get network configuration.

[YOCTO #10833]

(From OE-Core rev: cf66a1850677548aa63a54276fa4917f40259daf)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-22 11:35:19 +00:00
Robert Yang 766dccf975 runqemu: only boot ramfs when specified
This can fix a problem:
IMAGE_FSTYPES += "iso"
$ bitbake core-image-minimal
$ runqemu qemux86

It may boot core-image-minimal-initramfs rather than core-image-minimal, this
is not what we want usually. This patch makes it avoid booting ramfs when there
are other choices, or when it is specified, for example, "runqemu qemux86 ramfs"

(From OE-Core rev: 614bde6774f4dfd414066bbaf75ed422943e37ab)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-17 16:53:04 +00:00
Robert Yang 0079607387 runqemu: add -h and --help
Fixed:
$ runqemu -h
runqemu - INFO - Assuming MACHINE = -h
runqemu - INFO - Running MACHINE=-h bitbake -e...
[snip]
Exception: FSTYPE is NULL!

[YOCTO #10941]

(From OE-Core rev: 6b9dd7a589537b12da648be50298cf7d36461797)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-16 22:11:33 +00:00
Robert Yang ac451ea84c runqemu: improve when no machine specified
Fixed:
$ runqemu core-image-minimal
[snip]
Exception: FSTYPE is NULL!
[snip]

Get DEPLOY_DIR_IMAGE from "bitbake -e" to make it work.

[YOCTO #10471]

(From OE-Core rev: ca551b72a020782f164703765b97156000b908d2)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-16 22:11:32 +00:00
Juro Bystricky 68e57fa748 runqemu: independent network and rootfs setup
Presently, runqemu sets up rootfs as part of network setup.
In case there is no network desired we will end up without rootfs
as well.
This patch sets up network and rootfs independently.
It is also possible to bypass setup of rootfs if QB_ROOTFS is set to "none".

(From OE-Core rev: 006ab8c6bcfe9d065c215cab15289357cefc9259)

Signed-off-by: Juro Bystricky <juro.bystricky@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-16 22:11:32 +00:00
Patrick Ohly ef0c54ab70 yocto-compat-layer: improve error handling in signature creation
When "bitbake -k -S none world" failed, the error printed by
yocto-compat-layer.py contained the stack trace multiple times and did not
contain the stderr output from bitbake, making the error hard to understand
and debug:

  INFO: ======================================================================
  INFO: ERROR: test_signatures (common.CommonCompatLayer)
  INFO: ----------------------------------------------------------------------
  INFO: Traceback (most recent call last):
    File "/fast/work/poky/scripts/lib/compatlayer/__init__.py", line 144, in get_signatures
      stderr=subprocess.PIPE)
    File "/usr/lib/python3.4/subprocess.py", line 620, in check_output
      raise CalledProcessError(retcode, process.args, output=output)
  subprocess.CalledProcessError: Command 'bitbake -k -S none world' returned non-zero exit status 1

  During handling of the above exception, another exception occurred:

  Traceback (most recent call last):
    File "/fast/work/poky/scripts/lib/compatlayer/cases/common.py", line 51, in test_signatures
      curr_sigs = get_signatures(self.td['builddir'], failsafe=True)
    File "/fast/work/poky/scripts/lib/compatlayer/__init__.py", line 149, in get_signatures
      raise RuntimeError(msg)
  RuntimeError: Traceback (most recent call last):
    File "/fast/work/poky/scripts/lib/compatlayer/__init__.py", line 144, in get_signatures
      stderr=subprocess.PIPE)
    File "/usr/lib/python3.4/subprocess.py", line 620, in check_output
      raise CalledProcessError(retcode, process.args, output=output)
  subprocess.CalledProcessError: Command 'bitbake -k -S none world' returned non-zero exit status 1

  Loading cache...done.
  Loaded 1328 entries from dependency cache.
  NOTE: Resolving any missing task queue dependencies
  NOTE: Runtime target 'zlib-qat' is unbuildable, removing...
  Missing or unbuildable dependency chain was: ['zlib-qat']
  ...
  Summary: There were 5 ERROR messages shown, returning a non-zero exit code.

The yocto-compat-layer.log was incomplete, it only had the first part
without the command output.

stderr was missing due to stderr=subprocess.PIPE.

Instead of the complicated try/except construct it's better to check
the return code ourselves and raise just a single exception. The
output (both on stderr and in the yocto-compat-layer.log) now is:

  INFO: ======================================================================
  INFO: ERROR: test_signatures (common.CommonCompatLayer)
  INFO: ----------------------------------------------------------------------
  INFO: Traceback (most recent call last):
    File "/fast/work/poky/scripts/lib/compatlayer/cases/common.py", line 51, in test_signatures
      curr_sigs = get_signatures(self.td['builddir'], failsafe=True)
    File "/fast/work/poky/scripts/lib/compatlayer/__init__.py", line 147, in get_signatures
      raise RuntimeError(msg)
  RuntimeError: Generating signatures failed. This might be due to some parse error and/or general layer incompatibilities.
  Command: bitbake -k -S none world
  Output:
  Loading cache...done.
  Loaded 1328 entries from dependency cache.
  NOTE: Resolving any missing task queue dependencies
  ERROR: Nothing PROVIDES 'qat16' (but /fast/work/meta-intel/common/recipes-extended/openssl-qat/openssl-qat_0.4.9-009.bb DEPENDS on or otherwise requires it)
  ERROR: qat16 was skipped: incompatible with machine qemux86 (not in COMPATIBLE_MACHINE)
  ...
  Missing or unbuildable dependency chain was: ['openssl-qat-dev']
  ...
  Summary: There were 5 ERROR messages shown, returning a non-zero exit code.

(From OE-Core rev: 5b9ac62ab535d2791b9713857e1016f49f53dd8d)

Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-16 22:11:32 +00:00
Alexander Kanavin ba6106a3e2 scripts/oe-setup-rpmrepo: replace createrepo with createrepo_c
(From OE-Core rev: bab25f09d34a32301e483ff0700add1209809b14)

Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-14 14:42:17 +00:00
Alexander Kanavin aa17362d33 scripts/rpm2cpio.sh: replace 5.x version with 4.x version
(From OE-Core rev: a7da1aade118d1ccf1b286f82556cd9f706bd2a4)

Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-14 14:42:17 +00:00
Kristian Amlie 0cfcdf2e95 wic/direct.py: Avoid exception if using multiple rawcopy/no-table entries.
If we are both having a bootloader and a U-Boot environment file, we
can end up with two entries using "--source rawcopy" and "--no-table",
and since they reuse the same file [1], their cleanup handlers will
try to delete the same file twice. So make sure we only do it once.

[1] Although they reuse the same file, the resulting output is
correct, so it appears the file is accessed in properly sequential
order.

(From OE-Core rev: f1a3eac376c4600cdb128d870ad9b7e9d51ed9c0)

Signed-off-by: Kristian Amlie <kristian.amlie@mender.io>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-11 16:09:14 +00:00
Richard Purdie 0efe58df2e runqemu-gen-tapdevs: Improve help text with an example
Figuring how the correct commandline isn't trivial, improve the help
text with RSS in mind.

(From OE-Core rev: 056a9da9f3ac2bc175f19243b11864ca90eee28b)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-10 14:50:11 +00:00
Daniel Schultz 8dddce21c4 wic: filemap: Fixed spared_copy skip
This patches removes the empty space in front of the copied file which
was skipped. Without this reduction it's not possible to place a
partition with rawcopy and skip parameter on a desired alignment.

(From OE-Core rev: 5c024d71f9413b81ee1707dbc41f0721f8f27bdb)

Signed-off-by: Daniel Schultz <d.schultz@phytec.de>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-10 14:50:10 +00:00
Daniel Schultz 52de84763d wic: plugins: rawcopy: Fixed wrong variable type
Without the int() function this variable will be a string. This will led
to a error in Filemap on line 545 due wrong types.

> [...]
>   File
> ".../poky/scripts/lib/wic/filemap.py", line 545, in sparse_copy
>     if start < skip < end:
> TypeError: unorderable types: int() < str()

(From OE-Core rev: 46b5814bcdc0e7e3cb293e877e2aa949baf5fef8)

Signed-off-by: Daniel Schultz <d.schultz@phytec.de>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-10 14:50:10 +00:00
Nikunj Kela d49dba9a6a relocate_sdk.py: skip debug files from relocation
Debug files only have debug symbols hence don't need
to be relocated. Relocation script throws errors when
run on the debug files. This change skips these files
that have zero size.

(From OE-Core rev: 132e8bfd499c713eb63075fd6380317b60f0bd27)

(From OE-Core rev: 93b73b2495f9cb18741837c5437de629adfd3780)

Signed-off-by: Nikunj Kela <nkela@cisco.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-08 11:52:57 +00:00