Commit Graph

2221 Commits

Author SHA1 Message Date
Joshua Lock c0f2890c01 scripts: remove True option to getVar calls
getVar() now defaults to expanding by default, thus remove the True
option from getVar() calls with a regex search and replace.

Search made with the following regex: getVar ?\(( ?[^,()]*), True\)

(From OE-Core rev: 0a36bd96e6b29fd99a296efc358ca3e9fb5af735)

Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-16 10:23:23 +00:00
Mariano Lopez 3f6e6f4e69 scripts/runqemu: Allow to use qemu from host.
This will add support to use qemu from the running host,
with this is possible to put qemu-native in ASSUME_PROVIDED
variable.

By default it will try to get qemu from the build sysroot,
and only if it fails will try to use the host's qemu.

(From OE-Core rev: fe7fd2cd3a9c4fb5b31bd3cab81c96a3b81cb540)

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-16 10:23:23 +00:00
Paul Eggleton 229091babb devtool: prevent BBHandledException from showing traceback
If we don't catch this then attempting to run devtool in non-memres mode
when bitbake is already running will produce a traceback instead of just
an error message.

(From OE-Core rev: e01b75dff599ffa2b66e6608b28bbb3564365eee)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-14 12:30:50 +00:00
Paul Eggleton 4448b2a122 devtool: extract: disable basehash mismatch errors
Using the setVariable commands here followed by buildFile will result in
"basehash mismatch" errors, and that's expected since we are deviating
*at runtime* from what was previously seen by changing these variable
values. Set BB_HASH_IGNORE_MISMATCH to turn off the errors.

(From OE-Core rev: b0169796f294bbec0397b7eae86454a46b68cdc5)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-14 12:30:50 +00:00
Paul Eggleton 85de43fccd devtool: fix extraction of source to work in memres mode
Extracting the source for a recipe (as used by devtool's extract, modify
and upgrade subcommands) requires us to run do_fetch, do_unpack,
do_patch and any tasks that the recipe has inserted inbetween, and do so
with a modified datastore primarily so that we can redirect WORKDIR and
STAMPS_DIR in order to have the files written out to a place of our
choosing and avoid stamping the tasks as having executed in a real build
context respectively. However, this all gets much more difficult when in
memres mode since we can't call internal functions such as
bb.build.exec_func() directly - instead we need to execute the tasks on
the server. To do this we use the buildFile command which already exists
for the purpose of supporting bitbake -b, and setVariable commands to
set up the appropriate datastore.

(I did look at passing the modified datastore to the buildFile command
instead of using setVar() on the main datastore, however its use of
databuilder makes that very difficult, and we'd also need a different
method of getting the changes in the datastore over to the worker as
well.)

(From OE-Core rev: eb63b5339014fc72ba4829714e0a96a98e135ee2)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-14 12:30:50 +00:00
Paul Eggleton 86b55e6026 recipetool: add OE lib path
The autotools code imports oe.package; we weren't experiencing a problem
with this probably due to OE itself adding that path previously.

(From OE-Core rev: a61d7bf8447b2d2c65eb34315c86086ff35c8bc9)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-14 12:30:50 +00:00
Paul Eggleton 45adbe3709 classes/patch: move in logic to commit for additional tasks
If PATCHTOOL is "git", and PATCH_COMMIT_FUNCTIONS is set to "1", for
additional tasks between do_unpack and do_patch, make a git commit. This
logic was previously implemented in devtool itself, but it makes more
sense for it to be implemented in the patch class since that's where the
rest of the logic is for this (or in lib/oe/patch.py). It also makes
it possible for this to work with tinfoil2.

(From OE-Core rev: f24f59ea1d8bc335ea8576f6a346d0935f4a3548)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-14 12:30:49 +00:00
Paul Eggleton 70ce595237 devtool / recipetool: use tinfoil parsing API
Use Tinfoil.parse_recipe_file() and Tinfoil.parse_recipe() instead of
the recipeutils equivalents, and replace any local duplicate
implementations. This not only tidies up the code but also allows these
calls to work in memres mode.

(From OE-Core rev: f13b56266ee96dfab65a3a7db50e8051aa9f071a)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-14 12:30:49 +00:00
Paul Eggleton 1e29b57e92 devtool: package: don't try to initialise tinfoil twice
setup_tinfoil() already calls prepare(), we don't need to call it again
ourselves and doing so with tinfoil2 results in "ERROR: Only one copy of
bitbake should be run against a build directory". Calling prepare()
twice should probably still be allowed, so that ought to be fixed
separately, but in the mean time this code is still wrong so fix it
here.

(From OE-Core rev: 38b8a7d4aff096ea0a62f2ddf3fe2de1df591bf5)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-14 12:30:49 +00:00
Ola x Nilsson d65886bb69 oe-pkgdata-util: Make read-value handle override variables
Some variables in pkgdata files have a package-name override.  When
the bare variable can not be found, try with the override-variant.

PKGSIZE is one such variable, and already had special code to handle this.

Test included.

(From OE-Core rev: 6df99cda894033cba68bc6ab91e47f67e0d788a5)

Signed-off-by: Ola x Nilsson <ola.x.nilsson@axis.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-14 12:30:49 +00:00
Ed Bartosh a5438f8f74 oe-buildenv-internal: show usage output
Show usage text if script is not sourced.
Tested in bash, zsh and dash.

[YOCTO #10751]

(From OE-Core rev: ac7a905b18acb8bd9b2412b6682afbe1d7e18d7b)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-13 22:55:21 +00:00
Ed Bartosh 556bdb6642 oe-find-native-sysroot: create usage output
Created usage output for oe-find-native-sysroot script.

[YOCTO #10751]

(From OE-Core rev: 8ddfc48c7f3e2ca45c035cec492fdc31c6ad484f)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-13 22:55:21 +00:00
Ed Bartosh 0ebe2e64ba oe-git-proxy: create usage output
Created usage output for oe-git-proxy script.

[YOCTO #10751]

(From OE-Core rev: ec0fdb5e896fc20dbafcc8ae507b17c011dc56fd)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-13 22:55:21 +00:00
Ed Bartosh ca916f279f oepydevshell-internal.py: standardize usage output
Made usage output of oepydevshell-internal.py to look
similar to the output of other oe scripts.

[YOCTO #10751]

(From OE-Core rev: e6480af22a7a12c655efed14f8f1aea658f26b1c)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-13 22:55:21 +00:00
Ed Bartosh 8351dc41a4 oe-setup-builddir: create usage output
Created usage output for oe-setup-builddir script.

[YOCTO #10751]

(From OE-Core rev: 77606455df7d45fd014c3603e1cf1b24efd37695)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-13 22:55:20 +00:00
Ed Bartosh 78087eba73 oe-setup-rpmrepo: standardize usage output
Made usage output of oe-setup-rpmrepo to look similar to the
output of other oe scripts.

[YOCTO #10751]

(From OE-Core rev: 5423c9a412c680b781417a64b412838845b5d075)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-13 22:55:20 +00:00
Ed Bartosh 3c3f5c5160 oe-trim-schemas: create usage output
Created usage output for oe-trim-schemas script.

[YOCTO #10751]

(From OE-Core rev: a9fac12157ed434a6711de1af77eda1f4a8e9e8a)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-13 22:55:20 +00:00
Ed Bartosh e8fbd7158b oe-run-native: standardize usage output
Made usage output of oe-run-native to look similar to the
output of other oe scripts.

[YOCTO #10751]

(From OE-Core rev: e1c96125ea674509fbc9b36dc671b7a53bd848ac)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-13 22:55:20 +00:00
Ola x Nilsson 8dfc417780 recipetool: Load plugins in a well defined order
To allow recipetool plugins in one layer to shadow another in a well
defined way, first search BBPATH/lib/recipetool directories and then
scripts/lib/recipetool and load only the first found.

The previous search and load loop would load all found plugins with the
ones found later replacing any found before.

(From OE-Core rev: 7a8726a3662a3909dc6ef6e8d1029d0b1aa938c3)

Signed-off-by: Ola x Nilsson <ola.x.nilsson@axis.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-13 22:55:20 +00:00
Ola x Nilsson 14abe7982c devtool: Load plugins in a well defined order
To allow devtool plugins in one layer to shadow another in a well
defined way, first search BBPATH/lib/devtool directories and then
scripts/lib/devool and load only the first found.

The previous search and load loop would load all found plugins with the
ones found later replacing any found before.

(From OE-Core rev: 1b2b8a0a80de17ea053002fdd124055d2798029a)

Signed-off-by: Ola x Nilsson <ola.x.nilsson@axis.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-13 22:55:20 +00:00
Ed Bartosh 07dcfef1ea wic: rename command line option -p -> -s
Short variant of wic command line option --skip-build-check
is incorretly named -p. It's named -s in wic help and Yocto
documentation.

(From OE-Core rev: 8d4121459334349f10bfb0f4cda00a8461aab245)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-13 22:55:19 +00:00
Alessio Igor Bogani 8c1c43b790 wic: Create a logical partition only when it is really mandatory
Don't worth bother with logical partition on MBR partition type (aka
msdos) if disk image generated by wic should have 4 partitions.

(From OE-Core rev: 36a558fbdc96094626e7de1a3510691e30885368)

Signed-off-by: Alessio Igor Bogani <alessio.bogani@elettra.eu>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-13 22:55:19 +00:00
Mariano Lopez dc721f7590 oe-selftest: Add option to submit test result to a git repository.
This new option allows to commit the result to a git repository,
along with the results it will add a metadata file for information
of the current selftest run, such as: hostname, machine, distro,
distro version, host version, and layers.

This implementation will have a branch per different hostname,
testing branch, and machine.

To use this feature use:

oe-selftest <options> --repository <repository_link>

[YOCTO #9954]

(From OE-Core rev: 758e4b7e619f9aaa79e3910b5539ff8fb2d1064a)

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-13 22:55:19 +00:00
Paul Eggleton 8fe1f25ea0 recipetool: fix encoding-related errors creating python recipes
Yet another instance of us expecting a string back from subprocess when
in Python 3 what you get back is bytes. Just decode the output within
run_command() so we avoid this everywhere.

(From OE-Core rev: 103faae78cdff5280c7b7cdb7ca01e0868d02ec9)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-08 10:31:29 +00:00
Patrick Ohly b8a4eb5265 pybootchartgui: support reading reduced /proc logs
Pre-processing /proc data during the build considerably reduces the
amount of data written to disk: 176KB instead of 4.7MB for a 20
minuted build. Parsing also becomes faster.

buildstats.bbclass only writes the reduced logs now, but support for
the full /proc files is kept around as reference.

(From OE-Core rev: b5e47df9af1ebbb477074587fdeae17eb2f55582)

Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-07 10:38:00 +00:00
Patrick Ohly 820c042b36 pybootchartgui: simplify drawing of memory usage
The internal representation after parsing now matches exactly
what the drawing code needs, thus speeding up drawing a bit.
However, the main motivation is to store exactly that required
information in a more compact file.

(From OE-Core rev: ca06e67a0bb5820b38fda4c8dfee20764c1e59ae)

Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-07 10:37:59 +00:00
Patrick Ohly 6b5037bf2c pybootchartgui: render disk space usage
This adds a new, separate chart showing the amount of disk space used
over time for each volume monitored during the build. The hight of the
graph entries represents the delta between current usage and minimal
usage during the build.

That's more useful than showing just the current usage, because then a
graph showing changes in the order of MBs in a volume that is several
GB large would be just flat.

The legend shows the maximum of those deltas, i.e. maximum amount of
space needed for the build. Minor caveat: sampling of disk space usage
starts a bit later than the initial task, so the displayed value may
be slightly lower than the actual amount of space needed because
sampling does not record the actual initial state.

(From OE-Core rev: 263d189d066b578debf08b2bd07494a69b70f70d)

Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-07 10:37:59 +00:00
Patrick Ohly 0cd48fcef4 pybootchartgui/parsing.py: fix error handling in meminfo parser
When matching fails, m.group(0) is invalid and can't be used in the
error message.

(From OE-Core rev: ddfea21e06a2e6b1a1e766969f8c134a6de7388a)

Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-07 10:37:59 +00:00
Patrick Ohly 1416bb3244 pybootchartgui/draw.py: skip empty CPU and disk usage charts
The only real change is the addition of two if checks that skips the
corresponding drawing code when there is no data.

(From OE-Core rev: 1658fd5e9ca1ba793cae604c2a395d54e3ec9056)

Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-07 10:37:59 +00:00
Patrick Ohly 521887ea61 pybootchartgui: show system utilization
This enables rendering of the original bootchart charts for CPU, disk
and memory usage. It depends on the /proc samples recorded by the
updated buildstats.bbclass. Currently, empty charts CPU and disk usage
charts are drawn if that data is not present; the memory chart already
gets skipped when there's no data, which will also have to be added
for the other two.

(From OE-Core rev: 233d3e50b361feea07803a9c0f2a691e687c6cd5)

Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-07 10:37:59 +00:00
Patrick Ohly a2c2092195 pybootchartgui/draw.py: fix drawing of samples not starting at zero
The code did not handle x scaling correctly when drawing starts at
some time larger than zero, i.e. it worked for normal bootchart data,
but not for the system statistics recorded by buildstats.bbclass.

(From OE-Core rev: 166f8f9aaa1f01fc6d6a5451f8f06b815c51ffae)

Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-07 10:37:59 +00:00
Patrick Ohly 7b8ffa18fd pybootchartgui/draw.py: allow moving process chart up and down
Substracting curr_y when determining the hight of the process chart is
wrong because the height is independent of the position where the
chart is about to be drawn. It happens to work at the moment because
curr_y is always 10 when render_processes_chart() gets called. But it
leads to a negative height when other charts are drawn above it, and
then the grid gets drawn on top of those other charts.

Substracting some constant is relevant because otherwise the box is
slightly larger than the process bars. Not sure exactly where that
comes from (text height?); leg_s seems a suitable constant and happens
to be 10, so everything still gets rendered exactly as before.

(From OE-Core rev: b6bb690728c329ae448f89a1b68298c6dd8a573a)

Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-07 10:37:59 +00:00
Patrick Ohly 6d373fdf7b scripts/send-pull-request: Avoid multiple chain headers
When creating a patch set with cover letter using the
send-pull-request script, both the "In-Reply-To" and "References"
headers are appended twice in patch 2 and subsequent.

That's because git-format-patch already inserted them and then
git-send-email repeats that. Suppressing mail threading in
git-send-email with --no-thread avoids the problem and is the
right solution because it works regardless whether git-send-email is
called once or twicee.

Repeating these headers is a violation of RFC 2822 and can confuse
mail programs. For example, Patchwork does not detect a patch series
problem when there are these extra headers.

[YOCTO #10718]

(From OE-Core rev: 303a1aa3df43eb0b693d8602062fa33c4a08fdd6)

Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-07 10:37:59 +00:00
Leonardo Sandoval 70f99b427b bb-perf: plot histograms base on buildstats data
Scripts that produces script data to be consumed by gnuplot.
There are two possible plots depending if either the
-S parameter is present or not:

    * without -S: Produces a histogram listing top N recipes/tasks versus
      stats. The first stat defined in the -s parameter is the one taken
      into account for ranking
    * -S: Produces a histogram listing tasks versus stats.  In this case,
      the value of each stat is the sum for that particular stat in all recipes found.
      Stats values  are in descending order defined by the first stat defined on -s

EXAMPLES

1. Top recipes' tasks taking into account utime

    $ buildstats-plot.sh -s utime | gnuplot -p

2. Tasks versus utime:stime

    $ buildstats-plot.sh -s utime:stime -S | gnuplot -p

3. Tasks versus IO write_bytes:IO read_bytes

    $ buildstats-plot.sh -s 'IO write_bytes:IO read_bytes' -S | gnuplot -p

(From OE-Core rev: 74408c19fba89de54c093fccf65b3a072d6a197b)

Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-11-30 15:48:05 +00:00
Dominic Sacré 3f2ab3f7cc python3: Build and package precompiled modules
Remove the patch that was applied in the python3 and python3-native
recipes to skip compilation of python modules.

Modify generate-manifest-3.5.py to match '__pycache__' directories in
FILES_*.
This is necessary because Python3 puts .pyc files in '__pycache__'
subdirectories one level below the corresponding .py files, whereas in
Python2 they used to be right next to the sources.

This change significantly reduces the startup overhead of Python3
scripts. For example, on a Cortex-A9, "python3 -c pass" took 0.40s
before, and 0.19s after.

(From OE-Core rev: bb4d689769703177dbb0df0935e15016b879f42b)

Signed-off-by: Dominic Sacré <dominic.sacre@gmx.de>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-11-30 15:48:05 +00:00
Jiajie Hu bfd8c35c3f devtool: fix handling of unicode characters from subprocess stdout
In previous implementation, a UnicodeDecodeError exception will be
raised if multi-byte encoded characters are printed by the subprocess.
As an example, the following command will fail in an en_US.UTF-8
environment because wget quotes its saving destination with '‘'(0xE2
0x80 0x98), while just the first byte is provided for decoding:

    devtool add recipe http://example.com/source.tar.xz

The patch fixes the issue by avoiding such kind of incomplete decoding.

(From OE-Core rev: 1875ea92546d23abcab1b40b562477a0016f712d)

Signed-off-by: Jiajie Hu <jiajie.hu@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-11-23 11:10:15 +00:00
Leonardo Sandoval 4bc5353c92 scripts: Specify the stats to take into account
There are many more stats on buildstats that 'Elapsed time', so make the script
more flexible to support all stats. Some cmd line examples:

$ buildstats.sh -s 'utime'

Buildstats' data covers proc's stats in different areas, including CPU times,
IO, program system resources and child program system resources. In order
to print values on each of these sets from command line, one can use the
following:

$ buildstats.sh -H -s 'TIME' | less

$ buildstats.sh -H -s 'IO' | less

and 'RUSAGE' and 'CHILD_RUSAGE' for program and program's child system
resources.

(From OE-Core rev: 81479b191287ccbf4cf94fa2d0ad46813091bca1)

Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-11-23 11:10:14 +00:00
Paul Eggleton 6e182aaa76 devtool: modify: support recipes with only local files as source
The hello-mod recipe is unusual in that it has only local files in
SRC_URI and builds these out of ${WORKDIR}. When you use devtool modify
on it, devtool puts all of those files in an "oe-local-files"
subdirectory of the source tree, which is not ${S} (or ${B}) any more
and thus building the recipe afterwards fails. It's a bit of a hack, but
symlink the files in oe-local-files into the source tree (and commit the
symlinks with an ignored commit so that the repo is clean) to work
around the problem. We only do this at time of extraction, so any files
added to or removed from oe-local-files after that won't be handled, but
I think there's a limit to how far we should go to support these kinds
of recipes - ultimately they are anomalies.

I initially tried a hacky workaround where I set effectively set B =
"${WORKDIR}" and that allowed it to build, but other things such as the
LIC_FILES_CHKSUM checks still broke because they expected to find files
in ${S}. Another hack where I set the sourcetree to point to the
oe-local-files subdirectory works for hello-mod but not for makedevs
since whilst that is similar, unlike hello-mod it does in fact have
files in the source tree (since it has a patch that adds COPYING) and
thus the same issue occurred.

Also tweak one of the tests that tries devtool modify / update-recipe on
the makedevs recipe to try building it since that would have caught this
issue.

Fixes [YOCTO #10616].

(From OE-Core rev: 857c06d6a1d161bf5a01311d07758bd4241929a3)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-11-23 11:10:14 +00:00
Stephano Cetola 8b686f281c recipetool: add postinst to .deb import
The .deb import feature did not import postinst, postrm, preinst, or
prerm functions. This change checks to see if those files exist, and
if so, adds the appropriate functions.

[ YOCTO #10421 ]

(From OE-Core rev: ebb73aa6ad920bfd6a23f8c20105d6bcf07dd3d5)

Signed-off-by: Stephano Cetola <stephano.cetola@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-11-23 11:10:14 +00:00
Maciej Borzecki 40c07e1041 oe-selftest: enforce en_US.UTF-8 locale
Replicate bitbake and eforce en_US.UTF-8 locale so that ouptut of locale-aware
tools remains stable.

(From OE-Core rev: 17cd2cb99d3610fd77595ff116b2168188c250cd)

Signed-off-by: Maciej Borzecki <maciej.borzecki@rndity.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-11-23 11:10:14 +00:00
Paul Eggleton 13c3a4bfa2 devtool: update-recipe: support replacing remote patches
If you have a patch remotely fetched in a recipe (e.g. from an http
server) that needs updating then add a local version and substitute the
entry in SRC_URI to point to it.

One can argue about how desirable it is to be modifying patches fetched
in this way, but then one can argue about how desirable it is to have
such patches in the recipe in the first place - and in any case if
devtool update-recipe is to correctly transfer changes to such patches
made in the git repository within the source tree to the recipe then
there isn't much choice but to do it this way.

(From OE-Core rev: a19c26cc78a181f9dd2706dd42e7e450d7ad4082)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-11-23 11:10:13 +00:00
Paul Eggleton 55a157f4e6 devtool: update-recipe: fix handling of compressed local patches
It is possible to use gzip or bzip2 to compress patches and still refer
to them in compressed form in the SRC_URI value within a recipe. If you
run "devtool modify" on such a recipe, make changes to the commit for
the patch and then run devtool update-recipe, we need to correctly
associate the commit back to the compressed patch file and re-compress
the patch, neither of which we were doing previously.

Additionally, add an oe-selftest test to ensure this doesn't regress in
future.

Fixes [YOCTO #8278].

(From OE-Core rev: e47d21624dfec6f71742b837e91da553f18a28c5)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-11-23 11:10:13 +00:00
Paul Eggleton 0ffd92707b devtool: update-recipe: check output before treating it as a string
As of the move to Python 3 and the fixes we applied at that time,
bb.process.run() will return a byte array of length 0 rather than an
empty string if the output is empty. That may be a bug that we should
fix, but for now it's easiest to just check the result here before
treating it as a string. This fixes running "devtool update-recipe" or
"devtool finish" on a recipe which has no source tree, for example
initramfs-framework.

Fixes [YOCTO #10563].

(From OE-Core rev: 66bf6978fc807ecc422fb6b6328f68bc3406cf15)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-11-23 11:10:12 +00:00
Jussi Kukkonen 0cdbd2ad94 conf: Remove legacy X input drivers
xf86-input-libinput is RRECOMMENDed by xserver-xorg, these
legacy drivers are not needed.

(From meta-yocto rev: 715f72d7842b4a789a78e7889b2f01cd41f02df8)

Signed-off-by: Jussi Kukkonen <jussi.kukkonen@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-11-15 15:19:55 +00:00
Maciej Borzecki 59c8ae1742 oe-selftest: fix handling of test cases without ID in --list-tests-by
Running `oe-selftest --list-tests-by module wic` will produce the
following backtrace:

Traceback (most recent call last):
  File "<snip>/poky/scripts/oe-selftest", line 668, in <module>
    ret = main()
  File "<snip>/poky/scripts/oe-selftest", line 486, in main
    list_testsuite_by(criteria, keyword)
  File "<snip>/poky/scripts/oe-selftest", line 340, in list_testsuite_by
    ts = sorted([ (tc.tcid, tc.tctag, tc.tcname, tc.tcclass, tc.tcmodule) for tc in get_testsuite_by(criteria, keyword) ])
TypeError: unorderable types: int() < NoneType()

The root cause is that a test case does not necessarily have an ID
assigned, hence its value is None. Since Python 3 does not allow
comparison of heterogeneous types, TypeError is raised.

(From OE-Core rev: 71c6790689e2cbd3c4e882335b3b03e635ad46ed)

Signed-off-by: Maciej Borzecki <maciej.borzecki@rndity.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-11-15 15:19:55 +00:00
Maciej Borzecki 2eeed4838b wic: fix function comment typos
Fix typos in documentation of Image.add_partition() and
Image.__format_disks().

(From OE-Core rev: f5bf7bf253224912c66bab89f48ff63a73e0d698)

Signed-off-by: Maciej Borzecki <maciej.borzecki@rndity.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-11-15 15:19:55 +00:00
Maciej Borzecki 294410cce3 wic: check that filesystem is specified for a rootfs partition
We explicitly check for --fstype if no source was provided for a
partition. However, this was not the case for rootfs partitions. Make
sure to raise an error if filesystem was left unspecified when preparing
a rootfs partition image.

(From OE-Core rev: b8c35fcad57810a87aa25ebeb533adf286eed565)

Signed-off-by: Maciej Borzecki <maciej.borzecki@rndity.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-11-15 15:19:55 +00:00
Maciej Borzecki b92de9a6db wic: use partition size when creating empty partition files
It seems that prepare_empty_partition_ext() and
prepare_empty_partition_btrfs() got broken in commit
c8669749e3, thus one could observe the
following backtrace:

Backtrace:
  File "<snip>/poky/scripts/lib/wic/plugins/imager/direct_plugin.py", line 93, in do_create
    creator.create()
  File "<snip>/poky/scripts/lib/wic/imager/baseimager.py", line 159, in create
    self._create()
  File "<snip>/poky/scripts/lib/wic/imager/direct.py", line 290, in _create
    self.bootimg_dir, self.kernel_dir, self.native_sysroot)
  File "<snip>/poky/scripts/lib/wic/partition.py", line 146, in prepare
    method(rootfs, oe_builddir, native_sysroot)
  File "<snip>/poky/scripts/lib/wic/partition.py", line 325, in prepare_empty_partition_ext
    os.ftruncate(sparse.fileno(), rootfs_size * 1024)
NameError: name 'rootfs_size' is not defined

(From OE-Core rev: 0bf686739a5e8a1193f5be5aa60afbf2ea1e5074)

Signed-off-by: Maciej Borzecki <maciej.borzecki@rndity.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-11-15 15:19:55 +00:00
Maciej Borzecki f0181e79ed wic: make sure that partition size is always an integer in internal processing
The size field of Partition class is expected to be an integer and ought
to be set inside prepare_*() method. Make sure that this is always the
case.

(From OE-Core rev: a37838f995ae642b0b8bdd47a605a264fb3bf200)

Signed-off-by: Maciej Borzecki <maciej.borzecki@rndity.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-11-15 15:19:55 +00:00
Ed Bartosh 2ef30ef624 systemd-bootdisk.wks: use PARTUUID
Root device name in systemd-bootdisk.wks is 'sda'. This can cause
images, produced using this wks to refuse booting if real device
name is not 'sda'. For example, when booting MinnowBoard MAX from
MicroSD card the boot process stucks with this message on the boot
console output: Waiting for root device /dev/sda2...
This happens because real device name of MicroSD card on this device
is mmcblk1.

Used --use-uuid option for root partition. This should make
wic to put partiion UUID instead of device name into kernel command
line.

[YOCTO #10485]

(From OE-Core rev: 5b73d5f484cc844affe91ec19d881d42e187f30c)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-11-15 15:19:54 +00:00