It is currently possible to specify a file (e.g. a tarball) on the local
disk as the source, but you have to know to put file:// in front of it.
There's really no need to force users to jump through that hoop if they
really want to do this so check if the specified source is a file and
prefix it with file:// if that's the case.
Also ensure the same works for "devtool add" at the same time.
(From OE-Core rev: 71350003790c38e84b0e525a71a2fe5d24e3d083)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
For a while now, Github hasn't been advertising a specific repository
URL since cloning the web URL with git works. Armed with this knowledge
and fully expecting people to just paste the github URL, we need to
handle this situation specially. If it looks like a github URL to the
root of a repository then treat it as a git repository instead of a
normal https URL to be fetched by the wget fetcher.
(From OE-Core rev: 7998dc3597657229507e5c140fceef1e485ac402)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
move graph-tool to python3
(From OE-Core rev: 0d0864ae0ff9e53623ad1c7146b071f2a046f21f)
Signed-off-by: Maxin B. John <maxin.john@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
These recipes can't be used with devtool because they can't be unpacked
in the normal way.
(From OE-Core rev: b2cf098969b8b800a78d650cf60c0b5ad31c85b5)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If devtool returns exit code 4 then record the recipes as "skipped"
rather than "failed" - these are recipes we know cannot work (usually
because they don't provide any source).
(From OE-Core rev: 8fc109f1cb6eb437c12d2d11a6937de6f035e296)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Certain recipes cannot be used with devtool extract / modify / upgrade -
usually because they don't provide any source. Return a specific exit
code (4) so that scripts such as scripts/contrib/devtool-stress.py know
the difference between this and a genuine failure.
(From OE-Core rev: ffd295fed4ab81fc0bd00bb145ef4d72c49584bf)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We were attempting to open the recipe file unconditionally here - we
need to account for the possibility that the recipe file has been
deleted or moved away by the user.
(From OE-Core rev: 47822a2aff56fd338c16b5ad756feda9f395a8a1)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In OE-Core revision 7baf57ad896112cf2258b3e2c2a1f8b756fb39bc I changed
the default update-recipe behaviour to only update patches for commits
that were changed; unfortunately I failed to handle the --initial-rev
option which was broken after that point. Rework how the initial
revision is passed in so that it now operates correctly.
(From OE-Core rev: b2ca2523cc9e51a4759b4420b07b0b67b3f5ac43)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently gnome-terminal just returns straight away, opening a terminal in a new
separate process we have no insight into. For patch resolution, this leads to
spawning many different terminal windows, for pydevshell, it just flashes a window
up and then closes.
We need to block until the command completes but gnome-terminal gives us no way
to do this. We therefore write the pid to a file using a "phonehome" wrapper
script, then monitor the pid until it exits.
[YOCTO #7254]
(also fixing do_devpyshell)
(From OE-Core rev: 76e8ab47c936674b8bb9bf1c48de53b30f5bf74a)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There is a typo in the logging parameters, "filname" is being used instead of "filename" for yocto-kernel, yocto-layer and wic scripts. This didn't cause issues before since python 2 didn't validate unused parameters but with python >= 3.4.3 an exception is thrown. This patch fixes this parameter name.
[YOCTO #9834]
(From OE-Core rev: 5d123a6ec0e97652b2ec4295428797a336ef357a)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When detecting the second serial options we shouldn't append the
custom QEMU options otherwise we will end duplicating those.
(From OE-Core rev: 79798f20b2c0b98d84c3c4b14600635ff8ddfdad)
Signed-off-by: Otavio Salvador <otavio@ossystems.com.br>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Code cleanup, no functional changes - this code was never used.
(From OE-Core rev: 397b76c7f26e38e761b94b1f7987aafd55048e10)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We're opening source files with the default encoding (utf-8) but we
can't necessarily be sure that they are UTF-8 clean - for example,
recipetool create ftp://mama.indstate.edu/linux/tree/tree-1.7.0.tgz
prior to this patch resulted in a UnicodeDecodeError. Use the
"surrogateescape" mode to avoid this.
Fixes [YOCTO #9822].
(From OE-Core rev: 50fcd9d1b9a20d49bc873467a82a071f2f2f8b5a)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
As /dev/pty opened in binary mode its content has to
be decoded when reading from it and encoded when writing to it.
(From OE-Core rev: 211870ddbce5c966b2882e97cb2efe29b72a62a4)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Opening text stream in unbuffered mode raises the following
exception In Python 3:
ValueError: can't have unbuffered text I/O
Fixed by leaving std* streams in text mode and flushing
stdout explicitly.
(From OE-Core rev: 732001cb268683f5b56e251e2964ec5b694a2147)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If BB_SETSCENE_ENFORCE is set to "1" and an sstate package fails to
download outside of the whitelist specified by
BB_SETSCENE_ENFORCE_WHITELIST, then fail immediately so you can tell
that the problem was caused by failing to restore the task from sstate.
Part of the implementation of [YOCTO #9367].
(From OE-Core rev: 9e711b54487c3141d7264b8cf0d74f9465020190)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
With the changes to migrate from optparse to argparse there was a syntax change that needs an update in the help. This is basically just the change of 'properties' and 'property' from positional arguments to options. This patch makes the required changes.
[YOCTO #8321]
(From meta-yocto rev: b171379b5ca54d55ea763421794a651e71bbda91)
Signed-off-by: Humberto Ibarra <humberto.ibarra.lopez@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Optparse is deprecated and should be avoided. The arparse library is better suited and has more tools to handling the parsing of arguments. This patch makes necessary changes to migrate to the better library and uses arparse subcommand feature to improve organization of this script.
[YOCTO #8321]
(From meta-yocto rev: 3f45993b96d4d960da0efe8672dc323c9db091a2)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The version option is not maintained and is useless inside this script.
There is no reason for this script to have an independent version value.
[YOCTO #8321]
(From meta-yocto rev: 8f8790e56d00f2eaaf6508fb1909335f1fbef5ff)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Format of properties file is expected to be a simple json detailing properties, if this format fails an exception is thrown. This patch adds a graceful error message to the case when the properties file has a wrong format.
[YOCTO #9750]
(From meta-yocto rev: 7e543aa19d0d4b2112e6316783fb31b76df3493e)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Python3 requires strings to be encoded as bytes before sending them through a subprocess pipe. The help.py file is not considering this and fails when issuing paged help commands. This patch adds this encoding to solve the problem.
[YOCTO #9868]
(From meta-yocto rev: 35b487a47f0cbb99fdee2ec9cc8b56b814c8860e)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There is a typo in the logging parameters, "filname" is being used instead of "filename" for yocto-kernel, yocto-layer and wic scripts. This didn't cause issues before since python 2 didn't validate unused parameters but with python >= 3.4.3 an exception is thrown. This patch fixes this parameter name.
[YOCTO #9834]
(From meta-yocto rev: 844fbba63b146a2ded3fced0d62bf047bf844af3)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
A shell script wrapper around oe-build-perf-test script. The purpose of
this wrapper is to act as a executor script, making it possible to run
the tests with a single command. The wrapper script initializes the
build environment, runs oe-build-perf-test and archives the results.
(From OE-Core rev: 946a076c2ce20dd8f7cfa1acbdab1268d406d3e1)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The new option defines the output directory for the test result data.
(From OE-Core rev: bc865b5cb5a9a76048ee9c55a29f5e1a926bb543)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Makes it possible to guard that multiple tests are not run in parallel.
(From OE-Core rev: 181e92e7a1bccf678b3eb1bf547608a142784f97)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Using this option the script appends test results into a 'global results
file'. A CSV-formatted output of the results. This option is to provide
compatibility with the old build-perf-test.sh.
(From OE-Core rev: e9f18e63220e452f2b0c878998e57d944ae83980)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The new class is responsible for actually running the tests and
processing their results. This commit also adds a decorator function for
adding new tests. No automatic test discovery, at least yet.
(From OE-Core rev: bf90aecb7e150d6bfac7240286c797b79d26528b)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a new utility class for dropping Linux kernel caches. It uses sudo
and tee to write to the drop_caches file. Checking if the user has the
permissions to drop caches (without a password) is done by trying to
writing an invalid value to the drop_caches file. This way, we will find
if writing (with tee) is possible but not really dropping caches, yet.
User can avoid giving the password by adding something like:
<user> ALL = NOPASSWD: /usr/bin/tee /proc/sys/vm/drop_caches
to the system sudoers file.
(From OE-Core rev: c9cb248429ced50c96d11ba5361c272d4c9b9323)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The script will be required to be run in an initialized bitbake build
environment.
(From OE-Core rev: 1bce7b10283255a4498d11ead920c1f3b1dec4de)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Initial wireframe for re-writing build-perf-test.sh in Python.
(From OE-Core rev: 764eb2d011305b84501cc183531a2a5353b0b5ab)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
GNU make looks for "makefile" and "GNUmakefile" in addition to
"Makefile", so add these other names to the heuristic for detecting a
make-based project.
(From OE-Core rev: 204d19b02265e5b2241888e4c92c0a730f3d3472)
Signed-off-by: Nathan Lynch <nathan_lynch@mentor.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* format of bitbake tasks changed in:
2c88afb taskdata/runqueue: Rewrite without use of ID indirection
-ERROR: Task 4 (/OE/build/oe-core/openembedded-core/meta/recipes-devtools/rpm/sftp.bb, do_fetch) failed with exit code '1'
+ERROR: Task /OE/build/oe-core/openembedded-core/meta/recipes-devtools/rpm/sftp.bb:do_fetch (/OE/build/oe-core/openembedded-core/meta/recipes-devtools/rpm/sftp.bb:do_fetch) failed with exit code '1'
so strip not only '\.bb, .*' used before, but also '\.bb;.*' to drop
the task name to get recipe name.
* for more details see:
http://lists.openembedded.org/pipermail/openembedded-core/2016-June/123132.html
* without this change you can see test-dependencies.sh trying to rebuild packages
like:
Building recipe: fbprogress (6/21)
Building recipe: fbprogress.bb:do (7/21)
where the later of course doesn't exist as a recipe
(From OE-Core rev: b7d6d4203cf2021ee2b9b84c8faf15198bfb536d)
Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
With Python 2, argparse subparsers behaviour in Python 2 was to print
the usage information if the subparsers argument wasn't specified.
However, with Python 3.2.3 and later a subparsers argument is not
required by default, leading to errors when no arguments are specified:
AttributeError: 'Namespace' object has no attribute 'func'
Restore the previous desired behaviour of showing the help text for
devtool, recipetool and the devtool-stress script by setting
subparsers.required to True.
(From OE-Core rev: d36fdea1a7f32d97187e0e9e6d701ae8fa304e8f)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Allow specifying more than one recipe on the devtool reset command line.
Also tweak the help text slightly.
(From OE-Core rev: ad92ed8e4f7f48a3d212962531d596b36f6b284f)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Clarify slightly the intended usage of the workspace layer.
(From OE-Core rev: d9f3af6f4e6d3df30b411bbcc3c2b6f7f62c52ad)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Commit ab75075e79 broke the string formatting
for multiple arguments. Since parentheses were removed the formatting considers
only first argument instead of all of them. Fixed parentheses on the formatting.
[YOCTO #9723]
(From meta-yocto rev: d6d641d5b4bb0c8b9acd5cd1341a4a3214bff086)
Signed-off-by: Humberto Ibarra <humberto.ibarra.lopez@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Have to convert implicit relative imports to explicit imports since
implicit relative imports are not supported on python3.
[YOCTO #9723]
(From meta-yocto rev: 89cecc603d229768428f3cb68d21235dee80efda)
Signed-off-by: Humberto Ibarra <humberto.ibarra.lopez@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
According to https://docs.python.org/3/whatsnew/3.0.html python3
renamed 'raw_input' to 'input'. Making the appropiate changes for
this.
[YOCTO #9723]
(From meta-yocto rev: 76efeeeac7fb30b44ee7057b3b3d1fd84329dae1)
Signed-off-by: Humberto Ibarra <humberto.ibarra.lopez@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There is a typo in the logging parameters, "filname" is being used instead
of "filename". This didn't cause issues before since python 2 didn't
validate unused parameters but with python >= 3.4.3 an exception is
thrown. This patch fixes this parameter name.
[Yocto #9736]
(From meta-yocto rev: aae5c2c39f64be87152b2e5470b50681c0f54670)
Signed-off-by: Humberto Ibarra <humberto.ibarra.lopez@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If buildhistory-diff can't import git module it throws error
message suggesting to install it.
Due to the move to Python 3 the suggested package name and
version became outdated.
Changed package name to python3-git and version to >= 0.3.4
as GitPython supports Python 3 starting from 0.3.4.
[YOCTO #9741]
(From OE-Core rev: 0632c3e69070e340218c1c1c835b2343cd143d8d)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
New option can be used for displaying full information about binary
package(s), including name, full version, recipe name, recipe full
version and package size.
This information can be useful inside custom buildhistory class
to produce detailed image manifest for further analysis.
List of packages can be specified as command line arguments or can
be read from file (e.g. from existing image manifest).
Warning: In case of image manifest is used as input outside of build
process, be aware that pkgdata might change since the time image
was built.
Output format:
{PKG} [PKGE:]{PKGV}[-{PKGR}] {PN} [PE:]{PV}[-{PR}] {PKGSIZE}
(From OE-Core rev: bc0cdaa927124150a6c38cd47977ee4da8dd440e)
Signed-off-by: Alexander D. Kanevskiy <kad@kad.name>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The current qemuzynq options were causing QEMU to exit instead of
booting as the QEMU Zynq-7000 machine does not have virtio or
display devices.
(From OE-Core rev: e76f62a862a895147a89cea2519679aad337a679)
Signed-off-by: Alistair Francis <alistair.francis@xilinx.com>
Reported-by: Arno <star@gmx.li>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The default arparse behaviour in Python 2 and early versions of Python 3
was to print usage message if no subcommands provided by user.
In Python >= 3.2.3 subparser has to have attribute 'required' set to
True to behave similar way.
By default subparser.required attribute is set to False, usage message is
not printed and code continues to be executed.
oe-pkgdata-util code raises AttributeError exception in
this case:
AttributeError: 'Namespace' object has no attribute 'func'
Fixed this by setting subparser.required to True.
[YOCTO #9711]
(From OE-Core rev: 207dba5b6eb04c1ae2081bac06724b3b6a198cb3)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
swabber hasn't been used in years and never did work well in the first
place. Remove its recipes, class and configuration.
(From OE-Core rev: e18657df0b7e45a224fae17e68c447eae94258ac)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This was broken with the python3 fixes.
(From OE-Core rev: 0eadd008d08f9a6bb17a1f641b4735d0d2012580)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Jussi Kukkonen
Paul Eggleton
Maxin B. John
Richard Purdie
Humberto Ibarra
Otavio Salvador
Ed Bartosh
Markus Lehtonen
Nathan Lynch
Martin Jansa
Alexander D. Kanevskiy
Alistair Francis