The default arparse behaviour in Python 2 and early versions of Python 3
was to print usage message if no subcommands provided by user.
In Python >= 3.2.3 subparser has to have attribute 'required' set to
True to behave similar way.
By default subparser.required attribute is set to False, usage message is
not printed and code continues to be executed.
oe-pkgdata-util code raises AttributeError exception in
this case:
AttributeError: 'Namespace' object has no attribute 'func'
Fixed this by setting subparser.required to True.
[YOCTO #9711]
(From OE-Core rev: 207dba5b6eb04c1ae2081bac06724b3b6a198cb3)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
swabber hasn't been used in years and never did work well in the first
place. Remove its recipes, class and configuration.
(From OE-Core rev: e18657df0b7e45a224fae17e68c447eae94258ac)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This was broken with the python3 fixes.
(From OE-Core rev: 0eadd008d08f9a6bb17a1f641b4735d0d2012580)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
stdeout and stderr content returned by subprocess API has different
types in Python 3(bytes) and Python 2(string). Decoding it to
'utf-8' makes it unicode on both pythons.
(From meta-yocto rev: 1de9d0b4ad289c56907d082748cdc0111988cb4f)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Used items() as iteritems() doesn't exist in python 3.
(From meta-yocto rev: 0fd2e739cd14c6a02b6f4610176a0ad29b2ecc0e)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Used exec() builtin instead of 'exec' statement as
this statement doesn't exist in python 3.
(From meta-yocto rev: ceb6c6fac212c5e1e76bf58b793dc2b9420b4a66)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Implicit relative imports within packages are not supported in
python 3. They have to be converted to explicit imports.
Used 'from .module import' syntax for relative imports.
(From meta-yocto rev: 8eb2befc544e29fd87ac1fc75886cc6d7f40ad90)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Used metaclass=<metaclass> syntax instead old
__metaclass__ = <metaclass> as only new one is supported
in python 3.
(From meta-yocto rev: 70ba79981d904b4b67440c0b5755b5449cddb025)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Renamed raw_input to input as raw_input does not
exist in python 3.
(From meta-yocto rev: c9df9f2699885f2ba5b031c8761aefbf3c796067)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Encoded data before sending it through http as urllib expecting bytes.
Fixed TypeError: POST data should be bytes or an iterable of bytes. It
cannot be of type str.
(From OE-Core rev: b3f4de76d1b32c5079b0b857655cc2baad088519)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Since mtools has been patched to live with filesystems with sizes
not divisible by sectors-per-track, we no longer need to try to
set the size based on our guess of the sectors-per-track dosfstools is
going to use.
(From OE-Core rev: 334e32af88b310ff1ed950d127a6dedeb460f8d0)
Signed-off-by: Jussi Kukkonen <jussi.kukkonen@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
stdeout and stderr content returned by subprocess API has different
types in Python 3(bytes) and Python 2(string). Decoding it to
'utf-8' makes it unicode on both pythons.
(From OE-Core rev: ae4d36375683b6cfd48af25bfca70b2e77f7ade5)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Used tempfile.TemporaryFile() API instead of deprecated
os.tmpfile().
(From OE-Core rev: bf1b411eb1cd2cc960325d5fdb0cb4f4f7b1e40e)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Python 3 ignores the __cmp__() method and doesn't have cmp() builtin
function. This caused sorted() call to raise
TypeError: unorderable types: Record() < Record()
Removing __cmp__ method and implementing __lt__ should solve the
problem as __lt__ is the only method needed for sort[ed] to work.
(From OE-Core rev: 391cd33720e7d7e8e261193199272739293ad881)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Reduce is not a builtin function in python3.
It has to be imported from functools.
(From OE-Core rev: f9bda8b36decb5a837adcfeb1173092401988801)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Used metaclass=<metaclass> syntax instead old
__metaclass__ = <metaclass> as only new one is supported
in python 3.
(From OE-Core rev: 067a2156f36a12c731d49f88cf9d0b0bab7a0dcb)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Removed print_function and with_statement imports from __future__
as they're supported by python 3 by default.
(From OE-Core rev: 77ae2daad5d775d710b953cf0c623ce74cb2c274)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Renamed raw_input to input as raw_input does not
exist in python 3.
(From OE-Core rev: 32765150b860ecdea74b4494b9531f5bc40252bb)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Used join method instead of strings.join as stings.join
doesn't exist in python 3.
(From OE-Core rev: 10529d8fbc7254523f9749f4b35b07ebcccb6205)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Some functions and classes have been moved from urllib[2]
to urllib.request and urllib.error in python 3.
Used new imports to make the code working in python 3.
(From OE-Core rev: ec3f1759e8b491a44a1fc1ecb6f89919dd30da97)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Changed old syle except statements 'except <exception>, var'
to new style 'except <exception> as var' as old style is not
supported in python3.
(From OE-Core rev: 438eabc248f272e3d272aecaa4c9cec177b172d5)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Converted return value of items() keys() and values() to
lists when dictionary is modified in the loop and when
the result is added to the list.
(From OE-Core rev: 874a269eb1d70060c2f3b3f8b70800e2aea789f4)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This patch contains all the other misc pieces of the transition to
python3 which didn't make sense to be broken into individual patches.
(From OE-Core rev: fcd6b38bab8517d83e1ed48eef1bca9a9a190f57)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
As most of oe-test cases work for devtool and recipetool
it makes sense to switch both tools to python 3 by
explicitly referring to python3 in their shebangs.
(From OE-Core rev: dad9617809c60ec5f11d4780b0afa1cffa1efed5)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Used urllib.parse instead of urlparse to make code
working in python 3.
(From OE-Core rev: 0a064f2216895db0181ee033a785328e704ddc0b)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The ConfigParser API was renamed to configparser in python 3.
Renamed ConfigParser -> configparser in scripts/ to make the
code working in python 3.
(From OE-Core rev: de6e98f272e623ce72e724e66920eecf10cb2d41)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Moved call of decode('utf-8') as close as possible to
call of subprocess API to avoid calling it in a lot of
other places.
Decoded binary data to utf-8 where appropriate to fix devtool
and recipetool tests in python 3 environment.
(From OE-Core rev: 30d02e2aa2d42fdf76271234b2dc9f37bc46b250)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Python 3 doesn't have basestring type as all string
are unicode strings.
(From OE-Core rev: e8cfab060f4ff3c4c16387871354d407910e87aa)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Encoded help text before sending it to pager.communicate as
it expects binary.
[YOCTO #9412]
(From OE-Core rev: 23c27d9d936efaa17da00525f1d2e2f98c53abc7)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Switched scripts/wic to use python3 as a default
python interpreter.
(From OE-Core rev: ea6245d2383e2ba905ef9f1ba210e5dadc779ad8)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
xrange() no longer exists in python 3, use range()
(From OE-Core rev: d022b4335100612d6596cc4c4956cb98ed5873cc)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Iterators now return views, not lists in python3. Where we need
lists, handle this explicitly.
(From OE-Core rev: caebd862bac7eed725e0f0321bf50793671b5312)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
python3 standardises its use of iteration operations. Update
the code to match the for python3 requires.
(From OE-Core rev: 2476bdcbef591e951d11d57d53f1315848758571)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Check that 'python' is a python v2 since that is what we assume everywhere
and upstream python devs recommend. We can need both python2 and python3
available since we don't control the software we might download and run.
Also check that python 3 is >= 3.4.0, our minimum version for bitbake.
(From OE-Core rev: 3dd26cd6b3d731f7698f6fbcd1947969f360cdc4)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Used print function instead of print statement to make
the code work in python 3.
(From meta-yocto rev: d6eea5a794dd8802b773a9186479a863847e6e55)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Change shebang line to python3 and add parentheses to print
(From OE-Core rev: 85b6a53386382c0d92b5bea545c2db5e0204e629)
Signed-off-by: Allen Wild <allenwild93@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Used print function instead of print statement to make
the code work in python 3.
(From OE-Core rev: 80fecc44761fa38ccf2e4dc6897b9f1f0c9c1ed0)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If fetching source from a git repository, typically within OpenEmbedded
we encourage setting SRCREV to a fixed revision, so change to do that by
default and add a -a/--autorev option to use "${AUTOREV}" instead.
(From OE-Core rev: 000480c42797dd2f03ebc3bc6d1dabfc6a7b75f5)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If we use ${BP} for the subdirectory, the default value of S will work
rather than having to have an ugly value derived from the package
file name in both places. This does mean that we have to assume the
default though (we can't just let the normal logic work because the
value of BP is the default until later on, so the replacement doesn't
work).
(From OE-Core rev: 13bc2438d61c345a8f229b9d83bf36a14d08916f)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Extract the metadata from package files and use it to set variable
values in the recipe (including recipe name and version, LICENSE,
SUMMARY, DESCRIPTION, SECTION and HOMEPAGE). For LICENSE we take care
not to step on any value determined by our license file scan; if there
is one we simply add a comment above the LICENSE setting so the user can
resolve it.
(From OE-Core rev: 19e6b661d38c5ae9b19d6340762c289830baba59)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Allow plugins to set any variable value through the extravalues dict,
and use this to support extracting SUMMARY and HOMEPAGE values from spec
files included with the source; additionally translate "License:" to a
comment next to the LICENSE field (we have our own logic for setting
LICENSE, but it will often be useful to see what the spec file says if
one is present).
Also use the same mechanism for setting the same variables for node.js
modules; this was already supported but wasn't inserting the settings in
the appropriate place in the file which this will now do.
(From OE-Core rev: 91fc35ff5e89aa6d4c4ad945e45406fb4f71018e)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We want the system to boot on its own, rather than the user having to manually
input the correct file at the EFI prompt.
[YOCTO #9556]
(From OE-Core rev: aa215c12d729c001ad3b8b98a7e203a9ad192211)
Signed-off-by: Christopher Larson <chris_larson@mentor.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The -S / --srcrev option must be specified if fetching from a git
repository, so spell that out in the help text.
(From OE-Core rev: 1465f205c235a1688a85844ebf5259e8971038ae)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Make a couple of changes to the rebase operation:
1) Only wrap the actual rebase command in try...except since a failure
in any of the other commands should be an error, not a warning
2) If it's a conflict (which unfortunately we can only tell by checking
for the keyword "conflict" since git doesn't return error codes based
on the type of error) then print a message clarifying that the user
needs to resolve the issue themselves to finish the upgrade.
(From OE-Core rev: 2a76be958432a35a0de30e9a5433089a54a06cad)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The gdb recipe in OE-Core has an inc file with the version in it;
since the inc file is pulled in with a "require ${PV}.inc", when
upgrading the recipe we need to also rename the inc file it will fail to
parse and the upgrade itself will fail.
Fixes [YOCTO #9574].
(From OE-Core rev: 3c623aac9333d20a62475279c72b6b6ec3d7dd6b)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The script bitbake-whatchanged uses optparse library, which is
deprecated since python 2.7. This migrates to argparse library.
[Yocto #9634]
(From OE-Core rev: b6c71616e66708bb1c456b83f98913b198f49a4a)
Signed-off-by: Humberto Ibarra <humberto.ibarra.lopez@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
file() API doesn't exist in python 3, convert to open(). Also handle
some cases where files aren't closed. Compatible with python 2.7.
[Contributions from Ed and Richard]
(From OE-Core rev: 0f4ec13e11bb8abe21aba2a28547dfb9372bc377)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Used print function instead of print statement to make
the code work in python 3.
[Changes from both Ed and Richard]
(From OE-Core rev: ced1995694c394d92cb82fb9c25a33dc027a3b69)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Having two code paths here makes maintenance difficult, and it doesn't
seem likely that you would use the local case in real usage anyway, so
drop the local support entirely.
This should allow us to resolve [YOCTO #9301].
(From OE-Core rev: 7a4c9c96fee4fb514c2b69b52e811c4f896a16f1)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If the path contains 'scripts' more than once the first occurrence will be
incorrectly used. Use rfind instead of find to find the last occurrence.
(From OE-Core rev: f30c486c17060d2f21618612804a692512ad6a57)
Signed-off-by: George McCollister <george.mccollister@gmail.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The --list-tests option assumes a terminal of 150 characters wide,
which is clearly wrong. The output for this command is messy and
hard to understand for lower widths. Every command should look
good in a 80 characters terminal.
Unfortunately, this can't be done at the moment. The bad naming of
testcases have made the test names incredibly long.
This patch reduces spacing between columns and shows the tests
names in a concise and understandable way. The format is even the
same one required for running a testcase. Once the testcase naming
improves, this output will look even better.
[Yocto #9534]
(From OE-Core rev: 53da74155942febd520836cabf3aa727c53ce5ca)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Included description of -m/--bmap option to the help page
produced by 'wic help create'.
[YOCTO #9413]
(From OE-Core rev: 7c749d4cfe1642d70a6b3643422ef69dec1775fe)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This option enables generation of <image>.bmap file for the
result image using native bmaptool.
[YOCTO #9413]
(From OE-Core rev: d64c7b37c40b052510419b4d6629b83319c833e4)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Added bmaptool -> bmap-tools pair to the dictionary
executable -> recipe as bmaptool is going to be used by wic
to generate .bmap file.
[YOCTO #9413]
(From OE-Core rev: 4684ac5f4d823a6afedc57aad0aa417bb17ba950)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Mount the hddimg and rootfs.img as read-only when creating the bootable
image on the medium. Otherwise, the md5 checksum values of the hddimg will
be altered. As this changed checksum value might cause issue for users
whom would reuse the hddimg.
(From OE-Core rev: a1391c8a603f0ed972ee0bcc8c74999f5f43be43)
Signed-off-by: Raymond Tan <raymond.tan@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
So that user can easily make their own conf files such as conf/site.conf
work. For example, poky's default BBPATH in bblayers.conf is:
BBPATH = "${TOPDIR}"
Will change it to:
BBPATH_EXTRA ??= ""
BBPATH = "${BBPATH_EXTRA}${TOPDIR}"
When user sets BBPATH_EXTRA in env to their own dir which contains
conf/site.conf, it will work.
Note, BBPATH_EXTRA must end with ":", we can't set BBPATH as
"${BBPATH_EXTRA}:${TOPDIR}" since the sanity would fail when
BBPATH_EXTRA is null.
[YOCTO #7837]
(From OE-Core rev: a3d223284854a21c84e8f6d075d23b32789afa01)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Generators in Python 3 don't have .next method. It's recommended
to use 'next' builtin instead. As it also present in Python >= 2.6
it should make wic code compatible with both Python 2 and Python 3.
[YOCTO #9412]
(From OE-Core rev: 9b7ab632e47d786dd979262015dbfb1254103f83)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Removed check for unicode type as it doesn't work in Python 3.
This check is not needed for wic as all its output seem to be
strings. This allows to run code under both pythons.
[YOCTO #9412]
(From OE-Core rev: a56924b4a0102e401b5e37d857a08bab15da974e)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
stdeout and stderr content returned by communicate API has different
types in Python 3(bytes) and Python 2(string). Decoding it to 'utf-8'
makes it unicode on both pythons.
Decoded stdout and stderr output to utf-8 to make the code
working under both Python 2 and Python 3.
(From OE-Core rev: 5b556f58a171e3d45107bb56a1f780e5c1abba37)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Division operator works differently in Python 3. It results in
float unlike in Python 2, where it results in int.
Explicitly used "floor division" operator instead of 'division'
operator. This should make the code to result in integer under
both pythons.
[YOCTO #9412]
(From OE-Core rev: 997ff239bd753a7957cc14c6829b2f093d9bcef6)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
As bitbake is not ported to Python 3 yet it's better to
avoid using its APIs as much as possible to be able to
test wic under Python 3 at least partially.
Used distutils.spawn.find_executable API in favor of
bb.utils.which to get path of the command to run.
(From OE-Core rev: 9658956bf8a5da779e06f71941de9b3e89415cdc)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This suffix is not supported by Python 3. Wic code works
without it on Python 2 too, so it's safe to remove it.
[YOCTO #9412]
(From OE-Core rev: 296db7e33bd71585cac63dc78c2c95bc619b4a86)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Wic plugin machinery implemented using metaclasses.
Reimplemented plugin machinery using this advice from
https://wiki.python.org/moin/PortingToPy3k/BilingualQuickRef
Syntax for creating instances with different metaclasses is very
different between Python 2 and 3. Use the ability to call type instances
as a way to portably create such instances.
Now it should work under both Python 2 and Python 3.
[YOCTO #9412]
(From OE-Core rev: e62fe5a41bdcdd72b9b257fecff7ccdc59c76d33)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Removed 'raw', 'ask', 'choice' and 'pause' functions from
msger.py as they're not used in wic code and some of them
use raw_input, which is not present in Python 3.
[YOCTO #9412]
(From OE-Core rev: eb87d591ef67f1953b2689430ef6c5a6a27a5b6e)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
New syntax 'except Exception as err' is supported by Python >= 2.7.
Old syntax 'except Exception, err' is not supported by Python 3.
Used new syntax to be able to run wic on Python 3.
[YOCTO #9412]
(From OE-Core rev: 15e88714d6b0a93f72e8a19b083fcc1f2006e128)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
'with' statement is not used in baseimager.py
It's supported by Python 2.7, which is included into all target
distros. Other wic modules use this statement.
Removed useless 'from __future__ import with_statement' from
wic code.
(From OE-Core rev: 528a1f20939589949831efbb4de6336776efe7d5)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
All classes in Python3 are new style classes and don't need
to be inherited from object. Wic code works fine without
this inheritance even with Python2, so it's harmless to
remove it.
[YOCTO #9412]
(From OE-Core rev: a146b03ee7d0aa5bc1722da5977a5952782b69bf)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Replaced calls of dict.keys and dict.has_key methods with the
'key in dict' statement. 'key in dict' is more pythonic, faster
and readable. dict.has_key doesn't exist in Python 3.
[YOCTO #9412]
(From OE-Core rev: 003df7dfb932c551953fbf1bd769b3c31bd16fb4)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Dictionary method 'iteritems' doesn't exist in Python 3.
Replaced 'iteritems' with 'items' to be able to run the
code under both Python 3 and Python 2.
[YOCTO #9412]
(From OE-Core rev: 5b14eb8d68aaca82de4f8f6bcb28ad6f4a5125d0)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Print statements have been replaced with print function in
Python 3. Replaced them in wic code to be able to run it
under both Python 2 and Python 3.
[YOCTO #9412]
(From OE-Core rev: ee6979a19c77931c3cf6368e695e370d46192fef)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
As suspected, invoking "git archive" with all intended files as
parameters can run into command line length limitations. Splitting up
the parameters into multiple invocations (xargs-style) works and was
tested after encountering the situation in practice.
(From OE-Core rev: 1cb484ab99eabb5c24792757ab09d7f170f2e614)
Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When the "history" option is not set in the combo-layer.conf, the
intended default was to use the traditional method. Passing "True" as
default when querying the config was unintentional.
Also remove some left-over debugging code.
(From OE-Core rev: d0304acb05b926b08805d8652e12eaf19bf53ad6)
Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If an image with the filename foo.bb could be built using the name "bar"
instead, then build-sdk would fail to create the derivative sdk.
This was because the code assumed that the file name matched the target,
which is not necessarily the case.
(From OE-Core rev: d58a326b6960be14b8a049253559aec9582b7d0d)
Signed-off-by: Randy Witt <randy.e.witt@linux.intel.com>
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The core idea is that all commits get imported, including merge
commits, and joined into one big merge commit that imports the changes
from the individual components into the main branch of the combined
repository.
This is done by copying the files in each commit and removing deleted
ones, instead of trying to patch the combined repository.
The advantages of doing updates in this mode are:
- works for arbitrary upstream repos, not just those which
support conversion into a linear set of patches
- listing history shows that commits where developed
independently in the different components, instead of
artificially showing them as if they had been developed
one after the after (component "aaaa" before "bbbb", then "ccc", ...)
- bisecting becomes easier: when upstream repos only ensure consistency
when merging into their "master" branches, then those merge
commits are good candidates for test builds also in the combined
repo
- more data mining can be done, for example showing who merged a commit
and when
Selecting a subset of the files is supported, albeit with a slight
different semantic for wild card matching compared to other code paths
(/ is matched by * and ?). Empty commits get skipped because typically
they are a result of filtering (but that is not checked, so
intentionally empty commits also get skipped).
Other code paths are intentionally left unchanged, to avoid
regressions. However, the downside is that some opportunities for
refactoring (in particular regarding file filtering) were ignored.
(From OE-Core rev: 660f76b6fb0fb95738a2c8f50e0a99ffa5831c64)
Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Allow setting the environment. Due to a subprocess quirk, it must
always be set explicitly (reuses the one from the previous call if not
set, instead of falling back to os.environ).
Embedding nul characters will be useful for parsing git output more
reliably; support dumping such output a bit better.
(From OE-Core rev: 0af4fadafce690fc8357196cf7247bd222c08d10)
Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When setting "history = True" in combo-layer.conf consistently for the
components involved in an update or using "update" together with the
"--history" command line flag, a new mode for updating will be used
that does not rely on exporting/importing patches.
A config setting is used because it should be used consistently by
everyone using the same config, without having to remember to use an
additional command line parameter.
There are no real global settings, so the setting is checked
separately for each component although the setting has to be set
consistently. This restriction could be removed later.
In practice, putting "history" into the "[DEFAULT]" section is the
easiest approach for configuring it.
The actual code changes split up action_update and the
combo-layer.conf handling in preparation for this new mode, without
implementing the mode itself.
(From OE-Core rev: c9dab31f5f6dc225f5c2c2ca3ec9aeab2ff655d5)
Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When a branch is re-used, the kernel tools turns off any patch pushing unless
'mark patching' is explicitly set.
[YOCTO #9120]
(From meta-yocto rev: 427f5473722e15e288cbce251a9ce18989c23548)
Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This gives us a progress bar for the image write, which is quite helpful. See
https://www.ivarch.com/programs/pv.shtml.
(From OE-Core rev: 5b20cb2db929daaf0bf81c05368174e9c364ffab)
Signed-off-by: Christopher Larson <chris_larson@mentor.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fix two problems falling back to the "license" field from package.json
when no license file is present:
1) The function that was supposed to return the license field value was
always explicitly returning None, and this was never noticed (because
the test cases never exercised the fallback as they provided license
files for each module).
2) Fix the main package not falling back because it had a default of an
empty list, which evaluates to '' instead of 'Unknown'.
(From OE-Core rev: 59381a9450949ce6b4b03adb717e950b999830f3)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This aligns with our existing 'optional arguments' to 'options' change, and
seems more intuitive for users.
(From OE-Core rev: 8a1cd471210e5fb77952f28172084bf6a4fb73e8)
Signed-off-by: Christopher Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There's no need to iterate over the action groups here, as self._optionals and
self._positionals are available.
(From OE-Core rev: 408694f4320f3cb52a391e5b927fb8c8ba16c1d2)
Signed-off-by: Christopher Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
As an example, `recipetool create foo bar baz` shows `recipetool: error:
unrecognized arguments: bar baz` and then displays the main help, not the help
for the create command. Fix by saving the subparser name and using it in
parse_args() to look up the subparser.
(From OE-Core rev: 7fdaaedf4c63c8d019f03f84e22f9b838ef19aa6)
Signed-off-by: Christopher Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This aligns our subclassed error() with that in the original class, using
_print_message and self.prog. Also add a docstring based on the original.
(From OE-Core rev: cf0c5175136966eefde8c0d9aa0679e85779f713)
Signed-off-by: Christopher Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Used sparse_copy API in favor of dd/cp in rawcopy plugin to
preserve sparseness of the copied raw content.
[YOCTO #9099]
(From OE-Core rev: 04eca59068a79ae6a9969be495c4cdf0c5c3e466)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Copied partition images into final partitioned image using
sparse_copy API. This method preserves sparseness of the
final image. It also makes wic much faster, as unmapped
blocks of the partition images are not copied.
[YOCTO #9099]
(From OE-Core rev: 7f21427aca5df81d8881027fd98f71b821cf31d7)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In order to make wic images sparse sparse_copy function has been
copied from meta-ostro:
https://github.com/kad/meta-ostro/blob/master/meta-ostro/lib/image-dsk.py
This function uses filemap APIs to copy source sparse file into
destination file preserving sparseness.
The function has been modified to satisfy wic requirements:
parameter 'skip' has been added.
[YOCTO #9099]
(From OE-Core rev: bfde62bdc03152a4d3d383512479b974fa867f94)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In order to make wic images sparse set of APIs has been copied
from bmap-tools project.
filemap.py module is taken from bmap-tools project:
https://github.com/01org/bmap-tools/blob/master/bmaptools/Filemap.py
It implements two ways of get information about file block: FIEMAP
ioctl and the 'SEEK_HOLE / SEEK_DATA' features of the file seek
syscall.
Note that this module will be removed as soon as bmaptool utility
supports copying sparse source file into destination file (this is
already agreed with the maintainer of bmap-tools project).
[YOCTO #9099]
(From OE-Core rev: 182639ddc9cda85c896a54c1c64fd1fb145071a1)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Moved DiskImage class from utils/fs_related.py to
imager/direct.py as it's only used there.
Removed fs_related module as it doesn't contain anything
except of DiskImage.
(From OE-Core rev: b3cc471790784c28f9362fcd6fc6a81c4316754c)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Removed fs_related.makedirs as is not used anywhere. The name is
easy to confuse with os.makedirs.
(From OE-Core rev: 796b114863ef20fbc89da45dbe6780abe1256f5e)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There is no need in this inheritance as DiskImage class
is used only in one module and no other classes are inherited.
(From OE-Core rev: 5af1d9bedc2c961eb91faf80251f24c3df754d76)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Used truncate instead of dd to create wic images for the
following reasons:
- dd doesn't preserve sparseness
- truncate syntax is much more clear
- dd requires additional calculations of the image size
in blocks
- the way dd was used in the code is not always correct.
In some cases it was writing one block to the file which makes
it not 100% sparse.
[YOCTO #9099]
(From OE-Core rev: d2d0d18dfd3922411d856b98ab6ba5d64c9c1c9f)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Added explanation of --system-id option to the output of
wic help kickstart.
[YOCTO #9096]
(From OE-Core rev: 1a304afea4ad7be12ed5f0fcb397a538345a6b63)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Used sfdisk to set partition system id if --system-id parameter
is used for a partition in wks file.
[YOCTO #9096]
(From OE-Core rev: a1f7f7e61fd20fb6319825648930f7b6aa0e0cee)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Added new argument to add_partition call to pass partition
system id down the stack.
[YOCTO #9096]
(From OE-Core rev: f2733df697192c0010c17b7bbb02f8679cb8f313)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Added Partition.system_id attribute and initialized it
from parse result of wks option --system-id. It will be
used by the wic code below the call stack to set partition
system id.
[YOCTO #9096]
(From OE-Core rev: 4f195a5b7574ebff8fbdb3045daa71f173f97a30)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Added sfdisk -> util-linux pair to the dictionary
executable -> recipe as sfdisk is going to be used by wic
to set partition system id.
[YOCTO #9096]
(From OE-Core rev: 398aafa185acbc7239505f7107735e93a502f6d2)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Added new option --system-id to wks parser. The option
will be used to set partition system id.
[YOCTO #9096]
(From OE-Core rev: b9c56b1c95cd1d0fd809d257e0cd05a50c481bed)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
os.path.join discards the cr_workdir var contents if the path of the
second arguments is absolute.
(From OE-Core rev: dba099d77dcc66b239523a55f3ed26784f9a662a)
Signed-off-by: Ioan-Adrian Ratiu <adrian.ratiu@ni.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The latest wic kickstart refactoring introduced a bootloader option
"--configfile" which lets wks' specify a custom grub.cfg for use
while booting. This is very useful for creating stuff like boot menus.
This change lets isoimage-isohybrid use --configfile; if this option is
not specified in a wks, it generates a default cfg as before.
(From OE-Core rev: bf673a769514b13558ad9c785ae4da3a5adfd1e0)
Signed-off-by: Ioan-Adrian Ratiu <adrian.ratiu@ni.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If UNINATIVE_CHECKSUM changes over an SDK update, bitbake within the
extensible SDK will be broken because it will see that the matching
uninative tarball doesn't exist and if there is a default value of
UNINATIVE_URL it will attempt to download the file and will then fail
because the checksums don't match up; alternatively if no UNINATIVE_URL
is set then it'll also fail with an error about misconfiguration. To fix
this, add some logic to devtool sdk-update to download the matching
uninative tarball(s) for the checksum(s) in the newly fetched SDK
configuration.
Fixes [YOCTO #9301].
(From OE-Core rev: 14ff58ad98a5afac08db77068d80f152d8875766)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Replace git pull with fetch and reset to avoid the merge logic in the
event that the layers repo in the published SDK we're updating to isn't
fast-forward merge from the local repo.
Also add gitignore and committer info during publish to avoid errors and
to be sure that the first commit has a dummy user in it.
[ YOCTO #9368 ]
(From OE-Core rev: 4657bc9d165e51981e034e73e7b92552e873eef7)
Signed-off-by: Stephano Cetola <stephano.cetola@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
It is unusual but not impossible to find recipes whose first entry is
not the main source URL but instead some patch or other local file, for
example python-cryptography in meta-python (which sets SRC_URI before
inheriting pypi). There's nothing inherently wrong with this, and we
shouldn't assume that the first entry is the main source URL, so just
take the first non-local entry instead.
(From OE-Core rev: c868198c1f6006789707b497c2ae34d7cc5e706f)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
For example, support both:
$ runqemu qemux86-64 ramfs
$ runqemu qemux86-64 cpio.gz (new)
(From OE-Core rev: 6529264776701d4f5a1e4a8336ac2e01a6ddea85)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Partitions specifying --fsoptions were silently skipped by wic
due to the old bug introduced when removing code related to
subvolume handling:
- if mountpoint == "/" or not fsopts or fsopts.find("subvol=") == -1
+ if mountpoint == "/" or not fsopts:
[YOCTO #9396]
(From OE-Core rev: be7ff1741e8ab5f2724b3f64da1bed8b0d3dcb7c)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
It turns out that dd's conv=sparse doesn't look at the file extents, but simply
checks if a "block" is all zero. If the block of zero was meaningful it gets
lost and if the image is subsequently written to media using a sparse-aware
writer then the block of zeros won't be written at all.
This reverts commit 5fd592fbae2e046bcb8c3a6c3ef4993fe0400676.
(From OE-Core rev: 63d15764cc2014dba9fee2186f0c8b97c2ac5682)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The individual partitions created by wic are sparse but without
this change the assembled image is written as one (potentially
very) large file.
Preserve sparseness in the assembled image by passing the sparse
conversion symbol.
[YOCTO #9099]
(From OE-Core rev: 5fd592fbae2e046bcb8c3a6c3ef4993fe0400676)
Signed-off-by: Joshua Lock <joshua.lock@collabora.co.uk>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When a recipe is added to the workspace, the signatures for the tasks
will change. This means that bitbake must be told to allow the
signatures to be different if they are in locked-sigs.inc.
This is done by creating an unlocked-sigs.inc file which contains all
the recipes in the workspace each time devtool reads the workspace.
So not only will necessary things get added, previously added items will
be removed by virtue of them no longer being in the workspace.
This also makes sure that the extensible sdk picks up unlocked-sigs.inc
as part of the configuration.
[YOCTO #9195]
(From OE-Core rev: 6b2b5ffdcb8bf885a1c756ea132e9d2c55e13dcd)
Signed-off-by: Randy Witt <randy.e.witt@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
NetworkManager can clobber tap devices if left alone, this gives a
note about how to set tap* as unmanaged
[YOCTO #8587]
(From OE-Core rev: 503a80de1cc9097fdc8f6b5b7d0e3a3991b0ecd2)
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fixed when git > 2.1.0:
$ ./scripts/create-pull-request -r HEAD^ -u contrib -b rbt/git
fatal: Not a valid revision: rbt/git
ERROR: git request-pull reported an error
This is because newer git requires both local and remote branch named as
rbt/git, but usually, we only named the remote branch as rbt/foo, and
foo for local branch.
Add a option '-l' to fix the problem, default is HEAD.
(From OE-Core rev: 98faa3ec872e06774b5870fcfb52f3ff91494779)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Coverage subprocessing file is being created even when coverage
is not installed, which causes errors of "module not found" to
be send to the oe-selftest output.
This patch adds indent to the block of code creating this coverage
file, so it can only be executed when coverage is actually
installed.
[Yocto #9334]
(From OE-Core rev: 03d1711350895e941a476bd2d2a6cd389be07509)
Signed-off-by: Humberto Ibarra <humberto.ibarra.lopez@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Coverage temporal file for sub-processing is being removed only
when the --coverage option was found. This is wrong since the
file is created when any coverage option (source, include or omit)
is given, even if --coverage is not one of them.
This patch makes sure to remove the file if any coverage option was
given.
(From OE-Core rev: 7157fd3bfbf2d5ae07fd4e726884baf7138bb24f)
Signed-off-by: Humberto Ibarra <humberto.ibarra.lopez@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There is a message that warns the user about enabling subprocessing
for coverage to work. After the fix for Yocto #8930,
this task is done automatically, so the warning is not needed
anymore.
(From OE-Core rev: 1ccfdb31ea8a50e695d368c4fc78185254aac763)
Signed-off-by: Humberto Ibarra <humberto.ibarra.lopez@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The commit OE-Core:ada4639 (oe-buildenv-internal: simplify derivation
of BB_ENV_EXTRAWHITE) changed the format of BB_ENV_EXTRAWHITE
variable to:
,----
| BB_ENV_EXTRAWHITE='ALL_PROXY
| BB_NO_NETWORK
| BB_NUMBER_THREADS
| ...
| '
`----
Instead of:
,----
| BB_ENV_EXTRAWHITE='ALL_PROXY BB_NO_NETWORK BB_NUMBER_THREADS...'
`----
The old format allow for external script parsing easier and there is
no need to change the format as it has no benefit from usage
perspective.
(From OE-Core rev: 773e9ef9bacbf1e63fe758915d599d11762b8c1f)
Signed-off-by: Otavio Salvador <otavio@ossystems.com.br>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fix typo of the word "parameter" in for 'yocto-bsp create'.
Typo appears in both, usage and help.
Also, the word "description" is mispelled.
[Yocto #9282]
(From meta-yocto rev: 12c7243abd91b374b1b62c6a1ad13b0d25aa0e4c)
Signed-off-by: Humberto Ibarra <humberto.ibarra.lopez@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Invoking oe-pkgdata-util in turn for every package in the list was slow
with a large image. Modify oe-pkgdata-util's read-value command to take
an option to read the list of packages from a file, as well as prefix
the value with the package name; we can then use this to get all of the
package sizes at once. This reduces the time to gather this information
from minutes to just a second or two.
Fixes [YOCTO #7339].
(From OE-Core rev: 51c24904cc1bc823bccc3f179b8d7a192dace168)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
By default cpio preserves the uid&guid's of the original user which
leads to host contamination and boot failures because commands like
mount from initramfs expect to be run by root and the original host
user might not even exist on the target.
(From OE-Core rev: 28910ee2eacc15cf42b5e58bd43b3bd15c34eb97)
Signed-off-by: Ioan-Adrian Ratiu <adrian.ratiu@ni.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Some filenames can omit 'initramfs', or use other names. This makes
detection more flexible by using only the image name, machine arch and
image type in a glob wildcard.
(From OE-Core rev: ca516f5907a661606c35e1ca5c2ece9fc79c77ea)
Signed-off-by: Ioan-Adrian Ratiu <adrian.ratiu@ni.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Kernel recipes for linux-yocto_4.* and greater have outdated branches as default,
making it impossible to find the right branch if the user picks the default value.
The branches_base property uses these outdated branches also.
This updates standard/common-pc and standard/common-pc-64 branches to standard/base.
The fix was tested using 'yocto-bsp create' with each one of the following
archs:
-i386 with kernels 4.1 and 4.4
-x86_64 with kernels 4.1 and 4.4
-qemu (i386 and x86_64) with kernels 4.1 and 4.4
After the layer was created, it was added to local.conf and the MACHINE was set
accordingly.
'bitbake linux-yocto' ran successfully with each configuration tested.
[YOCTO #9160]
(From meta-yocto rev: d471e3dd7c5080a29f64b60b554f17ee706ee772)
Signed-off-by: Humberto Ibarra <humberto.ibarra.lopez@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There appears to have been a lot of copy and pasting of the code
which prints tracebacks upon failure and limits the stack trace to
5 entries. This obscures the real error and is very confusing to the user
it look me an age to work out why some tracebacks weren't useful.
This patch removes the limit, making tracebacks much more useful for
debugging.
[YOCTO #9230]
(From meta-yocto rev: 2f00b2605353b9757a40636870ad20ee70bcab9d)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When installing SDK in a non-default location and the path length
of the SDK install location is longer than the path length of the
default SDK location, relocation of .ldsochache section will overwrite
file location outside of the .ldsocache section size.
In addition, additional checks were added to ensure that any
path in sections .gccrelocprefix and .ldsochache will not exceed
the space allocated for it within the file, which would also result
in file corruption.
[YOCTO #9268]
(From OE-Core rev: 4d949da965a99ab33798af49e5584c8bb9f0f626)
Signed-off-by: Juro Bystricky <juro.bystricky@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Strictly speaking we ought to explicitly shut down a tinfoil instance
when we're done with it. This doesn't affect modify's operation but is
important if you want to be able to call into modify() from another
plugin (though anyone doing so should be advised that the function is
by no means a stable API and is subject to change in future releases).
(From OE-Core rev: 626dbadf22b57a22a8f8b9d1957937120f4ba4d5)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If SDK_EXT_TYPE is set to "minimal" then the SDK won't contain many
sstate artifacts, and you're required to set up an sstate mirror in this
case anyway so there's no point publishing the "stub" sstate-cache
directory from within the SDK since it won't be useful for update
purposes and may be confused with the real sstate-cache.
There is however a possibility that people might publish the real
sstate-cache directory under the same output directory provided to
oe-publish-sdk, thus deleting it after extracting (as we were doing with
other files we wanted to clean up at the end) would be problematic,
besides which extracting it and then deleting it is wasteful. Thus,
introduce a "-p" command line option to the SDK installer that we can
use to tell tar not to extract the items we don't want when publishing.
This has the added benefit of mostly keeping references to these in the
place they belong i.e. in populate_sdk_ext.bbclass.
(From OE-Core rev: 774b85d42db1d81936d4e4af4f6fb2c57cb51d2c)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The SDK argument is expected to be an installer .sh file; if a directory
is specified we can get an ugly failure later on; best to check up
front.
Fixes [YOCTO #9065].
(From OE-Core rev: ce71f5c2fb8a7b473988da30bbb9bec95e8a6f5e)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There appears to have been a lot of copy and pasting of the code
which prints tracebacks upon failure and limits the stack trace to
5 entries. This obscures the real error and is very confusing to the user
it look me an age to work out why some tracebacks weren't useful.
This patch removes the limit, making tracebacks much more useful for
debugging.
[YOCTO #9230]
(From OE-Core rev: 6069175e9bb97ace100bb5e99b6104d33163a3a2)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
It should be the similar type as hddimg, rather than ext234 or btrfs.
(From OE-Core rev: d5ddc5ec1628c94bd5edc45bc821da1ce616e80f)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This fixes the problem by changing the name to the coverage data file,
using the timestamp as an identifier.
The name for the coverage data file is constructed based on the tests
ran; this has created a couple of issues so far, affecting coverage report.
If --run-tests-by option is given, the data file name won't have any
identifier, causing following runs to overwrite themselves. On the
other hand, if too many tests are given, the file name exceeds linux
limits and fails to store the coverage data all together.
[Yocto #9253]
(From OE-Core rev: 7c0c9d1411e04935675f4c4be8768723c1d09b50)
Signed-off-by: Humberto Ibarra <humberto.ibarra.lopez@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When fixing paths for .gccrelocprefix section, it will corrupt the next
entry during updating the current one if "new_prefix" length is more
than "DEFAULT_INSTALL_DIR", this problem is obvious on the code, but it's
only found when install sdk onto a net file system.
(From OE-Core rev: c43192c46bb1759bf6909190b89d3b60918777c5)
Signed-off-by: Wenlin Kang <wenlin.kang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The new class uses gio-querymodules tool on postinst and postrm.
This regenerates the module cache which is useful to avoid loading
modules that are not needed at runtime: If a Gio module is not
listed in the cache file it will always get loaded.
* Add a postinst-intercept 'gio-module-cache': it runs
gio-querymodules using qemuwrapper. This is required because the tool
actually loads the modules to generate the cache.
* Add a gio-module-cache class that adds postinstall and postrm
scripts. In the sysroot population case use the new intercept.
* Inherit the new class in glib-2.0, glib-networking and gconf.
Fixes [YOCTO #9241].
(From OE-Core rev: 168cf5502a2dda78348cc9679e37bed0c69c0208)
Signed-off-by: Jussi Kukkonen <jussi.kukkonen@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We also make sure to correctly support usernames that contain spaces.
For simplicity sed + regex has been replaced with shell parameter expansion,
which works in both, bash and dash.
(From OE-Core rev: ce4ff3dd80379e1bf71a967e0512591aaa046308)
Signed-off-by: André Draszik <adraszik@tycoint.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Type of --no-table option was incorrectly set in new wks parser.
It causes parser to require argument for this option, which makes
wic to fail with wks files that use --no-table:
Error: argument --no-table: expected one argument
Changed action parameter to 'store_true' to fix the issue.
(From OE-Core rev: d483724cf3515f76e1b798a2018e2f3fa2bad0ba)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Use 'sort --unique' to combine the two lists and remove duplicates.
(From OE-Core rev: ada4639f1e2952e144ea7dfb2a784181d7fcc96a)
Signed-off-by: Andre McCurdy <armccurdy@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add the options --codedump and --skip-git-check to the yocto-bsp help and
yocto-bsp usage, since they are currently missing.
[YOCTO #8322]
(From meta-yocto rev: dfdf97b4239639affc8ce22e338a291d4b0dfc76)
Signed-off-by: Humberto Ibarra <humberto.ibarra.lopez@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Use config_<machine> to split the code into separate functions, so that
different machines won't affect each other, and they will have a better
structure.
(From OE-Core rev: 6098a72e8d21f6321636e727703ca7c29d9af142)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* remove akita and spitz related code
They are not supported by runqemu anymore:
$ runqemu spitz
Error: unable to classify arg [spitz]
So remove related code.
* Remove checking of 256M for qemuarm, qemu can check it, for example:
$ runqemu qemuarm qemuparams="-m 1024"
[snip]
qemu: Too much memory for this machine: 1024 MB, maximum 256 MB
[snip]
(From OE-Core rev: 36fb785bf8cd2f387c91d52f597602a5dbc3948b)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(From OE-Core rev: 69a1fca4374797dea56035ce56a17441a2ca9280)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
[YOCTO #9168]
(From OE-Core rev: 0f8306c77b4ebed1ff127b0786b7109abf0d57cd)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
They don't work, and the script can check the type correctly.
(From OE-Core rev: b5cc1e70dbd5df160ddedcaa40d0ab714a307561)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* So that we can add more image support easliy.
* I think that wic should be vm images.
(From OE-Core rev: 82d0014a0e1526ffa1ff7c8ea3903aeae31bada4)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* Make it can boot scsi and virtio block drive such as root=/dev/sdX and
/dev/vdX.
* Drop VM from help info, id doesn't work, and the script can check
whether it is a vm disk or not.
* Make it can be run by:
$ runqemu tmp/deploy/images/qemux86-64/core-image-minimal-qemux86-64.vmdk
or:
$ runqemu qemux86-64 vmdk
[YOCTO #9170]
(From OE-Core rev: 88c081b10902ec52928be78ad320c474bb918e01)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
A typo in the command prevented the last part of the publish from
succeeding.
(From OE-Core rev: 5d51bd1f02fb20e02d0de0ac9874d7f9a6f5ddc6)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This was just copy-and-pasted from devtool - set a reasonable
description and tweak the sdk parameter help text.
Also add a copyright statement and drop the opening comment describing
the command-line syntax that duplicated the help output (with at least
one mistake in it).
(From OE-Core rev: b18f9cb62e12d068fb840fd7ab9f35b4f039e649)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We don't really need the SDK installer in the published output, for two
reasons:
1) The directory produced is for the consumption of devtool sdk-update,
and the installer isn't used by that at all
2) It wouldn't really make sense to point users at the update directory
to download the SDK installer because it contains a bunch of things
that aren't meant for manual download, so it wouldn't be very tidy.
Leaving the file present can mislead you into thinking the opposite of
both of the above.
(From OE-Core rev: a9ec72b7897ffc0b901c1ccbcbe3cabbc7ac41ee)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When we add from a fetched URL we are supposed to turn the resulting
source tree into a git repository (if it isn't already one). However, we
were using the older deprecated option name here instead of the
positional argument, so "devtool add -f <url>" resulted in the repo
being created but "devtool add <url>" didn't, which was wrong.
Also update the oe-selftest tests to check that this worked.
(From OE-Core rev: a7b6b1f8cc1c096724f794ac9dee312b0f771f66)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The npm class just installs whatever is in ${S}; if you're using
externalsrc in conjunction with it the symlinks (oe-workdir and oe-logs
by default) plus singletask.lock will end up in the final package, which
isn't really right. Introduce a variable so we know the path the files
will be installed into within npm.bbclass, and append to do_install
within the workspace bbappend to delete the files.
(From OE-Core rev: 766845e06db9d7d595e836ea1364c16fa132a413)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The code here for running do_configure if it hadn't already been run was
using the wrong string substitution parameters; fix it and test it.
(From OE-Core rev: b2677a4448dbc42e523c731b953b44006749252c)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
By default the sdk-install subcommand expects to restore the requested
items from sstate and fails if it can't. If the user is OK with building
from source, add a -s/--allow-build option to allow them to do that. In
the process, ensure we show the status output while we're installing.
Also add the missing header to the top of the file.
(From OE-Core rev: a86b426cdd465ec5cb08bb5fa7729e4e673d94bb)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Without this, the code that adds paths to $PATH could cause it to end
up with a trailing : which would then cause an error from the sanity
checker.
(From OE-Core rev: 9c200760cbbe322ed884729eb395f389c863e1c8)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* Consistent indentation (four spaces)
* Use [ -z ...] and [ -n ... ] where possible
* Unset temporary variables
* Use $(...) instead of `...`
* Avoid an unnecessary call to expr
(From OE-Core rev: 791eec016792c3f4c04b12ae6ff93c1e23266f87)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Instead of adding all variables to BB_ENV_EXTRAWHITE as one, treat
them separately and add them one by one as needed.
(From OE-Core rev: 516b63fd9dea6fcc304fca920206467d2565dede)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Instead of assuming that the path to the scripts directory always is
in $PATH directly before the bitbake directory, treat them as separate
paths and add them individually to $PATH.
(From OE-Core rev: 5b9e91a3b1d4f56c8646d7d7937ab453cc9c40a0)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Otherwise (if the symlink is named .config) kernel build considers
source tree as dirty and fails.
[YOCTO #9270]
(From OE-Core rev: d36699b28c661880957d744420df9e23cdd76957)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The extra directory next to the recipe should only be created if there
are files to put into it; currently only the npm plugin does this. I
didn't notice the issue earlier because the test was actually able to
succeed under these circumstances if the recipe file came first in the
directory listing, which was a fault in my original oe-selftest test;
apparently on some YP autobuilder machines the order came out reversed.
With this change we can put the oe-selftest test that highlighted the
issue back to the way it was, with an extra check to reinforce that only
a single file should be created.
(From OE-Core rev: b8b778345eb0997c2cd952a1f61fdd2050b6b894)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add short names to most common options in oe-selftest. The options
changed were --run-tests, --run-all-tests, --list-tests and
--list-modules.
[Yocto #9079]
(From OE-Core rev: 05f699e3c2847f0c64118df8ffd43bf2f95ed4d1)
Signed-off-by: Humberto Ibarra <humberto.ibarra.lopez@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The compile() method returns objects that inherit from ast.AST so it's best that
python-core contains this class.
[YOCTO #8684]
(From OE-Core rev: 48bc643a4ebe74d0fa49fedbe7b0fd63fd0003e8)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The oe-selftest script required an exact matching for the parameters
passed to its run-tests-by and list-tests-by options. Many tests
can be retrieved here and filtering is a must.
This patch add this filtering functionality by enabling the use
of wildcards such as "*".
[Yocto #8916]
(From OE-Core rev: 01a242645cc12daed3f4da30d8c49813642d75e6)
Signed-off-by: Humberto Ibarra <humberto.ibarra.lopez@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
"npm shrinkwrap" creates a file that ensures that the exact same
versions get fetched the next time the recipe is built. lockdown is
similar but also includes sha1sums of the modules thus validating they
haven't changed between builds. These ensure that the build is
reproducible.
Fixes [YOCTO #9225].
(From OE-Core rev: 277377f13b2b771915eb853e336ca24b84523ed1)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Allow plugins to create additional files to go alongside the recipe. The
plugins don't know what the output filename is going to be, so they need
to put the files in a temporary location and add them to an "extrafiles"
dict within extravalues where the destination filename is the key and
the temporary path is the value.
devtool add was also extended to ensure these files get moved in and
preserved upon reset if they've been edited by the user.
(From OE-Core rev: 334b9451111b7e3efbb43b3a4eecebcab8ec6f0e)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If the user specifies an npm:// URL then the fetcher needs npm to be
available to run, so check if it's available early rather than failing
later.
(From OE-Core rev: a08d12ad867c292f7474731a0fe5e51e712446d6)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Rather than rolling all of an npm module's dependencies into the same
package, split them into one module per package, setting the SUMMARY and
PKGV values from the package.json file for each package. Additionally,
mark each package with the appropriate license using the license
scanning we already do, falling back to the license stated in the
package.json file for the module if unknown. All of this is mostly in
aid of ensuring all modules and their licenses now show up in the
manifests for the image.
Additionally we set the main LICENSE value more concretely once we've
calculated the per-package licenses, since we have more information at
that point.
(From OE-Core rev: 8226805f83d21e7c1d2ba21969f3e8ee4b137496)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Matching license texts directly to md5sums only goes so far. Some
licenses make the copyright statement an intrinsic part of the license
statement (e.g. MIT) which of course varies between projects. Also,
people often seem to take standard license texts such as GPLv2 and
reformat them cosmetically - re-wrapping lines at a different width or
changing quoting styles are seemingly popular examples. In order to
match license files to their actual licenses more effectively, "crunch"
out these elements before comparing to an md5sum. (The existing plain
md5sum matching has been left in since it's a shortcut, and our list of
crunched md5sums isn't a complete replacement for it.)
As always, this code isn't providing any guarantees (legal or otherwise)
that it will always get the license correct - as indicated by the
accompanying comments the LICENSE values it writes out to the recipe are
indicative and you should verify them yourself by looking at the
documentation supplied from upstream for the software being built if you
have any concerns.
(From OE-Core rev: 553bb4ea5d51be5179e7d8c019740cf61ece76ea)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
For example, this picks up a file named MIT-LICENSE.txt.
(From OE-Core rev: 103b4d26b340cbdf70bf43906e293f3497671fdc)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Make the following improvements to mapping items specified in
AC_CHECK_PROG, AC_PATH_PROG and AX_WITH_PROG to recipes/classes:
* Produce a map of native recipe -> binary for all binaries currently in
STAGING_BINDIR_NATIVE and use this when mapping items
* Add some more entries to the class map
* Ignore autotools binaries since they are covered by the inherit of
autotools
* Ignore coreutils-native since that would almost always be a bogus
dependency
(From OE-Core rev: 5614c5ae6a004d4367eccc34dd3cc7ee61fb7e57)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Allow for whitespace in appropriate places, and ensure we match all
whitespace chars not just the space character.
(This fixes extracting dependencies from tmux's configure.ac, for
example.)
(From OE-Core rev: 63524ac8093b734aa4f29f4ea47bcc036f748314)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add kernel 4.4 support. kernel 4.1 was left in the
tree for user to choose between 4.1 and 4.4.
[YOCTO #9047]
(From meta-yocto rev: c04093ec9b3cd151cf857bca9ad35bc9a0d4a4b1)
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Remove obsolete reference to enabling parallel build via local.conf.
Unify wording used to explain that local.conf and bblayers.conf have
been created based on defaults.
(From OE-Core rev: 0073679c60c813c7c8d62bff997feb30ce64543c)
Signed-off-by: Andre McCurdy <armccurdy@gmail.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The Hob UI is gone from BitBake, so we should remove this shortcut
script.
(From OE-Core rev: 414f0d1ef07bb782ffa15ffd9db648ee2b2d1448)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Clearly I didn't test this part of the code - lists don't have an "add"
method. Needless to say I have tested it now.
(From OE-Core rev: 063ed9058a14775f77e7875d4f6ef5719fa03f18)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a build-sdk command which is only available within the extensible
SDK that builds a derivative extensible SDK. The idea is recipes in the
workspace become a part of the new SDK - for example, this allows taking
a vendor provided SDK, adding a few libs and then producing a new SDK
with those included.
When normally building the extensible SDK, the workspace is excluded;
here we need to copy into the new SDK (renaming it in the process); the
recipes' task signatures become locked and thus the sources are no
longer needed, so they are removed along with the workspace bbappends
which would interfere with the locked signatures. Additionally we need
to just copy the configuration files (i.e. local.conf and auto.conf)
rather than filtering and appending to them since that work has already
been done when constructing the original SDK. The extra sstate artifacts
from workspace recipes are also determined and copied into the new SDK
in minimal mode (on the assumption that you won't set up a new sstate
mirror).
This reuses some code from build-image, so that needed to be
generalised to allow that.
Implements [YOCTO #8892].
(From OE-Core rev: 59e207ff6dd4b50a8905e14bc9292cf2794f4e7a)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Hyphens aren't allowed in python identifiers, so you shouldn't use them
in module names or they are more difficult to import.
(From OE-Core rev: e8f6db9436dfc923e236f2cbc08f357e3b24fd5d)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
"sdk_update" uses a variable newsdk_path, which was never declared.
This would cause the command:
devtool sdk-update <poky-sdk-latest>
to fail with an error:
NameError: global name 'newsdk_path' is not defined
The remedy is to declare newsdk_path as it was no doubt intended,
corresponding to the argument specifying <poky-sdk-latest>.
[YOCTO#9042]
(From OE-Core rev: 6ac589a4a9290665f8c5295e2e2a03a2b187b957)
Signed-off-by: Juro Bystricky <juro.bystricky@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add detection for npm modules and support for extracting the name and
version from package.json as is usually part of an npm module contents.
Note: this will likely only produce a buildable recipe if you use an
npm:// URL; simply pointing to a node.js source repository isn't going
to fetch the module's dependencies. It also doesn't set up the
shrinkwrap/lockdown automatically, so there is some room for improvement
later.
Implements [YOCTO #8690].
(From OE-Core rev: 41d0e4d75f13b53a6c1b6a8df9be4742be7534e0)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add support for detecting a Linux kernel source tree and generating a
basic kernel recipe using meta-skeleton's linux-yocto-custom recipe as a
base.
Implements [YOCTO #8981].
(From OE-Core rev: 39cab544b80ca4450106c9ede3180929ba24703c)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Detect kernel modules by looking for #include <linux/module.h>, and
handle the various styles of Makefile that appear to be used. I was able
to use this code to successfully build a number of external kernel
modules I found.
Implements [YOCTO #8982].
(From OE-Core rev: a85604f2eb2438b4caf0832c2ea15b5822f7e9a1)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When importing the components during the "combo-layer init" with full
history and relocation into a destination directory, components with
empty commits were not handled because the "mv" command was invoked
with just one parameter.
Replacing that with a find/xargs pair avoids the problem and should
also fix the handling of filenames with special characters (in
particular spaces).
(From OE-Core rev: fd9d08b8ba512151615a03fda61a103760a0683c)
Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We were creating the partition, mkswap'ing it, and then not dd'ing it into
place in the final image, as source_file wasn't being set for swap partitions.
This would result in a swap partition that couldn't be enabled on target until
mkswap was run.
(From OE-Core rev: 7580942a66f84c2120763c2d9665e2cec8e25138)
Signed-off-by: Christopher Larson <chris_larson@mentor.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Make this consistent with "devtool add" so that the user knows where to
find the new recipe.
(From OE-Core rev: 28fea4a104ba85aded2ecfe891e9bf997d72a103)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The PR value should be reset to the default when upgrading, so we need
to drop it from the newly created file.
(From OE-Core rev: 4acf8a6912106dd3da5ceee9e30ccce86211d04b)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We aren't modifying the datastore copy here, so we don't need a copy at
all.
(From OE-Core rev: 9f05bf98d4dbcb991b684a2ab480c6638c6be292)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fix several issues when extracting the new version source over the top
of the old one (when the recipe is not fetching from a git repo):
* Delete the old source first so we ensure files deleted in the new
version are deleted. This also has the side-effect of fixing any
issues where files aren't marked writeable in the old source and thus
overwriting them failed (harfbuzz 1.1.3 contains such files).
* Fix incorrect variable name in abspath statement that made it a no-op
(From OE-Core rev: 4f7595a514661f0e2418393286849a1cb6c46f0d)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When we do an upgrade from one tarball version to another we want to:
1) Check out the old version as a new branch
2) Record the changes between the old and new versions as a commit
3) Check out the old version with patches applied
4) Rebase that onto the new branch
Where we went wrong was step #1 where instead we checked out the old
version with patches applied as the new branch, which meant the rebase
didn't do anything and any changes made by the patches to files still in
the new version were wiped out.
(From OE-Core rev: a4f3e6d5f4c549e36cce1e18801a565a3a3c9b0a)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If the actual value of PV isn't in the name of the recipe (for example,
a git or svn recipe) there's no point trying to rename it. Additionally,
we already have the original filename, there's no need to guess it -
just pass it in.
(From OE-Core rev: e144db3650b4a7c0a59066621905f34a5400303e)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We were trying to move this from the current directory instead of the
path. Let's just use shutil.move() instead of shelling out to mv.
(From OE-Core rev: 60454a0ba154d6c777e0c2b05b887b4e4fcde986)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
For recipes that specify SRCREV, the code here wasn't quite doing the
right thing. If the recipe has a SRCREV then that needs changing on
upgrade, so ensure that the user specifies it. If it doesn't, then it'll
be "INVALID" not None since the former is the actual default, so handle
that properly as well. Additionally an unset variable was being
erroneously passed when raising the error about the version being the
same leading to a traceback, so fix that as well.
(From OE-Core rev: 1d0f821371d1cb93e30fad86f0c20e38cb93b54b)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The recipename argument to devtool upgrade specifies an existing recipe,
so by definition the name will be valid (or it won't exist) - we don't
need to validate it ourselves, that's only needed for situations like in
devtool add where we're creating a new recipe.
(From OE-Core rev: 1e2bc7d861555a04350a87f19047efdc717046be)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Make devtool upgrade consistent with devtool add/modify in defaulting to
sources/<recipename> under the workspace if no source tree path is
specified.
(From OE-Core rev: 8a952407b192313515e91570632446b6dff01665)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If you for example ran devtool modify virtual/libusb0 without specifying
a source tree path, the default was <workspace>/sources/virtual/libusb0
which isn't correct - it should be using the mapped name i.e.
libusb-compat (in the default OE-Core configuration). Reorder some of
the code to ensure that the mapped name is used.
(From OE-Core rev: c51736df17da8e6e561dd5b7ce59cb08254da870)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
As suggested by Khem Raj.
(From OE-Core rev: 36cc6b81d0281348a0f241a80ddd427745a6a678)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This patch adds functionality to allow creating subdirectories inside
lib/oeqa/selftest for all layers present in BBLAYERS. Like this, test
cases can be grouped into organized directories.
Addresses [YOCTO #7865]
(From OE-Core rev: 445b84456659ebb355149f6b0dca543b0bb2679c)
Signed-off-by: Costin Constantin <costin.c.constantin@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Create config fragment if the user makes modifications to kernel config.
User may change .config e.g. by directly editing it or by running the
'do_menuconfig' bitbake task. Devtool generates one monolithic fragment
by simply doing a diff between .config and .config.baseline files in the
source directory. If either of these files is missing, the config
fragment is not gerenrated or updated. The output is a file,
'devtool-fragment.cfg' that gets added to SRC_URI in the recipe (as well
as copied into the 'oe-local-files' directory if that is present in the
source tree).
${S}/.config will be a symlink to ${B}/.config. We need to do this as
devtool is not able to access ${B} because ${B} is set in a .bbappend in
the workspace layer which is not parsed by devtool itself.
[YOCTO #8999]
(From OE-Core rev: 524da136e5b837a60682516ac08f3092c635e934)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Copy kernel config is copied to the source directory at a later phase in
_extract_source() so that it gets copied when devtool sync is done, too.
(From OE-Core rev: ff895be7a46c4b3b1b791e5387490d90bb34fce2)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Coverage in oe-selftest currently requires to create or modify
a sitecustomize.py file according the coverage tool setup instructions
(http://coverage.readthedocs.org/). This file has to be located in
the system's python folder, which is not a good solution since this
folder is not accesible to non-privileged users.
The best solution so far is to create this file in the home directory.
This is implemented by creating the temporal file in the user site
default folder.
[Yocto #8930]
(From OE-Core rev: 3f9b1658d745b536eff1017b2c74a9dff46b6f4a)
Signed-off-by: Humberto Ibarra <humberto.ibarra.lopez@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When --coverage is used, oe-selftest gathers coverage data from the testcases
executed. The command lacks a way of filtering which files to gather coverage
data from.
This patch adds three options to specify which files should be considered.
The --coverage-source option specifies folders, while --coverage-include and
--coverage-omit specify patterns to have an extra level of filtering.
Some examples:
1. oe-selftest --run-all-tests --coverage
Gathers coverage data from the default poky folders
2. oe-selftest --run-all-tests --coverage --coverage-include /home/me/poky/scripts/*
Gathers coverage data only for the files located under '/home/me/poky/scripts'
3. oe-selftest --run-all-tests -coverage --coverage-omit /home/me/poky/meta*
Gathers coverage data. Files inside all the folders starting with 'meta' under
'/home/me/poky' are omited
4. oe-selftest --run-all-tests --coverage --coverage-source /home/me/poky/bitbake
Gathers coverage data only from files inside the folder: '/home/me/poky/bitbake'
[Yocto #8920]
(From OE-Core rev: 923481c7d8c09ed9b03109cf4debcc6b07845c59)
Signed-off-by: Humberto Ibarra <humberto.ibarra.lopez@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
"poky" is the reference distribution for the Yocto Project. This renames
the layer within the meta-yocto repository to meta-poky, better matching
what that layer contains.
A layer.conf file is left behind as this is the only way which allows
existing builds to migrate safely to the new name. It will be removed
at some future point.
This change requires the corresponding OE-Core change to handle the
migration and the changes to the infrastructure to support this.
(From meta-yocto rev: d0c88df2e14672fca4ebbde93c5efbcd0e4fa9b6)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* Package names are actually case sensitive near as I can tell, so we
shouldn't be lowercasing them everywhere.
* Look for CMake packages in pkgdata and map those back to recipes,
so we aren't dependent on the hardcoded mappings (though those are
still preserved).
* Avoid duplicates in the unmapped package list
(From OE-Core rev: 2ddad52ccca07245eea43d9b844c6c7d4b667ca3)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a means of extending the dependency extraction for autotools and
cmake.
Note: in order to have this work, you need to have an __init__.py in the
lib/recipetool directory within your layer along with the module
implementing the handlers, and the __init__.py needs to contain:
# Enable other layers to have modules in the same named directory
from pkgutil import extend_path
__path__ = extend_path(__path__, __name__)
(From OE-Core rev: 915dea9f89cd737e5ba167c384e8d314c5c23c49)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
I should have adjusted this in OE-Core commit
80a44e52609a89d9ffe816181ae193af491c06ac where the behaviour changed.
(From OE-Core rev: 13409a2b899bb74b8060c840b8c7ef8759d099cb)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If files would be overwritten by the deployment, preserve them in a
separate location on the target so that they can be restored if you
later run devtool undeploy-target.
At the same time, also check for sufficient space before starting the
operation so that we avoid potentially failing part way through.
Fixes [YOCTO #8978].
(From OE-Core rev: a2da55712691bcf1066c53d813107d75213d6b10)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If you want to put the target device back to exactly how it was before
devtool deploy-target started poking things into it, then it would make
things easier if you didn't have to figure out which recipes were
deployed. Now that we have the list stored on the target we can
determine this reliably, so add a -a/--all option to undeploy-target to
undeploy everything that has been deployed.
One of the side-effects of this is that the dry-run functionality for
undeploy-target had to be reimplemented to actually run the script on
the target, since we have no way of knowing what's been deployed from
the host side. We don't need to do the same for deploy-target though
since we know exactly which files will be deployed without referring to
the target.
(From OE-Core rev: 41fed83060f0041e14e455d1446397bda277d953)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When running devtool deploy-target, we save a list of deployed files,
and this list is used by devtool undeploy-target (or the next time
deploy-target is run if the list is present, in case any files have been
renamed or deleted since the first time). We were writing this file to
the host, but it makes more sense to write the list to the target
instead, so that if we for example swap in a different board, or switch
hosts, things will work as expected.
In order to do this properly we have to construct a shell script and
ship it over to the target so we can run it. The manifest is written out
to a hidden directory in the root (/.devtool).
Fixes [YOCTO #7908].
(From OE-Core rev: a16a0c9334b785e2df896266c8911a2c7a1806b8)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Get the default value for updateserver from the configuration file and
show it in the help; also only make the parameter optional if it's
specified. This means we can also drop the check in the function as
argparse will then ensure it's specified if there's no config setting.
(From OE-Core rev: 82497bde58fc8bdae8d8acfbf025a1a90b14cc3e)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a long description used when running --help on the specific command.
(From OE-Core rev: eb7787d1652fd84a149fd394969f4f1099406051)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Naming these as "optional arguments" is perhaps slightly confusing since
some of the positional arguments might also be optional; in addition
it's rare (though possible) for options to be mandatory - up until
recently we had a recipetool option (-o) that was mandatory. It's not
perfect, but change it to "options" so it's at least a bit more
appropriate.
(From OE-Core rev: 55e675de6191bf7eccd26df29189f2a6faa40a20)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The listing of subcommands in the --help output for devtool was starting
to get difficult to follow, with commands appearing in no particular
order (due to some being in separate modules and the order of those
modules being parsed). Logically grouping the subcommands as well as
being able to exercise some control over the order of the subcommands
and groups would help, if we do so without losing the dynamic nature of
the list (i.e. that it comes from the plugins). Argparse provides no
built-in way to handle this and really, really makes it a pain to add,
but with some subclassing and hacking it's now possible, and can be
extended by any plugin as desired.
To put a subcommand into a group, all you need to do is specify a group=
parameter in the call to subparsers.add_parser(). you can also specify
an order= parameter to make the subcommand sort higher or lower in the
list (higher order numbers appear first, so use negative numbers to
force items to the end if that's what you want). To add a new group, use
subparsers.add_subparser_group(), supplying the name, description and
optionally an order number for the group itself (again, higher numbers
appear first).
(From OE-Core rev: e1b9d31e6ea3c254ecfe940fe795af44761e0e69)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If we didn't make any changes to the file then there's no point warning
the user that we have done.
(From OE-Core rev: 391b9ba30d802ac420ddf382588e03e718861c01)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If files had been created next to the recipe (for example devtool add,
edit the source and commit and then devtool update-recipe), running
devtool reset failed to preserve those files and gave an error due
to trying to rmdir the directory containing them which wasn't empty.
Fix the preservation of files in the "attic" directory properly so
we catch anything under the directory for the recipe, and replicate
the same structure in the attic directory rather than slightly
flattening it as we were before.
(From OE-Core rev: bbe63eb97ae7f78959f117d6066ef821c4da1c77)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Looking at Chris Larson's code for starting the user's editor for
"recipetool newappend" it was slightly better than what I wrote for
"devtool edit-recipe" in that it checks VISUAL as well as EDITOR and
defaults to vi if neither are set, so break this out to its own function
and call it from both places. The broken out version passes shell=True
however in case it's a more complicated command rather than just a name
of an executable.
(From OE-Core rev: 184a256931e8cdc7bea97a905c4e67a435964de0)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There is no -N/--name option for devtool, that's a recipetool option -
with devtool you just specify the name as a positional argument.
(From OE-Core rev: d2bc0cba5ca8a7220ffe1ef96acf856fe972ce7c)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If the user specifies --basepath on the commandline, only the directory
specified should be searched for .devtoolbase. Otherwise when --basepath
is a child of the sdk directory, .devtoolbase will always be found and
devtool will only show options meant to be used within an sdk.
(From OE-Core rev: 3bb07c61220b89384a76c745e33be37aad5095c0)
Signed-off-by: Randy Witt <randy.e.witt@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We have an issue with PAT handling on older processes with limited PAT bits,
see the patch description for the full problem. This replaces the runqemu
workaround with a kernel patch until we can get the kernel trees sorted
out and discuss a proper fix with upstream. It should be safe everywhere
so is applied unconditionally.
(From OE-Core rev: e00f0794a535c8e68ae1c87c8b01dd65645d570b)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Processing of this option was lost during recent change of
wks parsting. It was discovered during the work on booting
wic images under qemu. Now, when -use-uuid is fixed it's
possible to specify root partition by partition uuid.
This will be done in the following commit.
(From OE-Core rev: b4882e0b84d7fd4c85ee95386e94722485eafc2b)
(From OE-Core rev: 73e9e3f150bf2de9b27c2ccc73e3dee334ee73fe)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Wic images should be boot as is, without pointing qemu to the kernel
binary. Current code doesn't use kernel, but sets KERNEL variable and
shows kernel path in the console output. This can confuse users.
Changed runqemu and runqemu-internal code to avoid setting KERNEL
variable and show kernel path.
(From OE-Core rev: 474caa7ed5ff05caa5d49d270b283882fa616ed1)
(From OE-Core rev: 35e776e00cce25f2c9c45500612fb66022ec4739)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Quemu should be able to run wic images this way:
runqemu <machine> <image recipe> wic
Tested with 'runqemu qemux86-64 wic-image-minimal wic'
(From OE-Core rev: 8716be799949cb8bde7fa49cbea61312a3a93bb7)
(From OE-Core rev: dd42931bf99b8bbd4ad452b3941d957f41b81796)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Use "out of" rather than "from" in the output.
(From OE-Core rev: c712529030c71b45bb8ae647c17319c2647aedff)
Signed-off-by: Jan Sarenik <jajomojo@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Since the upgrade of qemux86 to 4.4.1 we're seeing PAT issues when
starting the X server. We need to fix the problem but the failing
sanity tests mask out other issues and we need a workaround.
Merge this for now until we can figure out the full issue. This is
better than changing the kernel defconfig or reverting to old
kernel versions.
(From OE-Core rev: 2749ba318bf322cb5014689532372473004e92b9)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If you specify a local directory which happens to be a git repository
with an origin remote (and it is in fact remote), we can use that for
SRC_URI as implemented by OE-Core revision
b143d414846854dc8b3e0a47358daf5646eded38, however we also need to set S
if the recipe is going to be of any use fetching from that SRC_URI
later.
(From OE-Core rev: 8bbbd2d63f1bc752f9a30054a089dc2caf5fd84c)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When recipetool create is given a URL that starts with http(s):// and
contains /git/, such as the URLs at git.yoctoproject.org, it's fairly safe
to assume it's a git repository and not something that should be fetched
with wget, so rewrite the URL.
(From OE-Core rev: 3ca04757a670e8b6f78799cc0454d75691809ac4)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When auto-detecting the name for a recipe from the URL, strip off any
parameters (";name=value...") before parsing the URL, otherwise this
just ends up in the recipe name which is undesirable.
(From OE-Core rev: d3c46b5d0abd56bcadd4f2f1ef985f13d67f605b)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fix a regression introduced in in OE-Core revision
aedfc5a5db1c4b2b80a36147c9a13b31764d91dd where specifying a local source
tree without specifying a name resulted in a traceback.
Fixes [YOCTO #9086].
(From OE-Core rev: 67ea9d21f20a371a0ee443b46326018cb1527b62)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
It's going to be more common for users not to have the prepared source
tree for a recipe already, so the default behaviour ought to be to
extract it for them from the recipe. Change the default to extract
(effectively making the -x option a no-op) and add a --no-extract/-n
option to disable it. Later we can look at trying to be smart and
reusing an existing source tree instead of erroring out if it exists;
for now this is just the default reversal.
(From OE-Core rev: 80a44e52609a89d9ffe816181ae193af491c06ac)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Both BitBucket and GitHub provide "release" downloads that unfortunately
don't include the actual project name. For recipetool this means we have
to have special handling for these URLs or else the current code assumes
the name of the project is something like "v0.9.1". Borrow and adapt
some code from autospec to do this, and put it in its own function for
clarity.
(From OE-Core rev: e8435fde7f82c39cc0efe1d4bf043b29f63898e9)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
CMake supports a find_library() directive to find named libraries, so
detect dependencies from this.
(From OE-Core rev: d0265bc67f797ee4b7760cf37335994133809abf)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When extracting source for a recipe, if there are additional custom
tasks run that make changes to the source, create a commit in the
generated git branch so they are contained. This is particularly
useful for tasks that come before do_patch since otherwise the changes
might get incorporated in the first applied patch, but otherwise it
helps avoid the tree being dirty at any point.
Fixes [YOCTO #7626].
(From OE-Core rev: 997a77d9b20af1778b804778e5d8c8a7424f7582)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
python-xml depends on python-elementtree as the latter just contains a C library
used by the former. However there's no point to this split apart from
increasing the number of packages, so merge -elementtree into python-xml.
(From OE-Core rev: 5f7206eba3953b7f29148ecfb791995773ee5fc7)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Make path to newly generated README file legitimate
(From meta-yocto rev: 32286bb798c2778457b5578b4b590629c96a0ee2)
Signed-off-by: Maciej Borzecki <maciej.borzecki@open-rnd.pl>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Take `pwd` to be <initial-dir>. The %s path is relative to it. The value
of %s is "output_folder/build". The current code works as follows:
Changing directory to %s and finding the sources (after cd'ing) to cpio
with output redirection to %s/initrd.cpio triggers the following error
"Error: exec_cmd: cd output_folder/build/INITRD && find . | cpio -o -H
newc >output_folder/build/initrd.cpio returned '1' instead of 0"
This happens because after the cd, `pwd` is <initial-dir>/%s and by the
redirect we write the result to to <initial-dir>/%s/%s/initrd.cpio which
obviously does not exist.
Fix this by getting the sources with "find %s" instead of "cd && find ."
(From OE-Core rev: 07fa4783566d22d46ce719a621eee5404932dbbe)
Signed-off-by: Ioan-Adrian Ratiu <adrian.ratiu@ni.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
.iso image creation fails if during the image creation syslinux
is baked and syslinux-native is not.
Added new check to verify if both syslinux and syslinux-native
are baked and bake them if these are not installed.
(From OE-Core rev: fd5749832960ad3b85697c2878490d6f008982a3)
Signed-off-by: Mihaly Varga <mihaly.varga@ni.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
A kickstart file for non-x86 boards may have no 'bootloader' stanza. It
is the usual case if bootloader is setup using other mechanism than
through wic, and is for instance a part of u-boot configuration. In such
case the 'bootloader' field in the KickStart class will be
uninitialized. Instead of adding an empty bootloader line in every
kickstart file call the bootloader parser with empty argument list to
get defaults namespace.
(From OE-Core rev: 264c03e854f77c3b62acb710384f66716ccbf469)
Signed-off-by: Maciej Borzecki <maciej.borzecki@open-rnd.pl>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When used with --verbose, the heading for each task looks like
=== The verbose changes of example.do_do_compile:
This should instead be
=== The verbose changes of example.do_compile:
(From OE-Core rev: 628ad5e06d1136809d110a71148721095cb084dc)
Signed-off-by: Olof Johansson <olof.johansson@axis.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When copying the sstate-cache into the extensible SDK, if the source
path had a trailing / and the destination path did not, there would be a
missing / between the path and the subdirectory name, and you'd end up
with subdirectories like "sstate-cacheCentOS-6.7". There are functions
in os.path for this sort of thing so let's just use them and avoid the
problem.
(From OE-Core rev: 5eb8f15c48b5f39a10eb2b63b026cf1ebfd05533)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The uninative sysroot is in ${STAGING_DIR}-uninative so delete that alongwith
$STAGING_DIR.
(From OE-Core rev: 258668c3cb8f5c00e084e821dae05ba750768bfb)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When you need to set EXTRA_OECONF for a recipe, you need to know what
options the configure script actually supports; the configure script
however is only accessible from within a devshell and (at least in the
case of autotooled software fetched from an SCM repository) may not
actually exist until do_configure has run. Thus, provide a "devtool
configure-help" subcommand that runs the configure script for a recipe
with --help and shows you the output through a pager (e.g. less),
prefaced by a header describing the current options being specified.
There is basic support for autotools, cmake and bare configure scripts.
The cmake support is a little hacky since cmake doesn't really have a
concise help option that lists user-defined knobs (without actually
running through the configure process), however that being a design
feature of cmake there's not much I can think of to do about that at
the moment.
(From OE-Core rev: 0e5d84d9705091b338000ef02720cfa090f76888)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Alistair Francis
Ed Bartosh
Richard Purdie
Humberto Ibarra
Jussi Kukkonen
Allen Wild
Paul Eggleton
Christopher Larson
Ross Burton
George McCollister
Raymond Tan
Robert Yang
Patrick Ohly
Randy Witt
Leonardo Sandoval
Ioan-Adrian Ratiu
Peter Kjellerstedt
Stephano Cetola
Joshua Lock
Saul Wold
Otavio Salvador
Juro Bystricky
Wenlin Kang
André Draszik
Andre McCurdy
Peter Kjellerstedt
Daniel Istrate
Markus Lehtonen
Mariano Lopez
Costin Constantin
Jan Sarenik
Maciej Borzecki
Dominique Hunziker
Mihaly Varga
Olof Johansson