AX_PKG_SWIG is not the only commonly-used macro for detecting swig -
there's also AC_PROG_SWIG. As per AX_PKG_SWIG, add swig-native to
DEPENDS if AC_PROG_SWIG is found in configure.ac.
(From OE-Core rev: 847a1aa7153fc8a7b820353283a6f1e51d64f8de)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If python is required then we need to inherit pythonnative (or
python3native) otherwise do_configure will probably fail since it won't
be able to find python.
(From OE-Core rev: 63234cc45aee91b031657971f36997e1443f80ee)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When creating a recipe for an existing local git clone, we attempt to
use the fetcher to determine if it supports the SRCREV variable.
Unfortunately running this code does a network check to get the latest
revision as a direct result of us using '${AUTOREV}' as a default value.
If you don't have a network connection this will of course fail. Rather
than have this block creating the recipe, catch the exception and just
guess from the URL.
Ultimately this should probably be fixed in the fetcher but for now this
will at least resolve the issue on this end.
(From OE-Core rev: f7e43f931d7d6019a3b2509b2b2635978fbbae36)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
I ran into an example where recipetool was getting the name/version
completely wrong:
https://bitbucket.org/sortsmill/libunicodenames/downloads/libunicodenames-1.1.0_beta1.tar.xz
>From this it would create a libunicodenames-1.1.0-beta1_1.1.0-beta1.bb
file (likely because it couldn't split the file name and therefore took
all of it, then got the version from one of the files inside the
tarball). When this happens it's just irritating because you then have
to delete the recipe / run devtool reset and then run recipetool create
/ devtool add again and specify the version manually.
This patch is the result of systematically running the
determine_from_filename() function over the files on the Yocto Project
source mirror and my local downloads directory and fixing as many of the
generic issues as reasonably practical - it now gets the name and
version correct much more often. There are still cases where it won't,
but they are now in the minority.
(From OE-Core rev: 7b018b1d493a8d10fd02b8cc220990b191c87fe5)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Try to ensure that for Apache, GPL and LGPL where the values extracted
from the "Classifiers" field may not be version-specific, if there is a
versioned license in the free-form license field then use that instead.
Also insert the free-form license field as a comment in the recipe for
the user's reference.
(From OE-Core rev: 237f66042eedd906f654827b53bf9269738267ab)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Make use of the extravalues dict to send back other variable values from
the python handling plugin, and enable passing back PV and PN. This not
only places variable values in the final recipe a bit more consistently
with other types of source, it also allows the name and version to be
picked up fron a local source tree and not just when the recipe is
fetched from a remote URL that happens to have those in it.
(From OE-Core rev: 3e7029f28c6ea9bb1d283bcdc3fdfee11455af8e)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If we output extra blank lines (because of some automated editing) then
it makes the output recipe look a bit untidy. You could argue that we
should simply have the editing code not do that, but sometimes we don't
have enough context there for that to be practical. It's simple enough
to just filter out the extra blank lines when writing the file, so just
do it that way.
(From OE-Core rev: cbebc9a2edf7d7a422ee5c71219e79e3b349de3b)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If you have your own node.js application you may not publish it (or at
least not immediately) in an npm registry - it might just be in a
repository on github or on your local machine. Add support to recipetool
create for creating recipes to build such applications - extract their
dependencies, fetch them, and add corresponding npm:// URLs to SRC_URI,
and ensure that LICENSE / LIC_FILES_CHKSUM are updated to match. For
example, you can now run:
recipetool create https://github.com/diversario/node-ssdp
(I had to borrow some code from bitbake/lib/bb/fetch2/npm.py to
implement this functionality; this should be refactored out but now
isn't the time to do that refactoring.)
Part of the fix for [YOCTO #9537].
(From OE-Core rev: 4fb8b399c05a1b66986fc76e13525f6c5e0d9b58)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If you make adjustments to the source tree (as create_npm.py will be)
then you will need to re-run the license variable handling code at the
end so that we get all of the files that should go into
LIC_FILES_CHKSUM if nothing else. Split out the license variable
handling to a separate function in order to allow this.
(From OE-Core rev: f0d6f4b7e87ea781ac0dffcc8d0310570975811b)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
For debugging it's useful to be able to tell recipetool to keep the
temporary directory.
(From OE-Core rev: 480a6b745a85b2881e5cc1a0bbb572e3235ca008)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Ensure we fetch submodules and set SRC_URI correctly when pointing to a
git repository that contains submodules.
(From OE-Core rev: 65d5cc62d4ecfc78ce4b37b3886a7fe5aa05a75e)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When trying to map python module dependencies to the packages that
provide them, if we're looking for .so files that satisfy
dependencies then we need to exclude files found under the .debug
directory, otherwise the dependency will get mapped to the python-dbg
package which isn't correct.
For example, this fixes creating a recipe for pyserial and not getting
python-fcntl in RDEPENDS_${PN}, leading to errors when trying to use the
serial module on the target.
(From OE-Core rev: 46a068ca35975988a8e9c0310f71fdcee55937a4)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If AX_PKG_SWIG is found in configure.ac, then what's being looked for is
the swig binary, not swig for the target - so fix the dependency
accordingly.
(From OE-Core rev: 2600cd6f6c63ecf79804e2bc6eb6f198a012d5d6)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
It's rare but there are recipes that have individual files (as opposed
to archives) in SRC_URI using subdir= to put them under the source tree,
the examples in OE-Core being bzip2 and openssl. This broke devtool
update-recipe (and devtool finish) because the file wasn't unpacked into
the oe-local-files directory and thus when it came time to update the
recipe, the file was assumed to have been deleted by the user and thus
the file was erroneously removed. Add logic to handle these properly so
that this doesn't happen.
(We still have another potential problem in that these files become part
of the initial commit from upstream, which could be confusing because
they didn't come from there - but that's a separate issue and not one
that is trivially solved.)
(From OE-Core rev: 9069fef5dad5a873c8a8f720f7bcbc7625556309)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We should always shut down tinfoil when we're finished with it, either
by explicitly calling the shutdown() method or by using it as a
context manager ("with ...").
(From OE-Core rev: 5ec6d9ef309b841cdcbf1d14ac678d106d5d888a)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When using PATCHTOOL = "git", the user of the system is not really the
committer - it's the build system itself. Thus, specify "dummy" values
for username and email instead of using the user's configured values.
Various parts of the devtool code that need to make commits have also
been updated to use the same logic.
This allows PATCHTOOL = "git" and devtool to be used on systems where
git user.name / user.email has not been set (on versions of git where
it doesn't default a value under this circumstance).
If you want to return to the old behaviour where the externally
configured user name / email are used, set the following in your
local.conf:
PATCH_GIT_USER_NAME = ""
PATCH_GIT_USER_EMAIL = ""
Fixes [YOCTO #8703].
(From OE-Core rev: 765a9017eaf77ea3204fb10afb8181629680bd82)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The missing split() causes dev and dbg packages to match.
(From OE-Core rev: bf83e0f0a3d52958c4380599f1afc4b8e058afd7)
Signed-off-by: Ola x Nilsson <ola.x.nilsson@axis.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The --wilcard-version flag was only used in the srcrev variant of the
update-recipe command.
(From OE-Core rev: d3057cba0b01484712fcee3c52373c143608a436)
Signed-off-by: Ola x Nilsson <ola.x.nilsson@axis.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Unfortunately to implenent multiconfig support in bitbake some APIs
had to change. This updates code in OE to match the changes in bitbake.
Its mostly periperhal changes around devtool/recipetool
[Will need a bitbake version requirement bump which I'll make when merging]
(From OE-Core rev: 041212fa37bb83acac5ce4ceb9b7b77ad172c5c3)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
With some recent changes in the kern tools, we can drop some changes in
the yocto-bsp and yocto-kernel tools that ensured proper patching and
BSP inheritance.
In particular, we no longer need to signify the start of patching, and
we must instruct the tools that we only want configuration fragments
via inheritance .. no patches (since they are already applied).
(From meta-yocto rev: 34ed5eebd0b5baab98b6b2d7b3f06ca40932b37d)
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The regex here needs to be anchored to the end or it'll match longer
URLs, which was exactly what I was trying to avoid. This regression was
introduced in OE-Core revision 7998dc3597657229507e5c140fceef1e485ac402.
Fixes [YOCTO #10023].
(From OE-Core rev: 9291c5d3c257d5ada7605dfe46ababda08f6d3c1)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a comment to the recipe listing license files that were found but
not able to be identified, so that the user can find and examine them
by hand fairly easily.
Fixes [YOCTO #9882].
(From OE-Core rev: 4b7d1bf8172533e9ac91a49ade152a05e2ee4146)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Rawcopy plugin copies source files to build folder before using them
to assemble result image. After assembling the image wic renames
source files to <image>.p<partition number>. If the same source file
is used in multiple partitions wic breaks trying to rename file that
doesn't exist.
Added <line number> suffix to the files when copying them to the
build dir. This should make filename unique even if the same source
file is used for multiple partitions.
[YOCTO #9826]
(From OE-Core rev: 43a809bfe99024083b4ab4eb9895b084c9c4fa80)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a subcommand which will "finish" the work on a recipe. This is
effectively the same as update-recipe followed by reset, except that the
destination layer is required and it will do the right thing depending
on the situation - if the recipe file itself is in the workspace (e.g.
as a result of devtool add), the recipe file and any associated files
will be moved to the destination layer; or if the destination layer is
the one containing the original recipe, the recipe will be overwritten;
otherwise a bbappend will be created to apply the changes. In all cases
the layer path can be loosely specified - it could be a layer name, or
a partial path into a recipe. In the case of upgrades, devtool finish
will also take care of deleting the old recipe.
This avoids the user having to figure out the correct actions when
they're done - they just do "devtool finish recipename layername" and
it saves their work and then removes the recipe from the workspace.
Addresses [YOCTO #8594].
(From OE-Core rev: fa550fcb9333d59b28fc0e4aebde888831410f5c)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This provides us with the information we need to remove the original
version recipe and associated files when running "devtool finish" after
"devtool upgrade".
(From OE-Core rev: 92eb42c347af919cd9f8739515fdf806c12b5ba8)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This will be called by "devtool finish" to allow it to update the recipe
or create the bbappend depending on the destination.
(From OE-Core rev: 5067cdc73483b53d46d9bf584723e41957c7ec54)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This will be called by "devtool finish" to allow it to reset the recipe
at the end.
(From OE-Core rev: b8d398516556eaf97679e28ad58448f570984b52)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If there are files in the oe-local-files directory which are identical
to the original version, then we shouldn't be copying them to the
destination layer. This is particularly important when using the -a
option to create a bbappend.
(From OE-Core rev: 9230bfcc839eb35630949f0a8ed058ca1fa944b1)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
devtool update-recipe was defaulting to the ${BPN} named directory when
adding patches next to a recipe, but that meant if you already had files
in a ${BP} named directory (i.e. name and version) or "files" then you'd
end up with two directories next to the recipe, which is usually not
what you want. To avoid this, look through FILESPATH and take the first
one that's the same level or one level down from the recipe and already
exists, if any.
(From OE-Core rev: c7a8190cf8bdf86ba850b6780b8e951e90232c06)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
According to wic documentation partition mount point is an optional
argument. Skipping mount point also makes sense in certain
configurations when one needs to specify a partition that is not mounted
by the running system, such as a recovery or a mirror partition (in dual
rootfs setups).
(From OE-Core rev: 5e063a4c6bb0e0623a4d25bb2bf6eecd9ad6b9f1)
Signed-off-by: Maciej Borzecki <maciej.borzecki@rndity.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This wks is just a copy of gummiboot wks with bootloader
changed to systemd-boot. A very basic boot test on a X86
target is done with a direct-boot image generated by:
wic create mksystemd-bootdisk -e core-image-minimal
Because we plan to replace gummiboot with systemd-boot at any
time in the future, we summarize history (as much as I can) of
the current gummiboot wks before it's gone:
-----------------------------------------------------------------
commit 7d4bb40905
Author: Ed Bartosh <ed.bartosh@linux.intel.com>
Date: Wed Sep 2 13:58:02 2015 +0300
wic: get rid of scripts/lib/image
Moved content of scripts/lib/image/ to scripts/lib/wic as
one directory with the same name as a tool is self-explanatory
and less confusing than two.
(From OE-Core rev: 5dc02d572794298b3362378cea3d7da654456c44)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
commit 77561e7191
Author: Ed Bartosh <ed.bartosh@linux.intel.com>
Date: Wed Sep 2 13:58:01 2015 +0300
wic: use ext4 in canned .wks files
Latest kernel doesn't have ext3 compiled in. Wic images produced
from canned .wks can't boot because of that. Switching to ext4
fixes this issue.
(From OE-Core rev: d281a65a81f369fc8d75023b8f911ce4106969c1)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
commit 072657ce98
Author: Tom Zanussi <tom.zanussi@linux.intel.com>
Date: Mon Aug 11 20:35:41 2014 -0500
wic: Add mkgummidisk kickstart file
This is the same as mkefidisk but uses gummiboot instead of grub-efi.
(From OE-Core rev: 5979409ebfab0bb07b3c2b2fcf14a722c441f07b)
Signed-off-by: Tom Zanussi <tom.zanussi@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
-----------------------------------------------------------------
(From OE-Core rev: 546cd8352b8adce074831ec31cfa3bb2bf2f0084)
Signed-off-by: Jianxun Zhang <jianxun.zhang@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The new systemd-boot enabled in OE is the old gummiboot
merged into systemd project. Our intention is to replace
gummiboot with systemd-boot in OE once every feature based
on gummiboot is supported with systemd-boot.
Before we can purge gummiboot, we temporarily keep both of
the two bootloaders supported.
Patch doesn't do replacement for every "gummi" occurrence.
We think cleaning can be done in background after we disable
people to use gummiboot, so we just keep change small and
safe this time.
(From OE-Core rev: daa5f8b886408eb6a17898b18ac97d5a0d76d2cc)
Signed-off-by: Jianxun Zhang <jianxun.zhang@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
It is currently possible to specify a file (e.g. a tarball) on the local
disk as the source, but you have to know to put file:// in front of it.
There's really no need to force users to jump through that hoop if they
really want to do this so check if the specified source is a file and
prefix it with file:// if that's the case.
Also ensure the same works for "devtool add" at the same time.
(From OE-Core rev: 71350003790c38e84b0e525a71a2fe5d24e3d083)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
For a while now, Github hasn't been advertising a specific repository
URL since cloning the web URL with git works. Armed with this knowledge
and fully expecting people to just paste the github URL, we need to
handle this situation specially. If it looks like a github URL to the
root of a repository then treat it as a git repository instead of a
normal https URL to be fetched by the wget fetcher.
(From OE-Core rev: 7998dc3597657229507e5c140fceef1e485ac402)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Certain recipes cannot be used with devtool extract / modify / upgrade -
usually because they don't provide any source. Return a specific exit
code (4) so that scripts such as scripts/contrib/devtool-stress.py know
the difference between this and a genuine failure.
(From OE-Core rev: ffd295fed4ab81fc0bd00bb145ef4d72c49584bf)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We were attempting to open the recipe file unconditionally here - we
need to account for the possibility that the recipe file has been
deleted or moved away by the user.
(From OE-Core rev: 47822a2aff56fd338c16b5ad756feda9f395a8a1)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In OE-Core revision 7baf57ad896112cf2258b3e2c2a1f8b756fb39bc I changed
the default update-recipe behaviour to only update patches for commits
that were changed; unfortunately I failed to handle the --initial-rev
option which was broken after that point. Rework how the initial
revision is passed in so that it now operates correctly.
(From OE-Core rev: b2ca2523cc9e51a4759b4420b07b0b67b3f5ac43)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Code cleanup, no functional changes - this code was never used.
(From OE-Core rev: 397b76c7f26e38e761b94b1f7987aafd55048e10)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We're opening source files with the default encoding (utf-8) but we
can't necessarily be sure that they are UTF-8 clean - for example,
recipetool create ftp://mama.indstate.edu/linux/tree/tree-1.7.0.tgz
prior to this patch resulted in a UnicodeDecodeError. Use the
"surrogateescape" mode to avoid this.
Fixes [YOCTO #9822].
(From OE-Core rev: 50fcd9d1b9a20d49bc873467a82a071f2f2f8b5a)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
With the changes to migrate from optparse to argparse there was a syntax change that needs an update in the help. This is basically just the change of 'properties' and 'property' from positional arguments to options. This patch makes the required changes.
[YOCTO #8321]
(From meta-yocto rev: b171379b5ca54d55ea763421794a651e71bbda91)
Signed-off-by: Humberto Ibarra <humberto.ibarra.lopez@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Optparse is deprecated and should be avoided. The arparse library is better suited and has more tools to handling the parsing of arguments. This patch makes necessary changes to migrate to the better library and uses arparse subcommand feature to improve organization of this script.
[YOCTO #8321]
(From meta-yocto rev: 3f45993b96d4d960da0efe8672dc323c9db091a2)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Format of properties file is expected to be a simple json detailing properties, if this format fails an exception is thrown. This patch adds a graceful error message to the case when the properties file has a wrong format.
[YOCTO #9750]
(From meta-yocto rev: 7e543aa19d0d4b2112e6316783fb31b76df3493e)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Python3 requires strings to be encoded as bytes before sending them through a subprocess pipe. The help.py file is not considering this and fails when issuing paged help commands. This patch adds this encoding to solve the problem.
[YOCTO #9868]
(From meta-yocto rev: 35b487a47f0cbb99fdee2ec9cc8b56b814c8860e)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
GNU make looks for "makefile" and "GNUmakefile" in addition to
"Makefile", so add these other names to the heuristic for detecting a
make-based project.
(From OE-Core rev: 204d19b02265e5b2241888e4c92c0a730f3d3472)
Signed-off-by: Nathan Lynch <nathan_lynch@mentor.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Allow specifying more than one recipe on the devtool reset command line.
Also tweak the help text slightly.
(From OE-Core rev: ad92ed8e4f7f48a3d212962531d596b36f6b284f)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Commit ab75075e79 broke the string formatting
for multiple arguments. Since parentheses were removed the formatting considers
only first argument instead of all of them. Fixed parentheses on the formatting.
[YOCTO #9723]
(From meta-yocto rev: d6d641d5b4bb0c8b9acd5cd1341a4a3214bff086)
Signed-off-by: Humberto Ibarra <humberto.ibarra.lopez@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Have to convert implicit relative imports to explicit imports since
implicit relative imports are not supported on python3.
[YOCTO #9723]
(From meta-yocto rev: 89cecc603d229768428f3cb68d21235dee80efda)
Signed-off-by: Humberto Ibarra <humberto.ibarra.lopez@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
According to https://docs.python.org/3/whatsnew/3.0.html python3
renamed 'raw_input' to 'input'. Making the appropiate changes for
this.
[YOCTO #9723]
(From meta-yocto rev: 76efeeeac7fb30b44ee7057b3b3d1fd84329dae1)
Signed-off-by: Humberto Ibarra <humberto.ibarra.lopez@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
stdeout and stderr content returned by subprocess API has different
types in Python 3(bytes) and Python 2(string). Decoding it to
'utf-8' makes it unicode on both pythons.
(From meta-yocto rev: 1de9d0b4ad289c56907d082748cdc0111988cb4f)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Used items() as iteritems() doesn't exist in python 3.
(From meta-yocto rev: 0fd2e739cd14c6a02b6f4610176a0ad29b2ecc0e)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Used exec() builtin instead of 'exec' statement as
this statement doesn't exist in python 3.
(From meta-yocto rev: ceb6c6fac212c5e1e76bf58b793dc2b9420b4a66)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Implicit relative imports within packages are not supported in
python 3. They have to be converted to explicit imports.
Used 'from .module import' syntax for relative imports.
(From meta-yocto rev: 8eb2befc544e29fd87ac1fc75886cc6d7f40ad90)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Used metaclass=<metaclass> syntax instead old
__metaclass__ = <metaclass> as only new one is supported
in python 3.
(From meta-yocto rev: 70ba79981d904b4b67440c0b5755b5449cddb025)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Renamed raw_input to input as raw_input does not
exist in python 3.
(From meta-yocto rev: c9df9f2699885f2ba5b031c8761aefbf3c796067)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Since mtools has been patched to live with filesystems with sizes
not divisible by sectors-per-track, we no longer need to try to
set the size based on our guess of the sectors-per-track dosfstools is
going to use.
(From OE-Core rev: 334e32af88b310ff1ed950d127a6dedeb460f8d0)
Signed-off-by: Jussi Kukkonen <jussi.kukkonen@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Converted return value of items() keys() and values() to
lists when dictionary is modified in the loop and when
the result is added to the list.
(From OE-Core rev: 874a269eb1d70060c2f3b3f8b70800e2aea789f4)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Used urllib.parse instead of urlparse to make code
working in python 3.
(From OE-Core rev: 0a064f2216895db0181ee033a785328e704ddc0b)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Moved call of decode('utf-8') as close as possible to
call of subprocess API to avoid calling it in a lot of
other places.
Decoded binary data to utf-8 where appropriate to fix devtool
and recipetool tests in python 3 environment.
(From OE-Core rev: 30d02e2aa2d42fdf76271234b2dc9f37bc46b250)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Python 3 doesn't have basestring type as all string
are unicode strings.
(From OE-Core rev: e8cfab060f4ff3c4c16387871354d407910e87aa)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Encoded help text before sending it to pager.communicate as
it expects binary.
[YOCTO #9412]
(From OE-Core rev: 23c27d9d936efaa17da00525f1d2e2f98c53abc7)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
xrange() no longer exists in python 3, use range()
(From OE-Core rev: d022b4335100612d6596cc4c4956cb98ed5873cc)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Iterators now return views, not lists in python3. Where we need
lists, handle this explicitly.
(From OE-Core rev: caebd862bac7eed725e0f0321bf50793671b5312)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
python3 standardises its use of iteration operations. Update
the code to match the for python3 requires.
(From OE-Core rev: 2476bdcbef591e951d11d57d53f1315848758571)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Used print function instead of print statement to make
the code work in python 3.
(From meta-yocto rev: d6eea5a794dd8802b773a9186479a863847e6e55)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If fetching source from a git repository, typically within OpenEmbedded
we encourage setting SRCREV to a fixed revision, so change to do that by
default and add a -a/--autorev option to use "${AUTOREV}" instead.
(From OE-Core rev: 000480c42797dd2f03ebc3bc6d1dabfc6a7b75f5)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If we use ${BP} for the subdirectory, the default value of S will work
rather than having to have an ugly value derived from the package
file name in both places. This does mean that we have to assume the
default though (we can't just let the normal logic work because the
value of BP is the default until later on, so the replacement doesn't
work).
(From OE-Core rev: 13bc2438d61c345a8f229b9d83bf36a14d08916f)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Extract the metadata from package files and use it to set variable
values in the recipe (including recipe name and version, LICENSE,
SUMMARY, DESCRIPTION, SECTION and HOMEPAGE). For LICENSE we take care
not to step on any value determined by our license file scan; if there
is one we simply add a comment above the LICENSE setting so the user can
resolve it.
(From OE-Core rev: 19e6b661d38c5ae9b19d6340762c289830baba59)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Allow plugins to set any variable value through the extravalues dict,
and use this to support extracting SUMMARY and HOMEPAGE values from spec
files included with the source; additionally translate "License:" to a
comment next to the LICENSE field (we have our own logic for setting
LICENSE, but it will often be useful to see what the spec file says if
one is present).
Also use the same mechanism for setting the same variables for node.js
modules; this was already supported but wasn't inserting the settings in
the appropriate place in the file which this will now do.
(From OE-Core rev: 91fc35ff5e89aa6d4c4ad945e45406fb4f71018e)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We want the system to boot on its own, rather than the user having to manually
input the correct file at the EFI prompt.
[YOCTO #9556]
(From OE-Core rev: aa215c12d729c001ad3b8b98a7e203a9ad192211)
Signed-off-by: Christopher Larson <chris_larson@mentor.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The -S / --srcrev option must be specified if fetching from a git
repository, so spell that out in the help text.
(From OE-Core rev: 1465f205c235a1688a85844ebf5259e8971038ae)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Make a couple of changes to the rebase operation:
1) Only wrap the actual rebase command in try...except since a failure
in any of the other commands should be an error, not a warning
2) If it's a conflict (which unfortunately we can only tell by checking
for the keyword "conflict" since git doesn't return error codes based
on the type of error) then print a message clarifying that the user
needs to resolve the issue themselves to finish the upgrade.
(From OE-Core rev: 2a76be958432a35a0de30e9a5433089a54a06cad)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The gdb recipe in OE-Core has an inc file with the version in it;
since the inc file is pulled in with a "require ${PV}.inc", when
upgrading the recipe we need to also rename the inc file it will fail to
parse and the upgrade itself will fail.
Fixes [YOCTO #9574].
(From OE-Core rev: 3c623aac9333d20a62475279c72b6b6ec3d7dd6b)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Having two code paths here makes maintenance difficult, and it doesn't
seem likely that you would use the local case in real usage anyway, so
drop the local support entirely.
This should allow us to resolve [YOCTO #9301].
(From OE-Core rev: 7a4c9c96fee4fb514c2b69b52e811c4f896a16f1)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If the path contains 'scripts' more than once the first occurrence will be
incorrectly used. Use rfind instead of find to find the last occurrence.
(From OE-Core rev: f30c486c17060d2f21618612804a692512ad6a57)
Signed-off-by: George McCollister <george.mccollister@gmail.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Included description of -m/--bmap option to the help page
produced by 'wic help create'.
[YOCTO #9413]
(From OE-Core rev: 7c749d4cfe1642d70a6b3643422ef69dec1775fe)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This option enables generation of <image>.bmap file for the
result image using native bmaptool.
[YOCTO #9413]
(From OE-Core rev: d64c7b37c40b052510419b4d6629b83319c833e4)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Added bmaptool -> bmap-tools pair to the dictionary
executable -> recipe as bmaptool is going to be used by wic
to generate .bmap file.
[YOCTO #9413]
(From OE-Core rev: 4684ac5f4d823a6afedc57aad0aa417bb17ba950)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Generators in Python 3 don't have .next method. It's recommended
to use 'next' builtin instead. As it also present in Python >= 2.6
it should make wic code compatible with both Python 2 and Python 3.
[YOCTO #9412]
(From OE-Core rev: 9b7ab632e47d786dd979262015dbfb1254103f83)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Removed check for unicode type as it doesn't work in Python 3.
This check is not needed for wic as all its output seem to be
strings. This allows to run code under both pythons.
[YOCTO #9412]
(From OE-Core rev: a56924b4a0102e401b5e37d857a08bab15da974e)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
stdeout and stderr content returned by communicate API has different
types in Python 3(bytes) and Python 2(string). Decoding it to 'utf-8'
makes it unicode on both pythons.
Decoded stdout and stderr output to utf-8 to make the code
working under both Python 2 and Python 3.
(From OE-Core rev: 5b556f58a171e3d45107bb56a1f780e5c1abba37)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Division operator works differently in Python 3. It results in
float unlike in Python 2, where it results in int.
Explicitly used "floor division" operator instead of 'division'
operator. This should make the code to result in integer under
both pythons.
[YOCTO #9412]
(From OE-Core rev: 997ff239bd753a7957cc14c6829b2f093d9bcef6)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
As bitbake is not ported to Python 3 yet it's better to
avoid using its APIs as much as possible to be able to
test wic under Python 3 at least partially.
Used distutils.spawn.find_executable API in favor of
bb.utils.which to get path of the command to run.
(From OE-Core rev: 9658956bf8a5da779e06f71941de9b3e89415cdc)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This suffix is not supported by Python 3. Wic code works
without it on Python 2 too, so it's safe to remove it.
[YOCTO #9412]
(From OE-Core rev: 296db7e33bd71585cac63dc78c2c95bc619b4a86)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Wic plugin machinery implemented using metaclasses.
Reimplemented plugin machinery using this advice from
https://wiki.python.org/moin/PortingToPy3k/BilingualQuickRef
Syntax for creating instances with different metaclasses is very
different between Python 2 and 3. Use the ability to call type instances
as a way to portably create such instances.
Now it should work under both Python 2 and Python 3.
[YOCTO #9412]
(From OE-Core rev: e62fe5a41bdcdd72b9b257fecff7ccdc59c76d33)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Removed 'raw', 'ask', 'choice' and 'pause' functions from
msger.py as they're not used in wic code and some of them
use raw_input, which is not present in Python 3.
[YOCTO #9412]
(From OE-Core rev: eb87d591ef67f1953b2689430ef6c5a6a27a5b6e)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
New syntax 'except Exception as err' is supported by Python >= 2.7.
Old syntax 'except Exception, err' is not supported by Python 3.
Used new syntax to be able to run wic on Python 3.
[YOCTO #9412]
(From OE-Core rev: 15e88714d6b0a93f72e8a19b083fcc1f2006e128)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
'with' statement is not used in baseimager.py
It's supported by Python 2.7, which is included into all target
distros. Other wic modules use this statement.
Removed useless 'from __future__ import with_statement' from
wic code.
(From OE-Core rev: 528a1f20939589949831efbb4de6336776efe7d5)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
All classes in Python3 are new style classes and don't need
to be inherited from object. Wic code works fine without
this inheritance even with Python2, so it's harmless to
remove it.
[YOCTO #9412]
(From OE-Core rev: a146b03ee7d0aa5bc1722da5977a5952782b69bf)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Replaced calls of dict.keys and dict.has_key methods with the
'key in dict' statement. 'key in dict' is more pythonic, faster
and readable. dict.has_key doesn't exist in Python 3.
[YOCTO #9412]
(From OE-Core rev: 003df7dfb932c551953fbf1bd769b3c31bd16fb4)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Dictionary method 'iteritems' doesn't exist in Python 3.
Replaced 'iteritems' with 'items' to be able to run the
code under both Python 3 and Python 2.
[YOCTO #9412]
(From OE-Core rev: 5b14eb8d68aaca82de4f8f6bcb28ad6f4a5125d0)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Print statements have been replaced with print function in
Python 3. Replaced them in wic code to be able to run it
under both Python 2 and Python 3.
[YOCTO #9412]
(From OE-Core rev: ee6979a19c77931c3cf6368e695e370d46192fef)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If an image with the filename foo.bb could be built using the name "bar"
instead, then build-sdk would fail to create the derivative sdk.
This was because the code assumed that the file name matched the target,
which is not necessarily the case.
(From OE-Core rev: d58a326b6960be14b8a049253559aec9582b7d0d)
Signed-off-by: Randy Witt <randy.e.witt@linux.intel.com>
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When a branch is re-used, the kernel tools turns off any patch pushing unless
'mark patching' is explicitly set.
[YOCTO #9120]
(From meta-yocto rev: 427f5473722e15e288cbce251a9ce18989c23548)
Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fix two problems falling back to the "license" field from package.json
when no license file is present:
1) The function that was supposed to return the license field value was
always explicitly returning None, and this was never noticed (because
the test cases never exercised the fallback as they provided license
files for each module).
2) Fix the main package not falling back because it had a default of an
empty list, which evaluates to '' instead of 'Unknown'.
(From OE-Core rev: 59381a9450949ce6b4b03adb717e950b999830f3)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This aligns with our existing 'optional arguments' to 'options' change, and
seems more intuitive for users.
(From OE-Core rev: 8a1cd471210e5fb77952f28172084bf6a4fb73e8)
Signed-off-by: Christopher Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There's no need to iterate over the action groups here, as self._optionals and
self._positionals are available.
(From OE-Core rev: 408694f4320f3cb52a391e5b927fb8c8ba16c1d2)
Signed-off-by: Christopher Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
As an example, `recipetool create foo bar baz` shows `recipetool: error:
unrecognized arguments: bar baz` and then displays the main help, not the help
for the create command. Fix by saving the subparser name and using it in
parse_args() to look up the subparser.
(From OE-Core rev: 7fdaaedf4c63c8d019f03f84e22f9b838ef19aa6)
Signed-off-by: Christopher Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This aligns our subclassed error() with that in the original class, using
_print_message and self.prog. Also add a docstring based on the original.
(From OE-Core rev: cf0c5175136966eefde8c0d9aa0679e85779f713)
Signed-off-by: Christopher Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Used sparse_copy API in favor of dd/cp in rawcopy plugin to
preserve sparseness of the copied raw content.
[YOCTO #9099]
(From OE-Core rev: 04eca59068a79ae6a9969be495c4cdf0c5c3e466)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Copied partition images into final partitioned image using
sparse_copy API. This method preserves sparseness of the
final image. It also makes wic much faster, as unmapped
blocks of the partition images are not copied.
[YOCTO #9099]
(From OE-Core rev: 7f21427aca5df81d8881027fd98f71b821cf31d7)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In order to make wic images sparse sparse_copy function has been
copied from meta-ostro:
https://github.com/kad/meta-ostro/blob/master/meta-ostro/lib/image-dsk.py
This function uses filemap APIs to copy source sparse file into
destination file preserving sparseness.
The function has been modified to satisfy wic requirements:
parameter 'skip' has been added.
[YOCTO #9099]
(From OE-Core rev: bfde62bdc03152a4d3d383512479b974fa867f94)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In order to make wic images sparse set of APIs has been copied
from bmap-tools project.
filemap.py module is taken from bmap-tools project:
https://github.com/01org/bmap-tools/blob/master/bmaptools/Filemap.py
It implements two ways of get information about file block: FIEMAP
ioctl and the 'SEEK_HOLE / SEEK_DATA' features of the file seek
syscall.
Note that this module will be removed as soon as bmaptool utility
supports copying sparse source file into destination file (this is
already agreed with the maintainer of bmap-tools project).
[YOCTO #9099]
(From OE-Core rev: 182639ddc9cda85c896a54c1c64fd1fb145071a1)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Moved DiskImage class from utils/fs_related.py to
imager/direct.py as it's only used there.
Removed fs_related module as it doesn't contain anything
except of DiskImage.
(From OE-Core rev: b3cc471790784c28f9362fcd6fc6a81c4316754c)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Removed fs_related.makedirs as is not used anywhere. The name is
easy to confuse with os.makedirs.
(From OE-Core rev: 796b114863ef20fbc89da45dbe6780abe1256f5e)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There is no need in this inheritance as DiskImage class
is used only in one module and no other classes are inherited.
(From OE-Core rev: 5af1d9bedc2c961eb91faf80251f24c3df754d76)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Used truncate instead of dd to create wic images for the
following reasons:
- dd doesn't preserve sparseness
- truncate syntax is much more clear
- dd requires additional calculations of the image size
in blocks
- the way dd was used in the code is not always correct.
In some cases it was writing one block to the file which makes
it not 100% sparse.
[YOCTO #9099]
(From OE-Core rev: d2d0d18dfd3922411d856b98ab6ba5d64c9c1c9f)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Added explanation of --system-id option to the output of
wic help kickstart.
[YOCTO #9096]
(From OE-Core rev: 1a304afea4ad7be12ed5f0fcb397a538345a6b63)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Used sfdisk to set partition system id if --system-id parameter
is used for a partition in wks file.
[YOCTO #9096]
(From OE-Core rev: a1f7f7e61fd20fb6319825648930f7b6aa0e0cee)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Added new argument to add_partition call to pass partition
system id down the stack.
[YOCTO #9096]
(From OE-Core rev: f2733df697192c0010c17b7bbb02f8679cb8f313)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Added Partition.system_id attribute and initialized it
from parse result of wks option --system-id. It will be
used by the wic code below the call stack to set partition
system id.
[YOCTO #9096]
(From OE-Core rev: 4f195a5b7574ebff8fbdb3045daa71f173f97a30)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Added sfdisk -> util-linux pair to the dictionary
executable -> recipe as sfdisk is going to be used by wic
to set partition system id.
[YOCTO #9096]
(From OE-Core rev: 398aafa185acbc7239505f7107735e93a502f6d2)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Added new option --system-id to wks parser. The option
will be used to set partition system id.
[YOCTO #9096]
(From OE-Core rev: b9c56b1c95cd1d0fd809d257e0cd05a50c481bed)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
os.path.join discards the cr_workdir var contents if the path of the
second arguments is absolute.
(From OE-Core rev: dba099d77dcc66b239523a55f3ed26784f9a662a)
Signed-off-by: Ioan-Adrian Ratiu <adrian.ratiu@ni.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The latest wic kickstart refactoring introduced a bootloader option
"--configfile" which lets wks' specify a custom grub.cfg for use
while booting. This is very useful for creating stuff like boot menus.
This change lets isoimage-isohybrid use --configfile; if this option is
not specified in a wks, it generates a default cfg as before.
(From OE-Core rev: bf673a769514b13558ad9c785ae4da3a5adfd1e0)
Signed-off-by: Ioan-Adrian Ratiu <adrian.ratiu@ni.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If UNINATIVE_CHECKSUM changes over an SDK update, bitbake within the
extensible SDK will be broken because it will see that the matching
uninative tarball doesn't exist and if there is a default value of
UNINATIVE_URL it will attempt to download the file and will then fail
because the checksums don't match up; alternatively if no UNINATIVE_URL
is set then it'll also fail with an error about misconfiguration. To fix
this, add some logic to devtool sdk-update to download the matching
uninative tarball(s) for the checksum(s) in the newly fetched SDK
configuration.
Fixes [YOCTO #9301].
(From OE-Core rev: 14ff58ad98a5afac08db77068d80f152d8875766)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Replace git pull with fetch and reset to avoid the merge logic in the
event that the layers repo in the published SDK we're updating to isn't
fast-forward merge from the local repo.
Also add gitignore and committer info during publish to avoid errors and
to be sure that the first commit has a dummy user in it.
[ YOCTO #9368 ]
(From OE-Core rev: 4657bc9d165e51981e034e73e7b92552e873eef7)
Signed-off-by: Stephano Cetola <stephano.cetola@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
It is unusual but not impossible to find recipes whose first entry is
not the main source URL but instead some patch or other local file, for
example python-cryptography in meta-python (which sets SRC_URI before
inheriting pypi). There's nothing inherently wrong with this, and we
shouldn't assume that the first entry is the main source URL, so just
take the first non-local entry instead.
(From OE-Core rev: c868198c1f6006789707b497c2ae34d7cc5e706f)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Partitions specifying --fsoptions were silently skipped by wic
due to the old bug introduced when removing code related to
subvolume handling:
- if mountpoint == "/" or not fsopts or fsopts.find("subvol=") == -1
+ if mountpoint == "/" or not fsopts:
[YOCTO #9396]
(From OE-Core rev: be7ff1741e8ab5f2724b3f64da1bed8b0d3dcb7c)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
It turns out that dd's conv=sparse doesn't look at the file extents, but simply
checks if a "block" is all zero. If the block of zero was meaningful it gets
lost and if the image is subsequently written to media using a sparse-aware
writer then the block of zeros won't be written at all.
This reverts commit 5fd592fbae2e046bcb8c3a6c3ef4993fe0400676.
(From OE-Core rev: 63d15764cc2014dba9fee2186f0c8b97c2ac5682)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The individual partitions created by wic are sparse but without
this change the assembled image is written as one (potentially
very) large file.
Preserve sparseness in the assembled image by passing the sparse
conversion symbol.
[YOCTO #9099]
(From OE-Core rev: 5fd592fbae2e046bcb8c3a6c3ef4993fe0400676)
Signed-off-by: Joshua Lock <joshua.lock@collabora.co.uk>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fix typo of the word "parameter" in for 'yocto-bsp create'.
Typo appears in both, usage and help.
Also, the word "description" is mispelled.
[Yocto #9282]
(From meta-yocto rev: 12c7243abd91b374b1b62c6a1ad13b0d25aa0e4c)
Signed-off-by: Humberto Ibarra <humberto.ibarra.lopez@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
By default cpio preserves the uid&guid's of the original user which
leads to host contamination and boot failures because commands like
mount from initramfs expect to be run by root and the original host
user might not even exist on the target.
(From OE-Core rev: 28910ee2eacc15cf42b5e58bd43b3bd15c34eb97)
Signed-off-by: Ioan-Adrian Ratiu <adrian.ratiu@ni.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Some filenames can omit 'initramfs', or use other names. This makes
detection more flexible by using only the image name, machine arch and
image type in a glob wildcard.
(From OE-Core rev: ca516f5907a661606c35e1ca5c2ece9fc79c77ea)
Signed-off-by: Ioan-Adrian Ratiu <adrian.ratiu@ni.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Kernel recipes for linux-yocto_4.* and greater have outdated branches as default,
making it impossible to find the right branch if the user picks the default value.
The branches_base property uses these outdated branches also.
This updates standard/common-pc and standard/common-pc-64 branches to standard/base.
The fix was tested using 'yocto-bsp create' with each one of the following
archs:
-i386 with kernels 4.1 and 4.4
-x86_64 with kernels 4.1 and 4.4
-qemu (i386 and x86_64) with kernels 4.1 and 4.4
After the layer was created, it was added to local.conf and the MACHINE was set
accordingly.
'bitbake linux-yocto' ran successfully with each configuration tested.
[YOCTO #9160]
(From meta-yocto rev: d471e3dd7c5080a29f64b60b554f17ee706ee772)
Signed-off-by: Humberto Ibarra <humberto.ibarra.lopez@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Strictly speaking we ought to explicitly shut down a tinfoil instance
when we're done with it. This doesn't affect modify's operation but is
important if you want to be able to call into modify() from another
plugin (though anyone doing so should be advised that the function is
by no means a stable API and is subject to change in future releases).
(From OE-Core rev: 626dbadf22b57a22a8f8b9d1957937120f4ba4d5)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Type of --no-table option was incorrectly set in new wks parser.
It causes parser to require argument for this option, which makes
wic to fail with wks files that use --no-table:
Error: argument --no-table: expected one argument
Changed action parameter to 'store_true' to fix the issue.
(From OE-Core rev: d483724cf3515f76e1b798a2018e2f3fa2bad0ba)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add the options --codedump and --skip-git-check to the yocto-bsp help and
yocto-bsp usage, since they are currently missing.
[YOCTO #8322]
(From meta-yocto rev: dfdf97b4239639affc8ce22e338a291d4b0dfc76)
Signed-off-by: Humberto Ibarra <humberto.ibarra.lopez@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When we add from a fetched URL we are supposed to turn the resulting
source tree into a git repository (if it isn't already one). However, we
were using the older deprecated option name here instead of the
positional argument, so "devtool add -f <url>" resulted in the repo
being created but "devtool add <url>" didn't, which was wrong.
Also update the oe-selftest tests to check that this worked.
(From OE-Core rev: a7b6b1f8cc1c096724f794ac9dee312b0f771f66)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The npm class just installs whatever is in ${S}; if you're using
externalsrc in conjunction with it the symlinks (oe-workdir and oe-logs
by default) plus singletask.lock will end up in the final package, which
isn't really right. Introduce a variable so we know the path the files
will be installed into within npm.bbclass, and append to do_install
within the workspace bbappend to delete the files.
(From OE-Core rev: 766845e06db9d7d595e836ea1364c16fa132a413)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The code here for running do_configure if it hadn't already been run was
using the wrong string substitution parameters; fix it and test it.
(From OE-Core rev: b2677a4448dbc42e523c731b953b44006749252c)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
By default the sdk-install subcommand expects to restore the requested
items from sstate and fails if it can't. If the user is OK with building
from source, add a -s/--allow-build option to allow them to do that. In
the process, ensure we show the status output while we're installing.
Also add the missing header to the top of the file.
(From OE-Core rev: a86b426cdd465ec5cb08bb5fa7729e4e673d94bb)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Otherwise (if the symlink is named .config) kernel build considers
source tree as dirty and fails.
[YOCTO #9270]
(From OE-Core rev: d36699b28c661880957d744420df9e23cdd76957)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The extra directory next to the recipe should only be created if there
are files to put into it; currently only the npm plugin does this. I
didn't notice the issue earlier because the test was actually able to
succeed under these circumstances if the recipe file came first in the
directory listing, which was a fault in my original oe-selftest test;
apparently on some YP autobuilder machines the order came out reversed.
With this change we can put the oe-selftest test that highlighted the
issue back to the way it was, with an extra check to reinforce that only
a single file should be created.
(From OE-Core rev: b8b778345eb0997c2cd952a1f61fdd2050b6b894)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
"npm shrinkwrap" creates a file that ensures that the exact same
versions get fetched the next time the recipe is built. lockdown is
similar but also includes sha1sums of the modules thus validating they
haven't changed between builds. These ensure that the build is
reproducible.
Fixes [YOCTO #9225].
(From OE-Core rev: 277377f13b2b771915eb853e336ca24b84523ed1)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Allow plugins to create additional files to go alongside the recipe. The
plugins don't know what the output filename is going to be, so they need
to put the files in a temporary location and add them to an "extrafiles"
dict within extravalues where the destination filename is the key and
the temporary path is the value.
devtool add was also extended to ensure these files get moved in and
preserved upon reset if they've been edited by the user.
(From OE-Core rev: 334b9451111b7e3efbb43b3a4eecebcab8ec6f0e)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If the user specifies an npm:// URL then the fetcher needs npm to be
available to run, so check if it's available early rather than failing
later.
(From OE-Core rev: a08d12ad867c292f7474731a0fe5e51e712446d6)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Rather than rolling all of an npm module's dependencies into the same
package, split them into one module per package, setting the SUMMARY and
PKGV values from the package.json file for each package. Additionally,
mark each package with the appropriate license using the license
scanning we already do, falling back to the license stated in the
package.json file for the module if unknown. All of this is mostly in
aid of ensuring all modules and their licenses now show up in the
manifests for the image.
Additionally we set the main LICENSE value more concretely once we've
calculated the per-package licenses, since we have more information at
that point.
(From OE-Core rev: 8226805f83d21e7c1d2ba21969f3e8ee4b137496)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Matching license texts directly to md5sums only goes so far. Some
licenses make the copyright statement an intrinsic part of the license
statement (e.g. MIT) which of course varies between projects. Also,
people often seem to take standard license texts such as GPLv2 and
reformat them cosmetically - re-wrapping lines at a different width or
changing quoting styles are seemingly popular examples. In order to
match license files to their actual licenses more effectively, "crunch"
out these elements before comparing to an md5sum. (The existing plain
md5sum matching has been left in since it's a shortcut, and our list of
crunched md5sums isn't a complete replacement for it.)
As always, this code isn't providing any guarantees (legal or otherwise)
that it will always get the license correct - as indicated by the
accompanying comments the LICENSE values it writes out to the recipe are
indicative and you should verify them yourself by looking at the
documentation supplied from upstream for the software being built if you
have any concerns.
(From OE-Core rev: 553bb4ea5d51be5179e7d8c019740cf61ece76ea)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
For example, this picks up a file named MIT-LICENSE.txt.
(From OE-Core rev: 103b4d26b340cbdf70bf43906e293f3497671fdc)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Make the following improvements to mapping items specified in
AC_CHECK_PROG, AC_PATH_PROG and AX_WITH_PROG to recipes/classes:
* Produce a map of native recipe -> binary for all binaries currently in
STAGING_BINDIR_NATIVE and use this when mapping items
* Add some more entries to the class map
* Ignore autotools binaries since they are covered by the inherit of
autotools
* Ignore coreutils-native since that would almost always be a bogus
dependency
(From OE-Core rev: 5614c5ae6a004d4367eccc34dd3cc7ee61fb7e57)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Allow for whitespace in appropriate places, and ensure we match all
whitespace chars not just the space character.
(This fixes extracting dependencies from tmux's configure.ac, for
example.)
(From OE-Core rev: 63524ac8093b734aa4f29f4ea47bcc036f748314)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add kernel 4.4 support. kernel 4.1 was left in the
tree for user to choose between 4.1 and 4.4.
[YOCTO #9047]
(From meta-yocto rev: c04093ec9b3cd151cf857bca9ad35bc9a0d4a4b1)
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Clearly I didn't test this part of the code - lists don't have an "add"
method. Needless to say I have tested it now.
(From OE-Core rev: 063ed9058a14775f77e7875d4f6ef5719fa03f18)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a build-sdk command which is only available within the extensible
SDK that builds a derivative extensible SDK. The idea is recipes in the
workspace become a part of the new SDK - for example, this allows taking
a vendor provided SDK, adding a few libs and then producing a new SDK
with those included.
When normally building the extensible SDK, the workspace is excluded;
here we need to copy into the new SDK (renaming it in the process); the
recipes' task signatures become locked and thus the sources are no
longer needed, so they are removed along with the workspace bbappends
which would interfere with the locked signatures. Additionally we need
to just copy the configuration files (i.e. local.conf and auto.conf)
rather than filtering and appending to them since that work has already
been done when constructing the original SDK. The extra sstate artifacts
from workspace recipes are also determined and copied into the new SDK
in minimal mode (on the assumption that you won't set up a new sstate
mirror).
This reuses some code from build-image, so that needed to be
generalised to allow that.
Implements [YOCTO #8892].
(From OE-Core rev: 59e207ff6dd4b50a8905e14bc9292cf2794f4e7a)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Hyphens aren't allowed in python identifiers, so you shouldn't use them
in module names or they are more difficult to import.
(From OE-Core rev: e8f6db9436dfc923e236f2cbc08f357e3b24fd5d)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
