getVar() now defaults to expanding by default, thus remove the True
option from getVar() calls with a regex search and replace.
Search made with the following regex: getVar ?\(( ?[^,()]*), True\)
(From OE-Core rev: 0a36bd96e6b29fd99a296efc358ca3e9fb5af735)
Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Extracting the source for a recipe (as used by devtool's extract, modify
and upgrade subcommands) requires us to run do_fetch, do_unpack,
do_patch and any tasks that the recipe has inserted inbetween, and do so
with a modified datastore primarily so that we can redirect WORKDIR and
STAMPS_DIR in order to have the files written out to a place of our
choosing and avoid stamping the tasks as having executed in a real build
context respectively. However, this all gets much more difficult when in
memres mode since we can't call internal functions such as
bb.build.exec_func() directly - instead we need to execute the tasks on
the server. To do this we use the buildFile command which already exists
for the purpose of supporting bitbake -b, and setVariable commands to
set up the appropriate datastore.
(I did look at passing the modified datastore to the buildFile command
instead of using setVar() on the main datastore, however its use of
databuilder makes that very difficult, and we'd also need a different
method of getting the changes in the datastore over to the worker as
well.)
(From OE-Core rev: eb63b5339014fc72ba4829714e0a96a98e135ee2)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Use Tinfoil.parse_recipe_file() and Tinfoil.parse_recipe() instead of
the recipeutils equivalents, and replace any local duplicate
implementations. This not only tidies up the code but also allows these
calls to work in memres mode.
(From OE-Core rev: f13b56266ee96dfab65a3a7db50e8051aa9f071a)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We should always shut down tinfoil when we're finished with it, either
by explicitly calling the shutdown() method or by using it as a
context manager ("with ...").
(From OE-Core rev: 5ec6d9ef309b841cdcbf1d14ac678d106d5d888a)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When using PATCHTOOL = "git", the user of the system is not really the
committer - it's the build system itself. Thus, specify "dummy" values
for username and email instead of using the user's configured values.
Various parts of the devtool code that need to make commits have also
been updated to use the same logic.
This allows PATCHTOOL = "git" and devtool to be used on systems where
git user.name / user.email has not been set (on versions of git where
it doesn't default a value under this circumstance).
If you want to return to the old behaviour where the externally
configured user name / email are used, set the following in your
local.conf:
PATCH_GIT_USER_NAME = ""
PATCH_GIT_USER_EMAIL = ""
Fixes [YOCTO #8703].
(From OE-Core rev: 765a9017eaf77ea3204fb10afb8181629680bd82)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Unfortunately to implenent multiconfig support in bitbake some APIs
had to change. This updates code in OE to match the changes in bitbake.
Its mostly periperhal changes around devtool/recipetool
[Will need a bitbake version requirement bump which I'll make when merging]
(From OE-Core rev: 041212fa37bb83acac5ce4ceb9b7b77ad172c5c3)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This provides us with the information we need to remove the original
version recipe and associated files when running "devtool finish" after
"devtool upgrade".
(From OE-Core rev: 92eb42c347af919cd9f8739515fdf806c12b5ba8)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The -S / --srcrev option must be specified if fetching from a git
repository, so spell that out in the help text.
(From OE-Core rev: 1465f205c235a1688a85844ebf5259e8971038ae)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Make a couple of changes to the rebase operation:
1) Only wrap the actual rebase command in try...except since a failure
in any of the other commands should be an error, not a warning
2) If it's a conflict (which unfortunately we can only tell by checking
for the keyword "conflict" since git doesn't return error codes based
on the type of error) then print a message clarifying that the user
needs to resolve the issue themselves to finish the upgrade.
(From OE-Core rev: 2a76be958432a35a0de30e9a5433089a54a06cad)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The gdb recipe in OE-Core has an inc file with the version in it;
since the inc file is pulled in with a "require ${PV}.inc", when
upgrading the recipe we need to also rename the inc file it will fail to
parse and the upgrade itself will fail.
Fixes [YOCTO #9574].
(From OE-Core rev: 3c623aac9333d20a62475279c72b6b6ec3d7dd6b)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
It is unusual but not impossible to find recipes whose first entry is
not the main source URL but instead some patch or other local file, for
example python-cryptography in meta-python (which sets SRC_URI before
inheriting pypi). There's nothing inherently wrong with this, and we
shouldn't assume that the first entry is the main source URL, so just
take the first non-local entry instead.
(From OE-Core rev: c868198c1f6006789707b497c2ae34d7cc5e706f)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Make this consistent with "devtool add" so that the user knows where to
find the new recipe.
(From OE-Core rev: 28fea4a104ba85aded2ecfe891e9bf997d72a103)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The PR value should be reset to the default when upgrading, so we need
to drop it from the newly created file.
(From OE-Core rev: 4acf8a6912106dd3da5ceee9e30ccce86211d04b)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We aren't modifying the datastore copy here, so we don't need a copy at
all.
(From OE-Core rev: 9f05bf98d4dbcb991b684a2ab480c6638c6be292)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fix several issues when extracting the new version source over the top
of the old one (when the recipe is not fetching from a git repo):
* Delete the old source first so we ensure files deleted in the new
version are deleted. This also has the side-effect of fixing any
issues where files aren't marked writeable in the old source and thus
overwriting them failed (harfbuzz 1.1.3 contains such files).
* Fix incorrect variable name in abspath statement that made it a no-op
(From OE-Core rev: 4f7595a514661f0e2418393286849a1cb6c46f0d)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When we do an upgrade from one tarball version to another we want to:
1) Check out the old version as a new branch
2) Record the changes between the old and new versions as a commit
3) Check out the old version with patches applied
4) Rebase that onto the new branch
Where we went wrong was step #1 where instead we checked out the old
version with patches applied as the new branch, which meant the rebase
didn't do anything and any changes made by the patches to files still in
the new version were wiped out.
(From OE-Core rev: a4f3e6d5f4c549e36cce1e18801a565a3a3c9b0a)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If the actual value of PV isn't in the name of the recipe (for example,
a git or svn recipe) there's no point trying to rename it. Additionally,
we already have the original filename, there's no need to guess it -
just pass it in.
(From OE-Core rev: e144db3650b4a7c0a59066621905f34a5400303e)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We were trying to move this from the current directory instead of the
path. Let's just use shutil.move() instead of shelling out to mv.
(From OE-Core rev: 60454a0ba154d6c777e0c2b05b887b4e4fcde986)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
For recipes that specify SRCREV, the code here wasn't quite doing the
right thing. If the recipe has a SRCREV then that needs changing on
upgrade, so ensure that the user specifies it. If it doesn't, then it'll
be "INVALID" not None since the former is the actual default, so handle
that properly as well. Additionally an unset variable was being
erroneously passed when raising the error about the version being the
same leading to a traceback, so fix that as well.
(From OE-Core rev: 1d0f821371d1cb93e30fad86f0c20e38cb93b54b)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The recipename argument to devtool upgrade specifies an existing recipe,
so by definition the name will be valid (or it won't exist) - we don't
need to validate it ourselves, that's only needed for situations like in
devtool add where we're creating a new recipe.
(From OE-Core rev: 1e2bc7d861555a04350a87f19047efdc717046be)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Make devtool upgrade consistent with devtool add/modify in defaulting to
sources/<recipename> under the workspace if no source tree path is
specified.
(From OE-Core rev: 8a952407b192313515e91570632446b6dff01665)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The listing of subcommands in the --help output for devtool was starting
to get difficult to follow, with commands appearing in no particular
order (due to some being in separate modules and the order of those
modules being parsed). Logically grouping the subcommands as well as
being able to exercise some control over the order of the subcommands
and groups would help, if we do so without losing the dynamic nature of
the list (i.e. that it comes from the plugins). Argparse provides no
built-in way to handle this and really, really makes it a pain to add,
but with some subclassing and hacking it's now possible, and can be
extended by any plugin as desired.
To put a subcommand into a group, all you need to do is specify a group=
parameter in the call to subparsers.add_parser(). you can also specify
an order= parameter to make the subcommand sort higher or lower in the
list (higher order numbers appear first, so use negative numbers to
force items to the end if that's what you want). To add a new group, use
subparsers.add_subparser_group(), supplying the name, description and
optionally an order number for the group itself (again, higher numbers
appear first).
(From OE-Core rev: e1b9d31e6ea3c254ecfe940fe795af44761e0e69)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Use bb.utils.edit_metadata() to replace some of the logic in this
function; this avoids us effectively having two implementations of the
same thing. In the process fix the following issues:
* Insert values before any leading comments for the next variable
instead of after them
* Insert overridden variables (e.g. RDEPENDS_${PN}) in the correct place
* Properly handle replacing varflag settings (e.g. SRC_URI[md5sum])
(From OE-Core rev: 0f81b83fc5fd908efa7f6b837137830ca65f6ed6)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* Make some minor clarifications to help text
* Drop ArgumentDefaultsHelpFormatter and just put the defaults in the
text itself where needed (because otherwise you get defaults shown for
store_true options which is somewhat confusing).
(From OE-Core rev: a90ffea30c4578fd6acda2c5945b816ad33b13f5)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This function is no longer required to be defined for a plugin, so drop
it where it's a no-op.
(From OE-Core rev: 49716a5e27e704598321bd6c82275b16afc444aa)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If you're upgrading a git recipe to a revision on a release branch
that's different to the branch for the current revision, then you'll
need to update the branch parameter in SRC_URI, so add a --srcbranch/-B
command-line parameter to let you do that easily. It handles both when
the branch is stated verbatim in the recipe, and when a reference to
another variable is used (a common convention is to use a SRCBRANCH
variable for this, though the code doesn't care what variable is used
if any).
(From OE-Core rev: e49a66fd898dd44e54c77a838ebef3d983ed2a03)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If we're upgrading a recipe that fetches from git, and we've simply
fetched a tarball of the repo instead of directly from the upstream repo
(this can happen if you have PREMIRRORS set up as in poky with a core recipe,
e.g. kernelshark) then we won't have any new revisions, and the checkout
will fail with "fatal: reference is not a tree: <hash>". To avoid this,
do a "git fetch" before checking out the new revision.
(From OE-Core rev: c4daebf3fe797a8063dcbc2ab229be2fbedc8134)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If we're upgrading a git recipe the recipe file usually won't need
renaming; for some unknown reason we were throwing an error here which
isn't correct.
(From OE-Core rev: 656348dff9bc9dd1cafc8fff11e5e374e3667f0f)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This code was clearly never tested. Fix the following issues:
* Actually set SRCREV if it's been specified
* Enable history tracking and reparse so that we handle if variables are
set in an inc file next to the recipe
* Use a more accurate check for PV being in the recipe which will work
if it's in an inc file next to the recipe
(From OE-Core rev: 8b8f04226ebf464fa61c05ca7af7c6cbda392339)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The sync command is similar to the extract command, except it
fetches the sync'ed and patched branch to an existing git repository.
This enables users to keep track the upstream development while
maintaining their own local git repository at the same time.
(From OE-Core rev: e2fe4c99f1020a20b579832c4feafcd0e9bc2f75)
Signed-off-by: Tzu-Jung Lee <roylee17@currantlabs.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
For modify / extract / upgrade, if the specified "recipe" is not
actually a recipe but a virtual target such as virtual/kernel, map it
correctly to the actual recipe and make sure we use that name within the
workspace. Thanks to Chris Larson for reminding me this was still broken
and for a hint on how to fix it.
(From OE-Core rev: 1f7752282ffb47d2621030ddb2fa42a5e491d6d2)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Do not change change current working directory permanently, but, only
for the duration of tinfoil initialization instead. The previous fix
caused very unintuitive behavior where using relative paths were solved
with respect to the builddir instead of the current working directory.
E.g. calling "devtool extract zlib ./zlib" would always create create
srctree in ${TOPDIR}/zlib, independent of the users cwd.
(From OE-Core rev: 4c7f159b0e17a0475a4a4e9dc4dd012e3d2e6a1f)
(From OE-Core rev: 05060699e63cd25d089e83e9aa56c11d5baa8fd8)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When we were adding a recipe for software that would typically be built
in the same directory as the source, we were always using a separate
build directory unless the user explicitly specified not to, leading to
errors for software that doesn't expect to be built that way (such as
Python modules using distutils). Split out the code that makes this
determination automatically from the "devtool modify" and "devtool
upgrade" code and re-use that here so the behaviour is consistent.
(From OE-Core rev: 320585b7ff6340df0b0dbc63f95ed3ca8fc3a993)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Upgrades a recipe to a particular version and downloads the source code
into a folder. User can avoid patching the source code.
These are the general steps of the upgrade function:
- Extract current recipe source code into srctree and create a branch
- Extract upgrade recipe source code into srctree and rebase with
previous branch. In case the rebase is not correctly applied, source
code will not be deleted, so user correct the patches
- Creates the new recipe under the workspace
[YOCTO #7642]
(From OE-Core rev: 4020f5d91b3e4d011150d5081d36215f8eab732e)
Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com>
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Joshua Lock
Paul Eggleton
Richard Purdie
Ed Bartosh
Tzu-Jung Lee
Markus Lehtonen
Leonardo Sandoval