Commit Graph

240 Commits

Author SHA1 Message Date
Paul Eggleton 293c82baa4 devtool: deploy-target: use tinfoil instead of bitbake -e
Using tinfoil here is quicker and tidier than shelling out to
bitbake -e and interpreting its output.

(From OE-Core rev: 986ad99aee98dd5b7f30d59098dd9275097b8276)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-16 22:31:51 +01:00
Paul Eggleton 1bab18f26f devtool: fix build env command execution error handling
If we execute an external command, we ought to prepare for the
possibility that it can fail and handle the failure appropriately. We
can especially expect this to happen when running bitbake in this
scenario. Ensure we return the appropriate exit code to the calling
process.

Fixes [YOCTO #7757].

(From OE-Core rev: 98a716d79bfc5434a5b42d3ca683eab3eea30a41)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-16 22:31:51 +01:00
Markus Lehtonen 18114bb8a6 devtool: deploy plugin: fix bad indentation
(From OE-Core rev: 6866b3027babcc390130f0cba4990c0f769cdb6a)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-16 22:31:51 +01:00
Markus Lehtonen 7883af3097 devtool: add missing docstrings
(From OE-Core rev: 07b2e731a378c56852e2715f6c001097b81abe46)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-16 22:31:50 +01:00
Markus Lehtonen c977cdf3d8 devtool: rename unused variables
(From OE-Core rev: 92b84d54292518a387460ee1ec5a994023eb26fc)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-16 22:31:50 +01:00
Markus Lehtonen 87fba31ad7 devtool: remove unused imports / re-imports
(From OE-Core rev: e3de8aca33f612f2dd124ff712fb6758bf32a573)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-16 22:31:50 +01:00
Markus Lehtonen 89fb8c58c1 devtool: extract: remove patches when S=WORKDIR
Before this change, all files from the recipe (SRC_URI), including
patches, were added to to srctree repository when S==WORKDIR. The patch
files are useless as they are automatically applied on top of the
srctree by devtool.

This change causes devtool extract to not commit these unnecessary (and
possibly confusing) patch file(s) into srctree repository.

[YOCTO #7602]

(From OE-Core rev: 3e0ffff619e49b1f0c13e5f6a663455be3ed26af)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-16 22:31:50 +01:00
Paul Eggleton 1f15117404 devtool: fix for rename of gcc-source
After OE-Core commit 67db7182faf6742b0d971d61d8c5ba34f69d2e12, PV is
appended to the end of the gcc-source PN, thus we need to handle that in
devtool and the corresponding test.

Part of the fix for [YOCTO #7729].

(From OE-Core rev: 440029dc229a566b9bead1481215d5e5760c5fe6)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-07 14:59:43 +01:00
Paul Eggleton 4502daefb8 devtool: add: use the appropriate file naming and versioning for SCM recipes
* Recipes that fetch from git, svn or hg by OpenEmbedded convention
  should normally be named with this as a suffix, since PV is meant to
  be set appropriately within the recipe, so follow this. In order to
  make this work we need to be able to have the version independent from
  the file name, so add a -V option to recipetool create to allow this
  to be specified.
* If -V is specified on the devtool add command line, ensure at PV gets
  set to include this version.

(From OE-Core rev: 2b997c41c6476a13bf516586d56a9051aceb93ec)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-01 17:08:50 +01:00
Paul Eggleton 266ea28183 devtool: add: add an option to fetch remote source
Add a -f/--fetch option to fetch a remote URI (into the already
specified source tree path) and set this as SRC_URI within the recipe.
This simply wraps around the existing functionality in recipetool.

Implements [YOCTO #7644].

(From OE-Core rev: f22fd77e735fc5f4a3434e3d1f567a9d7d191cf4)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-01 17:08:50 +01:00
Paul Eggleton a4fca1d523 recipetool: ensure git clone is standalone when extracting
If -x is specified and the specified URI was a git repository, we need
to ensure that the resulting clone is a sandalone and not one that has
pointers into the temporary fetch location or DL_DIR (since the git
fetcher does a local clone with -s). Split out the code from devtool
that already does this for "devtool modify -x" and reuse that.

(From OE-Core rev: fc47e8652ef32e7399f57c80593df90dc52d8b84)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-01 17:08:50 +01:00
Paul Eggleton 216d26aac2 devtool: update-recipe: check if source tree is a git repository
If you've done "devtool add" (or "devtool modify" without -x) then it's
possible that the external source tree is not a git repository, so we
should handle that case here instead of printing a traceback.

(From OE-Core rev: eb2147aa8facd4ef33a0749e9ae660ec686dad48)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-04-27 15:05:52 +01:00
Paul Eggleton d6cd355def devtool: update-recipe: handle unversioned bbappends
Use the proper bbappend file name, don't just assume it will have a
version suffix (because it won't if the original recipe doesn't).

Fixes [YOCTO #7651].

(From OE-Core rev: 3332d68ef7b2a300ce8dcf5021497d98e5b17baa)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-04-27 15:05:52 +01:00
Paul Eggleton 1d4069255b devtool: reset: avoid errors in case file no longer exists
If you manually delete files in the workspace layer (which you really
shouldn't) it was possible to get yourself into the situation where you
couldn't reset because we were attempting to check if the file had been
modified and erroring out if it couldn't be opened. If the file's not
there anymore there's not much point checking if it needs to be
preserved, just skip it.

(From OE-Core rev: d2d352f7b747a97a54df9d20eb1455d166aa1ee3)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-04-27 15:05:52 +01:00
Paul Eggleton e8db1ccef3 devtool: add: use correct bbappend file name with -V option
We weren't adding the version into the bbappend file name when -V was
specified which meant that building or resetting failed.

Also adjust one of the tests so that we're testing devtool add both with
and without this option.

Fixes [YOCTO #7647].

(From OE-Core rev: bdbeff0cd342e31053d7203d78fc5dda611052b1)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-04-27 15:05:52 +01:00
Markus Lehtonen ca1f4d7279 devtool: include bbappends in recipe parsing
In order to get correct metadata, SRCREV for example.

Fixes [YOCTO #7648].

(From OE-Core rev: 8b1794559dd7fd956716179d628e61cffdce1686)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-04-27 15:05:52 +01:00
Markus Lehtonen 240b4ef3c6 devtool: modify: implement --no-same-dir
This option can be used to have a separate build directory, in order to
keep the srctree directory clean for packages that do not need to be
built in the source directory.

(From OE-Core rev: 5cc348c0a3016e4ccc1e80d099e2e197b3b38bd2)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-04-27 15:05:51 +01:00
Markus Lehtonen ee0ff82b56 devtool: modify: use B=S if that is the default for the recipe
Makes the build succeed for packages which do not support separate build
directory, e.g. zlib. The same outcome could be achieved with the
--same-dir option, but, it's generally hard to tell if a random package
would need that option. The negative side effect of this patch is that
dev srctree (of some packages that build fine without this modification)
gets dirtied by build artefacts.

(From OE-Core rev: 6ac5692426956e276ba2119f917b0f30791e6cd6)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Acked-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-04-27 15:05:51 +01:00
Paul Eggleton 5b1a6abcde devtool: force use of bash when running build within extensible SDK
Ubuntu's default dash shell causes oe-init-build-env to behave a bit
differently - (a) it can't pick up the OE root directory and (b) it
can't see any build directory specified as a command-line argument
(since dash doesn't pass through any arguments specified to sourced
scripts). We could work around these but doing so requires some internal
knowledge of the script; a much simpler fix is just to force running the
command under bash since it's expected to be installed on every distro.

Thanks to Chen Qi <Qi.Chen@windriver.com> for this fix.

Fixes [YOCTO #7614].

(From OE-Core rev: 27942f546e6b08cdf9f2dbda2e24d237cde7f5f5)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-04-24 11:06:54 +01:00
Markus Lehtonen d7a9b5ea55 devtool: call parse_recipe with correct arguments
Give the correct data object ("config data" instead of "recipe data") as
an argument to oe.recipeutils.patch_recipe()

Fixes [YOCTO #7595]

(From OE-Core rev: 7d2b918019b0afe215a6489dad697afac34f73e9)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-04-19 13:34:43 +01:00
Paul Eggleton 8123b4aa62 devtool: deploy-target: add dry-run option
Add a dry-run option to the deploy-target and undeploy-target
subcommands so you can see the list of files to be deployed or
un-deployed before actually carrying out the operation.

(From OE-Core rev: b9625581f3fe72fc402632be2d87cf889301c6a2)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-03-20 11:27:47 +00:00
Paul Eggleton 3d7777f061 devtool: deploy-target: add an option to disable quiet mode
The -q option to scp does stop the progress being shown, which is mostly
superfluous, however it also stops errors from ssh being shown - if
there's a problem, you'll just get "lost connection" which really isn't
that helpful. As a compromise, add a -s/--show-status option and
advertise this when the command fails.

(From OE-Core rev: 5cbb026212b4c8f5206a07d70b94f57edeee0839)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-03-20 11:27:46 +00:00
Paul Eggleton 3f370f8f6f devtool: deploy-target: detect and error if D is empty
If you haven't built the recipe yet or if the output directory (${D}) is
empty, then we should tell the user rather than have scp error out.

(From OE-Core rev: 3fd9ac277393bef5c646f16b9d8f30c277881d70)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-03-20 11:27:45 +00:00
Paul Eggleton ad57dab31f devtool: deploy-target: allow disabling host key checking
If you're testing with multiple images/devices that have the same IP
address / hostname then it can be annoying to deal with host key
mismatches all of the time. As a MITM attack is unlikely in the local
test environment, provide a command line option to pass the appropriate
options to scp/ssh to disable the host key checking.

Note: if you wish to apply this permanently, the best way is to do it
through your ssh configuration e.g. by adding the following to your
~/.ssh/config:

Host 192.168.7.2
   UserKnownHostsFile=/dev/null
   StrictHostKeyChecking no

(From OE-Core rev: 81dd1319112a99bc38b7a7ced0663918ac5b09a4)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-03-20 11:27:45 +00:00
Paul Eggleton cb4b07838c devtool: modify/extract: prevent usage with incompatible recipes
Consolidate code for checking compatible recipes and consider meta and
packagegroup recipes as well as package-index and gcc-source to be
incompatible.

(From OE-Core rev: 4be9bf637583b341a89af1b9924752abc7d49c94)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-03-20 11:27:45 +00:00
Paul Eggleton 221705f014 devtool: modify: get correct initial revision from previously extracted source tree
If you point devtool modify to a source tree previously created by
devtool modify or devtool extract, then we need to try to pick up the
correct initial revision so that devtool update-recipe knows where to
start looking for commits that match up with patches in the recipe.

(From OE-Core rev: c20e10543e268ebb43074a3f8d6e7ed991e54ec8)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-03-20 11:27:44 +00:00
Paul Eggleton 9adae67e71 devtool: reset: add ability to reset entire workspace
Add a -a/--all option to allow you to quickly reset all recipes in your
workspace.

(From OE-Core rev: 0c83788b111a761f6f500b86780cc51aed255402)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-03-20 11:27:44 +00:00
Paul Eggleton e3cfb80f1e devtool: name command line parsers appropriately
No functional changes, just use a unique name for each parser.

(From OE-Core rev: 5fabc59d6221c3fe7137b70e31ec2761a4276a6c)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-03-20 11:27:43 +00:00
Paul Eggleton 4424440f42 devtool: deploy-target: fix deploying to previously deployed machine
* Pass correct arguments to undeploy() function
* If an error occurs during undeploy(), exit instead of continuing

(From OE-Core rev: c938dee4b28af7e6296c86347dfa533f85605033)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-03-20 11:27:43 +00:00
Paul Eggleton 122ee11bd1 devtool: fix broken clones of git recipes
Because we move the workdir when extracting source, then move the source
and delete the temporary workdir, you lose the indirection symlink
pointed to by the alternates file (which is created when the fetcher
clones it from DL_DIR with -s) and the resulting repository is broken.
In any case, for a source repo that the user may put their own changes
into, we can't really rely on a clone made with -s in case the
original goes away - because of cleanall, DL_DIR disappearing, etc. So
repack the repository so that it is a complete, non-shared clone after
unpacking.

(While I'm at it, add a test for devtool modify with a git recipe which
verifies that this works.)

(From OE-Core rev: dccdde4321c48a571677995a4e1dfc583e899cbe)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-02-23 17:35:29 +00:00
Paul Eggleton e7744de454 devtool: support source extraction for recipes that use a shared workdir
Enable source extraction used by devtool extract / devtool modify -x for
recipes that use a shared workdir (e.g. the kernel and gcc).

(From OE-Core rev: 73a5bdc7921791b954ca717293e76c7e3091a19e)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-02-23 17:35:29 +00:00
Paul Eggleton 62a41b412f devtool: modify: handle recipes that use a shared workdir
If S is outside of WORKDIR then we shouldn't try to get the relative
path in order to work out where the source subdirectory is as we do by
default.

(From OE-Core rev: 7392ecd559ef71241405c704a65da171ee216ca1)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-02-23 17:35:29 +00:00
Paul Eggleton e2ce29fef2 devtool: prevent extract/modify use with image recipes
There wouldn't be any point to using these with an image recipe, so
disallow it.

(From OE-Core rev: 611c27182a6e52e159a1ce1b5bac054405c611d2)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-02-23 17:35:29 +00:00
Paul Eggleton efedd4323b devtool: update-recipe: add handling for git recipes
When updating git-based recipes, in a lot of cases what you want is to
push the changes to the repository and update SRCREV rather than to
apply patches within the recipe. Updating SRCREV is now the default
behaviour for recipes that fetch from git, but this can be overridden
in both directions using a new -m/--mode option.

(From OE-Core rev: 654792bb87610ee3569d02a85fa9ec071bf8ab6d)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-02-23 17:35:29 +00:00
Paul Eggleton e5bf4ed333 devtool: build: run do_populate_sysroot instead of do_install
If you want to be able to make use of libraries in conjunction with
devtool then we need to install them into the sysroot for other recipes
to use. Make it a configuration option in case it needs to be changed at
runtime.

(From OE-Core rev: 94f517ad8e55edfbe6f06afd963bcfeb849626ff)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-02-23 17:35:28 +00:00
Paul Eggleton 845efa450c devtool: reset: run bitbake -c clean on recipe
If you've added a new recipe, you want the output cleaned when you do
devtool reset, otherwise cruft from building the recipe may remain which
could interfere with future builds.

(From OE-Core rev: 664d1a7fe8f8288fabc582d00f6e36ab29496ec5)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-02-23 17:35:28 +00:00
Paul Eggleton 8e7437b3f0 devtool: add/modify: add option to build in same directory
The default behaviour is to build in a separate directory to the source,
however some projects can't be built this way, so add an option to do
that (or override the automatic behaviour in the case of modify).

(From OE-Core rev: cf094ed2f616a581eb07d78db4366c83a441fde5)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-02-23 17:35:28 +00:00
Paul Eggleton 943353707b devtool: improve command-line help
Based on feedback from Scott Rifenbark <scott.m.rifenbark@intel.com>

(From OE-Core rev: ec3378f3a7013e289daa0f5c52329488b861f99c)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-02-08 08:00:27 +00:00
Junchun Guan 0d686ccc7c scripts/devtool: Support deploy/undeploy function
Deploy recipe output files to live target machine using scp
Store the files list and target machine info in localhost if deployment
is done
Undeploy recipe output files in target machine using the previous
deployment info

[YOCTO #6654]

(From OE-Core rev: cc10751aa3121218b21ec2058358f3ea02a4500a)

Signed-off-by: Junchun Guan <junchunx.guan@intel.com>
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-12-23 10:18:16 +00:00
Paul Eggleton cd5ca4a11d scripts/devtool: add development helper tool
Provides an easy means to work on developing applications and system
components with the build system.

For example to "modify" the source for an existing recipe:

  $ devtool modify -x pango /home/projects/pango
  Parsing recipes..done.
  NOTE: Fetching pango...
  NOTE: Unpacking...
  NOTE: Patching...
  NOTE: Source tree extracted to /home/projects/pango
  NOTE: Recipe pango now set up to build from /home/paul/projects/pango

The pango source is now extracted to /home/paul/projects/pango, managed
in git, with each patch as a commit, and a bbappend is created in the
workspace layer to use the source in /home/paul/projects/pango when
building.

Additionally, you can add a new piece of software:

  $ devtool add pv /home/projects/pv
  NOTE: Recipe /path/to/workspace/recipes/pv/pv.bb has been
  automatically created; further editing may be required to make it
  fully functional

The latter uses recipetool to create a skeleton recipe and again sets up
a bbappend to use the source in /home/projects/pv when building.

Having done a "devtool modify", can also write any changes to the
external git repository back as patches next to the recipe:

  $ devtool update-recipe mdadm
  Parsing recipes..done.
  NOTE: Removing patch mdadm-3.2.2_fix_for_x32.patch
  NOTE: Removing patch gcc-4.9.patch
  NOTE: Updating recipe mdadm_3.3.1.bb

[YOCTO #6561]
[YOCTO #6653]
[YOCTO #6656]

(From OE-Core rev: 716d9b1f304a12bab61b15e3ce526977c055f074)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-12-23 10:18:16 +00:00