Commit Graph

1925 Commits

Author SHA1 Message Date
Andre McCurdy 424466b91f oe-setup-builddir: tidy up local.conf and bblayers.conf commentary
Remove obsolete reference to enabling parallel build via local.conf.

Unify wording used to explain that local.conf and bblayers.conf have
been created based on defaults.

(From OE-Core rev: 0073679c60c813c7c8d62bff997feb30ce64543c)

Signed-off-by: Andre McCurdy <armccurdy@gmail.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-09 16:58:11 +00:00
Paul Eggleton e43fcdf1e8 scripts/hob: drop
The Hob UI is gone from BitBake, so we should remove this shortcut
script.

(From OE-Core rev: 414f0d1ef07bb782ffa15ffd9db648ee2b2d1448)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-07 17:17:43 +00:00
Paul Eggleton afb1d09ced recipetool: create: fix support for AX_CHECK_LIBRARY
Clearly I didn't test this part of the code - lists don't have an "add"
method. Needless to say I have tested it now.

(From OE-Core rev: 063ed9058a14775f77e7875d4f6ef5719fa03f18)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-07 00:11:38 +00:00
Paul Eggleton 25d9c4e02a devtool: add build-sdk subcommand
Add a build-sdk command which is only available within the extensible
SDK that builds a derivative extensible SDK. The idea is recipes in the
workspace become a part of the new SDK - for example, this allows taking
a vendor provided SDK, adding a few libs and then producing a new SDK
with those included.

When normally building the extensible SDK, the workspace is excluded;
here we need to copy into the new SDK (renaming it in the process); the
recipes' task signatures become locked and thus the sources are no
longer needed, so they are removed along with the workspace bbappends
which would interfere with the locked signatures. Additionally we need
to just copy the configuration files (i.e. local.conf and auto.conf)
rather than filtering and appending to them since that work has already
been done when constructing the original SDK. The extra sstate artifacts
from workspace recipes are also determined and copied into the new SDK
in minimal mode (on the assumption that you won't set up a new sstate
mirror).

This reuses some code from build-image, so that needed to be
generalised to allow that.

Implements [YOCTO #8892].

(From OE-Core rev: 59e207ff6dd4b50a8905e14bc9292cf2794f4e7a)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-07 00:11:38 +00:00
Paul Eggleton 41eb36dc4c devtool: build-image: rename module
Hyphens aren't allowed in python identifiers, so you shouldn't use them
in module names or they are more difficult to import.

(From OE-Core rev: e8f6db9436dfc923e236f2cbc08f357e3b24fd5d)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-07 00:11:37 +00:00
Juro Bystricky 14dd07caac sdk.py: Fix undefined variable
"sdk_update" uses a variable newsdk_path, which was never declared.
This would cause the command:

    devtool sdk-update <poky-sdk-latest>

to fail with an error:
NameError: global name 'newsdk_path' is not defined

The remedy is to declare newsdk_path as it was no doubt intended,
corresponding to the argument specifying <poky-sdk-latest>.

[YOCTO#9042]

(From OE-Core rev: 6ac589a4a9290665f8c5295e2e2a03a2b187b957)

Signed-off-by: Juro Bystricky <juro.bystricky@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-03 10:49:00 +00:00
Paul Eggleton 5c5c13d777 recipetool: create: add basic support for new npm fetcher/class
Add detection for npm modules and support for extracting the name and
version from package.json as is usually part of an npm module contents.

Note: this will likely only produce a buildable recipe if you use an
npm:// URL; simply pointing to a node.js source repository isn't going
to fetch the module's dependencies. It also doesn't set up the
shrinkwrap/lockdown automatically, so there is some room for improvement
later.

Implements [YOCTO #8690].

(From OE-Core rev: 41d0e4d75f13b53a6c1b6a8df9be4742be7534e0)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-02 23:08:51 +00:00
Paul Eggleton 2be37a9373 recipetool: create: add basic support for generating linux kernel recipes
Add support for detecting a Linux kernel source tree and generating a
basic kernel recipe using meta-skeleton's linux-yocto-custom recipe as a
base.

Implements [YOCTO #8981].

(From OE-Core rev: 39cab544b80ca4450106c9ede3180929ba24703c)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-02 23:08:51 +00:00
Paul Eggleton 5cf15ffecc recipetool: create: add support for out-of-tree kernel modules
Detect kernel modules by looking for #include <linux/module.h>, and
handle the various styles of Makefile that appear to be used. I was able
to use this code to successfully build a number of external kernel
modules I found.

Implements [YOCTO #8982].

(From OE-Core rev: a85604f2eb2438b4caf0832c2ea15b5822f7e9a1)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-02 23:08:51 +00:00
Patrick Ohly 431c1e171f combo-layer: handle empty commits during "init --history"
When importing the components during the "combo-layer init" with full
history and relocation into a destination directory, components with
empty commits were not handled because the "mv" command was invoked
with just one parameter.

Replacing that with a find/xargs pair avoids the problem and should
also fix the handling of filenames with special characters (in
particular spaces).

(From OE-Core rev: fd9d08b8ba512151615a03fda61a103760a0683c)

Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-02 22:39:43 +00:00
Christopher Larson d4da534faa wic: don't throw away our created swap partition
We were creating the partition, mkswap'ing it, and then not dd'ing it into
place in the final image, as source_file wasn't being set for swap partitions.
This would result in a swap partition that couldn't be enabled on target until
mkswap was run.

(From OE-Core rev: 7580942a66f84c2120763c2d9665e2cec8e25138)

Signed-off-by: Christopher Larson <chris_larson@mentor.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-28 11:33:04 +00:00
Paul Eggleton 10290f2245 devtool: upgrade: print new recipe name
Make this consistent with "devtool add" so that the user knows where to
find the new recipe.

(From OE-Core rev: 28fea4a104ba85aded2ecfe891e9bf997d72a103)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-28 11:33:01 +00:00
Paul Eggleton 5cd3be38f2 devtool: upgrade: drop PR on upgrade
The PR value should be reset to the default when upgrading, so we need
to drop it from the newly created file.

(From OE-Core rev: 4acf8a6912106dd3da5ceee9e30ccce86211d04b)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-28 11:33:00 +00:00
Paul Eggleton e6f684b10c devtool: upgrade: eliminate unnecessary datastore copy
We aren't modifying the datastore copy here, so we don't need a copy at
all.

(From OE-Core rev: 9f05bf98d4dbcb991b684a2ab480c6638c6be292)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-28 11:33:00 +00:00
Paul Eggleton 860574edb2 devtool: upgrade: fix several issues with extraction of new source
Fix several issues when extracting the new version source over the top
of the old one (when the recipe is not fetching from a git repo):

* Delete the old source first so we ensure files deleted in the new
  version are deleted. This also has the side-effect of fixing any
  issues where files aren't marked writeable in the old source and thus
  overwriting them failed (harfbuzz 1.1.3 contains such files).
* Fix incorrect variable name in abspath statement that made it a no-op

(From OE-Core rev: 4f7595a514661f0e2418393286849a1cb6c46f0d)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-28 11:33:00 +00:00
Paul Eggleton 66a781c7e1 devtool: upgrade: fix constructing new branch from tarball releases
When we do an upgrade from one tarball version to another we want to:

1) Check out the old version as a new branch
2) Record the changes between the old and new versions as a commit
3) Check out the old version with patches applied
4) Rebase that onto the new branch

Where we went wrong was step #1 where instead we checked out the old
version with patches applied as the new branch, which meant the rebase
didn't do anything and any changes made by the patches to files still in
the new version were wiped out.

(From OE-Core rev: a4f3e6d5f4c549e36cce1e18801a565a3a3c9b0a)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-28 11:33:00 +00:00
Paul Eggleton d30cc76039 devtool: upgrade: fix renaming of recipe if PV is not in name
If the actual value of PV isn't in the name of the recipe (for example,
a git or svn recipe) there's no point trying to rename it. Additionally,
we already have the original filename, there's no need to guess it -
just pass it in.

(From OE-Core rev: e144db3650b4a7c0a59066621905f34a5400303e)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-28 11:33:00 +00:00
Paul Eggleton 75eeeabe3c devtool: upgrade: fix moving version-specific files directory
We were trying to move this from the current directory instead of the
path. Let's just use shutil.move() instead of shelling out to mv.

(From OE-Core rev: 60454a0ba154d6c777e0c2b05b887b4e4fcde986)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-28 11:33:00 +00:00
Paul Eggleton 81ebb0b61e devtool: upgrade: fix version argument checking
For recipes that specify SRCREV, the code here wasn't quite doing the
right thing. If the recipe has a SRCREV then that needs changing on
upgrade, so ensure that the user specifies it. If it doesn't, then it'll
be "INVALID" not None since the former is the actual default, so handle
that properly as well. Additionally an unset variable was being
erroneously passed when raising the error about the version being the
same leading to a traceback, so fix that as well.

(From OE-Core rev: 1d0f821371d1cb93e30fad86f0c20e38cb93b54b)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-28 11:33:00 +00:00
Paul Eggleton e953b5711f devtool: upgrade: drop superfluous call to validate_pn
The recipename argument to devtool upgrade specifies an existing recipe,
so by definition the name will be valid (or it won't exist) - we don't
need to validate it ourselves, that's only needed for situations like in
devtool add where we're creating a new recipe.

(From OE-Core rev: 1e2bc7d861555a04350a87f19047efdc717046be)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-28 11:33:00 +00:00
Paul Eggleton 492b1eb1d2 devtool: upgrade: make source tree path optional
Make devtool upgrade consistent with devtool add/modify in defaulting to
sources/<recipename> under the workspace if no source tree path is
specified.

(From OE-Core rev: 8a952407b192313515e91570632446b6dff01665)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-28 11:33:00 +00:00
Paul Eggleton 942ae25279 devtool: modify: fix source tree default name when mapping virtuals
If you for example ran devtool modify virtual/libusb0 without specifying
a source tree path, the default was <workspace>/sources/virtual/libusb0
which isn't correct - it should be using the mapped name i.e.
libusb-compat (in the default OE-Core configuration). Reorder some of
the code to ensure that the mapped name is used.

(From OE-Core rev: c51736df17da8e6e561dd5b7ce59cb08254da870)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-28 11:33:00 +00:00
Paul Eggleton e2334e126f devtool: add: tweak auto-determining name failure message
As suggested by Khem Raj.

(From OE-Core rev: 36cc6b81d0281348a0f241a80ddd427745a6a678)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-28 11:32:59 +00:00
Costin Constantin d2a563c32c oe-selftest: Add support for lib/oeqa/selftest subdirectories
This patch adds functionality to allow creating subdirectories inside
lib/oeqa/selftest for all layers present in BBLAYERS. Like this, test
cases can be grouped into organized directories.

Addresses [YOCTO #7865]

(From OE-Core rev: 445b84456659ebb355149f6b0dca543b0bb2679c)

Signed-off-by: Costin Constantin <costin.c.constantin@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-28 11:32:58 +00:00
Markus Lehtonen 73bf792c85 devtool: update-recipe: create config fragment
Create config fragment if the user makes modifications to kernel config.
User may change .config e.g. by directly editing it or by running the
'do_menuconfig' bitbake task. Devtool generates one monolithic fragment
by simply doing a diff between .config and .config.baseline files in the
source directory.  If either of these files is missing, the config
fragment is not gerenrated or updated. The output is a file,
'devtool-fragment.cfg' that gets added to SRC_URI in the recipe (as well
as copied into the 'oe-local-files' directory if that is present in the
source tree).

${S}/.config will be a symlink to ${B}/.config. We need to do this as
devtool is not able to access ${B} because ${B} is set in a .bbappend in
the workspace layer which is not parsed by devtool itself.

[YOCTO #8999]

(From OE-Core rev: 524da136e5b837a60682516ac08f3092c635e934)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-28 11:32:58 +00:00
Markus Lehtonen 2fbd1d77bd devtool: sync: update kernel config
Copy kernel config is copied to the source directory at a later phase in
_extract_source() so that it gets copied when devtool sync is done, too.

(From OE-Core rev: ff895be7a46c4b3b1b791e5387490d90bb34fce2)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-28 11:32:58 +00:00
Humberto Ibarra 3f7aa6fc5d scripts/oe-selftest: Use site.USER_SITE to run coverage configuration code for sub-process
Coverage in oe-selftest currently requires to create or modify
a sitecustomize.py file according the coverage tool setup instructions
(http://coverage.readthedocs.org/). This file has to be located in
the system's python folder, which is not a good solution since this
folder is not accesible to non-privileged users.

The best solution so far is to create this file in the home directory.
This is implemented by creating the temporal file in the user site
default folder.

[Yocto #8930]

(From OE-Core rev: 3f9b1658d745b536eff1017b2c74a9dff46b6f4a)

Signed-off-by: Humberto Ibarra <humberto.ibarra.lopez@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-28 11:32:57 +00:00
Humberto Ibarra 1c6c76e1df scripts/oe-selftest: Add filtering to the coverage data gathered by oe-selftest
When --coverage is used, oe-selftest gathers coverage data from the testcases
executed. The command lacks a way of filtering which files to gather coverage
data from.

This patch adds three options to specify which files should be considered.
The --coverage-source option specifies folders, while --coverage-include and
--coverage-omit specify patterns to have an extra level of filtering.

Some examples:

1. oe-selftest --run-all-tests --coverage

Gathers coverage data from the default poky folders

2. oe-selftest --run-all-tests --coverage --coverage-include /home/me/poky/scripts/*

Gathers coverage data only for the files located under '/home/me/poky/scripts'

3. oe-selftest --run-all-tests -coverage --coverage-omit /home/me/poky/meta*

Gathers coverage data. Files inside all the folders starting with 'meta' under
'/home/me/poky' are omited

4. oe-selftest --run-all-tests --coverage --coverage-source /home/me/poky/bitbake

Gathers coverage data only from files inside the folder: '/home/me/poky/bitbake'

[Yocto #8920]

(From OE-Core rev: 923481c7d8c09ed9b03109cf4debcc6b07845c59)

Signed-off-by: Humberto Ibarra <humberto.ibarra.lopez@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-28 11:32:57 +00:00
Richard Purdie 9d5483c375 meta-yocto: Rename to meta-poky to better match its purpose
"poky" is the reference distribution for the Yocto Project. This renames
the layer within the meta-yocto repository to meta-poky, better matching
what that layer contains.

A layer.conf file is left behind as this is the only way which allows
existing builds to migrate safely to the new name. It will be removed
at some future point.

This change requires the corresponding OE-Core change to handle the
migration and the changes to the infrastructure to support this.

(From meta-yocto rev: d0c88df2e14672fca4ebbde93c5efbcd0e4fa9b6)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-28 11:31:17 +00:00
Paul Eggleton 524ee088b3 recipetool: create: improve CMake package mapping
* Package names are actually case sensitive near as I can tell, so we
  shouldn't be lowercasing them everywhere.
* Look for CMake packages in pkgdata and map those back to recipes,
  so we aren't dependent on the hardcoded mappings (though those are
  still preserved).
* Avoid duplicates in the unmapped package list

(From OE-Core rev: 2ddad52ccca07245eea43d9b844c6c7d4b667ca3)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-21 09:32:43 +00:00
Paul Eggleton 7b6e5b025e recipetool: create: add additional extension mechanisms
Add a means of extending the dependency extraction for autotools and
cmake.

Note: in order to have this work, you need to have an __init__.py in the
lib/recipetool directory within your layer along with the module
implementing the handlers, and the __init__.py needs to contain:

  # Enable other layers to have modules in the same named directory
  from pkgutil import extend_path
  __path__ = extend_path(__path__, __name__)

(From OE-Core rev: 915dea9f89cd737e5ba167c384e8d314c5c23c49)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-21 09:32:43 +00:00
Paul Eggleton b2d44729e9 devtool: modify: tweak help description for behaviour change
I should have adjusted this in OE-Core commit
80a44e52609a89d9ffe816181ae193af491c06ac where the behaviour changed.

(From OE-Core rev: 13409a2b899bb74b8060c840b8c7ef8759d099cb)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-21 09:32:43 +00:00
Paul Eggleton a8e0e5ecd3 devtool: deploy-target: preserve existing files
If files would be overwritten by the deployment, preserve them in a
separate location on the target so that they can be restored if you
later run devtool undeploy-target.

At the same time, also check for sufficient space before starting the
operation so that we avoid potentially failing part way through.

Fixes [YOCTO #8978].

(From OE-Core rev: a2da55712691bcf1066c53d813107d75213d6b10)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-21 09:32:43 +00:00
Paul Eggleton 2059a344b6 devtool: undeploy-target: support undeploying all recipes
If you want to put the target device back to exactly how it was before
devtool deploy-target started poking things into it, then it would make
things easier if you didn't have to figure out which recipes were
deployed. Now that we have the list stored on the target we can
determine this reliably, so add a -a/--all option to undeploy-target to
undeploy everything that has been deployed.

One of the side-effects of this is that the dry-run functionality for
undeploy-target had to be reimplemented to actually run the script on
the target, since we have no way of knowing what's been deployed from
the host side. We don't need to do the same for deploy-target though
since we know exactly which files will be deployed without referring to
the target.

(From OE-Core rev: 41fed83060f0041e14e455d1446397bda277d953)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-21 09:32:42 +00:00
Paul Eggleton b95c72c4dc devtool: deploy-target: write deployed files list to target
When running devtool deploy-target, we save a list of deployed files,
and this list is used by devtool undeploy-target (or the next time
deploy-target is run if the list is present, in case any files have been
renamed or deleted since the first time). We were writing this file to
the host, but it makes more sense to write the list to the target
instead, so that if we for example swap in a different board, or switch
hosts, things will work as expected.

In order to do this properly we have to construct a shell script and
ship it over to the target so we can run it. The manifest is written out
to a hidden directory in the root (/.devtool).

Fixes [YOCTO #7908].

(From OE-Core rev: a16a0c9334b785e2df896266c8911a2c7a1806b8)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-21 09:32:42 +00:00
Paul Eggleton 62989efe6d devtool: sdk-update: tweak command-line handling of updateserver
Get the default value for updateserver from the configuration file and
show it in the help; also only make the parameter optional if it's
specified. This means we can also drop the check in the function as
argparse will then ensure it's specified if there's no config setting.

(From OE-Core rev: 82497bde58fc8bdae8d8acfbf025a1a90b14cc3e)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-21 09:32:42 +00:00
Paul Eggleton cada5a83c7 devtool: (un)deploy-target: add help descriptions
Add a long description used when running --help on the specific command.

(From OE-Core rev: eb7787d1652fd84a149fd394969f4f1099406051)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-21 09:32:42 +00:00
Paul Eggleton 6bd88e6171 scripts/lib/argparse_oe: tweak title above options
Naming these as "optional arguments" is perhaps slightly confusing since
some of the positional arguments might also be optional; in addition
it's rare (though possible) for options to be mandatory - up until
recently we had a recipetool option (-o) that was mandatory. It's not
perfect, but change it to "options" so it's at least a bit more
appropriate.

(From OE-Core rev: 55e675de6191bf7eccd26df29189f2a6faa40a20)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-21 09:32:42 +00:00
Paul Eggleton 32ef523898 devtool: categorise and order subcommands in help output
The listing of subcommands in the --help output for devtool was starting
to get difficult to follow, with commands appearing in no particular
order (due to some being in separate modules and the order of those
modules being parsed). Logically grouping the subcommands as well as
being able to exercise some control over the order of the subcommands
and groups would help, if we do so without losing the dynamic nature of
the list (i.e. that it comes from the plugins). Argparse provides no
built-in way to handle this and really, really makes it a pain to add,
but with some subclassing and hacking it's now possible, and can be
extended by any plugin as desired.

To put a subcommand into a group, all you need to do is specify a group=
parameter in the call to subparsers.add_parser(). you can also specify
an order= parameter to make the subcommand sort higher or lower in the
list (higher order numbers appear first, so use negative numbers to
force items to the end if that's what you want). To add a new group, use
subparsers.add_subparser_group(), supplying the name, description and
optionally an order number for the group itself (again, higher numbers
appear first).

(From OE-Core rev: e1b9d31e6ea3c254ecfe940fe795af44761e0e69)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-21 09:32:42 +00:00
Paul Eggleton 9f7df76eb4 devtool: update-recipe: don't show workspace recipe warning if no update
If we didn't make any changes to the file then there's no point warning
the user that we have done.

(From OE-Core rev: 391b9ba30d802ac420ddf382588e03e718861c01)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-21 09:32:42 +00:00
Paul Eggleton 51972edbb0 devtool: reset: fix preserving patches/other files next to recipes
If files had been created next to the recipe (for example devtool add,
edit the source and commit and then devtool update-recipe), running
devtool reset failed to preserve those files and gave an error due
to trying to rmdir the directory containing them which wasn't empty.
Fix the preservation of files in the "attic" directory properly so
we catch anything under the directory for the recipe, and replicate
the same structure in the attic directory rather than slightly
flattening it as we were before.

(From OE-Core rev: bbe63eb97ae7f78959f117d6066ef821c4da1c77)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-21 09:32:42 +00:00
Paul Eggleton e54f9c159d devtool / recipetool: use common code for launching editor
Looking at Chris Larson's code for starting the user's editor for
"recipetool newappend" it was slightly better than what I wrote for
"devtool edit-recipe" in that it checks VISUAL as well as EDITOR and
defaults to vi if neither are set, so break this out to its own function
and call it from both places. The broken out version passes shell=True
however in case it's a more complicated command rather than just a name
of an executable.

(From OE-Core rev: 184a256931e8cdc7bea97a905c4e67a435964de0)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-21 09:32:42 +00:00
Paul Eggleton dd35f69340 devtool: minor fix for error message
There is no -N/--name option for devtool, that's a recipetool option -
with devtool you just specify the name as a positional argument.

(From OE-Core rev: d2bc0cba5ca8a7220ffe1ef96acf856fe972ce7c)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-21 09:32:42 +00:00
Randy Witt 4c5c40df3c devtool: Don't recursively look for .devtoolbase in --basepath
If the user specifies --basepath on the commandline, only the directory
specified should be searched for .devtoolbase. Otherwise when --basepath
is a child of the sdk directory, .devtoolbase will always be found and
devtool will only show options meant to be used within an sdk.

(From OE-Core rev: 3bb07c61220b89384a76c745e33be37aad5095c0)

Signed-off-by: Randy Witt <randy.e.witt@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-21 09:32:41 +00:00
Richard Purdie 94abdb2eea linux-yocto: Work around PAT issue on qemux86
We have an issue with PAT handling on older processes with limited PAT bits,
see the patch description for the full problem. This replaces the runqemu
workaround with a kernel patch until we can get the kernel trees sorted
out and discuss a proper fix with upstream. It should be safe everywhere
so is applied unconditionally.

(From OE-Core rev: e00f0794a535c8e68ae1c87c8b01dd65645d570b)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-15 16:28:45 +00:00
Ed Bartosh ab7cb65ab5 wic: fix processing of --use-uuid
Processing of this option was lost during recent change of
wks parsting. It was discovered during the work on booting
wic images under qemu. Now, when -use-uuid is fixed it's
possible to specify root partition by partition uuid.
This will be done in the following commit.

(From OE-Core rev: b4882e0b84d7fd4c85ee95386e94722485eafc2b)

(From OE-Core rev: 73e9e3f150bf2de9b27c2ccc73e3dee334ee73fe)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-15 16:28:44 +00:00
Ed Bartosh c26a9c3afd runqemu: support path/to/<image>-<machine>.wic
Supported providing wic image path to runqemu:
  runquemu path/to/<image>-<machine>.wic

[YOCTO #8691]

(From OE-Core rev: 58a3bfb1e4b493200820cdf0bf3fc79e31e792de)

(From OE-Core rev: e6150971ea4eea49b802a12aea5ab733e894c92d)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-15 16:28:44 +00:00
Ed Bartosh c7f0578b78 runqemu: don't set KERNEL for wic images
Wic images should be boot as is, without pointing qemu to the kernel
binary. Current code doesn't use kernel, but sets KERNEL variable and
shows kernel path in the console output. This can confuse users.

Changed runqemu and runqemu-internal code to avoid setting KERNEL
variable and show kernel path.

(From OE-Core rev: 474caa7ed5ff05caa5d49d270b283882fa616ed1)

(From OE-Core rev: 35e776e00cce25f2c9c45500612fb66022ec4739)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-15 16:28:44 +00:00
Ed Bartosh 2c3a009b20 runqemu: add support for wic images
Quemu should be able to run wic images this way:
    runqemu <machine> <image recipe> wic

Tested with 'runqemu qemux86-64 wic-image-minimal wic'

(From OE-Core rev: 8716be799949cb8bde7fa49cbea61312a3a93bb7)

(From OE-Core rev: dd42931bf99b8bbd4ad452b3941d957f41b81796)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-15 16:28:43 +00:00
Jan Sarenik 64d2f13d1a scripts/sstate-cache-management.sh: Change wording
Use "out of" rather than "from" in the output.

(From OE-Core rev: c712529030c71b45bb8ae647c17319c2647aedff)

Signed-off-by: Jan Sarenik <jajomojo@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-15 16:28:43 +00:00
Richard Purdie 4e0b33413f scripts/runqemu-internal: Work around qemux86 PAT bugs in linux 4.4.1
Since the upgrade of qemux86 to 4.4.1 we're seeing PAT issues when
starting the X server. We need to fix the problem but the failing
sanity tests mask out other issues and we need a workaround.

Merge this for now until we can figure out the full issue. This is
better than changing the kernel defconfig or reverting to old
kernel versions.

(From OE-Core rev: 2749ba318bf322cb5014689532372473004e92b9)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-12 18:03:46 +00:00
Paul Eggleton a5095d146d recipetool: create: set S when we set SRC_URI from local git repo
If you specify a local directory which happens to be a git repository
with an origin remote (and it is in fact remote), we can use that for
SRC_URI as implemented by OE-Core revision
b143d414846854dc8b3e0a47358daf5646eded38, however we also need to set S
if the recipe is going to be of any use fetching from that SRC_URI
later.

(From OE-Core rev: 8bbbd2d63f1bc752f9a30054a089dc2caf5fd84c)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-11 12:33:03 +00:00
Paul Eggleton ca5a36cbd1 recipetool: create: convert http git URLs that don't end in .git but contain /git/
When recipetool create is given a URL that starts with http(s):// and
contains /git/, such as the URLs at git.yoctoproject.org, it's fairly safe
to assume it's a git repository and not something that should be fetched
with wget, so rewrite the URL.

(From OE-Core rev: 3ca04757a670e8b6f78799cc0454d75691809ac4)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-11 12:33:03 +00:00
Paul Eggleton 4c71afb378 recipetool: create: ensure URL parameters don't make it into the name
When auto-detecting the name for a recipe from the URL, strip off any
parameters (";name=value...") before parsing the URL, otherwise this
just ends up in the recipe name which is undesirable.

(From OE-Core rev: d3c46b5d0abd56bcadd4f2f1ef985f13d67f605b)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-11 12:33:03 +00:00
Paul Eggleton 86f3464b47 devtool: add: fix adding from a local source directory
Fix a regression introduced in in OE-Core revision
aedfc5a5db1c4b2b80a36147c9a13b31764d91dd where specifying a local source
tree without specifying a name resulted in a traceback.

Fixes [YOCTO #9086].

(From OE-Core rev: 67ea9d21f20a371a0ee443b46326018cb1527b62)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-11 12:33:03 +00:00
Paul Eggleton fa50153bc2 devtool: modify: make -x the default behaviour
It's going to be more common for users not to have the prepared source
tree for a recipe already, so the default behaviour ought to be to
extract it for them from the recipe. Change the default to extract
(effectively making the -x option a no-op) and add a --no-extract/-n
option to disable it. Later we can look at trying to be smart and
reusing an existing source tree instead of erroring out if it exists;
for now this is just the default reversal.

(From OE-Core rev: 80a44e52609a89d9ffe816181ae193af491c06ac)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-11 12:33:03 +00:00
Paul Eggleton f767757b3e recipetool: create: determine name/version from github/bitbucket URLs
Both BitBucket and GitHub provide "release" downloads that unfortunately
don't include the actual project name. For recipetool this means we have
to have special handling for these URLs or else the current code assumes
the name of the project is something like "v0.9.1". Borrow and adapt
some code from autospec to do this, and put it in its own function for
clarity.

(From OE-Core rev: e8435fde7f82c39cc0efe1d4bf043b29f63898e9)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-11 12:33:03 +00:00
Paul Eggleton d94c7e367c recipetool: create: support cmake find_library directive
CMake supports a find_library() directive to find named libraries, so
detect dependencies from this.

(From OE-Core rev: d0265bc67f797ee4b7760cf37335994133809abf)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-11 12:33:03 +00:00
Paul Eggleton ddfe74447b devtool: commit for extra tasks that modify source when extracting
When extracting source for a recipe, if there are additional custom
tasks run that make changes to the source, create a commit in the
generated git branch so they are contained. This is particularly
useful for tasks that come before do_patch since otherwise the changes
might get incorporated in the first applied patch, but otherwise it
helps avoid the tree being dirty at any point.

Fixes [YOCTO #7626].

(From OE-Core rev: 997a77d9b20af1778b804778e5d8c8a7424f7582)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-11 12:33:02 +00:00
Christopher Larson 6fab5fc1bb recipetool.newappend: add -e/--edit argument
(From OE-Core rev: 2bd518fe3a8cb0022a816a7d1ac8d9a3dedee6d9)

Signed-off-by: Christopher Larson <chris_larson@mentor.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-11 12:27:49 +00:00
Ross Burton 9237097b79 python: merge python-elementtree into python-xml
python-xml depends on python-elementtree as the latter just contains a C library
used by the former.  However there's no point to this split apart from
increasing the number of packages, so merge -elementtree into python-xml.

(From OE-Core rev: 5f7206eba3953b7f29148ecfb791995773ee5fc7)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-11 12:27:45 +00:00
Maciej Borzecki 7d5d8d0cd9 scripts/lib/bsp/engine: trailing whitespace cleanup
(From meta-yocto rev: 3f19a080b2370c0317f88299d392a5b1a5f9c830)

Signed-off-by: Maciej Borzecki <maciej.borzecki@open-rnd.pl>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-08 10:54:56 +00:00
Maciej Borzecki dfeda17ca1 scripts/lib/bsp/engine: fix path separator
Make path to newly generated README file legitimate

(From meta-yocto rev: 32286bb798c2778457b5578b4b590629c96a0ee2)

Signed-off-by: Maciej Borzecki <maciej.borzecki@open-rnd.pl>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-08 10:54:56 +00:00
Ioan-Adrian Ratiu eb134c6ee6 isoimage-isohybrid.py: fix cpio working directory
Take `pwd` to be <initial-dir>. The %s path is relative to it. The value
of %s is "output_folder/build". The current code works as follows:

Changing directory to %s and finding the sources (after cd'ing) to cpio
with output redirection to %s/initrd.cpio triggers the following error

"Error: exec_cmd: cd output_folder/build/INITRD && find . | cpio -o -H
newc >output_folder/build/initrd.cpio  returned '1' instead of 0"

This happens because after the cd, `pwd` is <initial-dir>/%s and by the
redirect we write the result to to <initial-dir>/%s/%s/initrd.cpio which
obviously does not exist.

Fix this by getting the sources with "find %s" instead of "cd && find ."

(From OE-Core rev: 07fa4783566d22d46ce719a621eee5404932dbbe)

Signed-off-by: Ioan-Adrian Ratiu <adrian.ratiu@ni.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-07 17:29:59 +00:00
Richard Purdie 32b498cba3 scripts/devtool: Add getVarFlag expand argument
(From OE-Core rev: afa9b1081848cf0dbc40bbaf565bcc2ee19e8f6e)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-06 23:10:44 +00:00
Dominique Hunziker 149cb1769b python-distutils: add missing dependency on python-email
(From OE-Core rev: 1a8efbf4f06d9274cd60ada61d34b6bf42c15dc0)

Signed-off-by: Dominique Hunziker <dominique.hunziker@gmail.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-04 23:39:02 +00:00
Mihaly Varga 0b82af2aed wic: isoimage-isohybrid: check for syslinux-native
.iso image creation fails if during the image creation syslinux
is baked and syslinux-native is not.

Added new check to verify if both syslinux and syslinux-native
are baked and bake them if these are not installed.

(From OE-Core rev: fd5749832960ad3b85697c2878490d6f008982a3)

Signed-off-by: Mihaly Varga <mihaly.varga@ni.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-04 23:39:02 +00:00
Maciej Borzecki 789463333f wic: default to empty bootloader config
A kickstart file for non-x86 boards may have no 'bootloader' stanza. It
is the usual case if bootloader is setup using other mechanism than
through wic, and is for instance a part of u-boot configuration. In such
case the 'bootloader' field in the KickStart class will be
uninitialized. Instead of adding an empty bootloader line in every
kickstart file call the bootloader parser with empty argument list to
get defaults namespace.

(From OE-Core rev: 264c03e854f77c3b62acb710384f66716ccbf469)

Signed-off-by: Maciej Borzecki <maciej.borzecki@open-rnd.pl>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-04 23:39:01 +00:00
Olof Johansson 5c35883efc bitbake-whatchanged: avoid double do_ task name prefix
When used with --verbose, the heading for each task looks like

  === The verbose changes of example.do_do_compile:

This should instead be

  === The verbose changes of example.do_compile:

(From OE-Core rev: 628ad5e06d1136809d110a71148721095cb084dc)

Signed-off-by: Olof Johansson <olof.johansson@axis.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-30 11:37:00 +00:00
Paul Eggleton 2c14be3507 gen-lockedsig-cache: fix bad destination path joining
When copying the sstate-cache into the extensible SDK, if the source
path had a trailing / and the destination path did not, there would be a
missing / between the path and the subdirectory name, and you'd end up
with subdirectories like "sstate-cacheCentOS-6.7". There are functions
in os.path for this sort of thing so let's just use them and avoid the
problem.

(From OE-Core rev: 5eb8f15c48b5f39a10eb2b63b026cf1ebfd05533)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-26 22:32:00 +00:00
Ross Burton 2495dfafad scripts/wipe-sysroot: also delete uninative sysroot
The uninative sysroot is in ${STAGING_DIR}-uninative so delete that alongwith
$STAGING_DIR.

(From OE-Core rev: 258668c3cb8f5c00e084e821dae05ba750768bfb)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-26 22:31:58 +00:00
Markus Lehtonen aadb879e5b devtool: extract: use the correct datastore for builddir
(From OE-Core rev: 6f5bec4570a9237681fcee3922209af2a48f6c07)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-26 22:31:58 +00:00
Paul Eggleton 89a7ed5b67 devtool: add configure-help subcommand
When you need to set EXTRA_OECONF for a recipe, you need to know what
options the configure script actually supports; the configure script
however is only accessible from within a devshell and (at least in the
case of autotooled software fetched from an SCM repository) may not
actually exist until do_configure has run. Thus, provide a "devtool
configure-help" subcommand that runs the configure script for a recipe
with --help and shows you the output through a pager (e.g. less),
prefaced by a header describing the current options being specified.

There is basic support for autotools, cmake and bare configure scripts.
The cmake support is a little hacky since cmake doesn't really have a
concise help option that lists user-defined knobs (without actually
running through the configure process), however that being a design
feature of cmake there's not much I can think of to do about that at
the moment.

(From OE-Core rev: 0e5d84d9705091b338000ef02720cfa090f76888)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-26 18:05:14 +00:00
Paul Eggleton 84720c8ce9 devtool: properly handle bb.build.FuncFailed when extracting source
When we run the tasks required to extract the source for a recipe (e.g.
within "devtool modify" or "devtool extract") if one of those tasks
fails you get a bb.build.FuncFailed exception; handle this properly so
you don't see a traceback.

(From OE-Core rev: 95d8631b3bdf216001e57f48277535c65a4cc49e)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-26 18:05:13 +00:00
Paul Eggleton c3f0f7be84 devtool: add: warn if modified recipe found in attic directory
If a recipe generated by "devtool add" has been modified since then when
you run "devtool reset", it will be moved into the "attic" subdirectory
of the workspace in case those modifications need to be preserved. It
seems natural that if those modifications were worth preserving we
should warn the user if such a file exists when they run "devtool add"
to create the same recipe again, so they can pick up where they left off
if they want to.

(From OE-Core rev: 0a39b907ff997c3a62c92ab22325c726b612de5b)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-26 18:05:13 +00:00
Paul Eggleton e559b66289 devtool: build-image: allow specifying packages to add to image
Provide an option to devtool build-image to specify the list of packages
instead of taking the list of packages produced by recipes in the
workspace. Sometimes you don't want all of these packages; other times
you want to add more.

This is the most immediate fix for [YOCTO #8855], though it is a little
crude so I would like to provide better means of customising the image
contents later.

(From OE-Core rev: b3a44951a74fe58714b72e71a7a558b67a71e1e3)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-26 18:05:13 +00:00
Paul Eggleton e00eac862e devtool: move edit-recipe to a separate module
standard.py is getting a bit large; move the "utility" commands to
another module.

(From OE-Core rev: 5089b93f5b341dc28c343f7afe15efda2081ed36)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-26 18:05:13 +00:00
Markus Lehtonen 7a52f67e59 build-perf-test.sh: add eSDK testing
Add simple initial eSDK test. Currently, only download size and
installation time of eSDK is measured. The eSDK to be tested is
generated from the same image that the other tests are run for. This
patch will add two new fields to the global results log and that needs
to be taken into account when examining the results.

(From OE-Core rev: c903c1e1f36a4dd1dc1b7a621fa7a6ffe7411119)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-25 12:44:13 +00:00
Markus Lehtonen 5c367eccf8 build-perf-test.sh: more generic timing function
Make it possible to time also other than bitbake commands. The name of
the log file is changed from bitbake.log to commands.log.

(From OE-Core rev: 7b355dc96255b06f3108a7d02ab0ed408d64bf1b)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-25 12:44:13 +00:00
Paul Eggleton c1f4e929c9 recipetool: create: better fix for fetch error handling
I was a little bit hasty in OE-Core revision
c2cc5abe34169eae92067d97ce1e747e7c1413f5 - it turns out BitBake's
fetcher code is not consistent in whether it logs something useful or
not; when fetching from an http URL it does but with a git repository
it doesn't. In advance of any major reworking of fetch error handling in
BitBake, let's just print the text of the exception and then we know we
have shown something to the user.

Additionally, we were only catching FetchException here but there are
several other classes of exception that the fetcher can raise (e.g.
MalformedUrl); catch the parent BBFetchException class instead so we
avoid tracebacks for those other classes as well.

(From OE-Core rev: 578d3873a6415c9203c185c21cff472f7d2dab02)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-24 09:40:33 +00:00
Paul Eggleton 10c8d14d1a recipetool: create: fix extraction of name from URLs ending in /
If the URL ends in a / then we want to strip that off the path we split
out of the URL before calling os.path.basename() on it.

(From OE-Core rev: 308189beda8a31541481d09e3d5e86187e843d8d)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-24 09:40:33 +00:00
Paul Eggleton b307e0a604 recipetool: create: extract SRC_URI from local git repositories
If you specify a local directory which happens to be a git repository
with an origin remote (and it is in fact remote), we can use that for
SRC_URI rather than leaving it blank in the recipe.

(From OE-Core rev: b143d414846854dc8b3e0a47358daf5646eded38)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-24 09:40:33 +00:00
Paul Eggleton 50e40fc91f devtool / recipetool: support specifying a subdirectory within the fetched source
Sometimes you don't want to build an entire project, just a subdirectory
of it; add a --src-subdir option to make that easier. (We still look for
a single subdirectory in what gets unpacked, e.g. what you might find
within a tarball, so whatever you specify with this option is added onto
the end of that.)

(From OE-Core rev: 59682d78f95732e014f78f13e0a05f843860d9bb)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-24 09:40:33 +00:00
Paul Eggleton 7e1691de0a recipetool: create: strip quotes from values extracted from CMakeLists.txt
Quoting is optional in CMakeLists.txt and is occasionally used, so strip
out quotes if they are present.

(From OE-Core rev: 4ffe2e1ec9df05b92a2ad5746fb0ca6d218fd77e)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-24 09:40:33 +00:00
Paul Eggleton 477fa84390 gen-lockedsig-cache: copy correct native sstate into ext SDK
When constructing the sstate-cache directory for the extensible SDK,
we were copying in any matching native sstate packages, and as the
signature doesn't actually change when the distro changes (since
NATIVELSBSTRING is just a path separator for the artifacts and is not
part of the signature) we ended up copying duplicated packages when the
distro changed e.g. upon host distro upgrade. Only search in the
NATIVELSBSTRING-named subdirectory for native packages and the issue
goes away.

Fixes [YOCTO #8885].

(From OE-Core rev: 6c6baf6aa1823b8b20123f505e45c2768a193ad5)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-24 09:40:33 +00:00
Paul Eggleton b435225e8a devtool: add sdk-install subcommand
Add the ability to install additional pre-built items (from shared
state) into the extensible SDK. This can already be done implicitly by
adding something to DEPENDS within a recipe you're working on and then
running "devtool build", but it's useful to be able to explicitly
install things particularly if you're using the extensible SDK as a
traditional toolchain.

Note that for this command to be useful you need to have SSTATE_MIRRORS
set in your SDK configuration, and that mirror needs to be populated
with sstate artifacts for recipes you wish to be able to install.

(From OE-Core rev: 3474a42954908d1688fd3a6cb600eed315b27833)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-24 09:40:32 +00:00
Paul Eggleton 44d1a2a45c devtool: sdk-update: improve SDK update process robustness
Make the following improvements to the SDK update process:

* Use a manifest file with sha256sums to track files other than sstate
  and metadata that we need to update - e.g. conf files. This allows us
  to handle where files such as auto.conf may or may not be present,
  as well as the configuration changing without affecting task signatures
  - we still want the config files copied in that case rather than it
  saying nothing needs to be done.
* Write the SSTATE_MIRRORS_append to site.conf rather than local.conf
  so that local.conf remains static (since we don't want to trigger an
  update every time). Also, If there is an SSTATE_MIRRORS value already
  set in the configuration we can skip this and assume it contains the
  needed packages.
* Allow the update process to be run in any directory, don't assume
  we're already at the base of the SDK
* Where practical, fetch remote files into a temporary location and
  then move them to the desired location at the end, to avoid a
  failed update leaving the SDK in a broken state.
* Update all installed do_populate_sysroot / do_packagedata tasks
  instead of using the SDK targets. This ensures any item installed
  through dependencies after installation (e.g. when running
  "devtool build") won't go stale.

(From OE-Core rev: 3d35631121f0e030bc8151f5c23d84008d06f44b)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-24 09:40:32 +00:00
Paul Eggleton 3360baa96b devtool: sdk-update: improve temp directory handling
* Use tempfile.mkdtemp() instead of hardcoding temp dir
* Set a variable early for the temp locked sigs file and use that
  everywhere
* Delete the temp dir at the end

(From OE-Core rev: bad5d1a8c047a8118d30d9fa708b021d1599e0dc)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-24 09:40:32 +00:00
Paul Eggleton d193531d5e devtool: build: ensure pkgdata is written out
When you run devtool build, you need to have the pkgdata written out at
the end, so that if what you're adding is a library and the next thing
you add is something that depends on that library, the necessary
information to map the dependency back to the recipe is present. In
practical terms all this means is we need do_packagedata to run in
addition to do_populate_sysroot.

This does mean that do_package needs to run which wasn't running before,
and that means that the few package QA tests that run within do_package
such as installed-vs-shipped will now be run. This may be a bit
bothersome, and prompted a fix for one of our oe-selftest tests as a
result, but I don't see an easy way around it. Ultimately if you care
about using the recipe in an image you'll need to fix any such errors
anyway.

Fixes [YOCTO #8887].

(From OE-Core rev: 6579c7120ee5a541427ff5b6b07f838d52f9fe7c)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-24 09:40:32 +00:00
Paul Eggleton d3a4f72896 classes/populate_sdk_ext: add option to bring in pkgdata for world
Add a variable SDK_INCLUDE_PKGDATA which you can set to "1" to include
pkgdata for all recipes in the world target. There are a couple of uses
for this:

1) If you use "devtool add" to add a recipe that builds something which
   depends on anything in world, the dependency can then be correctly
   mapped to the recipe providing it and that recipe can be added to
   DEPENDS, since we have the pkg-config and shared library dependency
   data within pkgdata.
2) You'll be able to search for these recipes and any files they
   package for the target with "devtool search" since that also uses
   pkgdata

This of course assumes you've tailored world through EXCLUDE_FROM_WORLD
to only include recipes you'd want built in your distro, but I think
that's a reasonable assumption; failing that there is a
WORLD_PKGDATA_EXCLUDE variable that you can set to exclude any recipes
you don't want.

Note that this patch relies on functionality implemented in a recent
BitBake patch and will not work without it.

Implements [YOCTO #8600].

(From OE-Core rev: 67149ea097d6fab7496b43e85a40853f40bd527e)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-24 09:40:32 +00:00
Alejandro Hernandez 2a608cce9f python-setuptools: Upgrade to 19.2
Upgrades python-setuptools to 19.2, easy_install works out of the box
adds the package python-plistlib to the manifest as it is needed by
setuptools now, and also updates runtime dependencies

(From OE-Core rev: 25efefac9f68d34bbb109645a515010b846c3a8b)

Signed-off-by: Alejandro Hernandez <alejandro.hernandez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-20 17:07:14 +00:00
Ed Bartosh 35855a02f8 wic: pylinted ksparser module
Added missing docstrings, fixed wrong indentation and long lines.
Final pylint score is 9.89/10

(From OE-Core rev: 6e5dd42727b40c6b5ba6235026a6cfc78f482ac9)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-20 17:07:14 +00:00
Ed Bartosh e3b3bcf07a wic: add help for 'include' command
Added description of 'include' parser command to the
'wic help kickstart' output.

(From OE-Core rev: 7481f39382e63ecbb5de406559cc28e5689bd974)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-20 17:07:14 +00:00
Ed Bartosh bfaabe5209 wic: move parts of canned .wks into common.wks.inc
In order to give and example of 'include' feature of ks parser
and for testing purposes common parts of 3 canned wks files were
moved into common.wks.inc

(From OE-Core rev: 629c6381669bd4acdb1613229cd095881d2d9cd2)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-20 17:07:13 +00:00
Ed Bartosh 50a3dc5b2d wic: implement search of includes
Used custom argument type to implement search of include
.wks files in canned wks paths. Include files can be
specified either by full path or by name.

[YOCTO #8848]

(From OE-Core rev: 3695962ba4b685f304f1039978cec60d1b1712e3)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-20 17:07:13 +00:00
Ed Bartosh 15ea180414 wic: refactor get_boot_config
This function is going to be used by ks parser to find include .wks
files. get_boot_config name is a bit confusing as function is quite
generic. It looks if file is present in the canned wks directories.

Renamed get_boot_config -> get_canned.
Renamed parameter file_boot -> file_name.
Updated description.

(From OE-Core rev: 8ea9a4c0422c9600cd33ec6e815ebcf2d0aad364)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-20 17:07:13 +00:00
Ed Bartosh d304162269 wic: ksparser: add support for include
Extended parser to support inclusion of .ks files:
    recursively called self._parse to parse included .ks

(From OE-Core rev: 33dd323ec6a1a1ed4e1a04e51de182c89c7b6bd9)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-20 17:07:13 +00:00
Ed Bartosh 3fc6aaa186 wic: do not remove build dir in source plugins
Interesting bug was found during implementation of 'include'
parser command.

Build directory was removed in do_configure_partition method of
bootimg- source plugins. This can cause removal of previously
prepared partition images if /boot partition is mentioned after
other partitions in .ks file.

Moved work directory removal to direct.py before processing
partitions.

(From OE-Core rev: ba98262573cf1600e0d477317f51d488b5f8c4bd)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-20 17:07:13 +00:00
Ed Bartosh 8d34eea303 wic: use unique partition number
This is a preparation for 'include' support.

Used unique counter instead of line number for partitions
in .ks file. Line numbers can be equal for different .ks files,
which can cause problems if one .ks file is included into
another.

(From OE-Core rev: cc2233b51f1d22d4e540f4a3e9ceedd7ede9ffa9)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-20 17:07:13 +00:00
Ed Bartosh 43b4058508 wic: move wks parsing code to KickStart._parse
This is a preparation for implementation of include statement.
Parser will be called recursively to parse included .wks files,
so it should be available as a method.

(From OE-Core rev: 7778b9851758f4f782cb5f5d5fb36e68aed3b275)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-20 17:07:13 +00:00
Ross Burton a9f2e87bd5 postinst-intercepts: always use set -e
If a postinst has a problem (say, qemu crashes) and set -e isn't in operation,
the only mention of the problem is a single line in the rootfs log that doesn't
trigger any warnings.

(From OE-Core rev: 072800f89a136bb5da44627f25599d3060cca0a1)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-19 17:37:33 +00:00
Paul Eggleton c1492c4e4f recipetool: create: add a couple more license checksums
I found these when I was looking at libftdi and they seem to be
generic enough to show up in at least a couple of other packages so I
figure I'll add them.

(From OE-Core rev: 9fa3ff44e05930d4dfa153db777077e747ecbf45)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-19 17:24:53 +00:00
Paul Eggleton 2c8c9fe3b4 recipetool: create: add basic support for extracting dependencies from cmake
Add support for extracting dependencies from CMakeLists.txt. There's
still a bunch of things missing that are outside the scope of OE-Core
and we still lack a proper extension mechanism, but this is a good
start.

This also adds an oe-selftest test to exercise the new code a bit.

Implements [YOCTO #7635].

(From OE-Core rev: 77e73e6930381fdbd6e78d3913d6467572e16568)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-19 17:24:53 +00:00
Paul Eggleton 3eb397fab6 recipetool: create: force GL libraries to virtual/*
We want to specify dependencies on virtual/* rather than whatever
library is selected in the current configuration.

(From OE-Core rev: e1ac0c45b27ded9962edaf34597f827d0b41ba82)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-19 17:24:52 +00:00
Paul Eggleton 726dbda396 recipetool: create: move dependency mapping code to RecipeHandler
Some refactoring to allow access to the library/header/pkg-config
mappings and the DEPENDS / unmapped dependency output code from other
classes than AutotoolsRecipeHandler.

(From OE-Core rev: 40c10d998b90dd59c6d36c28f8ba11ec598bfa0f)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-19 17:24:51 +00:00
Paul Eggleton 788e4bb23e recipetool: create: fix overzealous mapping of git URLs
The regex for detecting git URLs was unanchored, leading to it matching
where it shouldn't have. An example of where this went wrong was
http://taglib.github.io/releases/taglib-1.9.1.tar.gz.

(From OE-Core rev: bacff751c88b680fbfb07843b18c59c8bc80a9ea)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-19 17:24:51 +00:00
Paul Eggleton ece0a2e1a0 recipetool: create: support additional autoconf macros from autoconf-archive
Support a number of macros from autoconf-archive when reading
configure.ac to extract dependencies.

(From OE-Core rev: ee977a62c58ded361c2abd78654bd25637fe9ea1)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-19 17:24:50 +00:00
Paul Eggleton 903d471355 recipetool: create: detect flex/bison dependency
There are a few different macros that can be used to pick up these
tools, add support for them all.

(From OE-Core rev: 7dfff4b7f05653aea230294ff1a7c023730deff9)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-19 17:24:50 +00:00
Paul Eggleton a66f4ac086 recipetool: create: pick up boost macros in configure.ac
The presence of BOOST_REQUIRE or AX_BOOST.* indicates that boost is a
dependency.

(From OE-Core rev: 02570b1fc31c7f4e9643aea8365806089622c0e7)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-19 17:24:49 +00:00
Paul Eggleton dbe91a3d6a recipetool: create: improve extraction of pkg-config / lib deps
* The regexes for PKG_CHECK_MODULES / AC_CHECK_LIB were a bit too strict
  and thus we were skipping some macros.
* Add support for PKG_CHECK_EXISTS
* Avoid duplicates in warning on missing pkg-config dependencies
* Ignore dependency on musl (since this may come up if it's the selected
  C library)

(From OE-Core rev: c58669fb0977f7f0cb79f252484d5c5ef0dfb7e4)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-19 17:24:48 +00:00
Ed Bartosh e7bedb91a7 wic: rename kickstarter.py -> ksparser.py
kickstarter.py was not the best name for this module as previously
there was a directory with the same name in scripts/lib/wic/.
All files were removed from it, but .pyc files could still stay there
causing imports from wic.kickstart to fail with
ImportError: cannot import name KickStart.

(From OE-Core rev: b9d400be06bc4a4bb9f9c6a6a0c8e5ecfd4e2dfb)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-19 17:24:48 +00:00
Ed Bartosh 3bb6ea63fc wic: override ArgumentParser.error
Overriden error method to throw exception instead of
printing usage error message. Exception is caught by
KickStart code to add .ks file name and line number.

(From OE-Core rev: 373016ba08c2ec4dbcd44649d9c8cd57d5574402)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-19 17:24:47 +00:00
Ed Bartosh d652203b2f wic: removed unused imports
Removed imports of wic.kickstart from plugins as they're
not used in the code.

(From OE-Core rev: 33d8784470c506fabcf9627e754628cdea61dd07)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-19 17:24:47 +00:00
Ed Bartosh d2090a6ba1 wic: improve processing of parseing errors
Caught argparse.ArgumentError
Included .ks file name and line number into the error messages.

(From OE-Core rev: 549c76ebda9afba0771d6d2c9b0b83f7a479c626)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-19 17:24:47 +00:00
Ed Bartosh 1ed97cc886 wic: catch KickStartError
Catch parsing errors and output them using msger.

(From OE-Core rev: 9c058f115583592f5cce2a969882fdd0c2ab535f)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-19 17:24:47 +00:00
Ed Bartosh bda77fd506 wic: add custom exception KickStartError
This exception will be raised by kickstart parser
on parsing errors and processed in the code which
calls parser to produce meaningful error output.

(From OE-Core rev: 13092793693c1c0ea172701578506f4a70a093d2)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-19 17:24:47 +00:00
Joshua Lock 4339a822e3 wic/help.py: document requirements for valid fstab generation
Without one of the --ondrive, --ondisk or --use-uuid options for a
partition with a mountpoint specified the automatically generated
fstab entry will be invalid.

[YOCTO #8844]

(From OE-Core rev: a524ced19db05e776834cd1f1db03c68a05f9c0b)

Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-18 11:47:08 +00:00
Ed Bartosh af8b00556e wic: get rid of 2 getters
Used partitions and configfile bootloader attributes instead of
using getters get_bootloader_file and get_partitions.

(From OE-Core rev: d34dd190e1da00ca8c9b2cb0adba15b9f0de2920)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-18 11:47:06 +00:00
Ed Bartosh 2573e285d0 wic: get rid of set_size and set_source_file setters
Used size and source_file attributes instead of using
setters. It's more pythonic, clear an consistent.

(From OE-Core rev: 7cf236047fa5d0d947fd214181c9a63ccb2509ac)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-18 11:47:06 +00:00
Ed Bartosh 5cd222b19f wic: get rid of get_rootfs and set_rootfs
Got rid of get_rootfs and set_rootfs java-like getter and
setter. Renamed rootfs to rootfs_dir to be consistent with
the name of kickstart parameter --rootfs-dir.

(From OE-Core rev: 51ec52a62ce49d1a1a83489379990f78cfe849f9)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-18 11:47:06 +00:00
Ed Bartosh 4d5d5dd428 wic: get rid of get_timeout getter
Used bootloader.timeout instead of kickstart.get_timeout getter.

Accessing attributes instead of getting them with getters is
more pythonic, shorter and readable. It also more consistent as
most of partition and bootloader attributes are used this way.

This change also takes care of appendLine bootloader attribute:
it's renamed to bootloader.append attribute provided by new parser.

(From OE-Core rev: 8088caeff5bf4ee9279b47a69c9f1e5537909601)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-18 11:47:06 +00:00
Ed Bartosh 26fb2a1a45 wic: adjust code for new data structure
New data structure is less nested than old one.
Adjusted bootloader and partitions fields:
self.ks.handler.bootloader -> self.ks.bootoader
self.ks.handler.partitions -> self.ks.partitions

(From OE-Core rev: b46da0731e3f7016d0ab2a67967f07c7e199f219)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-18 11:47:05 +00:00
Ed Bartosh c8272380ce wic: remove pykickstart code
Removed pykickstart-related code as it's replaced by
new kickstart parser.

(From OE-Core rev: 30bb1f3b6b832f9be691350581458c5fdaaaad70)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-18 11:47:05 +00:00
Ed Bartosh c15ea825e7 wic: use new kickstart parser
Used KickStart parser class instead of pykickstart API.

This commit breaks wic as data structures and field names
provided by new API are a bit different from old ones.
This issue will be addressed in the following commits.

(From OE-Core rev: f72439d321220b724bfe2b4a92fed46c2c19fb3f)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-18 11:47:05 +00:00
Ed Bartosh f572f44214 wic: add kickstart parser module
This module will replace existing pykickstart machinery
it contains only option used by wic, it's simple and
clear. And It will allow to remove a lot of old complex
code from 3rdparty/pykickstart/ and kickstart/custom_commands.

(From OE-Core rev: c7b67ccfda8b22c090aa74d96b7c9af5a97a9a98)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-18 11:47:05 +00:00
Ed Bartosh e5e1905c36 wic: add partition module
Copied partition-related API from kickstart/custom_commands/partition.py
to separate module in preparation for removal of all
pykickstart related code.

(From OE-Core rev: a915c4cbc33ef0a12546ac0000ecf8aedb6241d5)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-18 11:47:05 +00:00
Ross Burton 34f8db9b44 update_font_cache: only scan system font directories
By default fc-cache will scan both system and user directories, which means it
attempts to scan $HOME/.fonts.  As this is the build host's idea of $HOME this
generally doesn't exist, and causes fc-cache to exit with a failure.

Solve this by passing --system-only so that fc-cache will only scan system
directories, as is appropriate for a rootfs-time invocation.

(From OE-Core rev: 541315d6c56df6448f64c262f99d43d5c1e9400b)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-11 23:26:32 +00:00
Martin Jansa f04fb8806c scripts/create-pull-request: fix git request-pull syntax
* at least with git 2.6.3 I see git request-pull failing
  when there is only :{BRANCH} as ending commit

* $ git request-pull origin/dizzy git://git.openembedded.org/openembedded-core-contrib jansa/dizzy-backports:jansa/dizzy-backports
  The following changes since commit 7bb182bdd130266100fc541fd09b82d09c51cd80:
    build-appliance-image: Update to dizzy head revision (2015-09-29 14:56:04 +0100)
  ...
  And finds correct 7 changes there

* $ git request-pull origin/dizzy git://git.openembedded.org/openembedded-core-contrib :jansa/dizzy-backports
  warn: No match for commit 6068d1c90336ddc1fb32856efd1d9ccf07733896 found at git://git.openembedded.org/openembedded-core-contrib
  warn: Are you sure you pushed 'jansa/dizzy-backports' there?
  The following changes since commit 97756472d3a69eaca95d105494ffea78c6b077e0:
    build-appliance-image: Update to dizzy head revision (2014-10-18 16:16:27 +0200)
  ...
  and lists all commits in _current_ branch since origin/dizzy, then it refuses
  to continue, because there are too many changes.

* 6068d1c90336ddc1fb32856efd1d9ccf07733896 is this commit in jansa/master-submitted
  branch so it really shouldn't be included in pull request from jansa/dizzy branch.
* git help says:
  <end>
    Commit to end at (defaults to HEAD). This names the commit at the tip of the history you are asking to be pulled.

    When the repository named by <url> has the commit at a tip of a ref that is different from the ref you have locally, you can use
    the <local>:<remote> syntax, to have its local name, a colon :, and its remote name.
* maybe the syntax got changes since git 2.1.0 when Saul added :${BRANCH}
* I haven't found how to respect ${COMMIT_ID in the new syntax

(From OE-Core rev: 2336d1e5de671f538f0cd493b75d29e1dfdb0caf)

Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-11 23:26:32 +00:00
Richard Purdie 0269219fbe populate_sdk_ext: Use new --setscene-only option to bitbake instead of workarounds
Rather than horrible workarounds, use the new --setscene-only option
of bitbake to pre-populate the SDK structures.

(From OE-Core rev: 38237b7ac53c416f85c4a70a61acafc3404c8b5f)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-11 23:26:28 +00:00
Paul Eggleton b2fe2a8fdc devtool: build: support using BBCLASSEXTENDed names
It's logical that you would want to build BBCLASSEXTENDed items
separately through devtool build, so simply allow that - we're just
passing the name verbatim to bitbake, so all it means is adjusting the
validation.

(From OE-Core rev: 25dc5ac42c9da53c01416e7fdcc819d729281133)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-11 15:41:46 +00:00
Paul Eggleton 38ed039753 devtool: reset: support recipes with BBCLASSEXTEND
If the recipe file itself was created in the workspace, and it uses
BBCLASSEXTEND (e.g. through devtool add --also-native), then we need to
clean the other variants as well.

(From OE-Core rev: e1bf6a30679a83d4dbcf37276204f639541e95f9)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-11 15:41:46 +00:00
Paul Eggleton 532f42985f devtool: refactor code for getting local recipe file
We're doing this in a couple of places, let's just find the recipe file
if it exists within the workspace (which it will if it's been added
through "devtool add") when we read in the workspace.

(From OE-Core rev: 81cf5580993c99050e3f4d6d891bc67534721487)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-11 15:41:46 +00:00
Paul Eggleton ec9016821d devtool: add: support adding a native variant
Sometimes you need to build a variant of a recipe for the build
host as well as for the target (i.e. BBCLASSEXTEND = "native"); add a
 --also-native command line option to "recipetool create" that enables
this and plumb it through from an identical option for "devtool add".

(We could conceivably do the same for nativesdk, but I felt it might be
confusing within the context of the extensible SDK, where nativesdk
isn't really relevant to the user.)

(From OE-Core rev: f3bea83db173cce921a9a30f04e88b7e3ed98854)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-11 15:41:46 +00:00
Paul Eggleton 99e3872d89 devtool: reset: do clean for multiple recipes at once with -a
We need to run the clean for all recipes that are being reset before we
start deleting things from the workspace; if we don't, recipes providing
dependencies may be missing when we come to clean a recipe later (since
we don't and couldn't practically reset them in dependency order). This
also improves performance since we have the startup startup time for the
clean just once rather than for every recipe.

(From OE-Core rev: c10a2de75a99410eb5338dd6da0e0b0e32bae6f5)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-11 15:41:46 +00:00
Paul Eggleton 5ef716c280 recipetool: create: support creating standalone native/nativesdk recipes
If the recipe name ends with -native then we should inherit native;
likewise if it starts with nativesdk- then inherit nativesdk.

(Note that the recipe name must actually be specified by the user in
order to trigger this - we won't do it based on any name auto-detected
from e.g. the tarball name.)

Since we're doing this based on the name, "devtool add" will also gain
this functionality automatically.

(From OE-Core rev: a216fb1f5953327790dce3d2c1a9af616c0b410d)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-11 15:41:46 +00:00
Paul Eggleton 1e503c0a51 recipetool: create: lower case name when determining from filename
As a matter of general convention we expect recipe names to be lower
case; in fact some of the packaging backends insist upon it. Since in
this part of the code we're auto-determining the name, we should convert
the name to lowercase if it's not already so that we're following
convention and avoiding any validation issues later on.

(From OE-Core rev: 64986decbc11afa0d1e11251f5f7adcba1860d19)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-11 15:41:46 +00:00
Paul Eggleton 4deed25330 devtool: sdk-update: add option to skip preparation step
For debugging purposes it's useful to be able to skip the preparation
step so you can inspect what the state of the build system is first.

(From OE-Core rev: 0bba4b5afd2ce2c3a79445eee886979a77f1a4d8)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-11 15:41:46 +00:00
Paul Eggleton d586a11ef7 devtool: sdk-update: fix error checking
Running "raise" with no arguments here is invalid, we're not in
exception handling context. Rather than also adding code to catch the
exception I just moved the check out to the parent function from which
we can just exit.

(From OE-Core rev: 0164dc66467739b357ab22bf9b8c0845f3eff4a4)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-11 15:41:46 +00:00
Paul Eggleton c1b7d83d77 devtool: sdk-update: fix metadata update step
* Clone the correct path - we need .git on the end
* Pull from the specified path instead of expecting a remote to be set
* up in the repo already (it isn't by default)

(From OE-Core rev: 1a60ee8bd21e156022c928f12bb296ab5caaa766)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-11 15:41:45 +00:00
Paul Eggleton efead10e6f devtool: sdk-update: fix not using updateserver config file option
We read the updateserver setting from the config file but we never
actually used that value - the code then went on to use only the value
supplied on the command line.

Fix courtesy of Dmitry Rozhkov <dmitry.rozhkov@intel.com>

(From OE-Core rev: 1c85237803038fba539d5b03bf4de39d99380684)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-11 15:41:45 +00:00
Paul Eggleton d11051c489 scripts/oe-publish-sdk: add missing call to git update-server-info
We need to call git update-server-info here on the created repository or
we can't share it over plain http as we need to be able to for the
update process to function as currently implemented.

(From OE-Core rev: 3ab40bf9d5f19d91e45f7bae77f037b2544e889b)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-11 15:41:45 +00:00
Humberto Ibarra 0f3ad7c77a scripts/yocto-layer: Avoids duplication of "meta-" prefix
The yocto-layer script puts and extra "meta-" prefix on the given layer
name even when the prefix is already there. This fix avoids duplicating
the prefix in these situations.

The change was done inside the create subcommand since this is a parsing
specific to the layer creation. Parsing this in the main method of
yocto-layer was not the right way to go.

Before the change:

$ yocto-layer create meta-layer
[...]
New layer created in meta-meta-layer.

After the change:

$ yocto-layer create meta-layer
[...]
New layer created in meta-layer.

[YOCTO #8050]

(From meta-yocto rev: e21c79eb830ed1593e81f2d58815664109a10933)

Signed-off-by: Humberto Ibarra <humberto.ibarra.lopez@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-07 13:52:21 +00:00
Bruce Ashfield b82e228d39 yocto-bsp: remove 3.14 and 3.19 bbappends
The 3.14 and 3.19 kernel have been removed from oe-core, so we drop
our bbappends.

(From meta-yocto rev: e82a9e75b2d02387fc58232ee469ed0ae661b996)

Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-07 13:52:21 +00:00
Trevor Woerner b68d947e7c mkefidisk.sh: add boot log on console
Hooking up a serial console is a "developer mode", the chances are pretty good
developers are interested in watching the kernel boot log on the console so
they can spot any problems or diagnose any failed boots (e.g. can't find root
fs).

(From OE-Core rev: d03e399623b9320268a2b56e4928bb7effa9146c)

Signed-off-by: Trevor Woerner <twoerner@gmail.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-07 13:40:17 +00:00
Trevor Woerner 62d7c97912 mkefidisk.sh: add startup script for automated boot
Regardless of which image is built using which layers, try to ensure the image
on the SD device being prepared is the one that is booted automatically when
the board is powered.

(From OE-Core rev: 51dc79d55d0985e64838a08d39a22e8cd3fcd59a)

Signed-off-by: Trevor Woerner <twoerner@gmail.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-07 13:40:17 +00:00
Martin Jansa b900ec8c6c sstate-sysroot-cruft.sh: Extend the whitelist
* add more php5 entries
* add builder user (from builder.bb)
* add .bashrc and .profile in HOME directory of created user
* add imgdata directory generated by image.py

(From OE-Core rev: 49c744321ff1dfb16491d300ff46b6b5d5aa3f14)

Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-07 13:40:17 +00:00
Daniel Istrate c2bda6c000 scripts/oe-selftest: Allow to run tests on random/all MACHINEs
Add an option for random MACHINE into oe-selftest:
--machine [random/all]
1. random: will set a random MACHINE for each test
2. all: will run tests for all machines

Custom machine sets only weak default values (??=) for MACHINE in machine.inc.
This let test cases that require a specific MACHINE to be able to
override it, using (?= or =).

e.g.:
oe-selftest --run-tests signing --machine random -->
will run all tests switching MACHINE randomly for each test

oe-selftest --run-tests signing --machine all -->
for each machine will run all tests

oe-selftest --run-all-tests --machine random

Also update oeqa/selftest/base.py to accomodate this feature.

Fix for [YOCTO #5880].

(From OE-Core rev: 4a9c3653eecd9a3c1b45bab5e16c8d679c55f8bd)

Signed-off-by: Daniel Istrate <daniel.alexandrux.istrate@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-07 13:40:17 +00:00
Daniel Istrate 32f332c9d4 oe-selftest: New option --list-tests
This option will list all available tests in a comprehensive manner.

Fix for [YOCTO #8868]

(From OE-Core rev: 8b9ea3ee39bceef7ee6faeef64f33bd9dbad2e4b)

Signed-off-by: Daniel Istrate <daniel.alexandrux.istrate@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-07 13:40:16 +00:00
Daniel Istrate 17d886bef2 oe-selftest: Improved --list-classes when determining test names
--list-classes does a weak validation when determining test names:
(if method.startswith("test_") which could report any class attribute
that starts with 'test_' as a valid test case.

This fix checks that the class attribute that starts with 'test_'
is also callable (is a method).

Fix for [YOCTO #8862]

(From OE-Core rev: 175810503d5596370cf7d840539ebdf35cf30278)

Signed-off-by: Daniel Istrate <daniel.alexandrux.istrate@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-07 13:40:16 +00:00
Humberto Ibarra 02d259c1f4 scripts/oe-selftest: Remove extra coverage data added to unittests
Coverage data tracking initiates too early, causing coverage data from the
oe-selftest environment setting to be added to each run. Even when no tests are run
oe-selftest reports around 24% of coverage due to this extra data.

Change the custom resultclass used by the TextTestRunner to one generated from the
command arguments. The generated class processes coverage when needed, running
coverage setup just before the first testcase is run and reporting after the last
one finished.

[Yocto #8846]

(From OE-Core rev: d66a4c5cb77f745e973daf34b84724f91549f391)

Signed-off-by: Humberto Ibarra <humberto.ibarra.lopez@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-07 13:40:16 +00:00
Noor Ahsan a3e26f9a7a wic: rawcopy: Copy source file to build folder
When a file is given using --sourceparams then wic directly use that file
instead of copying them to build folder. At time of assembling it os.rename
is called which renames all the files to name. In that process the original
file is renamed. When image recipe is rebuilt then wic complains about
missing file which was renamed in previous build.

[YOCTO #8854]

(From OE-Core rev: 33c52b1f2d39feb641465bf42e8b16d0ab22a316)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-07 13:40:16 +00:00
Ed Bartosh 23302ee03a devtool: use cp instead of shutil.copytree
Copied layers with 'cp -a' instead of calling shutil.copytree as
copytree fails to copy broken symlinks.

More pythonic fix would be to use copytree with 'ignore' parameter,
but this could slow down copying complex directory structures.

[YOCTO #8825]

(From OE-Core rev: e5b841420b9fdd33829f7665a62cd06a3017f7e6)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-28 09:25:19 +00:00
Costin Constantin b6d191d95f scripts/oe-selftest: Add support for selftest log with timestamp
Each time oe-selftest runs, the oe-selftest.log file is overwritten.
This patch solves it by adding time stamp to each selftest log file
and doing a symlink named as oe-selftest.log to the last one created.

(From OE-Core rev: da8097480ad70e7a75608d733c63c3ae5a337974)

Signed-off-by: Costin Constantin <costin.c.constantin@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-28 09:25:17 +00:00
Paul Eggleton 891673101e recipetool: create: fix error when extracting source to a specified directory
Having fetched the source and unpacked it to a temporary directory, we
then move part of it to the destination directory, or if the source is at
the top level we move the whole temporary directory, but in the latter
case we were later attempting to delete the temporary directory which no
longer existed. Clear out the variable so that doesn't happen.

(From OE-Core rev: 91714a52e91cddba5a16c73cf5765d1f47f7856c)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-28 09:25:14 +00:00
Paul Eggleton fe28c251b1 recipetool: create: improve autotools support
* tar and binutils we can assume are there
* libsocket is only relevant on BSD systems, so we can ignore it.
* Detect more things implying gettext/intltool is needed
* Detect glib-2.0 requirement.

(From OE-Core rev: 2c4c78a6a9970533f3352f1067b2263f45098493)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-28 09:25:14 +00:00
Paul Eggleton 498e4834a8 devtool: sync: tweak help / messages
Much of this was copy/pasted from the extract subcommand code; make it
specific to sync.

(From OE-Core rev: 08ca966dd48db881d8bfb7d7fa0aba49f1f7cffe)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-28 09:25:14 +00:00
Paul Eggleton b272c5174f devtool: reset: print message about leaving source tree behind
We deliberately leave the source tree alone when resetting in case it
contains any work in progress belonging to the user; tell them that
we're doing this so they aren't surprised about it still existing later
on.

(From OE-Core rev: 1c616802ba2cfb37f8403f1dc6cf91ce3bd99ac2)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-28 09:25:14 +00:00
Paul Eggleton 95a234e7b3 devtool: status: list recipe file within workspace if one exists
If a recipe in the workspace actually exists as a file within the
workspace (e.g. after doing "devtool add" or "devtool upgrade") then
show the path to the recipe file on the status line for the recipe.

(From OE-Core rev: 29833828a90c8433af3b231b50e99cd97edf19ff)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-28 09:25:14 +00:00
Paul Eggleton e11673960f devtool: modify: default source tree path
As per the changes to "devtool add", make the source tree path optional
and use the default path if none is specified.

(From OE-Core rev: 83707d1334fb094fd1877bcfd07a83866601048a)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-28 09:25:14 +00:00
Paul Eggleton 110f4337f2 devtool: add: allow specifying URL as positional argument
Having to specify -f is a little bit ugly when a URI is distinctive
enough to recognise amongst the other positional parameters, so take it
as an optional positional parameter. -f/--fetch is still supported, but
deprecated.

(From OE-Core rev: aedfc5a5db1c4b2b80a36147c9a13b31764d91dd)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-28 09:25:14 +00:00
Paul Eggleton ceaa4bfd09 devtool: add: figure out recipe name from recipetool
recipetool create now has all the logic in it for auto-detecting the
name and version, and using those in the file name - so we can make the
name an optional parameter for devtool add and we pick up the file name
that recipetool has used after the fact.

(From OE-Core rev: 70ab08146e930f1fc55fdf5726a87303e20bd60f)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-28 09:25:13 +00:00
Paul Eggleton ee0d5a1080 devtool: add: allow source tree to be omitted
Assuming we're fetching source remotely (from a URI) we can default the
source tree that will be extracted from it to a "sources" directory
under the workspace in order to save the user specifying it if they
don't have a preferred location.

(From OE-Core rev: ffdad964c7271972e4b067e4898bf7c338c25b68)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-28 09:25:13 +00:00
Paul Eggleton 0d8751fbab scripts/lib/argparse_oe: handle intermixing of optional positional arguments
Python's argparse module can't handle when several optional positional
arguments (set with nargs='?') are intermixed with other options. If the
positional arguments aren't optional then this isn't an issue; thus when
changing positional arguments to optional (as we are doing with devtool)
we need this workaround.

This is a pretty horrible hack, but we don't want this flexibility of
ordering to disappear simply because we made some arguments optional.
Unfortunately the corresponding bug remains unresolved upstream even in
Python 3, and argparse is not really designed to be subclassed so it
doesn't make things like this easy.

(From OE-Core rev: 98fd5de373e16fe5d69a3065f844efc8037385bc)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-28 09:25:13 +00:00
Paul Eggleton 1bd779347b devtool: update-recipe: use correct method to get bbappend filename
The bbappend already exists at this point, so we know what its path is -
there's no need to figure it out from scratch here.

(From OE-Core rev: c0754d672966901f22dff1bcd40bbd08d1219c7a)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-28 09:25:13 +00:00
Paul Eggleton 2074654423 devtool: split out function for naming bbappend
We're repeating this in a couple of places, so we might as well have a
function to do it.

(From OE-Core rev: 67a28109a1ee1383d1b17a8dafa4fe510948238b)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-28 09:25:13 +00:00
Paul Eggleton 6acbdc9cf1 devtool: add: tweak help text
Add a few clarifying words.

(From OE-Core rev: 2103fa9dc7faf2189c8b426b87fb9d421a9983ac)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-28 09:25:13 +00:00
Paul Eggleton 316b57b62e devtool: edit-recipe: add new subcommand
Add an "edit-recipe" subcommand that runs your default editor (as
specified by the EDITOR environment variable) on the specified recipe in
the workspace. Note that by default the recipe file itself must be in
the workspace - i.e. as a result of "devtool add" or "devtool upgrade";
however there is a -a/--any-recipe option to override this.

(From OE-Core rev: dbfe8fa2e86c2bb50bef47c389017cdf93543321)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-28 09:25:13 +00:00
Paul Eggleton ebe5f0b872 recipetool: create: basic extraction of name/version from filename
Often the filename (e.g. source tarball) contains the name and version
of the software it contains.

(This isn't intended to be exhaustive, just to catch the common case.)

(From OE-Core rev: 944eacfb849ee69b41e12c9de4f264406281ac6a)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-28 09:25:13 +00:00
Paul Eggleton db5f9645ad recipetool: create: support extracting name and version from build scripts
Some build systems (notably autotools) support declaring the name and
version of the program being built; since we need those for the recipe
we can attempt to extract them. It's a little fuzzy as they are often
omitted or may not be appropriately formatted for our purposes, but it
does work on a reasonable number of software packages to be useful.

(From OE-Core rev: 3b3fd33190d89c09e62126eea0e45aa84fe5442e)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-28 09:25:13 +00:00
Paul Eggleton 6a7661b800 recipetool: create: set up priority system for recipe handlers
Sometimes we want to force one handler to run before another; if the two
handlers are in different plugins that's difficult without some kind of
priority number, so add one and sort by it.

(From OE-Core rev: 0219d4fb9cefcee635387b46fc1d215f82753d92)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-28 09:25:13 +00:00
Paul Eggleton 38803e38d6 recipetool: create: detect when specified URL returns a web page
If the user specifies a URL that just returns a web page, then it's
probably incorrect (or broken); attempt to detect this and show an error
if it's the case.

(From OE-Core rev: 83b1245b2638eb5d314fe663d33cd52a776a34a7)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-28 09:25:13 +00:00
Paul Eggleton e78a03980a recipetool: create: prevent attempting to unpack entire DL_DIR
If you specify a URL ending in /, BitBake's fetcher returns a localpath
of ${DL_DIR}, and if you then try to unpack that it will attempt to copy
the entire DL_DIR contents to the destination - which at least on my
system filled my entire /tmp. Obviously we should fix the fetcher, but
at least detect and stop that from happening here for now.

(From OE-Core rev: 7e63a672517518644a37ce006e05b5494c29cf6e)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-28 09:25:13 +00:00
Paul Eggleton e61645b504 recipetool: create: minor fix for potential issue in python handling
If SRC_URI happened not to be in the pre-generated lines then this code
would error out. This is unlikely to happen with the way the create code
is structured at the moment, but handle it just in case.

(From OE-Core rev: 95d33e90f2d5d9dd5ccc950856b8a939fefb831e)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-28 09:25:12 +00:00
Paul Eggleton ae2141b84f recipetool: create: fix do_install handling for makefile-only software
In my testing here it appears make -qn returns an error (exit code 2)
whereas make -n doesn't; I can't immediately tell why based on the
documentation. We don't actually care for it to be quiet since we're
capturing the output, so let's just leave -q off and have this work
properly as a result.

(From OE-Core rev: 30c4cd9efdac400d713dff645f23f2627277d75a)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-28 09:25:12 +00:00
Paul Eggleton c2f17428b8 recipetool: create: avoid traceback on fetch error
If a fetch error occurs, the fetcher already prints a reasonable error -
we don't need the traceback as well, so catch that and exit if it
occurs.

(From OE-Core rev: c2cc5abe34169eae92067d97ce1e747e7c1413f5)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-28 09:25:12 +00:00
Paul Eggleton 470f20bc02 recipetool: create: handle https://....git URLs
When you grab a URL for a github repository you'll almost certainly find
it in https://github.com/path/to/repository.git format; but bitbake's
fetcher can't handle that because it'll see https:// at the start and
assume it should use wget to fetch it. If the URL starts with http:// or
https:// and the path part ends with .git then assume it's a git
repository and adjust it accordingly.

(From OE-Core rev: bdbc4cf41d30eddb8a9ed882dedcc1670ce8fdd6)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-28 09:25:12 +00:00
Paul Eggleton 8e0a84c901 scripts: print usage in argparse-using scripts when a command-line error occurs
For scripts that use Python's standard argparse module to parse
command-line arguments, create a subclass which will show the usage
the usage information when a command-line parsing error occurs. The most
common case would be when the script is run with no arguments; at least
then the user immediately gets to see what arguments they might need to
pass instead of just an error message.

(From OE-Core rev: d62fe7c9bc2df6a4464440a3cae0539074bf99aa)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-28 09:25:12 +00:00
Paul Eggleton 2381f4a394 devtool: sdk-update: fix traceback without update server set
If the SDK update server hasn't been set in the config (when building
the extensible SDK this would be set via SDK_UPDATE_URL) and it wasn't
specified on the command line then we were failing with a traceback
because we didn't pass the default value properly - None is interpreted
as no default, meaning raise an exception if no such option exists.

Additionally we don't need the try...except anymore either because with
a proper default value, NoSectionError is caught as well.

(From OE-Core rev: 9763c1b83362f8445ed6dff2804dd7d282861f79)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-28 09:25:12 +00:00
Joshua Lock ca69643faa wic/help.py: document that mountpoint is optional for part command
If no mountpoint is specified for a partition command the partition
will be created but not mounted — mention this in the kickstart
help text.

[YOCTO #8820]

(From OE-Core rev: d1ff1fef987457eb1a5ffe42dbabc7808fa7d598)

Signed-off-by: Joshua Lock <joshua.lock@collabora.co.uk>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-22 16:08:51 +00:00
Juro Bystricky 75cec07a03 oe-buildenv-internal: fix return code
The script oe-buildenv-internal is called from oe-init-build-env.
Make sure oe-init-buildenv does not return an error if BB_ENV_EXTRAWHITE is
already set, otherwise this will cause oe-init-build-env to fail.

(From OE-Core rev: 9ae79973cfdabd1b4dacddce32735c65fe3544e4)

Signed-off-by: Juro Bystricky <juro.bystricky@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-22 16:08:50 +00:00
Christopher Larson 647e0e44ce buildhistory-collect-srcrevs: hide empty sections
Cc: Paul Eggleton <paul.eggleton@linux.intel.com>
(From OE-Core rev: 3c4de5430aff2d7443f064d698014615e867c58c)

Signed-off-by: Christopher Larson <chris_larson@mentor.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-22 16:08:50 +00:00
Richard Purdie e6f27619af combo-layer: Stop using filterdiff
I ran into an issue where a patch just deleting a single file
within the repository (meta/recipes-devtools/m4/m4/make.patch)
would get skipped by combo-layer.

It turns out this has the patch header (commented to avoid breaking scripts):

: diff --git a/meta/recipes-devtools/m4/m4/make.patch b/meta/recipes-devtools/m4/m4/make.patch
: deleted file mode 100644
: index 79fb415..0000000
: --- a/meta/recipes-devtools/m4/m4/make.patch
: +++ /dev/null
: @@ -1,42 +0,0 @@

and this is classed as > 5 headers in filterdiff. When we piped the path
through filterdiff, the --- line disappears, then the second time we pass
through filterdiff, it shows no lines changed and the patch is assumed
to be empty and skipped.

Changing MAX_HEADERS in filterdiff is one way to fix this, another would
be to grep out "deleted file mode" lines. Instead, we can use new
git syntax to exclude files from the git format-patch instead and avoid
filterdiff entirely.

(From OE-Core rev: 296c70afeef75396dea9ae436058314d406dc257)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-18 12:18:18 +00:00
Martin Jansa 19c76ad209 sstate-sysroot-cruft: Add php, python, lua, fontcache generated files to whitelist
(From OE-Core rev: bae08f4caaddc9f5c8e3cd803538853dddb9f8bd)

Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-16 12:12:15 +00:00
Daniel Istrate 220a78b47e scripts: oe-selftest Added new features.
[YOCTO #8750] Allow oe-selftest to run custom test suites based on different criteria

1. Can run custom lists of tests based on different criteria:
   --run-tests-by <name|class|module|id|tag> <list of tests|classes|modules|ids|tags>
   eg: --run-tests-by module imagefeatures signing recipetool
       --run-tests-by id 1377 1273 935
       --run-tests-by tag wic sstate bitbake
2. Can list tests based on different criteria:
   --list-tests-by <name|class|module|id|tag> <list of tests|classes|modules|ids|tags>
   eg: --list-tests-by module imagefeatures signing recipetool
       --list-tests-by id 1377 1273 935
       --list-tests-by tag wic sstate bitbake
3. Can list all tags that have been set to test cases:
   --list-tags
   The list of tags should be kept as minimal as possible.
   This helps preview the tags used so far.

   To take advantage of the 'tag' feature:
   - add @tag(feature=<>) to testcases
   eg: @tag(feature='signing') for a single tag
       @tag(feature=(('signing', 'sstate')) or
       @tag(feature=['signing', 'sstate']) for multiple tags

(From OE-Core rev: 2d3a6d22e155911e39e4b7e323317f4a7cb1cb95)

Signed-off-by: Daniel Istrate <daniel.alexandrux.istrate@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-12 23:42:51 +00:00
Juro Bystricky 98d2485fee oe-buildenv-internal: preserve existing BB_ENV_EXTRAWHITE
According to the BitBake User Manual the environment variable BB_ENV_EXTRAWHITE
specifies an additional set of variables to allow through (whitelist) from the
external environment into BitBake's datastore. However, running:

  $ source oe-init-build-env build-name

will overwrite any pre-existing BB_ENV_EXTRAWHITE variables.
This patch modifies this behaviour: all oe-build-internal BB_ENV_EXTRAWHITE
variables are appended to any potentially already existing BB_ENV_EXTRAWHITE
variables. (The variables are only appended if not already in BB_ENV_EXTRAWHITE)

(From OE-Core rev: 826ae02c08b2210aa39ed51e1a9e50d40f33afb4)

Signed-off-by: Juro Bystricky <juro.bystricky@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-12 23:42:51 +00:00
Markus Lehtonen fd4894fcb4 devtool: extract: update SRCTREECOVEREDTASKS for kernel
Add 'do_kernel_configme' and 'do_kernel_configcheck' to
SRCTREECOVEREDTASKS of kernel packages. These tasks should not be run
because kernel meta in the srctree is not necessarily up-to-date or
even present which causes build failures and/or invalid kernel config.
Especially so because 'do_patch' which is a dependency of
'do_kernel_configme' is not being run.

We now store .config in the srctree and 'do_configure' task is able to
run successfully.

(From OE-Core rev: 7ce4c18a4ba1ebcb9f46e652a881ace1f21d2292)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-09 08:48:25 +00:00
Markus Lehtonen 34f1d819f4 devtool: extract: copy kernel config to srctree
This makes the correct kernel config to be used when building kernel
from srctree (extrernalsrc). If no kernel config is present in the
builddir 'do_configure' task copies .config from the srctree.

(From OE-Core rev: 3b516332e038a587685f6e0c14a7f04990bdd6cc)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-09 08:48:25 +00:00
Mariano Lopez 8612f26004 directdisk-bootloader-config.wks: Add example for custom bootloader config
Add new wks file as a example for a custom bootloader configuration.
This change also includes the configuration that file that will be
used.

This example is using syslinux with MBR, the configuration file is
almost the same as the one generated by wic. As stated before this
is just an example.

[YOCTO #8728]

(From OE-Core rev: 4a9db893f721c0da5d103d28b97a0302cc9e2197)

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-09 08:48:25 +00:00
Mariano Lopez c59dc3bae0 wic/help.py: Document the new option "configfile"
This just adds the "configfile" option for the bootloader
to wic help.

[YOCTO #8728]

(From OE-Core rev: 597045657a1a635f667404306160f9ab2551e954)

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-09 08:48:25 +00:00
Mariano Lopez 70338732ac wic: Allow to use a custom config for bootloaders
This change will allow to use a user defined file as the
configuration for the bootloaders (grub, gummiboot, syslinux).

The config file is defined in the wks file with the "configfile"
option in the bootloader line.

[YOCTO #8728]

(From OE-Core rev: d56546b0f312fd042b1a7df3bef97ac1c9b6a5b4)

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-09 08:48:24 +00:00
Mariano Lopez f95f729518 wic/utils/misc.py: Added function to search for files in canned-wks
This change add two new function to search for files in the
canned-wks folder for all the layers included in bblayers.conf.
This will be used to search for custom configuration files for
the bootloaders.

There are similar functions in the wic engine, but these are
focused in wks files only, so it was needed to create new ones.

[YOCTO #8728]

(From OE-Core rev: 356a942e75ac1743290d2c360d1bb89e2225b6cc)

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-09 08:48:24 +00:00
Mariano Lopez 9773faadfe wic: Prepare wicboot to allow custom bootloader config
Currently wic does the bootloader configuration file on the fly.
This change introduce a configfile variable for the bootloader;
this is to have a user defined configuration file for the
bootloaders (grub, syslinux, and gummiboot). This is particular
useful when having a multiboot system or scripts embedded in the
configuration file.

[YOCTO #8728]

(From OE-Core rev: 8347aee95ea271921c15ea8e580f0ff62325aa26)

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-09 08:48:24 +00:00
Ross Burton 70d459ceba scripts/oe-pkgdata-util: sort the packages in list-pkg-files
Sort the list of packages in list-pkg-files to make the output easier to read.

(From OE-Core rev: 6c31655c5abf6ad4308848c116444cc7b1e798bb)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-08 10:20:47 +00:00
Matt Madison 80e39195c7 wic: insert local Python paths at front
This follows how bitbake performs path insertion, and fixes a
failure to start wic on Ubuntu 15.10 with the distribution's
version of python-ply installed.

(From OE-Core rev: 59b1eefb801dfb1d4afe9640e3c8a070d4e1867f)

Signed-off-by: Matt Madison <matt@madison.systems>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-08 10:20:47 +00:00
Alejandro Hernandez 2268a702f1 python3: Upgrade from 3.4.3 to 3.5
python3-native_3.4.3.bb -> python3-native_3.5.0.bb:
- changed version
- changed cheksum
- no license change, just dates

python3_3.4.3.bb -> python3_3.5.0.bb:
- changed version
- changed cheksum
- no license change, just dates

New:
- use_packed_importlib.patch: Fixes importlib on cross-compile environments

Rebased:
- Manifest
- 000-cross-compile.patch
- 020-dont-compile-python-files.patch
- 04-default-is-optimized.patch
- python-3.3-multilib.patch
- distutils3-base.bbclass
- distutils3-native-base.bbclass
- python3native.bbclass

Upstream:
- makerace.patch

Misc:
- pip2 is handled as default on major distros,
modified python3-pip to leave /usr/bin/pip available for pip2
- Fixed importing pip3 from python3 interpreter

(From OE-Core rev: 701ec1977ced1bb08461e6de98b4f63d21cba8a6)

Signed-off-by: Alejandro Hernandez <alejandro.hernandez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-01 21:32:06 +00:00
Leonardo Sandoval cc34104561 oe-selftest: Enable code coverage on unit tests
Enable code coverage through the library 'python coverage'. In case the environment
variable COVERAGE_PROCESS_START is present (one of the requisites for measuring
sub-processes; the second one is including some coverage statements into the
python sitecustomize.py file) it will be taken into account, otherwise it is
exported with value '.coveragerc'. The latter value is a configuration file
(also automatically created) with some default settings. Once tests are
executed, a coverage report is shown on the log and the coverage output data is stored
with name '.coverage.<args>' where '<args>' is the name of the  unit tests executed
or 'all_tests' when running with --run-all-tests. This output data can be latter used
for better reporting using the same tool (coverage).

As briefly indicate before, measuring sub-process implies setting the env variable
COVERAGE_PROCESS_START (done automatically by the oe-selftest code with this patch if
not already set) and creating a sitecustomize.py as explained on [1].
If either one of these is missing, complete coverage will be incomplete.

Current measurements for 'oe-selftest --run-all-tests' indicate that current coverage
is around 42 % taking into account BBLAYERS, bitbake and scripts folders. More details
on [2], indicating the coverage per file/module.

This tasks has been done together with Humberto Ibarra <humberto.ibarra.lopez@linux.intel.com>

[YOCTO #8679]

[1] http://coverage.readthedocs.org/en/latest/subprocess.html
[2] https://bugzilla.yoctoproject.org/attachment.cgi?id=2854

(From OE-Core rev: b3feee2cefbbd98b66dc395b651f47c5028c80a0)

Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-01 21:32:05 +00:00
Khem Raj 6be94ec72c runqemu-internal: Replace wacom-tablet with tablet for usbdevice
When booting weston-core-image with latest wayland/weston/libinput
mouse/touchpad would not work on qemux86, this fixes the issue

(From OE-Core rev: 4abf18a25ccb1a062a3775be1a70eaf6c155349c)

Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Jussi Kukkonen <jussi.kukkonen@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-01 21:32:05 +00:00
Paul Eggleton 0cc3810299 recipetool: make plugin registration function name consistent with devtool
This should have been register_commands rather than register_command;
I used register_commands in devtool so lets change this here to be
consistent with that. (Since this is extensible through layers though we
need to remain compatible with the old name, so fall back to that if the
new function name isn't there.)

(From OE-Core rev: 1047f6592ac81643cd847f104da766dc4a4c81ea)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-01 21:32:05 +00:00
Paul Eggleton b381f804a5 recipetool: add setvar subcommand
Add a recipetool subcommand "setvar" to set a variable in a file. This
uses our existing logic such that it doesn't matter if the variable is
already set in the recipe, if it's set in the recipe or some inc file,
and if the variable is not currently set that the line setting the
variable gets inserted in the right place in the file.

Implements [YOCTO #7676].

(From OE-Core rev: 7c33ef77fa165182d24f0a9ae769e9e630e6bd47)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-01 21:32:05 +00:00
Paul Eggleton 1fbd76093d lib/oe/recipeutils: refactor patch_recipe_file() to use edit_metadata()
Use bb.utils.edit_metadata() to replace some of the logic in this
function; this avoids us effectively having two implementations of the
same thing. In the process fix the following issues:

* Insert values before any leading comments for the next variable
  instead of after them
* Insert overridden variables (e.g. RDEPENDS_${PN}) in the correct place
* Properly handle replacing varflag settings (e.g. SRC_URI[md5sum])

(From OE-Core rev: 0f81b83fc5fd908efa7f6b837137830ca65f6ed6)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-01 21:32:04 +00:00