Commit Graph

1686 Commits

Author SHA1 Message Date
Ed Bartosh 675bb9ddb8 selftest: wic: stop using iso image type
Removed 'IMAGE_FSTYPES = "iso" as this functionality depends on
do_bootimg, which is going to be obsoleted soon.

As wic doesn't depend on bootimg it's safe to remove this.

(From OE-Core rev: a08e2019223d7c04916967d8fd42279e8e1e186f)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-02-05 09:22:18 +00:00
Ed Bartosh 23237b46dc selftest: wic: use wic-tools recipe to get STAGING_DIR
STAGING_DIR variable is used to get path to a boot dir.
It's better to use wic-tools recipe to it as it contains
all bootloader artifacts.

Modified test_build_artifacts and test_rootfs_artifacts to
use wic-tools target to get STAGING_DIR.

(From OE-Core rev: eb648ac5566fbcc679e8544fd8403a730ce71058)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-02-05 09:22:18 +00:00
Richard Purdie 08858ff213 oeqa/buildproject: Run gnu-configize by default
The runtime oeqa tests were failing in the on-target case, only for
aarch64. This was because an old config.sub was being used.

Similar to the SDK testing, call gnu-configize in the on-target
case too to resolve the failing tests.

(From OE-Core rev: 7918fe08d8e48f46cf2402afa66d35063eb72545)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-02-05 09:22:17 +00:00
Richard Purdie 6840618f1a oeqa/runtime: Improve failure log output
Printing a message which says "configure failed" without the log output
is effectively useless. If a command fails, print the output by default
and simplify the calling code which makes debugging any of these failures
much easier.

(From OE-Core rev: b6352ff001c29f0bff10c18879b92c5618ec645c)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-02-05 09:22:17 +00:00
Jose Perez Carranza 383fc7df31 selftest/runtime-tests: Modify positins test to reduce execution time
Currently test_postinst_roofs_and_boot is building a full-cmdline image
this is taking a lot of time to execute the test, so a minimal image
will be build instead.

(From OE-Core rev: 3a6b0b9f94b7097a825edf3fe258a2f86b73edd6)

Signed-off-by: Jose Perez Carranza <jose.perez.carranza@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-02-02 17:37:45 +00:00
Leonardo Sandoval d3c6c10de4 selftest/bbtests: use write_config instead of local.conf file
Extra configuration data should be write using the write_config
method instead of manually appending to the local.conf file

(From OE-Core rev: 65c26d51eedc687c92e8ade262d24dc53ca71cec)

Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-02-02 17:37:43 +00:00
Ed Bartosh 3bc5724ec7 selftest: wic: stop using hddimg in FSTYPES
Removed hddimg from FSTYPES in wic test suite as
wic doesn't depend on hddimg anymore.

[YOCTO #10835]

(From OE-Core rev: a332d9618c2d0f46b6c0c8fdeee8bec4cc73f2cf)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-02-02 17:37:43 +00:00
Aníbal Limón bfead495de oeqa/core/context.py: Add validation for run-tests option
The run-tests option is optional so if isn't specified set
to None instead of crash on split().

(From OE-Core rev: 877732406cad2ee4bf11293dc8ccddc2998ffbaf)

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-31 23:27:47 +00:00
Aníbal Limón 90b4e075f4 oeqa/runtime/cases: Rename syslog module to oe_syslog
Debian based distros has a builtin syslog module so when
try to load tests using unittest it references the builtin
module instead of runtime/cases.

[YOCTO #10964]

(From OE-Core rev: 9923e3cdb58c2b3c54ec5fe99b2cec4cdc9fff92)

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-31 23:27:47 +00:00
Mariano Lopez 5582b67f6d runtime/cases: Fix case numbers, missing cases and unused classes
gcc: Removed unneded lines.

multilib: fixed case number.

syslog: added a missing test.

[YOCTO #10964]

(From OE-Core rev: 3f1c57191802309e58a7b3b42e8ec812e83bd8ed)

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-31 23:27:47 +00:00
Ming Liu 3c89b61533 meta: remove remaining True option to getVarFlag calls
This is a complementary fix to commit 2dea9e49:
[ meta: remove True option to getVarFlag calls ]

it intended to remove all True option to getVarFlag calls, but there are
still some remaining.

Search made with the following regex: getVarFlag ?\((.*), True\)

(From OE-Core rev: 5b53b34dcb64dc65ee651cbf78cbbd9105dc4f4c)

Signed-off-by: Ming Liu <peter.x.liu@external.atlascopco.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-31 14:43:02 +00:00
Ming Liu e2280b9e12 meta: remove remaining True option to getVar calls
This is a complementary fix to commit 7c552996:
[ meta: remove True option to getVar calls ]

it intended to remove all True option to getVar calls, but there are
still some remaining.

Search made with the following regex: getVar ?\((.*), True\)

(From OE-Core rev: 87d03ffe03d6f01e360bfd51714be96e62506e0a)

Signed-off-by: Ming Liu <peter.x.liu@external.atlascopco.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-31 14:43:02 +00:00
Mariano Lopez 1e88c8fce4 testimage.bbclass: Allow to run tests on autobuilder's images
With the change to the new framework data store dependecy was
removed, instead a new file is generated and used in testimage.
When testing builds from the autobuilders the test data values
are from the autobuilder, including the paths.

Some tests require paths to current environment in order to run,
this commit will update such paths and fix the error of running
images donwloaded from autobuilders.

[YOCTO #10964]

(From OE-Core rev: 26ad5105fc2ce03b7ee8ecc6911fd40a52bd573a)

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-31 14:43:01 +00:00
Mariano Lopez e8935e20d6 runtime/cases/smart.py: Check for IMAGE_PKGTYPE instead of PACKAGE_CLASSES
smart test requires to build the image using rpm packages, this check was
included, but it checked for PACKAGE_CLASSES=='package_rpm', and this is
not true when building packages for rpm and deb/ipk. So this would check
IMAGE_PKGTYPE instead.

[YOCTO #10964]

(From OE-Core rev: 570f50b30072a10c5eadfbbfea2647339ee85176)

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-31 14:43:01 +00:00
Mariano Lopez 45ef4ef2c4 runtime/cases/connman.py: Stop using oeRuntimeTest
oeRuntimeTest class is not used anymore as part of runtime
migration, this particular case was missed, so fix it.

[YOCTO #10964]

(From OE-Core rev: d928e5166d8812e7e20bc95c0a8b5cb5439ec497)

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-31 14:43:01 +00:00
Francisco Pedraza aca3afcb59 oeqa/sdk: Updates sanity tests for minimal eSDK
Updates sanity tests to cope with minimal eSDK installer
1. Skips the validation of sanity if packagegroup-cross-canadian is in
host package.
2. Skips if SDK does not include toolchain at cases/devtool.py

This should fix [YOCTO #10794]

(From OE-Core rev: 59a99cd8661af594ee0c33e086578d29d50a6268)

Signed-off-by: Francisco Pedraza <francisco.j.pedraza.gonzalez@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-31 14:43:01 +00:00
Ed Bartosh 3e833c681a selftest: wic: fix test_iso_image test case
Added "iso" to IMAGE_FSTYPES to build iso artifacts required
to fix test of isoimage-isohybrid wic plugin.

(From OE-Core rev: 055d6814f6de2e54ae27c5fa2e0c40e3e531590e)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-31 14:38:32 +00:00
Ed Bartosh 18c0af5214 selftest: stop using hddimg in the wic test suite
Removed hddimg from IMAGE_FEATURES as wic code doesn't
use hddimg anymore.

[YOCTO #10835]

(From OE-Core rev: 885a5beae86a55cc228fb2aa998a8b1135371eb4)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-31 14:38:32 +00:00
Tom Zanussi 66d08e0fda wic: Look for image artifacts in a common location
Rather than have each image type look for artifacts in image-specific
locations, move towards having them look for artifacts in a common
location, in this case DEPLOY_DIR_IMAGE

Use the existing deploy.bbclass to have the bootloaders put their binaries
in DEPLOY_DIR_IMAGE and then wic will find them and place them in the image

(From OE-Core rev: 1c8acea91e775b1b6f8b25e774aaba3e790fc59b)

Signed-off-by: Alejandro Hernandez <alejandro.hernandez@linux.intel.com>
Signed-off-by: Tom Zanussi <tom.zanussi@linux.intel.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-31 14:38:32 +00:00
Ed Bartosh 62310b3726 image-wic: remove HDDDIR from WICVARS
Removed HDDDIR as it's not used by wic anymore.
Stopped usage of HDDDIR in wic test suite.

(From OE-Core rev: b3ed3a79852f85651c8fa192a8118f9904e88ef8)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-31 14:38:32 +00:00
Ed Bartosh a508cb384c wic: use INITRD_LIVE in isoimage-isohybrid
INITRD variable is not set if hddimg is disabled.
isoimage-isohybrid can't get correct name for initrd if INITRD
variable is not set.

Added INITRD_LIVE to WICVARS and used it in isoimage-isohybrid
code to get initrd artifact name. Used INITRD if INITRD_LIVE is not set.

(From OE-Core rev: c672753be60550b15d5d593c251b61776d5b104d)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-31 14:38:32 +00:00
Ed Bartosh de23b7bb2f selftest: check results in test_image_vars_dir*
Tested that result images are produced in test_image_vars_dir_long
and test_image_vars_dir_short test cases.

(From OE-Core rev: 9ed399b99e40ec9ec76df1fbdedfb8104a463acc)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-31 14:38:32 +00:00
Ed Bartosh 8c51ca5cd6 selftest: split wic tests cases
Split tests to run wic only once per test case.
This should fix failures caused by result images left
from the previous wic run.

(From OE-Core rev: 71e3c631bac859d00cef05bbf7bd901a96bac520)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-31 14:38:32 +00:00
Ed Bartosh c01cc6ff24 wic: change location of .env files
Current location of .env files $STAGING_DIR/imagedata. It doesn't
depend on machine and be rewritten by the builds for different
machines.

Changed location to $STAGING_DIR/$MACHINE/imagedata to avoid .env
files to be rewritten.

(From OE-Core rev: 94245144f5cef344d90bc2a7b3267cdae9d192e4)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-31 14:38:32 +00:00
Ed Bartosh 3d585a518c selftest: wic: test default output directory
As tests now explicitly specify output directory we don't
need test_alternate_output_dir test case. However, we need
to test wic output to default output location.

Removed test_alternate_output_dir test case.
Added test_default_output_dir test case.

(From OE-Core rev: c1be1161cf555727aa56ee2109ee77f420f67c9f)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-31 14:38:32 +00:00
Ed Bartosh 3f21101e72 selftest: wic: explicitly specify output directory
wic started to use current directory as a default output dir.
Specified output directory in wic command line to make tests
more predictable and easier to maintain.

(From OE-Core rev: 0f205aed90edd90fcba0a565a6cadfe96339b410)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-31 14:38:32 +00:00
Ed Bartosh e330cf4a38 selftest: wic: split test_debug test case
Splitted to test_debug_short and test_debug_long to make each
of the test cases to run wic once. This is consistent with the
rest of the test cases and ensures that test cases are set up
properly.

This also fixes the following test failure caused by the image
left from the first wic run:

FAIL: test_debug (oeqa.selftest.wic.Wic)
Test debug
----------------------------------------------------------------------
Traceback (most recent call last):
  File "meta/lib/oeqa/utils/decorators.py", line 109, in wrapped_f
    return func(*args, **kwargs)
  File "meta/lib/oeqa/selftest/wic.py", line 270, in test_debug
    self.assertEqual(1, len(glob(self.resultdir + "directdisk-*.direct")))
AssertionError: 1 != 2

(From OE-Core rev: a453009821d7d7cb1dd8a7ae05c0d32227b4753b)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-28 23:32:16 +00:00
Leonardo Sandoval 70a30d9770 selftest: sstatetests: skip glibc-initial tests in case of non-glibc distro
Some distros uses non-glibc so skip those tests requiring glibc-initial as target

(From OE-Core rev: 41f02cb7307faec32b13733e41f78a0ebdc1b52b)

Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-26 10:44:27 +00:00
Leonardo Sandoval 6702c7b089 selftest/imagefeatures: skip weston test if required features are not present
Also, remove hard-coded features because the image already contains them.

(From OE-Core rev: 899680840d497343095d0e7d7264e3e278ba4602)

Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-26 10:44:27 +00:00
Richard Purdie 6ab4c5033a classes/oeqa: Replace subprocess.check_call() with check_output()
If you use subprocess.check_output() the traceback will contain the output
when the command fails which is very useful for debugging. There is no
good reason not to use this everywhere.

(From OE-Core rev: ad750dd1cc9d789abe723daddd098ce41d8547f5)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-23 23:30:57 +00:00
Francisco Pedraza 51b6df76f2 oeqa/selftest: Adds test case for sdk-update eSDK
1. Test case adds perl recipe before build eSDK.
2. After this added recipe, the script verifies the update:

    ${SDK_DEPLOY}/${TOOLCHAINEXT_OUTPUTNAME}.sh
    oe-publish-sdk [esdk] [path-to-http]
    CORE_IMAGE_EXTRA_INSTALL = "perl"
    bitbake -c populate-sdk-ext [some-image]
    devtool sdk-update

This should fix [YOCTO #9369]

(From OE-Core rev: 21bd406bf89e9ceafe1a807877406be817cacca6)

Signed-off-by: Francisco Pedraza <francisco.j.pedraza.gonzalez@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-23 12:05:22 +00:00
Chen Qi 2a64701acf package_manager: default to have scriptlet output captured in log
We need to have scriptlet output captured in log. If we don't do so,
some useful information from scriptlets (especially postinstall script)
would be missing. In case a script has a warning message but it does not
necessarily have to fail, the message should be captured.

Opkg has already done that. Change for rpm and dpkg so that scriptlet
output is captured and no warning message is missing.

(From OE-Core rev: 0e52e0c619e02327602d83999a61d978d3a9240e)

Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-23 12:05:22 +00:00
Markus Lehtonen f4c96ea829 oeqa.buildperf: store measurements as a dict (object) in the JSON report
Store measurements as a dict, instead of an array, in the JSON report.
This change makes traversing of the report much easier. The change also
disallows identically named measurements under one test, as a sanity
check for the test cases.

[YOCTO #10590]

(From OE-Core rev: 81065092f38c9631dcf5917d70a25809a21de5f4)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-23 12:05:22 +00:00
Markus Lehtonen 7171710acd oeqa.buildperf: change sorting in json report
Use OrderedDict() instead of sort_keys=True (of json.dump()). Makes for
more logical sorting of the values in the report.

[YOCTO #10590]

(From OE-Core rev: 75e8aec0e0d81888be47b35c3c84df73edb91868)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-23 12:05:22 +00:00
Markus Lehtonen 8e01802c9f oe-build-perf-test: remove unused imports and fix indent
[YOCTO #10590]

(From OE-Core rev: 0b1892fa9165407a156609ff7cb3708e21bacd8c)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-23 12:05:22 +00:00
Markus Lehtonen 0981bcb098 oe-build-perf-test: save test metadata in a separate file
The patch introduces a new metadata (.json or .xml) file in the output
directory. All test meta data, e.g. git revision information and tester
host information is now stored there. The JSON report format is slightly
changed as the metadata is not present in results.json anymore.

[YOCTO #10590]

(From OE-Core rev: 2036c646019660e32f1bc277fdec0cdbff0afdd4)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-23 12:05:22 +00:00
Markus Lehtonen 0e2d84728b oeqa.buildperf: report results in chronological order
Write results in the report file in chronological order, instead of
random order dependent on test statuses.

[YOCTO #10590]

(From OE-Core rev: 91ba6ea9fe2eb82f992a6516d7971b435e1cfd32)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-23 12:05:22 +00:00
Markus Lehtonen fa4742f585 oeqa.buildperf: extend xml report format with test description
Add test description as an attribute to the <testcase> element.

[YOCTO #10590]

(From OE-Core rev: 7c23ddfeb4a46ee519cafdbd83ad1880621fba4d)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-23 12:05:21 +00:00
Markus Lehtonen 4a26ceaecf oeqa.buildperf: extend xml format to contain measurement data
Make the xml report format slightly non-standard by incorporating
measurement data into it.

[YOCTO #10590]

(From OE-Core rev: b7164d30fb125ff0c85a2ea508b0f1801aa57f66)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-23 12:05:21 +00:00
Markus Lehtonen 07c245792e oe-build-perf-test: enable xml reporting
Add --xml command line option to oe-build-perf-test script for producing
a test report in JUnit XML format instead of JSON.

[YOCTO #10590]

(From OE-Core rev: 21ae1c491b93675254b7733640662b566ed76f98)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-23 12:05:21 +00:00
Markus Lehtonen 96fcb0f7b4 oeqa.buildperf: include error details in json report
This will typically mean assert message and exception type plus a
traceback. In case of skipped tests the reason (i.e. skip message) is
included.

[YOCTO #10590]

(From OE-Core rev: bd5f5ab6f7350b4487c9cc2dbd100fa4b687d0fa)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-23 12:05:21 +00:00
Markus Lehtonen 184afbc6ce oeqa.buildperf: sync test status names with JUnit
Use 'failure' instead of 'fail'. Also, use 'expected' instead of 'exp'.

[YOCTO #10590]

(From OE-Core rev: 51ae18c64eee074478157a484f71a53faadca80b)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-23 12:05:21 +00:00
Markus Lehtonen ef988b3d06 oeqa.buildperf: prevent a crash on unexpected success
(From OE-Core rev: af205d9a13d182a0b590426ba8c4e3dff7f3f02b)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-23 12:05:21 +00:00
Mariano Lopez 6cea270c54 selftest/runtime-test.py: Adapt test to use new runtime framework
This adapt the current runtime selftest to use the new runtime
framework.

(From OE-Core rev: cfeec8a59ba03f98944fd3dca1a67d80e7edb4c9)

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-23 12:05:21 +00:00
Mariano Lopez 1cc2bac7ec oeqa/runtime/context.py: Add defaults for runtime context
This adds default values to OERuntimeTestContextExecutor class in
order to make easier the execution of exported test that were
generated with testexport class.

[YOCTO #10686]

(From OE-Core rev: c78aeaac3b75610bada62b138c9670815a07ee80)

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-23 12:05:21 +00:00
Mariano Lopez 58789be270 testimage.bbclass: Add package install feature
This allows to use the package install feature with
the new OEQA framework.

[YOCTO #10234]

(From OE-Core rev: 077dc19445574457769eb4f231de97e8059cb75e)

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-23 12:05:20 +00:00
Mariano Lopez f8d7db1905 testimage.bbclass: Add support for package extraction
testimage support the installation of packages without a package
manager in the target. This adds support for package extraction
required to support the installation feature.

[YOCTO #10234]

(From OE-Core rev: 8c7335290cb00ed0683241249297ca573ebd353a)

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-23 12:05:20 +00:00
Mariano Lopez 8970ed4b1f oeqa/core/utils/test.py: Add functions to get module path
This will add functions to get module file path from a test
case or a complete suite.

[YOCTO #10234]

(From OE-Core rev: 11dd49cb9faaf6588f045083ab5bd6679e2eafc1)

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-23 12:05:20 +00:00
Mariano Lopez 41e4db0eea runtime/cases/smart.py: Migrate smart tests
This migrates the smart test from the old framework to
the new one. This has its own commit because smart
test was using bb and oe libraries that are available
when exporting the test cases to run in a different host.

Because of the removal of bb and oe libraries index and
packages feeds creation will be managed in testimage bbclass.

[YOCTO #10234]

(From OE-Core rev: 8d64ac4208e8dcb8a6fde6ea2959c9b3edfe2172)

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-23 12:05:20 +00:00
Mariano Lopez b569aa0e00 oeqa/runtime/cases: Migrate runtime tests.
This migrates current runtime test suite to be used with the new framework.

[YOCTO #10234]

(From OE-Core rev: b39c61f2d442c79d03b73e8ffd104996fcb2177e)

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-23 12:05:20 +00:00
Mariano Lopez 211c5be54c core/target/qemu.py Adds qemu target
This adds qemu target to be used in testimage.
It uses the current QemuRunner class in order
to boot and control qemu.

[YOCTO #10231]

(From OE-Core rev: 44d4e9d0bb31fbc28d8c1fad8860ff56e5ae043a)

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-23 12:05:20 +00:00
Mariano Lopez 71e456add8 oeqa/runtime/context.py: Add logger to getTarget
Current targets (ssh and qemu) require a logger in their
constructors, so in order to get a new target we need
to provide the logger.

[YOCTO #10686]

(From OE-Core rev: 26bae3c255bc1e1cc8d81db0cffc809de0182a43)

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-23 12:05:20 +00:00
Aníbal Limón 0e8d0a03f0 oeqa/runtime/context.py: Prepare for qemu
This commit will prepare context to use qemu and not just
simpleremote.

(From OE-Core rev: 7485a08c967916fb6edff4cc573d9314ec577031)

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-23 12:05:20 +00:00
Aníbal Limón 2d34b08533 oeqa/utils/dump: Move get_host_dumper to OERuntimeTestContextExecutor class
To avoid getVar calls inside a utils module, also moves
get_host_dumper import inside testexport isn't needed.

[YOCTO #10231]

(From OE-Core rev: f8beaf94e943a8b20d146be47a756af312ef107c)

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-23 12:05:20 +00:00
Aníbal Limón 1247118870 oeqa/runtime/context: Move helper functions for process args to executor
(From OE-Core rev: 07ee2fa0008a50da87fd840aea5e8bb6051c68fa)

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-23 12:05:20 +00:00
Aníbal Limón 6c3ca00884 oeqa/runtime/context: Add runtime option group and options for target type and server ip.
(From OE-Core rev: d3cc6761a13fd1da673829797b45323aed7997d0)

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-23 12:05:20 +00:00
Mariano Lopez 32a3f4d6ab oeqa/core/context: Add option to select tests to run
This add the option to select what tests to run in the
<module>[.<class>[.<test>]] format.

Currently it just support modules

(From OE-Core rev: 0050565d71cee5a0cc22660b2398e4334d4b6719)

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-23 12:05:20 +00:00
Mariano Lopez 41b0b31a84 oeqa/core/decorator: Add skipIfNotDataVar and skipIfNotInDataVar
skipIfNotDataVar will skip a test if a variable doesn't have certain value.

skipIfNotInDataVar will skip a test if a value is not in a certain variable.

(From OE-Core rev: a81045f4e2b740173237f5ae4e80e2bc0b287faa)

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-23 12:05:20 +00:00
Mariano Lopez c35b42cdbb oeqa/runtime/utils/targetbuildproject.py: Don't use more than 80 characters per line
(From OE-Core rev: 95a55e0736fc59ecdcb88d43f08b447ffa5a43ed)

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-23 12:05:20 +00:00
Mariano Lopez 5f57fd0b08 oeqa/runtime/files: Move runtime files from old directory
As part of the refactor we require to move the files used
in runtime testing to the new directory. This also adds
the path to the runtime test context.

[YOCTO #10234]

(From OE-Core rev: f80c21b46f69c42ff0853bf8abd01cf0c082c346)

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-23 12:05:20 +00:00
Mariano Lopez bfe20fd23c oeqa/core/decorator/data.py: Add skipIfNotFeature decorator
This adds a new decorator to check if image under tests has
certain DISTRO_FEATURE or IMAGE_FEATURE.

[YOCTO #10234]

(From OE-Core rev: 8740803d0696a0e97b72210a56f4fbd3135826ed)

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-23 12:05:20 +00:00
Mariano Lopez 9ee0816ca9 oeqa/runtime: Add OEHasPackage decorator
This new decorator will be used to skip the test
if the image under test doesn't have the required
packages installed.

[YOCTO #10234]

(From OE-Core rev: 021449938ff0b4d182d7f02930a80693f109c8ba)

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-23 12:05:20 +00:00
Mariano Lopez 4cd982566b oeqa/runtime: Add case, context and loader classes for runtime testing
This adds OERuntimeTestCase, OERuntimeTestContext, and OERuntimeTestLoader
to be used for runtime testing.

As expected there are some changes in runtime context:

- Adds the target to be used for runtime testing, the default
is a SSH connection to the device under test running a OE image.
- Runtime context requires image manifest because several
tests are skipped if a package is missing or installed.
- Several tests require the output of the ps command and it changes
its output and arguments if busybox o procps is installed, so the
case must use the correct ps command.

[YOCTO #10234]

(From OE-Core rev: f995f178de79d6d11422cd879d06371811f50651)

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-23 12:05:20 +00:00
Mariano Lopez 6ad52a82ea oeqa/core/target Add OESSHTarget to sent commands to targets using SSH
With this commit now it is possible to add targets with SSH for testing.
Most of it was imported for existing code, with improvements in log
handling.

[YOCTO #10234]

(From OE-Core rev: 3bc13548df4adb85f09467d200530a9c9f60da04)

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-23 12:05:20 +00:00
Aníbal Limón b61326efb1 oeqa/runtime: Move to runtime_cases
The new oeqa core framework will modify the structure of the runtime
folder the new runtime folder will have python code inside to support
runtime test cases.

(From OE-Core rev: 637b712096e9d230e15b1a432a561e4118db34c8)

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-23 12:05:19 +00:00
Aníbal Limón ba1aec3407 oeqa: Fix files handling on runtime tests.
Common files was move to oeqa/files from oeqa/runtime/files
because the same files are used across Runtime,SDK,eSDK tests.

(From OE-Core rev: f099302efe8f222c3e4ae3604429f5ede4fd8c67)

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-23 12:05:19 +00:00
Aníbal Limón e5f05409d0 oeqa/runtime: Fix TargetBuildProject instances
TargetBuildProject was refactored to avoid bitbake dependency so
the instance don't allow to pass data store anymore.

classes/testimage: Export proxies before run tests

The TargetBuildProject based tests download archives from network.

(From OE-Core rev: e275f29de500a338a02402ecc570405309963b35)

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-23 12:05:19 +00:00
Aníbal Limón 260741847a oeqa/sdkext/cases: Migrate test case to new OEQA framework
Summary,

- Changes base case class to OESDKExtTest.
- Changes decorator classes to new ones.
- Chnages variable names sdktestdir -> sdk_dir.
- Added missing license to MIT.

(From OE-Core rev: 49568055df0a64e4228f27130b13ccafbba2a460)

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-23 12:05:19 +00:00
Aníbal Limón 14eee4fdf8 oeqa/sdkext: Adds case and context modules.
The extensible sdk context and case modules extends the sdk ones,
this means that the tests from sdk are run also the sdkext tests.

Enables support in context for use oe-test esdk command for run
the test suites, the same options of sdk are required for run esdk tests.

Removes old related to case and context inside oetest.py.

[YOCTO #10599]

(From OE-Core rev: 1f0bb99249744b87dd39227a4cf37f2341f5499c)

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-23 12:05:19 +00:00
Aníbal Limón 55f05fab2b oeqa/sdkext: Move test cases inside cases directory
For match with the new structure of the OEQA framework.

In the new framework Test component base directory in this case
sdk module will contain case and context implementations.

[YOCTO #10599]

(From OE-Core rev: bdb92fa4d9bd2e4a0a14e3adc62a6b9e9bf639d3)

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-23 12:05:19 +00:00
Aníbal Limón 3fac2417dd classes/testsdk: Remove the need of TEST_LOG_DIR variable
The TEST_LOG_DIR was used for store sdk_target_log this log
contains the output of the run of build commands now that information
could be found also on log.do_testsdk under WORKDIR.

The log will continue to store into SDK_DIR instead of TEST_LOG_DIR.

(From OE-Core rev: 1c9ba4b698bab916d42b58255692a7bf3d773bbc)

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-23 12:05:19 +00:00
Aníbal Limón c9fe59d7c5 oeqa/sdk/cases: Migrate tests to the new OEQA framework
Summary of the changes:

- Remove auto extend_path using pkgutil at __init__, is not needed.
- Change base class to OESDKTestCase.
- Add td_vars attr to set dependencies of certain variables in test
data.
- Change skips from module level to class level because Test context
  (tc)
now isn't at module level.
- Variable names changes to be consistent (i.e. sdktestdir ->
  sdk_dir).

[YOCTO #10599]

- Don't use bb.utils functions use instead remove_safe and shutil
  for copy files.
- SDKBuildProject pass test data variables instead of call getVar
  inside.

[YOCTO #10231]

(From OE-Core rev: 91cd1ed19a3f34c29cd77eb136036975fe465444)

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-23 12:05:19 +00:00
Aníbal Limón 51be188063 oeqa/utils: {Target,SDK,}BuildProject remove dependency of bb
Don't use bitbake references inside utils modules, in order todo
that changes getVar calls for arguments in the __init__ method like
dl_dir for all the classes and testlogdir, builddatetime in
SDKBUildProject.

Also don't export proxies inside _download_archive method, a good
practice is to setup the proxies at init of the process instead of
do it in this helper module.

[YOCTO #10231]
[YOCTO #10599]

(From OE-Core rev: 581c34d1efe9839f50ef322761269b4e4d8a56a6)

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-23 12:05:19 +00:00
Aníbal Limón 72e9ae377d oeqa/utils: Move targetbuild to buildproject module
The new buildproject module will contain only BuildProject class
a helper class for build source code.

The remaining classes TargetBuildProject and SDKBuildProject was
move to runtime and sdk respectively.

[YOCTO #10599]

(From OE-Core rev: 525fd2a5cda00890e921b63f7f608a10bc024d73)

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-23 12:05:19 +00:00
Aníbal Limón 8a37763ae9 oeqa/sdk: Add case and context modules for the SDK component
Adds case and context modules for SDK based on oetest.py old code.

Enables SDK Test component usage with oe-test, the SDK Test component
adds command line options for specify sdk installed dir, sdk environment
and target/hosts maniftest.

[YOCTO #10599]

(From OE-Core rev: 19e875dd81c42841666e6db5f6b665b4e1cddfe6)

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-23 12:05:19 +00:00
Aníbal Limón 9a23e0f703 oeqa/{runtime,sdk}/files: Move testsdkmakefile from runtime to sdk module
It doesn't make sense to have files related to sdk module into runtime
module.

[YOCTO #10599]

(From OE-Core rev: 9a3be58b17afbe4ef00030b0e6ad8b20b03adc49)

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-23 12:05:19 +00:00
Aníbal Limón 2726c26efe oeqa/sdk: Move test cases inside cases directory
For match with the new structure of the OEQA framework.

In the new framework Test component base directory in this case
sdk module will contain case and context implementations.

[YOCTO #10599]

(From OE-Core rev: 57af8ee4021c302bd351adf03e6d85274ad7efd5)

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-23 12:05:19 +00:00
Aníbal Limón 8d3640b90d oeqa: Move common files to oeqa/files instead of runtime only
Those files are used by runtime and sdk test cases, so move to
base directory of oeqa module.

[YOCTO #10599]

(From OE-Core rev: ec73e8a3d3149f3866b7bfc06f169c6e05e2d338)

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-23 12:05:19 +00:00
Mariano Lopez bbb7366db3 oeqa/utils/__init__.py: Adds compatibility with bitbake logger
The bitbake logger changes the way debug is logged and adds
different levels within debug, this is passed as argument
to the function and breaks compatibility with vanilla loggers.

This implements a way to handle this adding a new function for
debug, that will dispatch the correct logging method signature.

Also overrides info method to use logging.INFO + 1 in order to
see plain data.

Also this commit fix the issue of not showing the test summary
and results when running from bitbake.

[YOCTO #10686]

(From OE-Core rev: 619c9ab308fbef9e3563dc661e432603e764b562)

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-23 12:05:19 +00:00
Aníbal Limón 4054b25d5e oe/data: Add export2json function
The export2json function export the variables contained in
the data store to JSON format, the main usage for now will be
to provide test data to QA framework.

(From OE-Core rev: 57c7bf68ed66a56601e1431bb2db750c5742b5ce)

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-23 12:05:18 +00:00
Aníbal Limón 388503c032 oeqa/core: Add README
The README has an introduction and explains how to run the test suite
and creates a new Test component.

(From OE-Core rev: 9d474172c47695be1a61538f5b87ca8d9db25fa7)

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-23 12:05:18 +00:00
Aníbal Limón 9a5f285fe3 oeqa/core/cases: Add example test cases
Serves as an first input of how to the OEQA framework works.

(From OE-Core rev: 115f80adf1b230c5d0392e7833e9aeb274642bcb)

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-23 12:05:18 +00:00
Aníbal Limón 94cb20f20d oeqa/core/context: Add support of OETestContextExecutor
The OETestContextExecutor class supports to use oe-test for run core
test component also is a base class for the other test components
(runtime, sdk, selftest).

Te principal functionality is to support cmdline parsing and execution
of OETestContext, the test components could extend the common options
to provide specific ones. The common options between test components
are test data file, output log and test cases path's to scan.

Also it initializes the logger to be passed to the whole OEQA framework.

[YOCTO #10230]

(From OE-Core rev: 039deafa5f2c8fab31b8373b39f8bc219377b893)

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-23 12:05:18 +00:00
Mariano Lopez 7bf63b28f1 oeqa/core: Add tests for the OEQA framework
This test suite covers the current functionality for the OEQA
framework.

For run certain test suite,

$ cd meta/lib/oeqa/core/tests
$ ./test_data.py

(From OE-Core rev: 7d7d0dc3736fc12ae7848de2785f0066e6470cd1)

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-23 12:05:18 +00:00
Aníbal Limón 4da4091cee oeqa/core/decorator: Add support for OETestDataDepends and skipIfDataVar
The OETestDataDepends decorator skips a test case if a variable
isn't into test data (d).

The skipIfDataVar decorator skips a test case if a variable
has certain value.

(From OE-Core rev: 7dc519d20e835ee7693c31903e164c4bc0e5e598)

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-23 12:05:18 +00:00
Mariano Lopez b685a7c1a8 oeqa/core/decorator: Add support for OETimeout decorator
The OETimeout provides support for specify certain timeout
in seconds for a test case, if the timeout is reach the SIGALRM
is sent and an exception is raised to notify the timeout.

[YOCTO #10235]

(From OE-Core rev: 1bf66a370361912e9950d7ff45e382c93622a169)

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-23 12:05:18 +00:00
Mariano Lopez b8cebdb96c oeqa/core/decorator: Add support for OETestID and OETestTag
These two decorators stores certain TAG or ID for the test case
also provides support for filtering in loading step.

[YOCTO #10236]

(From OE-Core rev: 047af4ce864bbf98e2617b348ae9ccb77ac52871)

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-23 12:05:18 +00:00
Aníbal Limón 95a2ec6aab oeqa/core/decorator: Add support for OETestDepends
The OETestDepends decorator could be used over test cases to
define some dependency between them.

At loading time sorting the tests to grauntee that a test case
executes before also raise an exception if found a circular
dependency between test cases.

At before test case run reviews if the dependency if meet, in the
case of don't it skips the test case run.

(From OE-Core rev: 2385bd3c8a7c012fd1cad5465ec7d34675552c75)

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-23 12:05:18 +00:00
Aníbal Limón 13c8c08b95 oeqa/core: Add loader, context and decorator modules
loader: Implements OETestLoader handling OETestDecorator
and filtering support when load tests. The OETestLoader is
responsible to set custom methods, attrs of the OEQA
frameowork.

[YOCTO #10231]
[YOCTO #10317]
[YOCTO #10353]

decorator: Add base class OETestDecorator to provide a common
way to define decorators to be used over OETestCase's, every
decorator has a method to be called when loading tests and
before test execution starts. Special decorators could be
implemented for filter tests on loading phase.

context: Provides HIGH level API for loadTests and runTests
of certain test component (i.e. runtime, sdk, selftest).

[YOCTO #10230]

(From OE-Core rev: 275ef03b77ef5f23b75cb01c55206d1ab0261342)

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-23 12:05:18 +00:00
Mariano Lopez abb55ab304 oeqa/core: Add utils module for OEQA framework
misc: Functions for transform object to other types.
path: Functions for path handling.
test: Functions for operations related to test cases and suites.

[YOCTO #10232]

(From OE-Core rev: 102d04ccca3ca89d41b76a8c44e0ca0f436b7004)

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-23 12:05:18 +00:00
Aníbal Limón 08714d3b7e oeqa/core: Add base OEQA framework
case: Defines OETestCase base class that provides custom
    methods/attrs defined by the framework.
    Every OETestCase instance contains a reference to the test
    data (d), the test context (tc) and the logger.
    Also implements _oe{SetUp,TearDown}Class for make special
    handling of OEQA decorators and validations.

runner: Defines OETestRunner/OETestResult with support for RAW
    and XML result logs.

exception: Custom exceptions related to the OEQA framework based
    on class OEQAException.

[YOCTO #10230]
[YOCTO #10233]

(From OE-Core rev: c466086ccc4d4bb02d578a821cfb945945bfd529)

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-23 12:05:18 +00:00
Ed Bartosh 253b8d7d58 selftest: fixed 3 wic test cases
Fixed test_systemd_bootdisk, test_wic_image_type and test_qemu
test cases by building core-image-minimal with correct configuration.

(From OE-Core rev: 897fe85d34302953c98d07ade4fa2dd749ae2d22)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-23 12:05:18 +00:00
Ed Bartosh 78d4d09cf2 selftest: wic: don't set WKS_FILE
Setting WKS_FILE variable should be done only when
wic image is expected to be built by bitbake.
If it's set for all images it breaks image building in
some cases.

(From OE-Core rev: 46b84310f7df157c2ef290a60f5c2136d4206f09)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-23 12:05:18 +00:00
Ed Bartosh 657f1e44b1 wic: fixed test_iso_image test case
Fixed isoimage-isohybrid plulgin and correspondent wic tet case:
- used wic-tools target when getting varlue of STAGING_LIBDIR variable
- ensured that image is built with efi and hddimg enabled

(From OE-Core rev: 5878484da64c38c7fde45bb06d76e22e608eb022)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-23 12:05:18 +00:00
Ed Bartosh dc4a612e60 image.bbclass: put .env files to ${STAGING_DIR}/imgdata/
As STAGING_DIR_TARGET started to point to a recipe specific
sysroot wic is not able to add .env files when .wks file refers
to multiple rootfs recipes.

Used STAGING_DIR instead of STAGING_DIR_TARGET to make the
directory with .env files the same for all recipes.

(From OE-Core rev: 3797cfd7473d3f9b7c0d999dcf9cd9608c8c7c6c)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-23 12:05:17 +00:00
Ed Bartosh 168f62e427 wic: fix getting path of native sysroot
wic used STAGING_DIR_NATIVE variable as a path to native sysroot.
This doesn't work with recipe specific sysroots as STAGING_DIR_NATIVE
points to the native sysroot of the current recipe.

Used RECIPE_SYSROOT_NATIVE variable of wic-tools recipe
to fix the issue.

(From OE-Core rev: de9d7d14cd03e4dfc5812890a53c79b706b56537)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-23 12:05:17 +00:00
Ed Bartosh a79b32c75a selftest: build wic-tools in the Wic.setUp
Built wic-tools recipe instead of set of tools recipes
to ensure that all tools are available from one recipe sysroot.

(From OE-Core rev: 52ce14d5ea667ffd319fdb3e147b6eb29505cf8e)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-23 12:05:17 +00:00
Richard Purdie 209a21ed46 selftest/devtool: Update to account for recipe specific sysroot
There is no common sysroot any more so add the libusb dependency using DEPENDS
and check for the output in the sysroot output directory so the tests
work with recipe specific sysroots.

(From OE-Core rev: ff30b833a423d300ec2b81bf80ef6733a6d8039b)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-23 12:05:17 +00:00
Richard Purdie 9107d6ca14 Switch to Recipe Specific Sysroots
This patch is comparatively large and invasive. It does only do one thing, switching the
system to build using recipe specific sysroots and where changes could be isolated from it,
that has been done.

With the current single sysroot approach, its possible for software to find things which
aren't in their dependencies. This leads to a determinism problem and is a growing issue in
several of the market segments where OE makes sense. The way to solve this problem for OE is
to have seperate sysroots for each recipe and these will only contain the dependencies for
that recipe.

Its worth noting that this is not task specific sysroots and that OE's dependencies do vary
enormously by task. This did result in some implementation challenges. There is nothing stopping
the implementation of task specific sysroots at some later point based on this work but
that as deemed a bridge too far right now.

Implementation details:

* Rather than installing the sysroot artefacts into a combined sysroots, they are now placed in
  TMPDIR/sysroot-components/PACKAGE_ARCH/PN.

* WORKDIR/recipe-sysroot and WORKDIR/recipe-sysroot-native are built by hardlinking in files
  from the sysroot-component trees. These new directories are known as RECIPE_SYSROOT and
  RECIPE_SYSROOT_NATIVE.

* This construction is primarily done by a new do_prepare_recipe_sysroot task which runs
  before do_configure and consists of a call to the extend_recipe_sysroot function.

* Other tasks need things in the sysroot before/after this, e.g. do_patch needs quilt-native
  and do_package_write_deb needs dpkg-native. The code therefore inspects the dependencies
  for each task and adds extend_recipe_sysroot as a prefunc if it has populate_sysroot
  dependencies.

* We have to do a search/replace 'fixme' operation on the files installed into the sysroot to
  change hardcoded paths into the correct ones. We create a fixmepath file in the component
  directory which lists the files which need this operation.

* Some files have "postinstall" commands which need to run against them, e.g. gdk-pixbuf each
  time a new loader is added. These are handled by adding files in bindir with the name
  prefixed by "postinst-" and are run in each sysroot as its created if they're present.
  This did mean most sstate postinstalls have to be rewritten but there shouldn't be many of them.

* Since a recipe can have multiple tasks and these tasks can run against each other at the same
  time we have to have a lock when we perform write operations against the sysroot. We also have
  to maintain manifests of what we install against a task checksum of the dependency. If the
  checksum changes, we remove its files and then add the new ones.

* The autotools logic for filtering the view of m4 files is no longer needed (and was the model
  for the way extend_recipe_sysroot works).

* For autotools, we used to build a combined m4 macros directory which had both the native and
  target m4 files. We can no longer do this so we use the target sysroot as the default and add
  the native sysroot as an extra backup include path. If we don't do this, we'd have to build
  target pkg-config before we could built anything using pkg-config for example (ditto gettext).
  Such dependencies would be painful so we haven't required that.

* PKDDATA_DIR was moved out the sysroot and works as before using sstate to build a hybrid copy
  for each machine. The paths therefore changed, the behaviour did not.

* The ccache class had to be reworked to function with rss.

* The TCBOOTSTRAP sysroot for compiler bootstrap is no longer needed but the -initial data
  does have to be filtered out from the main recipe sysroots. Putting "-initial" in a normal
  recipe name therefore remains a bad idea.

* The logic in insane needed tweaks to deal with the new path layout, as did the debug source
  file extraction code in package.bbclass.

* The logic in sstate.bbclass had to be rewritten since it previously only performed search and
  replace on extracted sstate and we now need this to happen even if the compiled path was
  "correct". This in theory could cause a mild performance issue but since the sysroot data
  was the main data that needed this and we'd have to do it there regardless with rss, I've opted
  just to change the way the class for everything. The built output used to build the sstate output
  is now retained and installed rather than deleted.

* The search and replace logic used in sstate objects also seemed weak/incorrect and didn't hold
  up against testing. This has been rewritten too. There are some assumptions made about paths, we
  save the 'proper' search and replace operations to fixmepath.cmd but then ignore this. What is
  here works but is a little hardcoded and an area for future improvement.

* In order to work with eSDK we need a way to build something that looks like the old style sysroot.
  "bitbake build-sysroots" will construct such a sysroot based on everything in the components
  directory that matches the current MACHINE. It will allow transition of external tools and can
  built target or native variants or both. It also supports a clean task. I'd suggest not relying on
  this for anything other than transitional purposes though. To see XXX in that sysroot, you'd have
  to have built that in a previous bitbake invocation.

* pseudo is run out of its components directory. This is fine as its statically linked.

* The hacks for wayland to see allarch dependencies in the multilib case are no longer needed
  and can be dropped.

* wic needed more extensive changes to work with rss and the fixes are in a separate commit series

* Various oe-selftest tweaks were needed since tests did assume the location to binaries and the
  combined sysroot in several cases.

* Most missing dependencies this work found have been sent out as separate patches as they were found
  but a few tweaks are still included here.

* A late addition is that extend_recipe_sysroot became multilib aware and able to populate multilib
  sysroots. I had hoped not to have to add that complexity but the meta-environment recipe forced my
  hand. That implementation can probably be neater but this is on the list of things to cleanup later
  at this point.

In summary, the impact people will likely see after this change:

* Recipes may fail with missing dependencies, particularly native tools like gettext-native,
  glib-2.0-native and libxml2.0-native. Some hosts have these installed and will mask these errors

* Any recipe/class using SSTATEPOSTINSTFUNCS will need that code rewriting into a postinst

* There was a separate patch series dealing with roots postinst native dependency issues. Any postinst
  which expects native tools at rootfs time will need to mark that dependency with PACKAGE_WRITE_DEPS.

There could well be other issues. This has been tested repeatedly against our autobuilders and oe-selftest
and issues found have been fixed. We believe at least OE-Core is in good shape but that doesn't mean
we've found all the issues.

Also, the logging is a bit chatty at the moment. It does help if something goes wrong and goes to the
task logfiles, not the console so I've intentionally left this like that for now. We can turn it down
easily enough in due course.

(From OE-Core rev: 809746f56df4b91af014bf6a3f28997d6698ac78)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-23 12:05:17 +00:00
Paul Eggleton b1b59b8a4b oe-selftest: devtool: remove use of git -C
The -C option isn't available in versions of git older than 1.8.5,
and officially we only require git 1.8.3.1 or newer (and the latter is
the version you'll find on CentOS 7, so the test fails there). In any
case we can simply specify the working directory to runCmd() so just
do that instead.

(From OE-Core rev: ce5c7075d530c0950f2feed35f95fbcd9f50721f)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-20 11:53:51 +00:00
Richard Purdie eeb30764e9 lib/oe/utils: Add build_depends_string function
This is useful when manipulating depends strings for task [depends]
flags and is slightly easier to parse than some inline python.

(From OE-Core rev: 7b05ea65a8db8a27b2a5579675775ee34ceb63c2)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-20 11:53:49 +00:00
Ross Burton a304b94eae oeqa/selftest/devtool: rewrite modify testcase
The modify testcase had to be updated as it started failing when mdadm was
upgraded due to hardcoding version numbers in the test.  I then noticed how
inefficient the test was and mostly rewrote it.

Start by changing the minor modification to change "Linux Software RAID" (the
subtitle of the man page) to "antique pin sardine" (a nonsense phrase that is
unlikely to appear upstream), and neaten the logic.

Start by not removing sstate at the beginning of the test. To ensure builds
happen we can use -f and -C, and iterating the sstate cache is time consuming.

Don't bitbake mdadm repeatedly until it stabilizes, we can start with bitbake -C
unpack to ensure that a full build is done from scratch.

os.path.join has the interesting quirk that join(/foo, /bar) results in /bar, so
use oe.path.join instead of working around that manually.

Don't repeatedly call get_bb_var(), each call results in a call to bitbake.

These changes reduce the runtime of the test from over 600 seconds to around 160
seconds on my machine.

(From OE-Core rev: fc97963bc61bf16112859fe1d7e460a13d34baca)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-19 22:47:22 +00:00
Mariano Lopez 81c3d4824a oeqa/utils/qemurunner.py: Be sure to stop qemu-system
When runqemu fails, qemu-system process would keep running
and won't be killed, setpgrp() was used when runqemu was
a shell script but it seems it doesn't work always with python.

This would kill qemu-system explicity and to avoid leaving
it behind.

(From OE-Core rev: 9d2b1aa1bcfb2f1933a8eeb9470b4174d5da2f0d)

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-19 22:47:22 +00:00
Mariano Lopez de17891185 oeqa/utils/qemurunner.py: Add missing sys module
This adds the missing sys module used by the child process
to exit. It seems the exception was cached in testimage and
selftest. It seems nobody noticed this because the module
is only used for sys.exit().

(From OE-Core rev: 66f66d1d763ff7bbaab9e8fcdf7fc882f2dfbb13)

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-19 22:47:22 +00:00
Markus Lehtonen e530a95c5a oeqa.utils.metadata: include BB_NUMBER_THREADS and PARALLEL_MAKE
Inlude values of BB_NUMBER_THREADS and PARALLEL_MAKE in the metadata.

[YOCTO #10590]

(From OE-Core rev: 97c07a55815c2fc3915705317b6f30d212fa1d45)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-19 22:47:22 +00:00
Markus Lehtonen 56a7f41711 oeqa.utils.metadata: allow storing any bitbake config variables
Make it possible to store any bitbake config variables in the metadata.
Config values will be stored under a new config element in the xml report:
    <config>
        <variable name="MACHINE">qemux86</variable>
    </config>

The value of MACHINE is moved there instead of having a dedicated
<machine> element.

[YOCTO #10590]

(From OE-Core rev: 6e7e6e37664b0a86111272f5f6f4a4e1d0f23302)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-19 22:47:22 +00:00
Markus Lehtonen c74af56d4b oeqa.utils.metadata: add bitbake revision information
[YOCTO #10590]

(From OE-Core rev: 71ca7dab08fc500b231054249aacc90ae4aa85da)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-19 22:47:22 +00:00
Markus Lehtonen 47aac40869 oeqa.utils.metadata: have layer name as an attribute in xml
Have the layer name as an attribute instead of of the name of the
element itself. That is, have <layer name="layer_name"/> instead of
<layer_name/>. A bit better XML design.

[YOCTO #10590]

(From OE-Core rev: 50ea44c19005b536a2791113f8b536fd10548ead)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-19 22:47:22 +00:00
Markus Lehtonen 28858af49f oeqa.utils.metadata: add commit count information
Makes it easier to put the commits into a timeline.

[YOCTO #10590]

(From OE-Core rev: 7757970bced4ecd6503991c0cf11f4d9158f650c)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-19 22:47:22 +00:00
Markus Lehtonen ceed2c7eb5 oeqa.utils.metadata: rename 'revision' to 'commit'
Revision is a bit vague and could point to a tag, for example. Git
commit objects are unambiguous and persistent so be explicit that the
element should contain git commit hash.

[YOCTO #10590]

(From OE-Core rev: a6521b89e86ddba6bd646dd3974d7b0390323b56)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-19 22:47:22 +00:00
Markus Lehtonen 927e759bb2 oeqa.utils.metadata: fix retrieval of git branch and revision
Always return a valid branch name, or, '(nobranch)' if the current HEAD
is detached. Also, always return the hash of the commit object that HEAD
is pointing to. Previous code returned an incorrect branch name (or
crashed) e.g. in the case of detached HEAD.

[YOCTO #10590]

(From OE-Core rev: 02d3ba17a8090bd088beb973980651d664f713bb)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-19 22:47:22 +00:00
Markus Lehtonen 4e67232f34 oeqa.utils.metadata: drop 'unknown' git data elements
It's better just to not have the xml elements than to have elements with
faux data. One could have git branch named 'unknown', for example.

[YOCTO #10590]

(From OE-Core rev: fce531c21f5e56d0f416b3405a0b0fc5ba567679)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-19 22:47:22 +00:00
Markus Lehtonen df8c0540e4 oeqa.utils.metadata: re-organise distro information
Use the same format, based on /etc/os-release, as for host distro
information.

[YOCTO #10590]

(From OE-Core rev: 0156ef46ccf5334ee72f0202f1089249c62af37b)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-19 22:47:22 +00:00
Markus Lehtonen 8d51f62cd4 oeqa.utils.metadata: re-organise host distro information
Put all host distro data under one <host_distro> element. In addition
take the data directly from /etc/os-release instead of the "lsb API".
The /etc/os-release file is virtually ubiquitous, now, and using its
field names and values provides a more standardized and extensible
format.

[YOCTO #10590]

(From OE-Core rev: 98cad0b4063772dad94fea96edce1a5422256c32)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-19 22:47:22 +00:00
Jose Perez Carranza 2cd7613582 oeqa/selftest/tinfoil: add test IDs to tinfoil test cases
Testopia entrances were created and the IDs retrieved are
added to their corresponding test case on tinfoil script.

(From OE-Core rev: 486e5ad8270a4d3897f477ae5ae61422826f93ce)

Signed-off-by: Jose Perez Carranza <jose.perez.carranza@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-19 22:47:22 +00:00
Mariano Lopez 55aa669750 lib/oe/package_manager.py: Fix extract for ipk and deb
With the move to use lists instead of strings in subprocess
calls, package extraction was broken for ipk and deb. This
fixes this issue.

(From OE-Core rev: 3e1d8e5c7ac3238eda85ee95dfef044bef2a6411)

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-19 22:47:20 +00:00
André Draszik 08fd7c4fc2 lib/oe/rootfs: reliably handle alternative symlinks
When removing unneeded packages from a (read-only) rootfs
during rootfs creation, alternative symlinks from those
packages may or may not be removed.

The reason is as follows:

update-alternatives(-native) is used during package
installation as part of the image creation. It uses
a database which contains entries for all the
alternative symlinks possible, and the -native version
uses the target's database by means of $OPKG_OFFLINE_ROOT,
i.e. the rootfs we're in the process of creating.

Once the rootfs has been created, OE removes certain
packages because we have a read-only rootfs - in
particular ROOTFS_RO_UNNEEDED which includes
VIRTUAL-RUNTIME_update-alternatives, i.e. the
update-alternatives. Recently, a change was made in
OE, where uninstallation of update-alternatives from the
rootfs causes removal of its database, too, to save space
(700KiB (uncompressed) in a busybox system)
  b24a63d71b517af701dfedbc7f7b541d25af708f
  http://git.openembedded.org/openembedded-core/commit/meta/recipes-devtools/opkg-utils/opkg-utils_git.bb?id=b24a63d71b517af701dfedbc7f7b541d25af708f

Following from that, if update-alternatives is removed
from the target file system, update-alternatives-native
has no database anymore, meaning it can't manage any of
the alternative symlinks anymore.

Because the order of packages to uninstall is
non-deterministic, and update-alternatives could well
be removed before any packages that use the mechanism
provided, sometimes the extra symlinks are removed,
sometimes not.

By sorting the list of packages to be removed such that
update-alternatives is removed last, we can ensure that
that tings work reliably. (Certainly opkg seems to
uninstall packages in the order given on the command
line.)

[YOCTO #10916]

(From OE-Core rev: 5263dd3eac9d9fbdb7ef654d0cd532c192baed16)

Signed-off-by: André Draszik <adraszik@tycoint.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-16 18:05:14 +00:00
Leonardo Sandoval 0bb40f0bb9 selftest: runtime-test: skip image-install test for poky-tiny
poky-tiny cannot build full-cmdline image, so skip this test in this case.

(From OE-Core rev: c2b1c562db160876fc3e1ee8b15bd07136d6ea7a)

Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-16 18:05:14 +00:00
Leonardo Sandoval 5c5db3a477 selftest: devtool: use distro agnostic recipes for devtool checks
The recipes being replaced are not compatible with all distros, so
use mraa and virtual/make for some checks.

(From OE-Core rev: bb32b232d686d1c057dede6f61ed4051b0088673)

Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-16 18:05:14 +00:00
Ross Burton 55228a1ca4 selftest/base: don't fetch DISTRO variable in constructor
Fetching the DISTRO variable in the base constructor means that we have to start
bitbake for every test case instance, which adds minutes to the startup time.

(From OE-Core rev: ac1c118dcb3cb27807b55115ef274a92bb512dd6)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-16 18:05:14 +00:00
Ross Burton 2abc37dbe8 selftest/bblayers: don't fetch a variable that is never used
(From OE-Core rev: f812f9518ef01abbdd9a6f415e5dc92263135f87)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-16 18:05:13 +00:00
Ross Burton cf8fcba8e7 selftest/archiver: don't build an image for a basic test
This test only exercises the include/exclude behaviour so it only needs to build
the two recipes that it tests against, not an entire image.

Part of #10874.

(From OE-Core rev: 9b02216be6c9dbf2f680db1ad1309bcb9fb32b23)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-16 18:05:13 +00:00
Chen Qi 988f2efd65 selftest/buildoptions.py: fix path assumption for DEPLOY_DIR_SRC
Fix path assumption for DEPLOY_DIR_SRC, otherwise, the testcase may fail
even if the functionality works well.

(From OE-Core rev: dab5d3901755a965cdd8f5b5e8ffb8e4cb79f2e5)

Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-16 18:05:13 +00:00
Chen Qi 0967a78689 selftest/bbtests.py: fix path assumption for LICENSE_DIRECTORY
Fix path assumption for LICENSE_DIRECTORY, otherwise, the test case
may fail even if the functionality it tests works well.

(From OE-Core rev: ae388652b8de0665390560e78429e10119d4d537)

Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-16 18:05:13 +00:00
Ross Burton 1800b1ba7a rootfs: don't put /usr/lib/ssl and /etc into debugfs
The /etc and /usr/lib/ssl directories were only put into the opkg-generated
debugfs because of a bug in opkg which means that a conffile has to exist if
we're running 'opkg status'.  This is now fixed, so the workaround can be
reverted.

(From OE-Core rev: 7267b1f6fa25e290eac070263355aa7f30b2ebcb)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-16 18:05:13 +00:00
Ola x Nilsson e97422f400 oe-selftest: devtool: Add test for externalsrc buildclean
Test both for S == B and S != B.

(From OE-Core rev: 3b46c1ac203717d85a1e2e8da067a69f066b7037)

Signed-off-by: Ola x Nilsson <olani@axis.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-16 18:05:13 +00:00
Chen Qi 9a1e8b9cf5 selftest/eSDK.py: fix sstate dir not found error
Fix the error below when SSTATE_DIR is not "${BUILDDIR}/sstate-cache".

  FileNotFoundError: [Errno 2] No such file or directory: '/xxx/../sstate-cache'

(From OE-Core rev: 785f0343d04c1684363b5289a3012cf7e1caa95f)

Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-16 18:05:12 +00:00
Jose Perez Carranza 970b76a728 runtime: Add cleanup for logrotate tests
Delete logrotate dir to avoid errors
when test are executed more than 1
time on the same target.

(From OE-Core rev: 02c98c848163a6837692fe74a91754701a44a6c0)

Signed-off-by: Jose Perez Carranza <jose.perez.carranza@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-16 18:05:12 +00:00
Patrick Ohly 753471db45 rootfs-postcommands.bbclass: sort passwd entries
The /etc passwd files in a rootfs consist of the default entries from
base-passwd plus anything that gets added via package installation,
EXTRA_USERS_PARAMS and/or system sysusers.

The execution order of preinst scripts is not perfectly deterministic,
or at least unrelated changes caused it to change in a
non-deterministic way, resulting in irrelevant changes in the order of
passwd entries.

useradd-staticids.bbclass ensures that the numeric IDs don't change,
but re-ordering can still occur, which is bad for reproducible builds
and file-based update mechanisms like swupd which work best if changes
are as minimal as possible.

To achieve that, the files get sorted in a post-processing command,
enabled by default. Sorting is based primarily on the numeric IDs, so
for example, the "root" user continues to be listed first. "nobody"
now is at the end, which wasn't the case before.

The order of the entries should not matter, but in obscure cases where
it does (like having multiple entries for the same numeric ID) this
behavior can be disabled by setting SORT_PASSWD_POSTPROCESS_COMMAND to
an empty string.

Fixes: YOCTO #10520

(From OE-Core rev: ba684f436908ac2300a00c174d5aa06b4f824367)

Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-16 18:05:12 +00:00
Randy Witt f64e321baf image_typedep.py: Add a test that ensures conversion type deps get added
Add a test that ensures if IMAGE_TYPEDEP_* contains a conversion type,
that the corresponding CONVERSION_DEPENDS_ for that type gets added to
the dependency tree for do_rootfs.

(From OE-Core rev: 9cf9c725f7d534c326ffd95ec539b041f4ad286f)

Signed-off-by: Randy Witt <randy.e.witt@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-16 18:05:12 +00:00
Ross Burton 949c7062b7 selftest/devtool: update test to work with new mtd-utils
The new mtd-utils (version 2.0) has been autotooled so the test needs to touch
Makefile.am instead of Makefile.

(From OE-Core rev: acce512a0b85853b5acf2ef07e4163a3b4f33a98)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-09 13:39:13 +00:00
Ola x Nilsson cfa6a7aa6f oe-selftest: devtool: Reverting a change should trigger rebuild
Add code to verify that not only does a change trigger a build, but so
does reverting that change.

Reverting a change in a devtool managed git repo may cause the current
checksum to match the checksum of a previous build, which will cause
bitbake to skip builds that are needed.

(From OE-Core rev: 58a31d8dd7293f14c70e56ec9639c420d15e7dfc)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-09 13:39:11 +00:00
Jianxun Zhang c782cb40d7 lib/oe.sstatesig: make locked sig file consistent
Sort keys of dict 'types' prior to dumping, in order to have
identical output every time. This could make it a little easier
to diff these human-readable dumps.

(From OE-Core rev: 8abbaba1931e2cb2b87aa733aa9a3e8eb359b500)

Signed-off-by: Jianxun Zhang <jianxun.zhang@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-09 13:39:11 +00:00
Alejandro Hernandez 3f990e1b17 wic: Remove gummiboot test
Weve now migrated to systemd-boot, the gummiboot test on wic is no longer necessary

(From OE-Core rev: 0a8abc7681edec5f128ceb757559c5a50f139a9c)

Signed-off-by: Alejandro Hernandez <alejandro.hernandez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-09 13:39:11 +00:00
Alejandro Hernandez 2dce2648e3 gummiboot: Remove/change gummiboot references with systemd-boot
After systemd-boot was introduced, its been tested for a while with no major
issues being found until now, this patch completely replaces all gummiboot
instances with systemd-boot ones, taking the next step into cleaning
up systemd-boot/gummiboot.

[YOCTO #10332]

(From OE-Core rev: f9a61d3400ad9068a6d83b8eb6aefe3098c58e68)

Signed-off-by: Alejandro Hernandez <alejandro.hernandez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-09 13:39:11 +00:00
Paul Eggleton fff7563799 lib/oe/path: add warning comment about oe.path.remove() with wildcarded filenames
Add a warning in the doc comment for oe.path.remove() about using that
function on paths that may contain wildcards in the actual
file/directory names.

(From OE-Core rev: 18cc0965741102bccc62dfb32ed7753cdacbadc7)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-09 13:39:11 +00:00
Richard Purdie 022a8b58c8 meta/scripts: Various getVar/getVarFlag expansion parameter fixes
There were a few straggling expansion parameter removals left for
getVar/getVarFlag where the odd whitespace meant they were missed
on previous passes. There were also some plain broken ussages such
as:

d.getVar('ALTERNATIVE_TARGET', old_name, True)
path = d.getVar('PATH', d, True)
d.getVar('IMAGE_ROOTFS', 'True')

which I've corrected (they happend to work by luck).

(From OE-Core rev: 688f7a64917a5ce5cbe12f8e5da4d47e265d240f)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-09 13:39:11 +00:00
Richard Purdie d77a135e18 lib/oe/utils: Drop python2 compatibility code
We've moved to python3, we don't need this compatibility code which just makes
the code less readable.

(From OE-Core rev: 425afe2484707640ac71194885fdb263e95e9950)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-22 12:36:40 +00:00
Jair Gonzalez dd37336b4d selftest/wic: extending test coverage for WIC script options
The previous WIC script selftest didn't cover all of its command line
options. Some option variants were included in existing test cases and
the following tests were added to complete covering them:

1552 Test wic --version
1553 Test wic help create
1554 Test wic help list
1555 Test wic list images
1556 Test wic list source-plugins
1557 Test wic listed images help
1558 Test debug
1563 Test skip build check
1564 Test build rootfs
1559 Test image vars directory selection
1562 Test alternate output directory

(From OE-Core rev: b4d52c3f1e0ad2c14028ff08c0938d1b24b7f648)

Signed-off-by: Jair Gonzalez <jair.de.jesus.gonzalez.plascencia@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-22 08:50:17 +00:00
Jair Gonzalez f8745bb519 selftest/wic: reorganizing test methods by functionality
Part of the test methods were rearranged to group them by
functionality and identify more easily opportunities to extend
coverage.

(From OE-Core rev: f290a2c9b946e00dbc451592691596c656d49042)

Signed-off-by: Jair Gonzalez <jair.de.jesus.gonzalez.plascencia@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-22 08:50:17 +00:00
Jair Gonzalez 285c2ed90e selftest/wic: code cleanup
The following changes were added to improve the code readability:

- Removed redundant backlashes between brackets
- Aligned continuation lines according to PEP8 style
- Refactored command execution in the next methods for enhanced
  legibility:

  - test_build_image_name(self)
  - test_gpt_image(self)
  - test_qemux86_directdisk(self)
  - test_rootfs_indirect_recipes(self)
  - test_iso_image(self)
  - test_mkgummidisk(self)
  - test_mkefidisk(self)
  - test_directdisk_bootloader_config(self)
  - test_qemu(self)
  - test_bmap(self)
  - test_systemd_bootdisk(self)
  - test_sdimage_bootpart(self)

(From OE-Core rev: 5ba21836fe3c061a6271f855a9e26c66004d330a)

Signed-off-by: Jair Gonzalez <jair.de.jesus.gonzalez.plascencia@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-22 08:50:17 +00:00
Jair Gonzalez c48c0f38b1 selftest/wic: adding Testopia ID numbers to test cases missing it
The following test cases were assigned an ID number on Testopia:

1496 Test generation of .bmap file
1560 Test creation of systemd-bootdisk image
1561 Test creation of sdimage-bootpart image

(From OE-Core rev: b53e432206eaba7c6c67e4689c25b5e62d7ee9b2)

Signed-off-by: Jair Gonzalez <jair.de.jesus.gonzalez.plascencia@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-22 08:50:17 +00:00
Maciej Borzecki 5903182484 oeqa/utils/commands.py: allow use of binaries from native sysroot
Tests may need to run a native tool that is not available on the host
filesystem, but can be built using one of the *-native recipes. In such case,
the tool will be available in native sysroot, and running in from that location
will require adjustments to PATH.

runCmd() can now take a path to native sysroot as one of its arguments and
setup PATH accordingly.

(From OE-Core rev: f2a04631949db72d4261d1c142c5044fad3741f9)

Signed-off-by: Maciej Borzecki <maciej.borzecki@rndity.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-22 08:50:16 +00:00
Richard Purdie 8ccf396c72 Revert "selftest/wic: extending test coverage for WIC script options"
This reverts commit 68cb3180c1b0dcee50812b21f98850d188d8621b as this wasn't
ready for merge and there are new better versions.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-20 17:06:38 +00:00
Ed Bartosh 2f4008d694 oe-selftest: import git module only when needed
git module is not included into standard Python
library and therefore causes import errors on the systems
where PythonGit is not installed.

As git module only used in the code implementing --repository
functionality it's better to import git only in the scope
that requires it.

[YOCTO #10821]

(From OE-Core rev: 66be32c1a075201d6ee0e9b9e10b84e6a2ace745)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-20 15:22:17 +00:00
Jair Gonzalez aa1c50a32b selftest/wic: extending test coverage for WIC script options
The previous WIC script selftest didn't cover all of its command
line options. The following test cases were added to complete
covering them:

1552 Test wic --version
1553 Test wic help create
1554 Test wic help list
1555 Test wic list images
1556 Test wic list source-plugins
1557 Test wic listed images help
1558 Test wic debug, skip-build-check and build_rootfs
1559 Test image vars directory selection
1562 Test alternate output directory

In addition, the following test cases were assigned an ID number on
Testopia:

1560 Test creation of systemd-bootdisk image
1561 Test creation of sdimage-bootpart image

Finally, part of the test methods were rearranged to group them by
functionality, and some cleanup was made to improve the code's
compliance with PEP8 style guide.

Fixes [YOCTO 10594]

(From OE-Core rev: 3d2ac67765020885a0996ebdd97a576ba37dbec0)

(From OE-Core rev: 68cb3180c1b0dcee50812b21f98850d188d8621b)

Signed-off-by: Jair Gonzalez <jair.de.jesus.gonzalez.plascencia@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-20 15:22:16 +00:00
Ross Burton de43650c29 lib/oe/package_manager: bail if createrepo can't be found
If createrepo isn't found then the errors later are mysterious, so explicitly
check and error out early if it isn't there.

(From OE-Core rev: e09636bbb3ea8ec58984197fd9c691bb908efe00)

(From OE-Core rev: c87361fc886432a9db584712bf3e41ecd0541960)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-17 09:57:04 +00:00
Ross Burton 1969854ec3 utils: Always use datastore's PATH for host_gcc_version
BUILD_CC may reference something like ccache and expect this to come from
ccache-native, we at least have some selftests which assume this. Modify the
code to use PATH when runnig BUILD_CC to ensure the tests continue to work
as expected.

(From OE-Core rev: f3e753372baac43d0921186340cf260df056de20)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-16 10:23:23 +00:00
Joshua Lock 3c59b1bf93 meta: remove True option to getVarFlag calls
getVarFlag() now defaults to expanding by default, thus remove the
True option from getVarFlag() calls with a regex search and
replace.

Search made with the following regex:
getVarFlag ?\(( ?[^,()]*, ?[^,()]*), True\)

(From OE-Core rev: 2dea9e490a98377010b3d4118d054814c317a735)

Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-16 10:23:23 +00:00
Joshua Lock c4e2c59088 meta: remove True option to getVar calls
getVar() now defaults to expanding by default, thus remove the True
option from getVar() calls with a regex search and replace.

Search made with the following regex: getVar ?\(( ?[^,()]*), True\)

(From OE-Core rev: 7c552996597faaee2fbee185b250c0ee30ea3b5f)

Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-16 10:23:23 +00:00
Ed Bartosh 46e787b248 buildhistory-diff: report directory renames
The script detects directory renaming if two different
directories with the same set of files are added and removed.

[YOCTO #10691]

(From OE-Core rev: 944db779a9f45cbeeebc976c00da37a517eea237)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-16 10:23:23 +00:00
Ed Bartosh c21cec8488 uninative: rebuild uninative for gcc 4.8 and 4.9
Some c++ libraries fail to build if uninative is built
with gcc 5.x and host gcc version is either 4.8 or 4.9.

The issue should be solved by making separate uninative sstate
directory structure sstate-cache/universal-<gcc version> for host gcc
versions 4.8 and 4.9. This causes rebuilds of uninative if host gcc
is either 4.8 or 4.9 and it doesn't match gcc version used to build
uninative.

[YOCTO #10441]

(From OE-Core rev: d36f41e5658bbbb6080ee833027879c119edf3e0)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-16 10:23:23 +00:00
Jose Perez Carranza 333890953d postinst: Add a test case to verify postinst scripts behavior
Add test case that verify behavior of postinst scripts at
roofts time and when is delayed to the first boot directly
on the target.

(From OE-Core rev: 82b171f3b37e6733997fc1e7685b7cac5a3476e7)

Signed-off-by: Jose Perez Carranza <jose.perez.carranza@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-16 10:23:23 +00:00
Francisco Pedraza 23d1c4ffb7 selftest: Test needed to verify postinst order
It verifies the following:

1. Compile a minimal image.
2. The compiled image will add the layer with the recipe postinst,
previously created at:
"meta-selftest/recipes-test"
3. Run QEMU.
4. Validate the task execution order.
[YOCTO #5319]

(From OE-Core rev: a8ff789a3bfedcbc4358db7907a45270d8b1b76a)

Signed-off-by: Francisco Pedraza <francisco.j.pedraza.gonzalez@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-16 10:23:23 +00:00
Ross Burton d535aba974 oeqa: move lib/oe tests to oe-selftest
These tests don't get ran often (as demonstrated by the fact that some were not
ported to Python 3), so move them to oeqa/selftest so they get executed
frequently and can be extended easily.

[ YOCTO #7376 ]

(From OE-Core rev: 2001979ad41e6fdd5a37b0f90a96708f39c9df07)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-16 10:23:22 +00:00
Leonardo Sandoval 030f62e9c8 selftest: buildoptions: skip read-only-image test depending on distro
Poky-tiny cannot build core-image-sato, so skip test (read-only-image)
in this case.

(From OE-Core rev: bcee8c614f28b38054f5d8c1c5251b3702cf113c)

Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-16 10:23:22 +00:00
Leonardo Sandoval e2ea2597c9 selftest: bblayers: remove linux kernel checks for show-recipes check
Preferred kernel recipes depends on the distro, so remove the kernel
checks to avoid failures on non-poky distros and make the test
distro agnostic.

(From OE-Core rev: ae92b72990b3ac804791b501d08126491fdddb7c)

Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-16 10:23:22 +00:00
Leonardo Sandoval 279c8ce310 selftest: bbtests: use minimal image so all distros can execute it
poky-tiny distro cannot build full-cmdline image, so use an image
(core-image-minimal) that can be built in all distros.

(From OE-Core rev: b293dd4200bbb3705c88af6113be7f43fbd0ed72)

Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-16 10:23:22 +00:00
Leonardo Sandoval 009adf73fa selftest: base: new object member to store the DISTRO value
Instead of quering it multiple times, query once and use it on
test method skip checks. Also, rename current distro sstate object
member to a more meaninful name.

(From OE-Core rev: bde9d99575a63ad2d7fd5974ce6ce19aad9a8984)

Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-16 10:23:22 +00:00
Mariano Lopez a92d27acd8 oeqa/utils/commands.py: Fix get_bb_vars() when called without arguments
Commit 9d55e9d489 broke calling get_bb_vars()
when called without arguments. This fix this issue.

(From OE-Core rev: 91f856426c7523e1ebdf6d6f93f5fa7e509d6e49)

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-14 16:15:04 +00:00
Paul Eggleton 66adb6bdf7 oe-selftest: add basic tinfoil tests
Add some tests to verify that the new tinfoil API is operating
correctly.

(From OE-Core rev: 16afda66b861ba028c1152dcdcab2b7ebfbff965)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-14 12:30:50 +00:00
Paul Eggleton 0cba3cbcb4 oe-selftest: devtool: improve test_devtool_modify slightly
* Check that man .in file actually gets modified, since sed -i doesn't
  fail if it it doesn't
* Use a variable for man file path

(From OE-Core rev: 9ad36e945fa5b03726f78ba99e823eade8daa710)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-14 12:30:50 +00:00
Paul Eggleton 078ef4361d lib/oe/recipeutils: drop parse_recipe_simple()
This was intended to be used with tinfoil, but tinfoil now has its own
parse_recipe() method to do this which works properly in the memres
case.

(From OE-Core rev: cdfc6173cb06ca374b7d927442a0fdde8373ba48)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-14 12:30:50 +00:00
Paul Eggleton 275b3fe5f4 classes/patch: move several functions to oe.patch
Move patch_path(), src_patches() and should_apply() to oe.patch, making
them easier to call from elsewhere (particularly across the
UI/server boundary).

(From OE-Core rev: 2724511e18810cc8082c1b028e3b7c8a8b5def56)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-14 12:30:50 +00:00
Paul Eggleton 1f8cf63f8a oe-selftest: use tinfoil.parse_recipe()
Use tinfoil.parse_recipe() in order to allow oe-selftest to be used in
memres mode.

(From OE-Core rev: 499ee9006271112f22cfe08fa5ba5c21be95380b)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-14 12:30:49 +00:00
Paul Eggleton 93d675304d oe-selftest: make tinfoil quiet when using to start QEMU
We don't need to see the parsing/cache loading message in the
oe-selftest output, so use the newly added quiet option to disable it.

(From OE-Core rev: c023bc55ff000d1de891d1a8e2a163e94bf63de6)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-14 12:30:49 +00:00
Paul Eggleton 557382ab42 lib/oe/recipeutils: use cooker function instead of bb.providers
We now have a function in cooker itself that can do this lookup;
additionally, the rewritten tinfoil's cooker adapter has its own
implementation that can work remotely, so if we use it then this
function can work in that scenario as well.

(From OE-Core rev: 0a6a4be99c1e4ef3c0da53d63f18ad579545d6a8)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-14 12:30:49 +00:00
Paul Eggleton af867199a5 classes/image: suppress log_check mechanism for warnings/errors logged through BitBake
If you printed a warning through bb.warn() / bbwarn or an error through
bb.error() / bberror, this was also being picked up by our log_check
mechanism that was designed to pick up warnings and errors printed by
other programs used during do_rootfs. This meant you saw not only the
warning or error itself, you saw it a second time through log_check,
which is a bit ugly. Use the just-added BB_TASK_LOGGER to access the
logger and add a handler that we can use to find out if any warning or
error we find in the logs is one we should ignore as it has already been
printed.

Fixes [YOCTO #8223].

(From OE-Core rev: fb37304d27857df3c53c0867e81fbc8899b48089)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-14 12:30:49 +00:00
Paul Eggleton 13916a4fab lib/oe/rootfs: fix log_check warnings being printed twice with RPM packaging
We were calling _log_check() in the RPM-specific rootfs class as well as
in the base class; this is unnecessary and resulted in any errors/warnings
generated during the actual package installation time triggering two warnings
instead of one. Drop the call from RpmRootfs._create() to fix this.

(From OE-Core rev: 541c56d755ba0354297673e857628026ad9e4df2)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-14 12:30:49 +00:00
Ola x Nilsson d65886bb69 oe-pkgdata-util: Make read-value handle override variables
Some variables in pkgdata files have a package-name override.  When
the bare variable can not be found, try with the override-variant.

PKGSIZE is one such variable, and already had special code to handle this.

Test included.

(From OE-Core rev: 6df99cda894033cba68bc6ab91e47f67e0d788a5)

Signed-off-by: Ola x Nilsson <ola.x.nilsson@axis.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-14 12:30:49 +00:00
California Sullivan 840ea755f2 parselogs.py: Don't clog QA with Joule errors
The Joule is very new hardware and there is ongoing kernel and firmware
work to fix these issues, which will be available in future kernel and
firmware releases. In the meantime, don't clog QA reports.

[YOCTO #10611]

(From OE-Core rev: facf9fa905100945738c13f9f79e938ed4a81030)

Signed-off-by: California Sullivan <california.l.sullivan@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-13 22:55:21 +00:00
Patrick Ohly 9eb89487ff buildstats.py: skip collecting unavailable /proc data
Some virtualized environments like Linux-VServer do not have the
entries under /proc that the new system usage sampling expected,
leading to an exception when trying to open the files.

Now the presence of these files is checked once before enabling the
corresponding data collection. When a file is missing, the
corresponding log file is not written either and pybootchart will not
draw the chart that normally displays the data.

Errors while reading or writing of data samples is intentionally still
a fatal error, because that points towards a bigger problem that
should not be ignored.

Reported-by: Andreas Oberritter <obi@opendreambox.org>
(From OE-Core rev: daeee2d6731014c33f0d1f8a3846830c099932b4)

Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-13 22:55:20 +00:00
Ola x Nilsson 2765231499 recipetool: selftest: Add test for recipetool plugin loading
Test that recipetool plugins are loaded in a well defined order.

(From OE-Core rev: 044de8424a454a7057906e44eb56e2134ebb17e4)

Signed-off-by: Ola x Nilsson <ola.x.nilsson@axis.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-13 22:55:20 +00:00
Ola x Nilsson 9170a88cbd devtool: selftest: add test for devtool plugin loading
Test that devtool plugins are loaded in a well defined order.

(From OE-Core rev: 0de81f0c8b29d8b442b3d099c3bec3fd345b6bfe)

Signed-off-by: Ola x Nilsson <ola.x.nilsson@axis.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-13 22:55:20 +00:00
Mariano Lopez 904b915369 oeqa/utils/metadata.py: Add metadata library
Adds functions to get metadata from the host running the tests.

[YOCTO #9954]

(From OE-Core rev: 10b05794254886e55c76f29f7778d783c550befa)

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-13 22:55:19 +00:00
Mariano Lopez 9d55e9d489 oeqa/utils/commands.py: Make a copy of variables in get_bb_vars
The function get_bb_vars will remove items for the list passed
as the function argument, this will leave the caller with an
empty list and the function never says it will consume the items.

This hasn't been found before because only get_bb_var uses this
function.

(From OE-Core rev: 22b7fa24fefcc3974806d1b282c93b8c5880f6a4)

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-13 22:55:19 +00:00
Juro Bystricky 90404e1f24 targetloader.py: drop test for ClassType
ClassType was removed from python3.
The code testing for ClassType kept throwing AttributeError exceptions:

    module 'types' has no attribute 'ClassType'

The exceptions prevented loading of any dynamically resolved target
controllers.

(From OE-Core rev: d62f18c39bc0ed3b0f5ac8465b393c15f2143ecf)

Signed-off-by: Juro Bystricky <juro.bystricky@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-12 15:16:42 +00:00
Stephano Cetola d8fbaebc48 package_manager: remove strings and migrate to direct arrays
When using subprocess call and check_output, it is better to use arrays
rather than strings when possible to avoid whitespace and quoting
problems.

[ YOCTO #9342 ]

(From OE-Core rev: b12cec9a5ef14ecb02be7feec65508cf5d65c795)

Signed-off-by: Stephano Cetola <stephano.cetola@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-08 10:31:30 +00:00
Ed Bartosh 1b32c6ed02 selftest: wic: fix test_qemu
Setting WKS_FILE variable in qemux86-64 made wic test to
use wrong wks file to produce an image and resulted in
test_qemu failure.

Used conditional assignment in qemux86-64 and explicitly
set WKS_FILE in wic testing suite to make the suite to use
wic-image-minimal.wsk. This should fix test_qemu failure.

(From OE-Core rev: 3bca4d18c2712e3b154bacfb917f0a749ebaddeb)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-07 10:38:06 +00:00
Patrick Ohly f1a527c5f6 buildstats: reduce amount of data stored for system utilization
Pre-processing /proc data during the build considerably reduces the
amount of data written to disk: 176KB instead of 4.7MB for a 20
minuted build. Parsing also becomes faster.

The disk monitor log added another 16KB in that example build. The
overall buildstat was 20MB, so the overhead for monitoring system
utilization is small enough that it can be enabled by default.

(From OE-Core rev: b17812385cd55e81066d3ceda92dffdc6e5564da)

Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-07 10:37:59 +00:00
Patrick Ohly 25a5536a6e buildstats: record disk space usage
Hooks into the new monitordisk.py event and records the used space for
each volume. That is probably the only relevant value when it comes to
visualizing the build and recording more would only increase disk
usage.

(From OE-Core rev: 21a5b569370f47cc02291e1d8b76fe43faa04ea6)

Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-07 10:37:59 +00:00
Patrick Ohly 8f475b78c9 buildstats: add system state sampling
/proc/[diskstats|meminfo|stat] get sampled and written to the same
proc_<filename>.log files as during normal bootchat logging. This will
allow rendering the CPU, disk and memory usage charts.

Right now sampling happens once a second, triggered by the heartbeat
event.That produces quite a bit of data for long builds, which will be
addressed in a separate commit by storing the data in a more compact
form.

(From OE-Core rev: 6f4e8180b5b4857eaf6caf410fd3a4a41ed85930)

Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-07 10:37:59 +00:00
Jair Gonzalez 5956492c20 parselogs: Whitelist GPT warnings as the device is fully functional
The warning occurs when the GPT image is not the same size than the
media into which it's being flashed, causing the backup GPT table
not being at the end of the disk. However, this is expected as the
image is created before having the information about the destination
media. The error is harmless, so it will be whitelisted.

Fixes [YOCTO 10481].

(From OE-Core rev: 5cc5cdc788308a79f8f0706e6d794c602ef427ed)

Signed-off-by: Jair Gonzalez <jair.de.jesus.gonzalez.plascencia@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-07 10:37:59 +00:00
California Sullivan 23d2a7d156 parselogs.py: Whitelist iwlwifi firmware load error messages
The iwlwifi module of any given kernel has a minimum and maximum
supported firmware version. The kernel begins by attempting to load the
maximum version, and decrements until it is successful. The 4.8 kernel's
maximum supported firmware version is 24, but thus far only 22 has been
released, meaning we get errors for 24 and 23.

Filter out iwlwifi firmware load error messages, as they are not
necessarily indicative of real problems.

(From OE-Core rev: 7df570c2310efac8f9898da15deaac2b7df16655)

Signed-off-by: California Sullivan <california.l.sullivan@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-07 10:37:59 +00:00
Martin Vuille b7deb1f046 terminal.py: Pass string instead of bytes to ExecutionError to avoid exception
Based on run() in bitbake/lib/bb/process.py, ExecutionError() expects strings
not bytes. Passing bytes results in a "TypeError: Can't convert 'bytes' object
to str implicitly" exception.

Fixes Bug 10729

(From OE-Core rev: 063b63d4d324c23322ac1b6b7c7928e725d7b968)

Signed-off-by: Martin Vuille <jpmv27@yahoo.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-11-30 15:48:08 +00:00
Jose Perez Carranza b81477d329 runtime: Update test cases numbers for runtime tests
Update test case numbers on runtime tests to do match
with templates defined on Testopia for 2.3 release

(From OE-Core rev: d9df762b4c62b74f6d3a1521642ea86c26793a22)

Signed-off-by: Jose Perez Carranza <jose.perez.carranza@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-11-30 15:48:08 +00:00
Michael Wood cfdc5b853e oe-selftest: toaster Remove redundant Toaster test
This test has been ported to be run as part of Toaster's own tests.

(From OE-Core rev: f6366781a34dbdb8ec2d73f4fb36359bc15c4f42)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-11-30 15:48:06 +00:00
California Sullivan f3c9e66e8d parselogs: Whitelist NUC6 firmware load error message on genericx86-64
This was already whitelisted, but the 4.8 kernel changed the error
message, causing it to get caught by parselogs again.

Fixes [YOCTO #10494].

(From OE-Core rev: e1bad14231115f3b1a2bf844ef5b2022c648b55d)

Signed-off-by: California Sullivan <california.l.sullivan@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-11-23 11:10:14 +00:00
Joshua Lock 333d300fba lib/oe/lsb: better handle missing fields
Some rolling release distros, such as Arch Linux, don't include a
VERSION_ID field in their os-release file.

Change release_dict_osr() to better handle this optional field
being absent.

Further improve the resilience of the release_dict_*() methods by
always returning a dict and using dict.get() in distro_identifier()
to supply a default, empty string, value when then key is missing.

(From OE-Core rev: e36066dcc3b56cac1c695370ea178b566c0ebfd6)

Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-11-23 11:10:14 +00:00
Paul Eggleton 6e182aaa76 devtool: modify: support recipes with only local files as source
The hello-mod recipe is unusual in that it has only local files in
SRC_URI and builds these out of ${WORKDIR}. When you use devtool modify
on it, devtool puts all of those files in an "oe-local-files"
subdirectory of the source tree, which is not ${S} (or ${B}) any more
and thus building the recipe afterwards fails. It's a bit of a hack, but
symlink the files in oe-local-files into the source tree (and commit the
symlinks with an ignored commit so that the repo is clean) to work
around the problem. We only do this at time of extraction, so any files
added to or removed from oe-local-files after that won't be handled, but
I think there's a limit to how far we should go to support these kinds
of recipes - ultimately they are anomalies.

I initially tried a hacky workaround where I set effectively set B =
"${WORKDIR}" and that allowed it to build, but other things such as the
LIC_FILES_CHKSUM checks still broke because they expected to find files
in ${S}. Another hack where I set the sourcetree to point to the
oe-local-files subdirectory works for hello-mod but not for makedevs
since whilst that is similar, unlike hello-mod it does in fact have
files in the source tree (since it has a patch that adds COPYING) and
thus the same issue occurred.

Also tweak one of the tests that tries devtool modify / update-recipe on
the makedevs recipe to try building it since that would have caught this
issue.

Fixes [YOCTO #10616].

(From OE-Core rev: 857c06d6a1d161bf5a01311d07758bd4241929a3)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-11-23 11:10:14 +00:00
Stephano Cetola 93e8db2224 devshell: list commands when throwing NoSupportedTerminals
When attempting to run devshell, if no terminal is available, the
error being thrown was not very specific. This adds a list of
commands that failed, informing the user of what they can install to
fix the error.

[ YOCTO #10472]

(From OE-Core rev: c077f4aab2fc956408d4ad45c4e2e2ea6e480624)

Signed-off-by: Stephano Cetola <stephano.cetola@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-11-23 11:10:13 +00:00
Paul Eggleton 39fba4d70f oe-selftest: devtool: test that updating a file with subdir= works
If you have a file:// entry in SRC_URI with a subdir= parameter that
makes it extract into the source tree, then when you update that file in
oe-local-files and run devtool update-recipe then you want the original
file to be updated. This was made to work by OE-Core commit
9069fef5dad5a873c8a8f720f7bcbc7625556309 together with
31f1bbad248c36a8c86dde4ff57ce42efc664082, however until now there was no
oe-selftest test to verify it.

Note that in order to succeed this test also requires the fix
"lib/oe/recipeutils: ignore archives by default in
get_recipe_local_files()" since the test recipe uses a local tarball.

(From OE-Core rev: 936eba3e1059d1dcd5e58c1ce76870fff7b11b3c)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-11-23 11:10:13 +00:00
Paul Eggleton e3193b76e0 lib/oe/recipeutils: ignore archives by default in get_recipe_local_files()
By default, have get_recipe_local_files() not return any archive
files. This prevents a local tarball from being erroneously removed
from SRC_URI if you run "devtool modify" on a recipe followed by
"devtool update-recipe". It doesn't actually help you to directly
update the contents of such tarballs, but at least now it won't break
the recipe.

(From OE-Core rev: e9c418d4704c1bed4c5880e176e5288485f4f5a6)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-11-23 11:10:13 +00:00
Paul Eggleton 55a157f4e6 devtool: update-recipe: fix handling of compressed local patches
It is possible to use gzip or bzip2 to compress patches and still refer
to them in compressed form in the SRC_URI value within a recipe. If you
run "devtool modify" on such a recipe, make changes to the commit for
the patch and then run devtool update-recipe, we need to correctly
associate the commit back to the compressed patch file and re-compress
the patch, neither of which we were doing previously.

Additionally, add an oe-selftest test to ensure this doesn't regress in
future.

Fixes [YOCTO #8278].

(From OE-Core rev: e47d21624dfec6f71742b837e91da553f18a28c5)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-11-23 11:10:13 +00:00
Paul Eggleton d316363b7b lib/oe/patch: fix handling of patches with no header
If a patch applied by a recipe has no header and we turn the recipe's
source into a git tree (when PATCHTOOL = "git" or when using devtool
extract / modify / upgrade), the commit message ends up consisting only
of the original filename marker ("%% original patch: filename.patch").
When we come to do turn the commits back into a set of patches in
extractPatches(), this first line ends up in the "Subject: " part of
the file, but we were ignoring it because the line didn't start with the
marker text. The end result was we weren't able to get the original
patch name. Strip off any "Subject [PATCH x/y]" part before looking for
the marker text to fix.

This caused "devtool modify openssl" followed by "devtool update-recipe
openssl" (without any changes in-between) to remove version-script.patch
because that patch has no header and we weren't able to determine the
original filename.

(From OE-Core rev: d9971f5dc8eb7de551fd6f5e058fd24770ef5d78)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-11-23 11:10:12 +00:00
Paul Eggleton e5a391795a oe-selftest: devtool: test update-recipe with only local files
Add a test to ensure devtool update-recipe works properly on recipes
that contain only local files (since the other tests we have didn't test
that).

Relates to [YOCTO #10563].

(From OE-Core rev: bdc844b3f0c3fbddcd3523095899a5bd29797704)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-11-23 11:10:12 +00:00
Linus Wallgren b91e47fccb lib/oe/package_manager: .deb pre/postinst args
The debian policy manual and MaintainerScripts wiki page states that the
postinst script is supposed to be called with the `configure` argument
at first install, likewise the preinst script is supposed to be called
with the `install` argument on first install.

https://www.debian.org/doc/debian-policy/ch-maintainerscripts.html
https://wiki.debian.org/MaintainerScripts

(From OE-Core rev: 3d9c3aae54589794ce3484fa1b21d1af2bd32661)

Signed-off-by: Linus Wallgren <linus.wallgren@scypho.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-11-23 11:10:11 +00:00
Joshua Lock 81afedc905 lib/oe/lsb: attempt to ensure consistent distro id regardless of source
The LSB Distributor ID and os-release NAME differ for most of the
distributions tested by the Yocto Project (CentOS, Debian, Fedora,
openSUSE and Ubuntu) however for all but openSUSE the os-release ID
matches the LSB Distributor ID when both are lowered before
comparison.

Therefore, in order to improve the consistency of identification of
a distribution, switch to using the os-release ID and converting
the ID value to lowercase.

Table showing comparison of LSB Distributor ID to os-release fields NAME
and ID for current Yocto Project supported host distributions:

Distribution | Version | Distributor ID   | NAME             | ID       |
-------------------------------------------------------------------------
CentOS       | 7       | CentOS           | CentOS Linux     | centos   |
Debian       | 8       | Debian           | Debian GNU/Linux | debian   |
Fedora       | 23      | Fedora           | Fedora           | fedora   |
Fedora       | 24      | Fedora           | Fedora           | fedora   |
openSUSE     | 13.2    | openSUSE project | openSUSE         | opensuse |
openSUSE     | 42.1    | SUSE LINUX       | openSUSE Leap    | opensuse |
Ubuntu       | 14.04   | Ubuntu           | Ubuntu           | ubuntu   |
Ubuntu       | 16.04   | Ubuntu           | Ubuntu           | ubuntu   |

[YOCTO #10591]

(From OE-Core rev: 8689e5618d45c2119134ea64754430c06a93ea09)

Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-11-15 15:19:54 +00:00
Joshua Lock 42441ea481 lib/oe/lsb: prefer /etc/os-release for distribution data
os-release(5) is an increasingly standard source of operating system
identification and more likely to be present on modern OS deployments, i.e.
many container variants of common distros include os-release and not the
lsb_release tool.

Therefore we should favour parsing /etc/os-release in distro_identifier(),
try lsb_release when that fails and finally fall back on various distro
specific sources of OS identification.

(From OE-Core rev: fc4eddecddec68d03a985086fa32db40ad0c7bfc)

Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-11-15 15:19:54 +00:00
Joshua Lock 545f5f96d9 lib/oe/lsb: make the release dict keys consistent regardless of source
Rather than have the distro_identifier method look for different keys in
the dict depending on the source ensure that each function for retrieving
release data uses the same key names in the returned dict.

(From OE-Core rev: 2ddd6ddaf0c5ba14ae83347eba877ac9ef179c76)

Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-11-15 15:19:54 +00:00