Commit Graph

775 Commits

Author SHA1 Message Date
Ross Burton bb42984277 oeqa/recipetool: update recipe test to pass SHA
(From OE-Core rev: 71dd4c05c41e8b363dc1ecac1f5105d316ee82dc)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-20 10:28:53 +01:00
Ross Burton 738f47a2d9 oeqa/devtool: update recipe test as libmatchbox changed
(From OE-Core rev: b36712eef14c20007e0adb01cc7d4bce9e7926bb)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-20 10:28:53 +01:00
Jussi Kukkonen 096b4c639b libxsettings-client: Remove as unneeded
xsettings-client is not meant to be a shared library and the only
user in oe-core (libmatchbox) now has an in-tree copy.

(From OE-Core rev: ed5cbf977dca8f3d85c8d933071c6e49fb4bd95f)

Signed-off-by: Jussi Kukkonen <jussi.kukkonen@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-20 10:28:50 +01:00
Ed Bartosh 05084640e6 wic: oe-selftest: build bmap-tools-native
bmap-tools-native is required to generate .bmap file. Without it
wic fails with this error when run with --bmap option:
    Error: A native program bmaptool required to build
           the image was not found

Added bmap-tools-native to the list of requirements to build
in Wic.setUpLocal method.

(From OE-Core rev: dd11ab551f9bf1d8761d0430f19510a9caa53a12)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-20 10:28:50 +01:00
Paul Eggleton d62fd7711d devtool: add finish subcommand
Add a subcommand which will "finish" the work on a recipe. This is
effectively the same as update-recipe followed by reset, except that the
destination layer is required and it will do the right thing depending
on the situation - if the recipe file itself is in the workspace (e.g.
as a result of devtool add), the recipe file and any associated files
will be moved to the destination layer; or if the destination layer is
the one containing the original recipe, the recipe will be overwritten;
otherwise a bbappend will be created to apply the changes. In all cases
the layer path can be loosely specified - it could be a layer name, or
a partial path into a recipe. In the case of upgrades, devtool finish
will also take care of deleting the old recipe.

This avoids the user having to figure out the correct actions when
they're done - they just do "devtool finish recipename layername" and
it saves their work and then removes the recipe from the workspace.

Addresses [YOCTO #8594].

(From OE-Core rev: fa550fcb9333d59b28fc0e4aebde888831410f5c)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-20 10:28:49 +01:00
Aníbal Limón ad6aaae4bf utils/qemurunner.py: QemuRunner.start() add support for specify extra kernel cmdline
Add ability to specify extra_bootargs (kernel cmdline) in order to enable systemd
debug log in images that enables systemd init.

[YOCTO #9299]

(From OE-Core rev: 09d62551c289b5607341a4f9c46eecd6390ad774)

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-20 10:28:47 +01:00
Ross Burton 38d2c59ab8 remove sukoku
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-20 10:28:44 +01:00
Ross Burton 9c4b143607 oeqa/sdk: remove buildsudoku test
This application needs GTK+ 2 which we're trying to move away from for Sato
images.

(From OE-Core rev: 5e52f208a17e7122209675fa96de6e7807c62feb)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-20 10:28:44 +01:00
Maxin B. John 78523811fb buildoptions.py: remove directfb image build test
removing this test since we move directfb out of oe-core

(From OE-Core rev: 2d8fda36ecfa1945f22b7139a2febd12ec59272b)

Signed-off-by: Maxin B. John <maxin.john@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-10 14:12:17 +01:00
Paul Eggleton a14d30f7de oe-selftest: drop test_prepare_unexpected
This test refers to a function that no longer exists after the eSDK
install double execution of bitbake has been removed, and since
test_prepare_unexpected is the only test in this module, drop the
entire module. We can easily resurrect it if we have unit tests to add
in the future.

(From OE-Core rev: 7e792a22e62904ed2dafb1ea214911235e3f3efc)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-08 10:29:44 +01:00
Richard Purdie 1bbd580a37 oeqa/selftest/bbtests: Fix failing test after progress changes
The progress patches change the output slightly, update the test to
deal with this.

(From OE-Core rev: 90dbd838fa97c89ace5cb147aa5cff39b94178b1)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-08 09:57:26 +01:00
Ross Burton c7147ceec8 oeqa/runtime/systemd: fix typo in skipUnlessPassed decorator
(From OE-Core rev: f0420d8c934154a94957cabcfdc993f0eae92cd7)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-08 09:57:26 +01:00
Ross Burton 5c11e365e1 lib/oeqa: add Galculator to SDK and runtime tests
This is a GTK+3 application, so we don't need to ship GTK+ 2 in Sato just for
the SDK test suite.

[ YOCTO #9780 ]

(From OE-Core rev: 53fcfe4348a2ca727844f2b0bd3fca2902cbdda0)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-01 16:22:48 +01:00
Ross Burton b305564028 oeqa/runtime/syslog: remove redundant skipUnlessPassed
The test test_syslog_help doesn't exist, so skipping unless it passed just
produced a warning.

(From OE-Core rev: 9856fe02bf30cd9e834a7d324e357070f4d7735f)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-01 16:22:48 +01:00
Ross Burton 076a93eed7 oeqa/utils/commands: fix single-character variable matching in get_bb_vars()
(From OE-Core rev: 2f7e4d0636ce1647edf2d82c65b82be0eb2c1db1)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-01 16:22:48 +01:00
Markus Lehtonen eb36f00002 oe-build-perf-test: implement --globalres-file option
Using this option the script appends test results into a 'global results
file'. A CSV-formatted output of the results. This option is to provide
compatibility with the old build-perf-test.sh.

(From OE-Core rev: e9f18e63220e452f2b0c878998e57d944ae83980)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-01 16:22:47 +01:00
Markus Lehtonen 8c596369dd oeqa.buildperf: add git revision and branch to result data
BuildPerfTestRunner determines these from the Git repository under which
it is being run (i.e. where the build directory exists). The branch and
revision may be defined/overridden with OE_BUILDPERFTEST_GIT_BRANCH
and OE_BUILDPERFTEST_GIT_BRANCH environment variables, if needed. This
makes it possible to run the build performance test script even if the
top directory is not a git repository clone, for example.

(From OE-Core rev: e6004582454d8c6a18f617c12e6e408ded5be8df)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-01 16:22:47 +01:00
Markus Lehtonen 964fffa514 oeqa.utils: add git module
A new helper module for easier interaction with Git repositories.
Provides GitRepo class that represents one local Git repository clone.
The GitRepo class currently only has one method, run_cmd(), for running
arbitrary git commands in the repository. More specialized methods for
commonly used git operations can be added later.

(From OE-Core rev: 4eaf434f885afbda03fe67ab6e9ff291c7a9c77e)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-01 16:22:47 +01:00
Markus Lehtonen 5599645d33 oeqa.buildperf: archive build/conf into test results
Have the build/conf directory as part of test results.

(From OE-Core rev: ea32a8a65f668450e0ec0e80483bd5e8c91b8a18)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-01 16:22:47 +01:00
Markus Lehtonen da3b9242fc oeqa.buildperf: add test Test4
Re-implement "test4" from build-perf-test.sh which measures eSDK metrics.

(From OE-Core rev: c449ca1b062426380a0572aed2da4ee4a83281f5)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-01 16:22:47 +01:00
Markus Lehtonen fe24b5a597 oeqa.buildperf: add test Test3
Re-implement "test3" from build-perf-test.sh which measures
bitbake parsing time.

(From OE-Core rev: 388c7160b90247df9c78d8803083821402c13af5)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-01 16:22:47 +01:00
Markus Lehtonen 0b2fae81fd oeqa.buildperf: add test Test2
Re-implement "test2" from build-perf-test.sh which measures
'bitbake core-image-sato -c rootfs'.

(From OE-Core rev: 8cb322d6a5be08f0bb2c9e5d3b73161b20d737a3)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-01 16:22:47 +01:00
Markus Lehtonen 1701a34f14 oeqa.buildperf: add test Test1P3
Re-implement "test1_p3" from build-perf-test.sh which measures
'bitbake core-image-sato' with rm_work enabled.

(From OE-Core rev: bfe48d343e8a281760da0b6744c08d47039f93e0)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-01 16:22:47 +01:00
Markus Lehtonen b5ca437714 oeqa.buildperf: add test Test1P2
Re-implement "test1_p2" from build-perf-test.sh which measures
'bitbake virtual/kernel'.

(From OE-Core rev: 5ff82c6e2b0ff41a9bfdd49c99e84a982cf6e467)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-01 16:22:47 +01:00
Markus Lehtonen 2c1b87482c oeqa.buildperf: add test Test1P1
Re-implement "test1_p1" from build-perf-test.sh which measures
'bitbake core-image-sato'.

(From OE-Core rev: 1ddd4a8ce4e2053def0e946d4e4b4b97bdc28e61)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-01 16:22:47 +01:00
Markus Lehtonen 1d88659ef6 oeqa.buildperf: implement BuildPerfTestRunner class
The new class is responsible for actually running the tests and
processing their results. This commit also adds a decorator function for
adding new tests. No automatic test discovery, at least yet.

(From OE-Core rev: bf90aecb7e150d6bfac7240286c797b79d26528b)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-01 16:22:46 +01:00
Markus Lehtonen 6512d6956b oeqa.buildperf: add method for saving buildstats
(From OE-Core rev: 15b44a484ce408b0a1acfb907e2b12ebc6cce296)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-01 16:22:46 +01:00
Markus Lehtonen 45c6a04a37 oeqa.buildperf: add method for measuring file disk usage
Add a new method to BuildPerfTest class for measuring the disk usage of
a file of directory.

(From OE-Core rev: 85cdc240e75d481e93238fbf75f8b8431da05f19)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-01 16:22:46 +01:00
Markus Lehtonen 1a0e20546e oeqa.buildperf: add method to log shell commands
Add new methods to BuildPerfTest class for running a shell
command and logging its output.

(From OE-Core rev: 8f0b11ba1266f9c650cf34d9b394d72009ee7207)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-01 16:22:46 +01:00
Markus Lehtonen 6e27b2ae0e oeqa.buildperf: method for measuring system resource usage
Extend BuildPerfTest class with a new method for measuring the system
resource usage of a shell command to BuildPerfTest class.  For now,
easurement of the elapsed time is done with the Gnu time utility,
similarly to the build-perf-test.sh shell script. And, it currently only
records the elapsed (wall clock).

The measured values (currently, only the elapsed time) is actually a
dictionary, making it possible to extend it with additional resource
values, e.g. cpu time or i/o usage, in the future.  In addition to the
actual values of the measurement each record contains a 'name' and
'legend' where name is supposed to function as a common key or id over
test runs, making comparison and trending easier, for example. Legend is
supposed to be a short human readable description.

(From OE-Core rev: ced156bfea4a6649d201f41275641a633f218322)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-01 16:22:46 +01:00
Markus Lehtonen 1b10ded015 oeqa.buildperf: add BuildPerfTest class
The new class will be used as an abstract base class for build
performance tests. This implementation contains some common
functionality used in multiple tests, "copied" from the
build-perf-test.sh shell script.

(From OE-Core rev: 35cd7363759a286e80ddca0d028db3d2bf524b17)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-01 16:22:46 +01:00
Markus Lehtonen cee685ca0d oeqa.buildperf: functionality to drop kernel caches
Add a new utility class for dropping Linux kernel caches.  It uses sudo
and tee to write to the drop_caches file. Checking if the user has the
permissions to drop caches (without a password) is done by trying to
writing an invalid value to the drop_caches file. This way, we will find
if writing (with tee) is possible but not really dropping caches, yet.

User can avoid giving the password by adding something like:
<user> ALL = NOPASSWD: /usr/bin/tee /proc/sys/vm/drop_caches
to the system sudoers file.

(From OE-Core rev: c9cb248429ced50c96d11ba5361c272d4c9b9323)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-01 16:22:46 +01:00
Markus Lehtonen 7d42ceae3a oe-build-perf-test: introduce oeqa.buildperf module
Wireframe of a new Python module for containing build performance tests
and utility functions.

(From OE-Core rev: b44af9051a9da8d2cddc7b071a896331967f15cc)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-01 16:22:46 +01:00
Markus Lehtonen ec8be2039e oeqa.utils.commands: runCmd: return stderr output, too
Useful if one wants to separate stdout and stderr.

(From OE-Core rev: de9744c91a997a5ab0e7a19dbe13d8def8d62800)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-01 16:22:46 +01:00
Markus Lehtonen cf4c8980d7 oeqa.utils.commands: use get_bb_vars() in get_bb_var()
Get rid of duplicate code.

(From OE-Core rev: cdd6b7386afd460337705d8117a4328d4993ecef)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-01 16:22:45 +01:00
Markus Lehtonen c9639e6523 oeqa.utils.commands: Introduce get_bb_vars()
A new function for getting values of multiple bitbake variables at the
same time.

(From OE-Core rev: fe3039322e2f846b336ac5af5177e9da27d79695)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-01 16:22:45 +01:00
Ross Burton 1960abc96c oeqa/targetbuild: use os.path.join instead of string concatenation
(From OE-Core rev: 1d40f6e3d85d0cd3be6dd784677686ed4dec89c4)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-06-23 14:26:16 +01:00
Ross Burton 40268fe27a oeqa/targetbuild: add extra_cmds argument to run_configure
Some upstreams need more than just gnu-configize ran before ./configure works,
such as ./autogen.sh or autoreconf.  Add extra_args (defaulting to
gnu-configize) so that this can be done in test cases.

(From OE-Core rev: 7096f2889f1623ce97a6696f6f4c7217f0efb972)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-06-23 14:26:16 +01:00
Ross Burton 0aa028f5a9 oeqa/targetbuild: fix folder name estimate logic
The re.sub() used to transform a tarball into a best guess folder name wasn't
right, as there isn't enough escaping and tar.xz was missing.

(From OE-Core rev: ac99135b009a1066486ed2afb2f298d0a5a3a854)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-06-23 14:26:16 +01:00
Ross Burton 675843d540 oeqa: fix hasPackage, add hasPackageMatch
hasPackage() was looking for the string provided as an RE substring in the
manifest, which resulted in a large number of false positives (i.e. libgtkfoo
would match "gtk+").

Rewrite the manifest loader to parse the files into a proper data structure,
change hasPackage to do full string matches, and add hasPackageMatch which does
RE substring matches.

(From OE-Core rev: b9409863af71899e02275439949e3f4cdfaf2d0f)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-06-23 14:26:14 +01:00
Mariano Lopez 4a842cc952 selftest/testexport.py: Add test test_testexport_basic
Exported tests is not been tested right now and it get
broken very easily with import errors from bitbake.

This adds a new test in order to verify in the CI if
the testexport gets broken.

[YOCTO #8384]

(From OE-Core rev: b6e0a762f8a17ee323cef02934f9a624faf706ed)

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-06-07 21:36:20 +01:00
Mariano Lopez 120f7067c8 testexport.bbclass: Add support for testexport-tarball
Add support to export the SDK tarball needed when a test
system doesn't have the required software to perform runtime
tests.

The support is when exporting the test and when running
the test on a remote system. The user of this feature just
need to set TEST_EXPORT_SDK_ENABLED to "1" and declare
the sdk packages in TEST_EXPORT_SDK_PACKAGES.

[YOCTO #7850]

(From OE-Core rev: a6041f81b81baa7564e4c712fc88de2b997e52e4)

(From OE-Core rev: 05e6c89f0f71311f8bd32cdb86a2deb789c58035)

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-06-07 15:22:38 +01:00
Mariano Lopez 938f05333b oetest.py: Add install/uninstall functionality for DUTs
Add the functionality to install/unistall packages in the
DUTs without the use of the package manager. This is possible
with the extraction introduced in package manager class.

testimage and testexport bbclasses has been modified in order
to support this new feature.

[YOCTO #8694]

(From OE-Core rev: b7111d9e9d64d21f57729d1ac1865aea6e54cc8b)

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-06-07 15:22:38 +01:00
Mariano Lopez d9b3ee8b38 lib/oeqa/otest.py: Fix import tests from other layers with python3
In python3 the functionality to import modules has been changed and
this broke the capability to add runtime tests from other layers.

This commit returns this capability to testimage and testexport.

[YOCTO #9705]

(From OE-Core rev: a26f23d3ce8f7e9f59dbc9bf27516377fd7a0a6d)

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-06-07 15:22:38 +01:00
Mariano Lopez 91a7f54e08 testexport.bbclass: Update to use python3
Remove deprecated attributes in order to use python3.

runexported was changed to use python3.

[YOCTO #9702]

(From OE-Core rev: 9129af6dc421455c0253be313bf5781b913dc5fd)

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-06-03 13:13:30 +01:00
Leonardo Sandoval 4f8959324d oeqa/selftest/recipetool.py: Tweak matching of warning line
We are observing cases (see below) where the 'WARNING:' prefix is not at the
beginning of a line, so instead of expecting it in the beginning, match it
within the string.

    ======================================================================
    FAIL: test_recipetool_appendfile_patch (oeqa.selftest.recipetool.RecipetoolTests)
    ----------------------------------------------------------------------
    Traceback (most recent call last):
      File "/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-oe-selftest/build/meta/lib/oeqa/utils/decorators.py", line 109, in wrapped_f
        return func(*args, **kwargs)
      File "/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-oe-selftest/build/meta/lib/oeqa/selftest/recipetool.py", line 285, in test_recipetool_appendfile_patch
        self.fail('Patch warning not found in output:\n%s' % output)
    AssertionError: Patch warning not found in output:
    Parsing recipes..WARNING: File /etc/selftest-replaceme-patched is added by the patch /home/pokybuild/yocto-autobuilder/yocto-worker/nightly-oe-selftest/build/meta-selftest/recipes-test/recipetool/files/add-file.patch - you may need to remove or replace this patch in order to replace the file.
    NOTE: Writing append file /tmp/recipetoolqaug0kdb4x/recipes-test/recipetool/selftest-recipetool-appendfile.bbappend
    NOTE: Copying /tmp/recipetoolqagci9tita/testfile to /tmp/recipetoolqaug0kdb4x/recipes-test/recipetool/selftest-recipetool-appendfile/testfile
    done.

(From OE-Core rev: 2289138bdaa17b764821f41a3b3fd0a01cda7440)

Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-06-02 08:24:02 +01:00
Richard Purdie 440c681646 qemurunner: Use surrogateescape decoding
Since the stream can contain invalid binary characters (e.g. from
ppc's bootloader) use surrogateescape decoding to ensure we do process
the character stream, else it can hang/timeout.

(From OE-Core rev: 28a0030430d4cfcaf5dfc3e71bda07cdbfbbf4a7)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-06-02 08:24:02 +01:00
Leonardo Sandoval 76aefc4c15 devtool.py: Fix parsing of bitbake-layers' output
Current parsing was picking wrong targets, leading to the following problem:

	AssertionError: Command 'bitbake  Parsing recipes..done. -e' returned non-zero exit status 1:

(From OE-Core rev: eaf83a58825d91c7445835b27d843da7532c208b)

Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-06-02 08:24:02 +01:00
Leonardo Sandoval 89d40a1f93 selftest/devtool: Compare sets instead of arrays on AssertEqual
Sets are safer when comparing internal elements and positions are not
important. This commit avoid errors observed on python3 builds as reported
on the below bugzilla entry.

[YOCTO #9661]

(From OE-Core rev: f6df164d09a4d4cf58977bf6cc0bc4f4bc71183a)

Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-06-02 08:24:01 +01:00
Richard Purdie 3b39971748 classes/lib: Complete transition to python3
This patch contains all the other misc pieces of the transition to
python3 which didn't make sense to be broken into individual patches.

(From OE-Core rev: fcd6b38bab8517d83e1ed48eef1bca9a9a190f57)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-06-02 08:24:01 +01:00
Ed Bartosh 63404baadb qemurunner: convert data when working with socket
Converted str to bytes before sending to socket.
Converted bytes to str after receiving from socket.

This should fix TypeError: 'str' does not support the buffer interface
for qemurunner.run_serial method.

(From OE-Core rev: 210e290c9251839dc74e3aabdcea3655dd707a50)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-06-02 08:24:01 +01:00
Richard Purdie 44e9a0d2fa classes/lib: Update to explictly create lists where needed
Iterators now return views, not lists in python3. Where we need
lists, handle this explicitly.

(From OE-Core rev: caebd862bac7eed725e0f0321bf50793671b5312)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-06-02 08:24:00 +01:00
Richard Purdie 8587bce564 classes/lib: Update to match python3 iter requirements
python3 standardises its use of iteration operations. Update
the code to match the for python3 requires.

(From OE-Core rev: 2476bdcbef591e951d11d57d53f1315848758571)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-06-02 08:24:00 +01:00
Richard Purdie a7309d5790 classes/lib: Update to use python3 command pipeline decoding
In python3, strings are unicode by default. We need to encode/decode
from command pipelines and other places where we interface with the
real world using the correct locales. This patch updates various
call sites to use the correct encoding/decodings.

(From OE-Core rev: bb4685af1bffe17b3aa92a6d21398f38a44ea874)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-06-02 08:24:00 +01:00
Paul Eggleton 49557a5e9d recipetool / devtool: set a fixed SRCREV by default when fetching from git
If fetching source from a git repository, typically within OpenEmbedded
we encourage setting SRCREV to a fixed revision, so change to do that by
default and add a  -a/--autorev option to use "${AUTOREV}" instead.

(From OE-Core rev: 000480c42797dd2f03ebc3bc6d1dabfc6a7b75f5)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-06-01 12:38:41 +01:00
Mariano Lopez d56ccf691b oetest.py: Add support to copy unextracted packages for runtime testing
Sometimes is needed to have a package without extraction when
running a test. This patch adds the functionality.

[YOCTO #8536]

(From OE-Core rev: 49234fe926224c21ef6c8292132620b4716c5263)

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-05-30 15:58:13 +01:00
Mariano Lopez 92afca7181 oetest.py: Add extract_packages() to RuntimeTestContext class
This new method extracts the content of package (RPM, DEB, or IPK)
to a directory inside of WORKDIR. The extraction is needed for later
install in the DUTs without using a package manager.

[YOCTO #8694]

(From OE-Core rev: 90d585f59f217f23694a9b02a73b79d18dfdb579)

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-05-30 15:58:13 +01:00
Mariano Lopez 8a68910958 oetest.py: Add json file support to specify packages needed in runtime tests
This adds the functionality to use a json file to
specify the packages needed for a particular test.

The content of the json file is a dictionary with
dictionaries inside, using the test name as the hash.

The json file must have the same name as the class
module name and must be in the same path.

[YOCTO #7850]

(From OE-Core rev: 1f24ef9a339a2ad34e010f39aa93abdc8d085c85)

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-05-30 15:58:13 +01:00
Mariano Lopez 6a7138ebb6 oeqa/utils/package_manager.py: Add get_package_manager()
This new file just contain one function to return an OE package
manager, this depends in the current packaging method.

[YOCTO #8694]

(From OE-Core rev: 9b4b150ccbcd2f7a0b27d083eb406578584af3fa)

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-05-30 15:58:11 +01:00
Mariano Lopez daf0cc3745 oetest.py: Move getTests() outside loadTests() method
The method getTests() can be useful to all the class,
not just to loadTests().

[YOCTO #8694]

(From OE-Core rev: 667a4549bd51a6e4b487006f4cc1542d2961757a)

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-05-30 15:58:11 +01:00
Richard Purdie 3ddde5f32a meta: Update to modern exception syntax
Update older exception syntax to modern one required by python 3.
Compatible with python 2.7.

(From OE-Core rev: d13f0ac614f1d1e2ef2c8ddc71cbfcf76a8dc3f2)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-05-21 22:26:33 +01:00
Richard Purdie ffe9604be2 meta/selftest/scripts: Use print function for python3 compatibility
Used print function instead of print statement to make
the code work in python 3.

[Changes from both Ed and Richard]

(From OE-Core rev: ced1995694c394d92cb82fb9c25a33dc027a3b69)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-05-21 22:26:32 +01:00
Paul Eggleton 02986886fc devtool: sdk-update: drop support for local updates
Having two code paths here makes maintenance difficult, and it doesn't
seem likely that you would use the local case in real usage anyway, so
drop the local support entirely.

This should allow us to resolve [YOCTO #9301].

(From OE-Core rev: 7a4c9c96fee4fb514c2b69b52e811c4f896a16f1)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-05-21 22:24:57 +01:00
Ed Bartosh 978d450a52 oe-selftest: wic: add test_bmap test case
Tested generation of .bmap file using wic --bmap command
line option.

(From OE-Core rev: 1f21c46e517685ff1363d3b6f4fd840300b1930b)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-05-19 09:05:19 +01:00
Mariano Lopez b4c23e9e95 oetest.py: Use the real ExportTestContext in exported tests
There are parts of the fuctionallity missing when using the
dummy ExportTestContext class in runexported.py.

This changes the use of ExportTestContext dummy class from
runexported.py to the real class in oetest.py.

(From OE-Core rev: b0f105eecf25d334dd5fd0d936ec98cb8430ed23)

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-05-19 09:05:18 +01:00
Mariano Lopez 86d65d79ad oetest.py: Add default pscmd to oeTest
pscmd is used by some tests to get the process
running on the target. If the test are exported
there won't be any pscmd attibute in the oeTest.

This adds "ps" as default pscmd.

(From OE-Core rev: 44aa8c3b6747179a0c4c156fc4922d838cdc19a4)

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-05-19 09:05:18 +01:00
Mariano Lopez 41fbf2c353 oeqa/runexported.py: Remove host dumper
The host dumper is used to get information about the host
running the test when a test fails. This is used for the
autobuilders of Yocto Project.

Now that exported tests have thier own class the host dumper
is not necessary anymore.

(From OE-Core rev: fab3e4865a356ceb73d7deab5b4fd38ed795ca72)

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-05-19 09:05:18 +01:00
Mariano Lopez a9eb22e3d3 oeqa/oetest.py: Add class ExportTestContext
Adding the class is needed to separate the exported test
from the test image; both test run under different conditions,
i.e. an exported test doesn't require to change the signal
handling.

This change adds clasess ExportTestContext and ImageTestContext,
both of them inherits from RuntimeTestContext. Also refactors
RuntimeTestContext class, to keep the code common in this class.

(From OE-Core rev: ef9618a6e7f99e485cc6b59484d11807475c7823)

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-05-19 09:05:17 +01:00
Richard Purdie 9094ea9fdd classes/oeqa: Update for print statements and file() -> open() for python3
Found some more syntax cleanups needed for python3.

(From OE-Core rev: 1181d86e8707c5b8e8d43d5e785d7d9cf01fa491)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-05-16 09:31:23 +01:00
Richard Purdie ca2edae7c3 oeqa: Print function python3 fixes
Use print functions for comptibility with python3.

(From OE-Core rev: 0c89a8a276b67a9292ee3100003c789126bd9ea9)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-05-16 08:13:11 +01:00
Richard Purdie bb44e5b99d oeqa/decorators: Use wraps consistently
We want the decorator to leave the function names of the test unchanged. Some
decorators are already using wraps for this but not all. Fix this to be consistent
allowing inspection of the test to give the wanted values.

(From OE-Core rev: 9e4d60b29ff5667d23a89953ce7139b34c11d40b)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-05-15 09:06:00 +01:00
Ed Bartosh ff5d6f8854 selftest: add bmap test
Added test_bmap to imagefeatures tests.
It tests if bmap file is generated for the images and
if the image is sparse.

[YOCTO #9414]

(From OE-Core rev: db27d8fbb44d2cdd524ac992630c781fd0c45b1b)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-05-14 23:05:14 +01:00
Richard Purdie 5052bf92e4 oeqa/lic-checksum: Update after recent LIC_FILES_CHKSUM changes
The check now runs at populate_lic time so change this from configure. We
also need to set a SRC_URI so that the check triggers.

(From OE-Core rev: a5970809a2f01dbd152684266e2a3d946d896620)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-05-11 10:33:43 +01:00
Mariano Lopez eb0ab04149 sshcontrol.py: Add methods to copy dirs and delete files
This patch add new methods to SSHControl class. These methods
include:
    - Copy a dir to DUT
    - Delete a file in the DUT
    - Delete a directory in the DUT (if empty)
    - Delete a directory structure in the DUT

[YOCTO #9565]

(From OE-Core rev: f22afb09fefdcb568d79ccd81277a43dacee2a82)

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-05-11 10:33:41 +01:00
Ross Burton 9efd5474ff oeqa/selftest/buildoptions: remove buildhistory signature test
This test is a subset of the new sstate_noop_samesigs test, and less helpful
when it breaks, so remove it.

(From OE-Core rev: 7157261014e1dcbe9a57e7504dbb0ab2a53aa4d8)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-05-06 10:31:15 +01:00
Ross Burton f22380e693 oeqa/sstatetests: remove temporary DL_DIRs in noop_samesigs
(From OE-Core rev: a98acf4840fc4888c0f4b8998a0a3983c639ecc2)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-04-29 07:58:45 +01:00
Ross Burton 19d8760032 oeqa/sstatetests: add http_proxy to no-op hash test
Add two values for http_proxy to verify that changing it doesn't change any
unexpected tasks.

As this causes uninative to fail to fetch, ensure that uninative is always
disabled.

(From OE-Core rev: 7d8ffd22303a5b89cb129e804c124a2d1dedf9ab)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-04-29 07:58:45 +01:00
Ed Bartosh ecaf12ea5b oetest: make console output more verbose
Added output of test runner to bitbake console output.
bitbake <image> -c testimage now reports test progress
to the console:
NOTE: Executing RunQueue Tasks
...
NOTE: recipe core-image-lsb-1.0-r0: task do_testimage: Started
core-image-lsb-1.0-r0 do_testimage: test_ping (oeqa.runtime.ping.PingTest) ... ok
core-image-lsb-1.0-r0 do_testimage: test_ssh (oeqa.runtime.ssh.SshTest) ... ok
...
----------------------------------------------------------------------
core-image-lsb-1.0-r0 do_testimage: Ran 38 tests in 785.100s
core-image-lsb-1.0-r0 do_testimage:
core-image-lsb-1.0-r0 do_testimage: OK (skipped=1)

Adding more output to the console should also prevent autobuilder to kill
long running tests as ab assumes that test is stuck if no console output
produced by it for a long time.

[YOCTO #8238]

(From OE-Core rev: 06aa68138ec67623892221fbe9516bbb3795b48e)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-04-18 16:28:22 +01:00
Peter Kjellerstedt ed07f43e68 lib/oeqa/selftest/base.py: Correct a reference to meta/lib/oeqa/selftest
(From OE-Core rev: 4ed2a6d43ea19a3026d94666fc342493689d7d46)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-04-18 16:28:22 +01:00
Bill Randle 8beb671843 ext-sdk-prepare.py: exclude do_rm_work from unexpected output; create unit test
When installing the esdk with INHERIT += "rm_work", the script complains
about do_rm_work as unexpected output from the bitbake run. This patch
ignores any output lines with do_rm_work and further refactors the
output comparison into its own function creates a new unit test to
verify the fix. The unit test can be run direct from the command line or
via oe-selftest.

[YOCTO #9019]

(From OE-Core rev: d41930e1daa933cf4bf063fa79a2e8fc9129e1b1)

Signed-off-by: Bill Randle <william.c.randle@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-04-18 16:28:21 +01:00
Ross Burton 76f4bbc30e oeqa/selftest/sstatetests: fix no-op sstate test
The test to verify that certain classes when inherited were no-ops only passed
if the build already inherited buildhistory.

To ensure that the test works as expected, use INHERITS_remove to ensure that
the classes are not inherited in the base run.

(From OE-Core rev: fc90c5db45933369b8fc0a33d6238876f5e5a087)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-04-15 06:59:43 +01:00
Ross Burton e80800e005 Revert "oeqa/selftest/wic: add test case for sparse images"
This reverts commit 43150ab7ec.

(From OE-Core rev: d7d06cd7390fa02f866ba5414efc3924ceecfb4e)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-04-13 10:12:52 +01:00
Mariano Lopez d27ca36e2a oeqa/runexported.py: Fix exported test
With the changes introduced to test the eSDK
the runexported test failed during the execution.

This change fix runexported test in the least invasive
way, because of the release cycle.

(From OE-Core rev: da0abb9679cb1fd639859a2fdbd82101d0a81259)

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-04-12 22:50:21 +01:00
Ross Burton 85dbd7bf9e oeqa/selftest/sstatetests: split 32/64 build host from no-op action tests
As more variables were being added it made sense to split this test into two:
one quick test that verifies that the host being 32-bit or 64-bit doesn't impact
the hashes, and another which compares stamps for two builds with several
variables changed and classes inherited.

(From OE-Core rev: d29ede773fc94da5ff932e0cd306295a8d3871a3)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-04-12 22:50:21 +01:00
Joshua Lock 43150ab7ec oeqa/selftest/wic: add test case for sparse images
Add a testcase to build a directdisk image and check that the
used disk size is less than the apparent size, as wic now
assembles images as sparse files.

(From OE-Core rev: 2b8299eb6b911e64d9a05b86c07b489d15eeaa5e)

Signed-off-by: Joshua Lock <joshua.lock@collabora.co.uk>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-04-09 23:00:45 +01:00
Randy Witt 7e902807cb sstatesig.py: Split single locked sigs check into multiple checks
Add the SIGGEN_LOCKEDSIGS_TASKSIG_CHECK and
SIGGEN_LOCKEDSIGS_SSTATE_EXISTS_CHECK variables to replace
SIGGEN_LOCKEDSIGS_CHECK_LEVEL.

SIGGEN_LOCKEDSIGS_TASKSIG_CHECK will no control whether there is a
warning or error if a task's hash in the locked signature file doesn't match
the computed hash from the current metadata.

SIGGEN_LOCKEDSIGS_SSTATE_EXISTS_CHECK will control whther there is a
warning or error if a task that supports sstate is in the locked
signature file, but no sstate exists for the task.

Previously you could only have warning/errors for both controlled by
SIGGEN_LOCKEDSIGS_CHECK_LEVEL. This was an issue in the extensible sdk,
because we know sstate won't exist for certain items in the reverse
dependencies list for tasks. However, we still want to error if task
signatures don't match.

[YOCTO #9195]

(From OE-Core rev: 0fe2a5e5ffd01e926d0f3d4c78ad9910296e2d1a)

Signed-off-by: Randy Witt <randy.e.witt@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-04-09 23:00:43 +01:00
Richard Purdie d110eba662 selftest/signing: Use packagedata to obtain PR value for signing test
Using PF to calculate the rpm filename doesn't work when PR server is
enabled and an extra PR value can be injected. Add code to use packagedata
to obtain the full name, allowing the test to work when PR server is
in use.

(From OE-Core rev: 322904f62f11e794543362f04212242567c556a0)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-04-01 07:14:30 +01:00
Richard Purdie c37d5426b1 scripts, lib: Don't limit traceback lengths to arbitrary values
There appears to have been a lot of copy and pasting of the code
which prints tracebacks upon failure and limits the stack trace to
5 entries. This obscures the real error and is very confusing to the user
it look me an age to work out why some tracebacks weren't useful.

This patch removes the limit, making tracebacks much more useful for
debugging.

[YOCTO #9230]

(From OE-Core rev: 6069175e9bb97ace100bb5e99b6104d33163a3a2)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-31 23:01:36 +01:00
Aníbal Limón ab5c62ea0d oeqa/runtime/parselogs.py: Add systemd unit circular dependencies errors.
When systemd is enabled as init we need to notice when circular
dependencies in units happen because systemd try to solve this situation
removing the unit itself.

(From OE-Core rev: 04b8fcc95f339282edc9ab405d0ba0e51dbc1d91)

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-31 09:15:05 +01:00
Daniel Istrate ede11b6b42 selftest: Added testcase decorator to tests
1418	test_recipetool_create_cmake
1422	test_qemu
1423	test_devtool_add_git_local
1433	test_devtool_upgrade_git
1434	test_sanity_unsafe_binary_references
1435	test_read_only_image

(From OE-Core rev: f3b46fe73b68db96ba245fcead74dc070ab10518)

Signed-off-by: Daniel Istrate <daniel.alexandrux.istrate@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-31 09:15:05 +01:00
Robert Yang dd31bcafb8 selftest/buildoptions.py: use INHERIT +=
* INHERIT = -> INHERIT +=

(From OE-Core rev: 727566ea0e4d19797ecb4cce5750f78e895f6293)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-30 21:24:02 +01:00
Daniel Istrate abb5b46ca3 oeqa/selftest/bbtests: Test bbappend order
BitBake should append to recipe in a predictable order.

fix for [YOCTO #9145]
test for [YOCTO #9138]

(From OE-Core rev: 51bef86ce52fdc2455cd1879e3e19d5ccc5c1c9c)

Signed-off-by: Daniel Istrate <daniel.alexandrux.istrate@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-30 12:01:29 +01:00
Ed Bartosh 528a890554 oe-selftest: generate .env only in test_image_env
Most of wic functionality doesn't depend on .wicenv file,
so it's better to generate it only in test_image_env
test case where it's used.

(From OE-Core rev: caf9b41e1db7b565ef977200195d57b385127de9)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-29 23:20:12 +01:00
Ross Burton 37683ef86e oeqa/utils/ftools: improve remove_from_file algorithm
The algorithm was sub-optimal so replace it with something more elegant.

(From OE-Core rev: 6119a90173f9222efa6df25aacf873af85d64bcd)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-28 15:55:50 +01:00
Richard Purdie 92b2bc5d27 wic: Update after task ordering changes
We need the wic env files to be available and this no longer happens automatically
so ensure we have them by specifying a specific task dependency.

(From OE-Core rev: 15e10957a0c9f65eaa119f8cb4f817c2fe3d580f)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-28 15:55:49 +01:00
Richard Purdie f81065f5b2 selftest/devtool: Update after make PROVIDER changes
Now that make/remake needs a PREFERRED_PROVIDER, we need to set this
correctly during tests.

(From OE-Core rev: 1a41953331f42d69c0201dcfcbb7d8dc12422fde)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-28 15:55:49 +01:00
Robert Yang 5988b5c5d7 selftest/signing.py: RPM_GPG_PASSPHRASE_FILE -> RPM_GPG_PASSPHRASE
(From OE-Core rev: ce14964d99741f1a4579bae18da5013498c365fd)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-25 10:29:14 +00:00
Paul Eggleton 0523378082 devtool: add: create git repository if URL specified as positional argument
When we add from a fetched URL we are supposed to turn the resulting
source tree into a git repository (if it isn't already one). However, we
were using the older deprecated option name here instead of the
positional argument, so "devtool add -f <url>" resulted in the repo
being created but "devtool add <url>" didn't, which was wrong.

Also update the oe-selftest tests to check that this worked.

(From OE-Core rev: a7b6b1f8cc1c096724f794ac9dee312b0f771f66)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-25 10:29:13 +00:00
Saul Wold 2f2f784727 parselogs: add new whitelist entries to address 4.4.3 issues
Adds intel-corei7-64 with x86-common, this makes the x86-common
whitelist available for intel-corei7-64 bsp also.

[YOCTO #9179]

(From OE-Core rev: 34e7292fb40635cee1f1237ac3156530f8dfce37)

Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-22 08:54:34 +00:00
Saul Wold 90234f1931 parselogs: add new whitelist entries to address 4.4.3 issues
This message appears on older hardware and is a benign warning

[YOCTO #9179]

(From OE-Core rev: e941853e3dd9e498dcf486686d30cd07d65d83fb)

Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-20 23:12:31 +00:00
Joshua Lock 10ef68f0f0 oeqa: remove RPM 4 self test
We're dropping RPM 4 as it's not well maintained, therefore remove
a selftest which tests RPM 4 availability.

(From OE-Core rev: ed005dd51c121c27a89a878bfc6abfca496bf51c)

Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-20 23:12:30 +00:00
Ross Burton 40a55f1594 oeqa/selftest/buildoptions: test read-only-rootfs
Add a test to build core-image-sato with read-only-rootfs enabled.

[ YOCTO #9214 ]

(From OE-Core rev: c23dc788386a8d3636f7f656667dc87052cf73d9)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-20 23:12:29 +00:00
Ross Burton f64fdd2734 oeqa/selftest/sstatetests: verify more variables don't impact the hash
Add DL_DIR and TIME to the hash tests.  We can't add DATE for some reason.

(From OE-Core rev: 206a95065628a839c589452de7aa646c90e02f5d)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-20 23:12:29 +00:00
Daniel Istrate 7e83af3727 selftest/buildoptions: Renamed one test case
'test_layer_git_revisions_are_displayed_and_do_not_fail_without_git_repo'
was renamed to 'test_layer_without_git_dir' which is shorter.

fix for [YOCTO #9243]

(From OE-Core rev: 7bd990e635e0b41f4ab9d71695a0309b5302178f)

Signed-off-by: Daniel Istrate <daniel.alexandrux.istrate@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-20 23:12:27 +00:00
Costin Constantin 427e369409 oeqa/utils/testexport.py: add functionality for exporting binaries
This new file is encapsulating functionality for both
running tests with binaries support via TestNeedsBin() decorator
and exporting these binaries via testimage.bbclass file.
Addresses [YOCTO #7850], [YOCTO #8478], [YOCTO #8481],
[YOCTO #8536], [YOCTO #8694].

(From OE-Core rev: 14640f16b5ce09a14f88b3fa641d4cf2780f8b97)

Signed-off-by: Costin Constantin <costin.c.constantin@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-20 23:12:27 +00:00
Paul Eggleton 9cc65ed020 Revert "selftest: Added MACHINE = "qemux86" to tests that use runqemu"
This is not the correct way to handle this - it significantly increases
the time taken to run oe-selftest anywhere MACHINE is set to some other
value (for example "qemux86-64"), because all of the artifacts for
qemux86 need to then be built as well when running the test. If we need
to skip these tests on non-QEMU machines, the devtool test already
demonstrates how to do that.

This reverts commit 169e1eaa4fc5ed03e2307b68686a7f5b1db37a36.

(From OE-Core rev: f60da6d5be5469ed5b834759c3822471f81fcdd2)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-20 23:12:26 +00:00
Paul Eggleton 600b7007f6 recipetool: create: don't create extra files directory unconditionally
The extra directory next to the recipe should only be created if there
are files to put into it; currently only the npm plugin does this. I
didn't notice the issue earlier because the test was actually able to
succeed under these circumstances if the recipe file came first in the
directory listing, which was a fault in my original oe-selftest test;
apparently on some YP autobuilder machines the order came out reversed.

With this change we can put the oe-selftest test that highlighted the
issue back to the way it was, with an extra check to reinforce that only
a single file should be created.

(From OE-Core rev: b8b778345eb0997c2cd952a1f61fdd2050b6b894)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-20 23:12:25 +00:00
Richard Purdie efa08813d7 oeqa/selftest/recipetool: Fix test_recipetool_create_simple
Correctly handle the any other files/directories that may exist
during the test.

======================================================================
FAIL: test_recipetool_create_simple (oeqa.selftest.recipetool.RecipetoolTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-oe-selftest/build/meta/lib/oeqa/utils/decorators.py", line 106, in wrapped_f
    return func(*args, **kwargs)
  File "/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-oe-selftest/build/meta/lib/oeqa/selftest/recipetool.py", line 414, in test_recipetool_create_simple
    self.fail('recipetool did not create recipe file; output:\n%s\ndirlist:\n%s' % (result.output, str(dirlist)))
AssertionError: recipetool did not create recipe file; output:
NOTE: Fetching http://www.dest-unreach.org/socat/download/socat-1.7.3.0.tar.bz2...
NOTE: Unpacking /srv/www/vhosts/autobuilder.yoctoproject.org/current_sources/socat-1.7.3.0.tar.bz2 to /tmp/recipetool-Uj7MIh/
NOTE: Recipe /tmp/recipetoolqaebTo9s/recipe/socat_1.7.3.0.bb has been created; further editing may be required to make it fully functional
dirlist:
['socat', 'socat_1.7.3.0.bb']

(From OE-Core rev: 4be0e15f74cff85edca9de55248939fb438f30ae)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-13 18:15:52 +00:00
Richard Purdie d24a39ae80 oeqa/recipetool: Fix syntax error
(From OE-Core rev: 680385e86cbeb629e9baa80dfadb432f1fcd6786)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-13 09:28:20 +00:00
Richard Purdie 55a1e52195 oeqa/recipetool: Improve debugging output by adding dirlist
(From OE-Core rev: f7b38ec43a1090ff2c56f26fe19637fba2b1a996)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-12 22:34:56 +00:00
Mariano Lopez a5fd57d971 selftest/bblayers.py: Remove harcoded recipe files
Currently the recipe files are hardcoded and if the recipe
change the version, the test will fail.

This will change from using a harcoded file to look for the
file using bitbake-layers. Now, just the recipe name must
be specified.

(From OE-Core rev: 1ee24e435353d93374895eead81fb281e1338739)

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-11 16:50:44 +00:00
Mariano Lopez dce7290a4f selftest/prservice.py: Sanitize package version when looking for stamp
Currently when using a git version the check for the stamp, using regex,
will fail because of plus sign in the version.

With this change the version is escaped before adding it to the regex.

(From OE-Core rev: 1aefa6a4dec84a5581aab70451bb84801b3b3615)

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-11 16:50:44 +00:00
Daniel Istrate f8a9774f79 oeqa/selftest/buildoptions: Test build does not fail without git rev
Test that layer git revisions are displayed and
do not fail without git repository.

fix for [YOCTO #8852]

(From OE-Core rev: 8adaad7f3a76d527f34d2caa4b032beba7e21840)

Signed-off-by: Daniel Istrate <daniel.alexandrux.istrate@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-09 16:58:12 +00:00
Richard Purdie 82d0c8ad1e oeqa/buildoptions: Improve unsafe references tests
Fixing one of the recipes not to contain unsafe references to paths
results in this QA test failing. Improve the test so that we test
the recipe works, then if we intentionally break the recipe, the
issue is detected.

Also split out the binaries test from the scripts test. The binaries
issue may also get 'fixed' in future and need the same fix.

(From OE-Core rev: 68db200ca5b404d6c0aa0cbf5a587397d0aa65da)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-07 00:11:37 +00:00
Juro Bystricky b04280ab5f sdk_update.py: Enable local sdk-update tests
Testing of local sdk updates was commented out.
Local sdk updates are functional now, so the tests should be re-enabled.

(From OE-Core rev: 0bc7cd0bd10f79907c4f41676801a113fb3df8a3)

Signed-off-by: Juro Bystricky <juro.bystricky@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-03 10:49:00 +00:00
Aníbal Limón c7e5a38d23 oeqa/sdkext: Add sdk_update.SDKUpdateTest class.
The SDKUpdateTest class test devtool sdk-update mechanism inside
eSDK.

The SDKUpdateTest class search for new sdk if not found uses
the main one then it publish the eSDK into known folder
inside work and it starts a web server for serve the eSDK.

Finally it executes sdk-update over http, the local test is
commented due to bug [1].

[1] https://bugzilla.yoctoproject.org/show_bug.cgi?id=9043

[YOCTO #9089]

(From OE-Core rev: be7f5036a7c86fe70d43526df529bc467a9cf7d9)

Signed-off-by: Aníbal Limón <limon.anibal@gmail.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-28 11:33:06 +00:00
Aníbal Limón 738bd1a640 classes/testsdk: Pass tcname to SDK and SDKExt contexts
tcname is needed for eSDK update testcase will be used for
 publish it and then try to update

(From OE-Core rev: a75944a63482597be88ff0f3ce55025647b78e2c)

Signed-off-by: Aníbal Limón <limon.anibal@gmail.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-28 11:33:06 +00:00
Aníbal Limón 2a410b225a classes/testsdk: Move the removal of bitbake PATH to eSDK context only
The removal of bitbake and scripts PATH is only needed by eSDK tests
so move to eSDK context only.

This also it's a support for eSDK update test because it needs to
execute oe-publish-sdk from scripts.

(From OE-Core rev: 4ae0c84568f39661722cbceba8ddab22cffe5003)

Signed-off-by: Aníbal Limón <limon.anibal@gmail.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-28 11:33:06 +00:00
Aníbal Limón eb1f8b9427 classes/testsdk: Move code for avoid PATHs to oeqa.utils
Due to the neeed to use in other modules.

(From OE-Core rev: a25aef2bdefae54c8b3eb2bd4afec5a86110ddc7)

Signed-off-by: Aníbal Limón <limon.anibal@gmail.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-28 11:33:06 +00:00
Paul Eggleton bf4d380b97 oe-selftest: devtool: add an additional test for devtool upgrade
Add a test for devtool upgrade with a recipe pointing to a git
repository, since this uses several different code paths.

(From OE-Core rev: d3ec74ff4db248ea11a568d5214708d6a1757012)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-28 11:33:01 +00:00
Paul Eggleton 4bae2f25b8 oe-selftest: devtool: rework devtool upgrade test
* Use a more real-world test of a recipe pointing to a remote file
* The cleanup tracking / teardown commands need to be added towards the
  top, or they won't have the desired effect of cleaning up if the test
  fails.
* Check that a versioned subdirectory gets renamed to match the new
  version
* Ensure the recipe contents gets changed as we expect it to
* Check that the recipe directory is deleted by devtool reset at the end

(From OE-Core rev: d0ae258e963f9bafffc4ca43c87497d27e57c127)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-28 11:33:01 +00:00
Cristina Agurida b748f40980 oeqa/parselogs: Updated whitelist
Moved an error to common, and whitelisted the BAR errors
and ati module. Also the uvesafb related errors will be
ignored in LSB images which do not have graphic interface.

Fix for [YOCTO #8387]
Fix for [YOCTO #8590]
Fix for [YOCTO #6820]
Fix for [YOCTO #7897]

(From OE-Core rev: 4e59b6f66eeae256d4e62f4741725cdebb53cbf7)

Signed-off-by: Lucian Musat <george.l.musat at intel.com>
Signed-off-by: Cristina Agurida <cristina-danielax.agurida@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-28 11:32:59 +00:00
Francisco Pedraza 8d07e14009 oeqa/selftest/prservice: Added new TC: check pr-server starts and stop correctly on localhost.
This test case validates if prservice start and stop properly the server.
Using bitbake commands also validates the exit status.

[YOCTO #8258]

(From OE-Core rev: 5125dd2a5c2e382c47ddeaf0327fc7dc39482023)

Signed-off-by: Francisco Pedraza <francisco.j.pedraza.gonzalez@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-28 11:32:59 +00:00
Francisco Pedraza 00a6f5afeb oeqa/utils: added new network module
A network module was added, and will contain network utility funcions for now.
with get_free_port that returns available network port in the system.

(From OE-Core rev: 72b336ad0d0a2994f00c57747686111a59fa8b29)

Signed-off-by: Francisco Pedraza <francisco.j.pedraza.gonzalez@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-28 11:32:58 +00:00
Daniel Istrate 4a21827415 oeqa/selftest/signing: Added test for locked signatures
fix for [YOCTO #8706]

(From OE-Core rev: 9b704ae44d9dc1d3d5c3aee6d7a5530b642070c4)

Signed-off-by: Daniel Istrate <daniel.alexandrux.istrate@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-28 11:32:57 +00:00
Ross Burton 7a95c2cb67 oeqa/selftest/buildoptions: build -minimal instead of -sato images
When checking enabling buildhistory doesn't change anything but rootfs stamps,
just build core-image-minimal instead of -sato to reduce the time this test
takes.

(From OE-Core rev: e9b44579007cbaa24c6b39ff788be3a927797660)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-26 17:16:19 +00:00
Daniel Istrate 9cd64edebf oeqa/selftest/bbtests: Test bitbake --setscene-only option
Bitbake option to restore from sstate only within a build
(i.e. execute no real tasks, only setscene)

fix for [YOCTO #8876]

(From OE-Core rev: 3821aee0129b87a859b3a540b82745fdae184721)

Signed-off-by: Daniel Istrate <daniel.alexandrux.istrate@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-18 07:39:30 +00:00
Richard Purdie 152914f298 oeqa/parselogs: Whitelist dmi firmware failure message in 4.4 kernels
(From OE-Core rev: 31af4a37c429bb1db8bfc64242b2963220f9a0e1)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-16 23:54:05 +00:00
Benjamin Esquivel 637b44ce7c runtime/systemd: Fix for boot time string parse error
boot time string can change its format of the output of the amount of time
it took to boot. It is required to handle graceful fail of the parsing
errors that it provokes

[YOCTO #8889]

(From OE-Core rev: d17f5079594cd74014f29054f9ad4f38c7ef03d8)

Signed-off-by: Benjamin Esquivel <benjamin.esquivel@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-16 09:04:22 +00:00
Richard Purdie d585a716d2 oeqa: Update to handle domain specific references in build logs
With the addition of the task name to recipe output, the sanity tests
need updates where they are looking for specific messages.

(From OE-Core rev: 0f2ef4304e6a6f18b4ed13f59000b4a1daa35f6b)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-15 16:28:46 +00:00
Ed Bartosh 5d156bc066 oe-selftest: don't use specific tasks
Image should be build usual way, i.e. using bitbake <image>.
Specifying do_mage_complete and do_rootfs_wicenv tasks should not
be needed anymore as those tasks should be run by bitbake for
every image.

Removed specifying do_mage_complete and do_rootfs_wicenv tasks
from bitbake calls.

(From OE-Core rev: d8d7bd1b41eb846f18378a2581ff172cb2cc52b8)

(From OE-Core rev: 00cd291605ff5f001979b350d95670e33012fdc9)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-15 16:28:45 +00:00
Ed Bartosh 80e8928942 oe-selftest: pylinted wic tests
Fixed the following pylint warnings:
  C: 45, 0: Line too long (91/80) (line-too-long)
  C:105, 0: Wrong continued indentation.
                            ignore_status=True).status)
                            ^         | (bad-continuation)
  W: 83, 8: Redefining built-in 'vars' (redefined-builtin)
  W:175, 8: Redefining built-in 'vars' (redefined-builtin)
  W: 27, 0: Unused import sys (unused-import)

(From OE-Core rev: ecb5320e766d0fa031c269d6d9b2a24c81c7aa7e)

(From OE-Core rev: c47af4a772883c86559673ba9b9f774f0ff0d54c)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-15 16:28:44 +00:00
Ed Bartosh 51e0a8a180 oe-selftest: add new wic testcase
Added test_qemu testcase to boot wic-image-minimal and test
that 2 partitions mentioned in .wks are mounted.

[YOCTO #8499]

(From OE-Core rev: 6fb015d0847fe7d259d654d4a99bf4c328f810ab)

(From OE-Core rev: be360c24649391235fd9547a8f2c1251b12e9c81)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-15 16:28:44 +00:00
Ed Bartosh 4b2660144d wic-image-minimal: change IMAGE_FSTYPES
Replaced wic.bz2 -> wic in the recipe.
Replaced wic.bz2 -> wic in oe-selftest test case.

wic-image-minimal is going to be booted in qemu and tested.
As runqemu doesn't support compressed images this recipe needs
to produce raw wic image.

(From OE-Core rev: 3ce91db94f4b921566dce0dc6f91a422009be06b)

(From OE-Core rev: 69d8dfe4fc1fec440e33b1bb6cef62e20f1f5c28)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-15 16:28:44 +00:00
Ed Bartosh f799e218ed oeqa/targetcontrol: support wic image type
Added 'wic' to the list image types supported by targetcontrol.

This is a preparation for booting and testing wic images
with oe-selftest.

[YOCTO #8498]

(From OE-Core rev: 7dda053fbd1ea1354b7720cfa691470ba88ef5b9)

(From OE-Core rev: 4b84328d3cd0d87ad146f034b58f68a5158313d7)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-15 16:28:44 +00:00
Ed Bartosh 7066f1671e oeqa/targetcontrol: make ssh control optional
Added new parameter 'ssh' to targetcontrol 'start' method
to be able to test images without running ssh server.

[YOCTO #8498]

(From OE-Core rev: 1c3c66aadd43092bc19242b0651ee810cc31fe7c)

(From OE-Core rev: 67ccf7413b2ac9f516dbdaa6a39d4cec38a6c94d)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-15 16:28:44 +00:00
Ed Bartosh 0ade65834b qemurunner: add parameter to method 'start'
QemuRunner requires pair of ip addresses provided through kernel
commandline for method 'start' to work. These ip addresses are
used to connect to the image using ssh and run tests there.
However, this functionality should not be mandatory as testing
doesn't always require ssh connection. Some tests can be run using
serial console.

Added new parameter 'get_ip' to QemuRunner.start to make it possible
to skip getting pair of ip addresses from kernel command line. This
should allow oe-selftest to test images without modifying kernel
command line.

[YOCTO #8498]

(From OE-Core rev: 3f8b734ebb81d035849288091bb0b97b9c4fba34)

(From OE-Core rev: 4c90daaeb946f1adf58b2f71f1af8eb7f5906474)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-15 16:28:44 +00:00
Ed Bartosh d083fec6d1 oe-selftest: remove unused parameter
Removed unused parameter 'test' from runqemu function.

(From OE-Core rev: c688b3bcbb57099fa72a9728bc708b109802f7fc)

(From OE-Core rev: a7af1eb28fab515180c0fa01a003ac7b2ce0cff4)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-15 16:28:44 +00:00
Paul Eggleton e64ce73b7e oe-selftest: devtool: add another devtool add test
Add a test so we validate the following recently implemented/fixed
aspects of "devtool add":

* Adding from a local directory with no name specified
* Auto-detecting the name and version from autoconf files
* Setting SRC_URI and S from the local git repo
* Showing the recipe file path in "devtool status" if it has been
  created within the workspace

Incidentally also tests:

* LICENSE and LIC_FILES_CHKSUM detection (though just for this piece of
  software)
* Extracting a dependency from autoconf (though just for this narrow
  case since there's only one)

(From OE-Core rev: d991fe7f17861dc5c21f7eac30d8c912b4d5dbed)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-11 12:33:03 +00:00
Paul Eggleton fa50153bc2 devtool: modify: make -x the default behaviour
It's going to be more common for users not to have the prepared source
tree for a recipe already, so the default behaviour ought to be to
extract it for them from the recipe. Change the default to extract
(effectively making the -x option a no-op) and add a --no-extract/-n
option to disable it. Later we can look at trying to be smart and
reusing an existing source tree instead of erroring out if it exists;
for now this is just the default reversal.

(From OE-Core rev: 80a44e52609a89d9ffe816181ae193af491c06ac)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-11 12:33:03 +00:00
Ross Burton 25d2956d97 oeqa/selftest/sstatetests.py: check that PARALLEL_MAKE doesn't change signatures
There are recipes that manage to have sstate signatures that depend on
PARALLEL_MAKE, so verify that changing this variable doesn't change signatures.

(From OE-Core rev: 14e8026cc25394b0a6d1d2063dc36b6b295857e5)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-11 12:27:48 +00:00
Richard Purdie 864797ab8d oeqa/prservice: Fix whitespace problem
Recent bitbake changes mean the whitespace formatting around the way functions are
defined, changed which broke the rather ugly construct this test uses to append to
a python function. This really needs to be rewritten and improved but fix the
whitespace so at least the tests work again and other regressions don't creep in.

(From OE-Core rev: 3279386edb2d231ea19ebc2b66c19062305ddf8c)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-11 12:27:43 +00:00
Richard Purdie ecb9c34bbc oeqa: Improve test failure messages
When the test fails, print the incorrect values so aid debugging.

(From OE-Core rev: 115f479ff91347ecb5069e89e193a5c678e7dc35)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-10 16:06:24 +00:00
Aníbal Limón e22fbce142 oeqa/sdkext/devtool.py: Add location test to ensure that devtool is the eSDK one.
(From OE-Core rev: 3e3abf0753a3bf1737dc4f476df4e70d31070391)

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-10 16:06:23 +00:00
Aníbal Limón 92d0cc582c oeqa/sdkext: Add devtool basic tests for eSDK.
Add simple myapp application is a C app that prints hello world
and exit.

Add devtool test for that this app to the workspace, build and
reset it.

(From OE-Core rev: 2ec513c00ab2ae0f7df631d32f8f248446c90184)

Signed-off-by: Aníbal Limón <limon.anibal@gmail.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-10 16:06:23 +00:00
Aníbal Limón a619ea293a oeqa/oetest: Fix compatibility SDK tests using eSDK.
When run SDK tests over eSDK we need to use SDKExtTestContext
instead of SDKTestContext because if we use SDKTestContext search
for SDK manifest and depends on the SDK manifest generation so
populate_sdk needs to be executed.

Adds a compatibility mode flag to SDKExtTestContext for search tests
over sdk module instead of sdkext module and change testsdk calls
to comply with this new param.

(From OE-Core rev: ebe743235b383b17225553d84bf8e6f80d364dcd)

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-10 16:06:23 +00:00
Aníbal Limón 062dbd6da7 classes/populate_sdk_ext: Add SDK_EXT_TARGET_MANIFEST and SDK_EXT_HOST_MANIFEST
Extensible SDK needs to point to the correct manifest so add
SDK_EXT_TARGET_MANIFEST and SDK_EXT_HOST_MANIFEST variables.

oeqa/oetest.py: Fix SDKExtTestContext for load the correct manifests.

(From OE-Core rev: 2310d5ad03531b7e1f9572c12c83c2fedc0291c9)

Signed-off-by: Aníbal Limón <limon.anibal@gmail.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-10 16:06:23 +00:00
Aníbal Limón 4cfdf174ce testsdkext: Add skeleton for support Extensible SDK tests.
oeqa/sdkext: Add module and __init__.py will contain eSDK tests.
classes/testsdk: Add support for run eSDK tests.
oeqa/oetest: Create oeSDKExtTest for now only inherit oeSDKTest,
             modified SDKExtTestContext now inherit SDKTestContext
             and set sdkext filesdir for store data fixtures.

(From OE-Core rev: f3781544a5c077610498a6b7dc5244ee4c5bc6df)

Signed-off-by: Aníbal Limón <limon.anibal@gmail.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-10 16:06:23 +00:00
Aníbal Limón 7181da7192 oeqa/oetest: oeSDKTest when run a command redirect env output to null
Some tests are failing because the eSDK env load script shows a help
message on the output so redirect this to /dev/null.

There was a discussion with Paul Eggleton [1], he don't want to have env
variables to change the behaviour of what the env script needs to print
so redirect the output in oeSDK.run() method. I didn't agree because
it can hides another messages like error ones and with the variable you
can control what prints or not.

[1] http://lists.openembedded.org/pipermail/openembedded-core/2016-February/116744.html

(From OE-Core rev: 7defb10d10861501947b4a686510e957c72313e6)

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-10 16:06:23 +00:00
Aníbal Limón 3577c35f6e oetest.py/TestContext: Move loadTests and runTests inside it.
Method's for loadTests and runTests make sense to define
inside TestContext because it can be different around
Image, SDK, SDKExt.

(From OE-Core rev: 03af7b99e3ce36ce3e29dc31e33d2cc74eb14849)

Signed-off-by: Aníbal Limón <limon.anibal@gmail.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-10 16:06:23 +00:00
Aníbal Limón 8009418d55 testimage/testsdk: Move get test suites routine inside TestContext.
In order to provide better abstraction move functions to get the test
suite inside the TestContext.

(From OE-Core rev: 552285688441330440bd1bc138aeaea87549024a)

Signed-off-by: Aníbal Limón <limon.anibal@gmail.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-10 16:06:22 +00:00
Aníbal Limón b588b80d66 testimage/testsdk: Modularize TestContext.
Move anonymous duplicated class TestContext from testimage/testsdk to
oeqa/oetest now we have two new classes ImageTestContext and
SDKTestContext with common code in TestContext class.

(From OE-Core rev: 593f2fdf6ee94c5f91761a669048da3598cbe3fa)

Signed-off-by: Aníbal Limón <limon.anibal@gmail.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-10 16:06:22 +00:00
Aníbal Limón 90590ab047 get_test_suites: Add sdkext type for load test suites.
(From OE-Core rev: 01a420dc9c0c04c793bbf5824db7c2b9d7c9de9b)

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-10 16:06:22 +00:00
Aníbal Limón 3d1d30b0a3 testimage: Modularize helper functions for get test lists.
Test lists functions can be used in other parts so modularize it and
move to oeqa/oetest.py library.

Testimage class was updated to meet the new sign of the functions.

(From OE-Core rev: c9f771533af70e7ccb1e7064e58926cfaee7367a)

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-10 16:06:22 +00:00
Ed Bartosh 947e5260f9 oeqa: setup bitbake logger after tinfoil.shutdown
Bitbake logger stops working after tinfoil.shutdown removes console
handler from it. This makes bb.{error,warn,note,critical} messages
disappear from the console. Adding console handler to bitbake logger
again should fix this issue.

(From OE-Core rev: ef372c1829b5818fd20224d305f6e20fba643acc)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-10 15:51:15 +00:00
Richard Purdie eb53c54675 sstatetests: Fix after change to sstate populate_lic SWSPEC
http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=9fd214d2e7a931cfb46f40cb76d49aeb07af612e
changed SSTATE_SWSPEC, correctly however it did mean that some sstate files
now appear where the tests don't expect them. Filter the test results
to ignore the correct behaviour.

(From OE-Core rev: e86947bb79a9cb1b59a564ee511739f8a2b2b913)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-07 17:30:01 +00:00
Ross Burton 956be0c858 oeqa/runtime/rpm: be more verbose if test_rpm_query_nonroot fails
If some parts of this test fail put the failing output in the logs to help
debugging.

(From OE-Core rev: aeee86050019caae5da3af8d31cf57bc811c5e80)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-30 11:36:57 +00:00
Paul Eggleton d193531d5e devtool: build: ensure pkgdata is written out
When you run devtool build, you need to have the pkgdata written out at
the end, so that if what you're adding is a library and the next thing
you add is something that depends on that library, the necessary
information to map the dependency back to the recipe is present. In
practical terms all this means is we need do_packagedata to run in
addition to do_populate_sysroot.

This does mean that do_package needs to run which wasn't running before,
and that means that the few package QA tests that run within do_package
such as installed-vs-shipped will now be run. This may be a bit
bothersome, and prompted a fix for one of our oe-selftest tests as a
result, but I don't see an easy way around it. Ultimately if you care
about using the recipe in an image you'll need to fix any such errors
anyway.

Fixes [YOCTO #8887].

(From OE-Core rev: 6579c7120ee5a541427ff5b6b07f838d52f9fe7c)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-24 09:40:32 +00:00
Markus Lehtonen a84058826d oeqa/selftest/signing: use temporary rpmdb
Use temporary rpmdb when importing gpg public key and checking rpm
signature. This patch should fix a problem where test_signing_packages()
sometimes fails with
"pmdb: BDB1540 configured environment flags incompatible with existing
 environment"

(From OE-Core rev: e2ddf409f405ab570ea7f6ed87180789b3a54607)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-24 09:40:20 +00:00
Yi Zhao 7a0b1c1850 oeqa/runtime/parselogs: use -F to search fixed strings for grep
The grep output is not correct if the log message contains special
characters. Use -F which tells grep to search for fixed strings, not
expressions and pass lists instead of strings to subprocess.

(From OE-Core rev: bfc8304d4c10b5601a618011893369831baaf511)

Signed-off-by: Yi Zhao <yi.zhao@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-19 17:37:33 +00:00
Paul Eggleton 0612ca4d0f oe-selftest: devtool: fix test_devtool_add_library if python was built first
If Python is in the sysroot then this test can fail due to some
brokenness in libftdi's CMakeLists.txt file for its python bindings.
Just disable it in order to have the test work more reliably.

(From OE-Core rev: 3e53e675bba75eb949be97a8d372e0134f6df935)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-19 17:24:53 +00:00
Paul Eggleton 2c8c9fe3b4 recipetool: create: add basic support for extracting dependencies from cmake
Add support for extracting dependencies from CMakeLists.txt. There's
still a bunch of things missing that are outside the scope of OE-Core
and we still lack a proper extension mechanism, but this is a good
start.

This also adds an oe-selftest test to exercise the new code a bit.

Implements [YOCTO #7635].

(From OE-Core rev: 77e73e6930381fdbd6e78d3913d6467572e16568)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-19 17:24:53 +00:00
Paul Eggleton dbe91a3d6a recipetool: create: improve extraction of pkg-config / lib deps
* The regexes for PKG_CHECK_MODULES / AC_CHECK_LIB were a bit too strict
  and thus we were skipping some macros.
* Add support for PKG_CHECK_EXISTS
* Avoid duplicates in warning on missing pkg-config dependencies
* Ignore dependency on musl (since this may come up if it's the selected
  C library)

(From OE-Core rev: c58669fb0977f7f0cb79f252484d5c5ef0dfb7e4)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-19 17:24:48 +00:00
Richard Purdie 12e37e7c79 selftest/buildhistory: Improve test to remove sources of error
When we add buildhistory, we expect do_rootfs to rerun, but depending
on IMAGE_FSTYPES, the number of tasks which would execute after
do_rootfs varies (e.g. live would add do_bootimg and we recently
added do_image).

Therefore limit the test to -c rootfs and then we're clear that only
one task should re-run.

(From OE-Core rev: 9e9a33281e6008c3b699b5bbb738921829dbc501)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-19 17:24:47 +00:00
Richard Purdie fb1654fee4 image: Fix wic environment issues
The wic environment function needs to run after the rootfs size is
setup. We move this code to a specific task, and depend on that task
from the wic images and other places its needed.
This fixes:

======================================================================
FAIL: test_image_env (oeqa.selftest.wic.Wic)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-oe-selftest/build/meta/lib/oeqa/utils/decorators.py", line 106, in wrapped_f
    return func(*args, **kwargs)
  File "/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-oe-selftest/build/meta/lib/oeqa/selftest/wic.py", line 218, in test_image_env
    self.assertTrue(var in content, "%s is not in .env file" % var)
AssertionError: False is not true : ROOTFS_SIZE is not in .env file

(From OE-Core rev: 606f9e2d7d8d389c8d4f5c3090139d3bb780e09c)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-19 17:24:46 +00:00
Richard Purdie 1da8f52b67 insane: Start to clean up do_configure_qa code
This cleans up the do_configure_qa code so that the exit status from
package_qa_handle_error is handled correctly.

It also converts package_qa_check_license to use our standard QA check
configuration interface through package_qa_handle_error rather than
a mix of return values, bb.fatal and bb.build.FuncFailed.

Due to the merging of multiple lines into one message, we need to
tweak a QA test to account for this.

(From OE-Core rev: fbdf977a8094bb1e0baca85ffb3ed7f326986639)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-19 17:24:46 +00:00
Yi Zhao 235606f31b oeqa/runtime/logrotate: fix hardcoded root directory
Use $HOME instead of /home/root in case user changes the default root
directory by ROOT_HOME

(From OE-Core rev: 123ca1c3457e472bd3dab5563f7ba5312a9864bf)

Signed-off-by: Yi Zhao <yi.zhao@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-11 23:26:30 +00:00
Yi Zhao cce6c3ed9a oeqa/runtime/smart: fix hardcoded root directory
Use $HOME instead of /home/root in case user changes the default root
directory by ROOT_HOME

(From OE-Core rev: c9adb8399687b60eb976ab32d960f4b32af71fea)

Signed-off-by: Yi Zhao <yi.zhao@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-11 23:26:30 +00:00
Ioan-Adrian Ratiu 8bec5c55a2 x11vnc: remove all references to moved package
Together with the move to meta-oe, all references to x11vnc should be
removed from oe-core. There are three of these: a distro alias, a
packagegroup rdepends and a runtime test.

(From OE-Core rev: cfd1e4bcd66a9a542007115647cadb8480330fab)

Signed-off-by: Ioan-Adrian Ratiu <adrian.ratiu@ni.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-07 13:40:20 +00:00
Ross Burton 5aa3b93009 oeqa/selftest/recipetool: update for libjpeg-turbo migration
The recipe creation test of libmatchbox now depends on libjpeg-turbo, not jpeg.

(From OE-Core rev: 58bb2d82f4f1b111641c480a27836be6119ee279)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-07 13:40:17 +00:00
Daniel Istrate c2bda6c000 scripts/oe-selftest: Allow to run tests on random/all MACHINEs
Add an option for random MACHINE into oe-selftest:
--machine [random/all]
1. random: will set a random MACHINE for each test
2. all: will run tests for all machines

Custom machine sets only weak default values (??=) for MACHINE in machine.inc.
This let test cases that require a specific MACHINE to be able to
override it, using (?= or =).

e.g.:
oe-selftest --run-tests signing --machine random -->
will run all tests switching MACHINE randomly for each test

oe-selftest --run-tests signing --machine all -->
for each machine will run all tests

oe-selftest --run-all-tests --machine random

Also update oeqa/selftest/base.py to accomodate this feature.

Fix for [YOCTO #5880].

(From OE-Core rev: 4a9c3653eecd9a3c1b45bab5e16c8d679c55f8bd)

Signed-off-by: Daniel Istrate <daniel.alexandrux.istrate@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-07 13:40:17 +00:00
Daniel Istrate 8e1435eb71 selftest: Added testcase decorators for 2 tests
1391 for test_devtool_add_fetch_simple from devtool
1392 for test_recipetool_create_simple from recipetool

(From OE-Core rev: d6ee070127a08be98958bbb8b11c80bb64165dc7)

Signed-off-by: Daniel Istrate <daniel.alexandrux.istrate@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-07 13:40:16 +00:00
Daniel Istrate 4ec2da71db selftest: moved tc test_buildhistory_does_not_change_signatures
Moved test_buildhistory_does_not_change_signatures from
buildhistory/BuildhistoryBase to buildoptions/BuildhistoryTests.

The test being in the base class was causing it to run
multiple times.

Fix for [YOCTO #8867]

(From OE-Core rev: 975e67e28ccba5dcb0fced43c1f9e7da183dc201)

Signed-off-by: Daniel Istrate <daniel.alexandrux.istrate@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-07 13:40:16 +00:00
Ross Burton 09705a4af0 oeqa/selftest: support sets in devtool comparisons
The devtool and recipetool tests do literal string comparisons, but for some
fields the ordering could be irrelevant and potentially non-deterministic.  For
example, the recipetool_create_simple test started failing with:

AssertionError: 'GPLv2 Unknown' != 'Unknown GPLv2' : values for LICENSE do not match

The ordering of the LICENSE field isn't relevant.  So, if the expected value is
a set(), split the string into a set too and compare those.

(From OE-Core rev: dab1b0ad42f08eb547732e854ba28a4cf0a23107)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-28 09:25:20 +00:00
Daniel Istrate ab79287688 selftest: Added MACHINE = "qemux86" to tests that use runqemu
It makes sense for tests that use runqemu to have MACHINE set as qemu.
This also avoid issues when running oe-selftest with --arch random/all
option.

(From OE-Core rev: 169e1eaa4fc5ed03e2307b68686a7f5b1db37a36)

Signed-off-by: Daniel Istrate <daniel.alexandrux.istrate@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-28 09:25:17 +00:00
Paul Eggleton 9da9308781 oe-selftest: devtool: add more explicit check for ls output
test_devtool_deploy_target is failing on the Yocto Project autobuilder
apparently when it attempts to cut out some fields from the list. It
doesn't fail here and I can't see what the problem lines are, so add a
check for lines with too few fields so we can get a look at them next
time it fails.

(From OE-Core rev: 2b19c74332c3899c84708946d749f88c4416cf73)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-28 09:25:14 +00:00
Paul Eggleton c2435b1181 oe-selftest: add tests for simple devtool add / recipetool create URL case
Add an oe-selftest test case for the newly supported syntax with only
the remote URL specified (auto-detecting name and version).

(From OE-Core rev: 7c7df9f62fe15578af0420c63e320c317e058708)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-28 09:25:14 +00:00
Paul Eggleton ebe5f0b872 recipetool: create: basic extraction of name/version from filename
Often the filename (e.g. source tarball) contains the name and version
of the software it contains.

(This isn't intended to be exhaustive, just to catch the common case.)

(From OE-Core rev: 944eacfb849ee69b41e12c9de4f264406281ac6a)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-28 09:25:13 +00:00
Paul Eggleton db5f9645ad recipetool: create: support extracting name and version from build scripts
Some build systems (notably autotools) support declaring the name and
version of the program being built; since we need those for the recipe
we can attempt to extract them. It's a little fuzzy as they are often
omitted or may not be appropriately formatted for our purposes, but it
does work on a reasonable number of software packages to be useful.

(From OE-Core rev: 3b3fd33190d89c09e62126eea0e45aa84fe5442e)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-28 09:25:13 +00:00
Daniel Istrate d4b5a1f171 selftest/buildhistory.py: Test buildhistory does not change sigs
[YOCTO #5953] Add a test to ensure buildhistory does not
change signatures.

Also removed unused imports.

(From OE-Core rev: 9e8f5ff6b0bd6cffcbb991d75487ab6005974000)

Signed-off-by: Daniel Istrate <daniel.alexandrux.istrate@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-22 16:08:49 +00:00
Daniel Istrate f80f8baa6f oeqa/selftest: Added testcase decorators for 2 testcases
bblayers: test_bitbakelayers_showrecipes    1384
wic:      test_directdisk_bootloader_config 1385

(From OE-Core rev: eeb3382f38f0cb9188cccb4d32deb38be0b50f67)

Signed-off-by: Daniel Istrate <daniel.alexandrux.istrate@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-16 12:12:15 +00:00
Benjamin Esquivel 23083e7e5e oeqa/systemd: get runtest target boot time and log
oeqa.runtime.systemd.SystemdJournalTests.test_systemd_boot_time
this test will query the target boot time from journactl and will
print it to the output, if the time is obtained, the test passes

it then compares the boot time against the default systemd's
timeout TimeoutStartSec and if the boot time is greater than the
it will print it too

this test prints the startup time in the test log like:
...
test_systemd_boot_time (oeqa.runtime.systemd.SystemdJournalTests) ...
Startup finished in 6.922s (kernel) + 52.089s (userspace) = 59.011s.

(From OE-Core rev: 59c6c13a8fa1bfc8e0615463e00ccdef04a87a47)

Signed-off-by: Benjamin Esquivel <benjamin.esquivel@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-12 23:42:51 +00:00
Benjamin Esquivel c6330a2783 oeqa/systemd: journalctl helper function
a function to request for the journalctl output to the current target
system with l_match_units support

(From OE-Core rev: b411fec74f39d1efa7276046a32d6048b35dd199)

Signed-off-by: Benjamin Esquivel <benjamin.esquivel@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-12 23:42:51 +00:00
Mariano Lopez d7baeb5fad selftest/wic.py: Add test for custom bootloader config
This change just add anoher test to the wic module.
It will try to create a image with a custom bootloader
configuration. This test will use the example image
directdisk-bootloader-config to test the configfile
option for the bootloaders.

[YOCTO #8728]

(From OE-Core rev: 567cb51f15354c0398b986e32011420a5f3dd090)

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-09 08:48:25 +00:00
Ross Burton 40cd22815c oeqa/sshcontrol: don't source profile
Instead of sourcing /etc/profile to get $PATH including /usr/sbin, just assign
to PATH in the ssh invocation.

The remote /etc/profile may not actually be manipulating PATH as we expect, and
there may be other commands which can interfere with the tests (such as resize
emitting a series of control characters on connection).

(From OE-Core rev: 0f3fb5bbf2fd7db82898fed3281af143387316ff)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-01 21:32:05 +00:00
Ross Burton d39192a848 oeqa/runtime/multilib: refactor ELF class extraction
Instead of duplicating the same code over and over, split it out to a separate
function.

(From OE-Core rev: 4f870f020bbf908ab87990803f3c278bf4e44843)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-01 21:32:05 +00:00
Paul Eggleton 0fe742674e devtool: disable creating workspace for extract and search subcommands
For subcommands that don't actually involve the workspace, don't
auto-create the workspace.

(From OE-Core rev: 90cba7992bc1d227e242666cd486414bd4a45f7e)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-01 21:32:03 +00:00
Ross Burton b3641838d6 oeqa/qemurunner: pass nographic to runqemu if DISPLAY isn't set
Not everyone wants to run the tests with a qemu that has a graphical output, so
allow display to be None and pass nographic to runqemu in that case.

(From OE-Core rev: b1d85f13e79f8a493b9849551357a1a5cf0d0fec)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-01 21:32:01 +00:00
Ross Burton a6c68d85a9 oeqa/selftest/sstatetests: prettier output for allarch test
Instead of creating two lists of full paths and comparing them which in failure
produces a list of every stamp file (so all tasks, twice), reduce the filename
down to a recipe/task->hash dictionary and compare those, meaning unittest
shows the differences in the dictionaries.

In the future get_files() should be generalised so all tests in this class can
use it, and find a pair of hashes that don't match and run diffsigs on them.

(From OE-Core rev: b612628081b81b50965ae9454df4b2747c6997b2)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-11-24 15:55:41 +00:00
Daniel Istrate 92328b43bc oeqa/selftest/signing: Added new test for signing sstate.
[YOCTO #8182] Optional signing sstate archives and signature verification
[YOCTO #8559] Signing sstate archives with custom dir for gpg keys

(From OE-Core rev: 6a462fbb11db2085e4b6763a601c7fc4ac0025c8)

Signed-off-by: Daniel Istrate <daniel.alexandrux.istrate@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-11-24 15:55:41 +00:00
Daniel Istrate fbb03a8c90 oeqa/selftest/signing: New test for Signing packages in the package feeds.
[YOCTO # 8134] This test verifies features introduced in bug 8134.

It requires as resources the files from meta-selftest/files/signing:
For 'gpg --gen-key' the used input was:
key: RSA
key-size: 2048
key-valid: 0
realname: testuser
email: testuser@email.com
comment: nocomment
passphrase: test123

(From OE-Core rev: 6b9d22bfd5414b517a1f0468e1229dfa2294b5fd)

Signed-off-by: Daniel Istrate <daniel.alexandrux.istrate@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-11-24 15:55:41 +00:00
Paul Eggleton 8b2b068064 oe-selftest: add test for bitbake-layers show-recipes
Add a test for bitbake-layers show-recipes including the recently
added -i option.

(From OE-Core rev: da2733ad91cdff8cae2848fec5377ca777820d6b)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-11-16 11:39:39 +00:00
Paul Eggleton 480bbaee58 oeqa/selftest/layerappend: fix test if build directory is not inside COREBASE
Fix test_layer_appends to work when build directory is not inside
COREBASE.

Fixes [YOCTO #8639].

(From OE-Core rev: 0f146e77655d153d3f9a59e489265450f08c6ad7)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-11-16 11:39:39 +00:00
Paul Eggleton a301f6e3b8 oeqa/selftest/devtool: fix test if build directory is not inside COREBASE
Fix test_devtool_update_recipe_git to work when build directory is not
inside COREBASE.

Fixes [YOCTO #8639].

(From OE-Core rev: 0225888207f82e5f1d9e3dffb7c342a10169aea3)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-11-16 11:39:39 +00:00
Mariano Lopez 3b95964bc1 qemurunner: Remove the timeout in run_serial
Sometmes when there is high load in the server the
commands executed in the target take a lot of time
to complete and this lead to incorrect dump files or
empty files. This is caused because run_serial has a
timeout of five seconds when running the commands in
the target.

This change removes the timeout and just keep reading
the socket until it finds the prompt from the target
or when the socket is not ready to be read.

[YOCTO #8510]

(From OE-Core rev: d1a97475b4e6c7066a3161cb9cec1d4b27644518)

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-11-16 11:39:36 +00:00
Leonardo Sandoval e63e1918ff qemurunner: Enable timestamps on kernel boot-up
Having the timestamps in the log is useful when debugging those cases when
the booting time takes longer than expected. printk.times=1 enables stamping
into every printk Kernel call.

(From OE-Core rev: 8f8db89a91142bf71d009df881107b4c1fc61237)

Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-11-16 11:39:34 +00:00
Ross Burton ecbe1359da oeqa/runtime/multilib: run the arch tests on connmand not connman-applet
(From OE-Core rev: 29945c12d43a1cc40b22caf04fe57f25b946e84f)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-11-16 11:39:30 +00:00
Ross Burton 2d1071edb1 oeqa/runtime: remove dmesg test
This has been obsoleted by parselogs.

(From OE-Core rev: 05132fd236835db71d3e763b4d6ce01fcf14741a)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-11-16 11:39:30 +00:00
Paul Eggleton fdfdfc8ccb oeqa/utils/decorators: fix missing keyword arguments on decorators
We need to handle keyword arguments here or sending a keyword argument
to a decorated function that accepts keyword arguments will trigger an
error. (This showed up when testcase decorators were added to the
recipetool.RecipetoolTests.test_recipetool_appendsrcfiles_basic_subdir
test).

(From OE-Core rev: f787b688f2884ce3fa888b4041030538c7d2bf55)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-10-27 07:24:30 +00:00
Daniel Istrate 426a9b78d0 oeqa/selftest: Added testcase decorators.
Added testcase decorators for testopia integration.

(From OE-Core rev: 9c7ffd397c8232d53c87017e58e03e3056863edf)

Signed-off-by: Daniel Istrate <daniel.alexandrux.istrate@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-10-27 07:24:28 +00:00