Lately autobuilders are experiencing hangs with selftest,
it seems it is cause if an error happens in setUpClass
method of oeSDKExtSelfTest class because HTTP server
keeps running in background.
This patch will ensure tearDownClass will be run if there
is an error in setUpClass.
(From OE-Core rev: eb1383949f76c6eb36f86c051057f761a71016a3)
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The idea on getTarget is to use kwargs to send custom variables
to different targets, instead of this, a new variable was added
(just used for custom targets) and this broke testexport. So
in order to fix it, just add the custom variable to kwargs.
This fixes the use of getTarget() in testexport class that was
introduced in 1dc8010afd71fe46fb28bb86fb7c07a5fbd3d7cf.
(From OE-Core rev: cf138029a1f18f991fc7a28c81d85884942e9d56)
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The word 'uninstall' was spelled incorrectly in various parts of
oeqa files.
(From OE-Core rev: af365e025030436f83b233fa51ecc9c58c58ce3c)
(From OE-Core rev: 9d25188e873645b849584b51a77b86588a51d4ba)
Signed-off-by: Humberto Ibarra <humberto.ibarra.lopez@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This test checks to make sure only the files expected exist in a
container image. Currently only ROOTFS_BOOTSTRAP_INSTALL, gets added to
all images without the user specifying it.
But this test should help if a developer in the future ever silently
adds more than just ROOTFS_BOOTSTRAP_INSTALL, and that the developer can
make sure it also gets removed from a container image.
[YOCTO #9502]
(From OE-Core rev: 4f5e5321fafe491f91d78a35d11afc0dc0527d4b)
Signed-off-by: Randy Witt <randy.e.witt@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Based partially on an earlier patch by Maciej Borzecki.
Note that because tools now reside in recipe specific sysroots, we
need to import the path from bitbake and apply it during the test.
(From OE-Core rev: c826233ad08ee5a4b9943a05d4e73f3fb3281588)
Signed-off-by: Kristian Amlie <kristian.amlie@mender.io>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
because of the way that the test cases are looked in the results,
the file logger is incompatible with extended implementations of the
unittest runner.
as the xml runner extends the unittest runner, it shares the id()
method which returns the full name of the test, not only the test
function name. With that, a single check of the full name reviews at
the same time the class name as well as the function name.
[YOCTO#11012]
(From OE-Core rev: 19b025333846018fd3e4ee4ca5cc18d375fa6213)
Signed-off-by: Benjamin Esquivel <benjamin.esquivel@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Stop capturing output of the shell commands into <test>/commands.log.
Redirecting output into a file prevented the unittest framework from
capturing it, causing useless errors (with empty output) like:
oeqa.utils.CommandError: Command '['bitbake', 'core-image-sato']'
returned non-zero exit status 1 with output:
In general, the console output of commands is only interesting when
something fails. Also, dropping the commands.log file is a huge saving
in disk space, and thus, repository size when results are archived in
Git.
(From OE-Core rev: e004664287ec03e7367a7bf553d9a3038444e82e)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This patch implements custom target loading for testimage, currently
missing due to major changes to the test framework.
Custom targets can be defined in various meta-layers, so we
need an extra path information in order to find them.
Any other additional info is retrieved as usual via the variables
TEST_TARGET and TEST_SUITES
(From OE-Core rev: 1dc8010afd71fe46fb28bb86fb7c07a5fbd3d7cf)
Signed-off-by: Juro Bystricky <juro.bystricky@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Without this an exception would occur
(From OE-Core rev: d1bda068d4bc0ad3ff81ab1f104dc12cedd20b3f)
Signed-off-by: Juro Bystricky <juro.bystricky@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add 'elf' as also supported by OEQemuTarget.
(From OE-Core rev: c1ec42ee9beb1603012db5cf990cdbf1596b2a3d)
Signed-off-by: Juro Bystricky <juro.bystricky@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Change conditional to avoid losing stack traces when log
into the logger. The logic only needs to don't log empty lines.
(From OE-Core rev: 545dd180cc27a69732f2332565038bd06d01c014)
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The pkgdata test broke due to oe-core cf0fd8b which added alternatives support
to bash, so the binary is now /bin/bash.bash.
Instead of fixing the test case for this new name, use the m4 recipe instead as
it has less build dependencies.
(From OE-Core rev: 68da6c98841f4450736ae170e443aa744c8b77eb)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In an attempt to make them more compact and coherent.
(From OE-Core rev: 7e7252108d5e3503888efe0e603b3883fdc793e8)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Updates decorators for eSDK in test cases at:
sdkext/cases/devtool.py and oeqa/selftest/eSDK.py
(From OE-Core rev: 36783b17085e73bbf47242e87bbf054352ae0a9d)
Signed-off-by: Francisco Pedraza <francisco.j.pedraza.gonzalez@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Removed 'IMAGE_FSTYPES = "iso" as this functionality depends on
do_bootimg, which is going to be obsoleted soon.
As wic doesn't depend on bootimg it's safe to remove this.
(From OE-Core rev: a08e2019223d7c04916967d8fd42279e8e1e186f)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
STAGING_DIR variable is used to get path to a boot dir.
It's better to use wic-tools recipe to it as it contains
all bootloader artifacts.
Modified test_build_artifacts and test_rootfs_artifacts to
use wic-tools target to get STAGING_DIR.
(From OE-Core rev: eb648ac5566fbcc679e8544fd8403a730ce71058)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The runtime oeqa tests were failing in the on-target case, only for
aarch64. This was because an old config.sub was being used.
Similar to the SDK testing, call gnu-configize in the on-target
case too to resolve the failing tests.
(From OE-Core rev: 7918fe08d8e48f46cf2402afa66d35063eb72545)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Printing a message which says "configure failed" without the log output
is effectively useless. If a command fails, print the output by default
and simplify the calling code which makes debugging any of these failures
much easier.
(From OE-Core rev: b6352ff001c29f0bff10c18879b92c5618ec645c)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently test_postinst_roofs_and_boot is building a full-cmdline image
this is taking a lot of time to execute the test, so a minimal image
will be build instead.
(From OE-Core rev: 3a6b0b9f94b7097a825edf3fe258a2f86b73edd6)
Signed-off-by: Jose Perez Carranza <jose.perez.carranza@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Extra configuration data should be write using the write_config
method instead of manually appending to the local.conf file
(From OE-Core rev: 65c26d51eedc687c92e8ade262d24dc53ca71cec)
Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Removed hddimg from FSTYPES in wic test suite as
wic doesn't depend on hddimg anymore.
[YOCTO #10835]
(From OE-Core rev: a332d9618c2d0f46b6c0c8fdeee8bec4cc73f2cf)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The run-tests option is optional so if isn't specified set
to None instead of crash on split().
(From OE-Core rev: 877732406cad2ee4bf11293dc8ccddc2998ffbaf)
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Debian based distros has a builtin syslog module so when
try to load tests using unittest it references the builtin
module instead of runtime/cases.
[YOCTO #10964]
(From OE-Core rev: 9923e3cdb58c2b3c54ec5fe99b2cec4cdc9fff92)
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This is a complementary fix to commit 7c552996:
[ meta: remove True option to getVar calls ]
it intended to remove all True option to getVar calls, but there are
still some remaining.
Search made with the following regex: getVar ?\((.*), True\)
(From OE-Core rev: 87d03ffe03d6f01e360bfd51714be96e62506e0a)
Signed-off-by: Ming Liu <peter.x.liu@external.atlascopco.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
With the change to the new framework data store dependecy was
removed, instead a new file is generated and used in testimage.
When testing builds from the autobuilders the test data values
are from the autobuilder, including the paths.
Some tests require paths to current environment in order to run,
this commit will update such paths and fix the error of running
images donwloaded from autobuilders.
[YOCTO #10964]
(From OE-Core rev: 26ad5105fc2ce03b7ee8ecc6911fd40a52bd573a)
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
smart test requires to build the image using rpm packages, this check was
included, but it checked for PACKAGE_CLASSES=='package_rpm', and this is
not true when building packages for rpm and deb/ipk. So this would check
IMAGE_PKGTYPE instead.
[YOCTO #10964]
(From OE-Core rev: 570f50b30072a10c5eadfbbfea2647339ee85176)
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
oeRuntimeTest class is not used anymore as part of runtime
migration, this particular case was missed, so fix it.
[YOCTO #10964]
(From OE-Core rev: d928e5166d8812e7e20bc95c0a8b5cb5439ec497)
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Updates sanity tests to cope with minimal eSDK installer
1. Skips the validation of sanity if packagegroup-cross-canadian is in
host package.
2. Skips if SDK does not include toolchain at cases/devtool.py
This should fix [YOCTO #10794]
(From OE-Core rev: 59a99cd8661af594ee0c33e086578d29d50a6268)
Signed-off-by: Francisco Pedraza <francisco.j.pedraza.gonzalez@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Added "iso" to IMAGE_FSTYPES to build iso artifacts required
to fix test of isoimage-isohybrid wic plugin.
(From OE-Core rev: 055d6814f6de2e54ae27c5fa2e0c40e3e531590e)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Removed hddimg from IMAGE_FEATURES as wic code doesn't
use hddimg anymore.
[YOCTO #10835]
(From OE-Core rev: 885a5beae86a55cc228fb2aa998a8b1135371eb4)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Rather than have each image type look for artifacts in image-specific
locations, move towards having them look for artifacts in a common
location, in this case DEPLOY_DIR_IMAGE
Use the existing deploy.bbclass to have the bootloaders put their binaries
in DEPLOY_DIR_IMAGE and then wic will find them and place them in the image
(From OE-Core rev: 1c8acea91e775b1b6f8b25e774aaba3e790fc59b)
Signed-off-by: Alejandro Hernandez <alejandro.hernandez@linux.intel.com>
Signed-off-by: Tom Zanussi <tom.zanussi@linux.intel.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Removed HDDDIR as it's not used by wic anymore.
Stopped usage of HDDDIR in wic test suite.
(From OE-Core rev: b3ed3a79852f85651c8fa192a8118f9904e88ef8)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
INITRD variable is not set if hddimg is disabled.
isoimage-isohybrid can't get correct name for initrd if INITRD
variable is not set.
Added INITRD_LIVE to WICVARS and used it in isoimage-isohybrid
code to get initrd artifact name. Used INITRD if INITRD_LIVE is not set.
(From OE-Core rev: c672753be60550b15d5d593c251b61776d5b104d)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Tested that result images are produced in test_image_vars_dir_long
and test_image_vars_dir_short test cases.
(From OE-Core rev: 9ed399b99e40ec9ec76df1fbdedfb8104a463acc)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Split tests to run wic only once per test case.
This should fix failures caused by result images left
from the previous wic run.
(From OE-Core rev: 71e3c631bac859d00cef05bbf7bd901a96bac520)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Current location of .env files $STAGING_DIR/imagedata. It doesn't
depend on machine and be rewritten by the builds for different
machines.
Changed location to $STAGING_DIR/$MACHINE/imagedata to avoid .env
files to be rewritten.
(From OE-Core rev: 94245144f5cef344d90bc2a7b3267cdae9d192e4)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
As tests now explicitly specify output directory we don't
need test_alternate_output_dir test case. However, we need
to test wic output to default output location.
Removed test_alternate_output_dir test case.
Added test_default_output_dir test case.
(From OE-Core rev: c1be1161cf555727aa56ee2109ee77f420f67c9f)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
wic started to use current directory as a default output dir.
Specified output directory in wic command line to make tests
more predictable and easier to maintain.
(From OE-Core rev: 0f205aed90edd90fcba0a565a6cadfe96339b410)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Splitted to test_debug_short and test_debug_long to make each
of the test cases to run wic once. This is consistent with the
rest of the test cases and ensures that test cases are set up
properly.
This also fixes the following test failure caused by the image
left from the first wic run:
FAIL: test_debug (oeqa.selftest.wic.Wic)
Test debug
----------------------------------------------------------------------
Traceback (most recent call last):
File "meta/lib/oeqa/utils/decorators.py", line 109, in wrapped_f
return func(*args, **kwargs)
File "meta/lib/oeqa/selftest/wic.py", line 270, in test_debug
self.assertEqual(1, len(glob(self.resultdir + "directdisk-*.direct")))
AssertionError: 1 != 2
(From OE-Core rev: a453009821d7d7cb1dd8a7ae05c0d32227b4753b)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Some distros uses non-glibc so skip those tests requiring glibc-initial as target
(From OE-Core rev: 41f02cb7307faec32b13733e41f78a0ebdc1b52b)
Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If you use subprocess.check_output() the traceback will contain the output
when the command fails which is very useful for debugging. There is no
good reason not to use this everywhere.
(From OE-Core rev: ad750dd1cc9d789abe723daddd098ce41d8547f5)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
1. Test case adds perl recipe before build eSDK.
2. After this added recipe, the script verifies the update:
${SDK_DEPLOY}/${TOOLCHAINEXT_OUTPUTNAME}.sh
oe-publish-sdk [esdk] [path-to-http]
CORE_IMAGE_EXTRA_INSTALL = "perl"
bitbake -c populate-sdk-ext [some-image]
devtool sdk-update
This should fix [YOCTO #9369]
(From OE-Core rev: 21bd406bf89e9ceafe1a807877406be817cacca6)
Signed-off-by: Francisco Pedraza <francisco.j.pedraza.gonzalez@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Store measurements as a dict, instead of an array, in the JSON report.
This change makes traversing of the report much easier. The change also
disallows identically named measurements under one test, as a sanity
check for the test cases.
[YOCTO #10590]
(From OE-Core rev: 81065092f38c9631dcf5917d70a25809a21de5f4)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Use OrderedDict() instead of sort_keys=True (of json.dump()). Makes for
more logical sorting of the values in the report.
[YOCTO #10590]
(From OE-Core rev: 75e8aec0e0d81888be47b35c3c84df73edb91868)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Mariano Lopez
Humberto Ibarra
Randy Witt
Richard Purdie
Kristian Amlie
Benjamin Esquivel
Markus Lehtonen
Juro Bystricky
Aníbal Limón
Ross Burton
Francisco Pedraza
Ed Bartosh
Jose Perez Carranza
Leonardo Sandoval
Ming Liu
Tom Zanussi