Commit Graph

32 Commits

Author SHA1 Message Date
Mariano Lopez 19d23814e4 selftest: Optimize get_bb_var use
get_bb_var calls bitbake every time it is used and every call
would take about 7 seconds. There are tests that calls get_bb_var
several times when they can use get_bb_vars. Also there are tests
that calls it to fetch the same variable over and over again.

This will optimize the use of get_bb_var and get_bb_vars for a
little speed up in the tests.

[YOCTO #11037]

(From OE-Core rev: e53f86ba8aeb6d2e9eb259329001d27d62401072)

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-01 23:27:10 +00:00
Mariano Lopez 9365e5f7a9 selftest: Avoid sstate corruption by calling cleansstate
Currently selftest doesn't use sstates because some tests
clean sstate cache; using sstates would give a performance
boost instead of building everything from scratch.

With this sstates are not corrupted using different methods
depending on tests:

devtool: These tests needed to delete the cache so SSTATE_DIR
as SSTATE_MIRRORS and set a temporal SSTATE_DIR.

sstatetests: This module already used a temporal SSTATE_DIR, so
just set up the SSTATE_MIRRORS.

Rest: Removed cleansstate, some of them required to force a
certain task, others were just removed or changed for another
task.

[YOCTO #10929]

(From OE-Core rev: 62c61087a10cc3b26fbff32c9e2efd1704a39724)

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-01 23:27:08 +00:00
Richard Purdie b57e93c5c8 oeqa/selftest/bbtests: Update after changes to dependency graph file generation
(From OE-Core rev: 36a7470cacfe56931948ee865bd9ef46a7303ced)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-02-19 06:47:06 -08:00
Leonardo Sandoval d3c6c10de4 selftest/bbtests: use write_config instead of local.conf file
Extra configuration data should be write using the write_config
method instead of manually appending to the local.conf file

(From OE-Core rev: 65c26d51eedc687c92e8ade262d24dc53ca71cec)

Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-02-02 17:37:43 +00:00
Chen Qi 0967a78689 selftest/bbtests.py: fix path assumption for LICENSE_DIRECTORY
Fix path assumption for LICENSE_DIRECTORY, otherwise, the test case
may fail even if the functionality it tests works well.

(From OE-Core rev: ae388652b8de0665390560e78429e10119d4d537)

Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-16 18:05:13 +00:00
Leonardo Sandoval 279c8ce310 selftest: bbtests: use minimal image so all distros can execute it
poky-tiny distro cannot build full-cmdline image, so use an image
(core-image-minimal) that can be built in all distros.

(From OE-Core rev: b293dd4200bbb3705c88af6113be7f43fbd0ed72)

Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-16 10:23:22 +00:00
Benjamin Esquivel d89e16d5cc oeqa/selftest: Update test after fetcher error changes
The following poky commit:

4359ef08 base.bbclass: Use bb.fatal() instead of raising FuncFailed

changed the way the fetcher error is reported.

Previous reporting:
...Function failed: Fetcher failure for URL:...

New reporting:
...Fetcher failure for URL:...

Updating how the check is done fixes the test error and accurately
confirms the tested scenario for test_invalid_recipe_src_uri.

[YOCTO #10370]

(From OE-Core rev: 197da17dc97cef87375ae9190c6d1495e1c615b9)

Signed-off-by: Benjamin Esquivel <benjamin.esquivel@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-10-05 10:10:10 +01:00
Ross Burton a191285b97 lib/oeqa/selftest/bbtests: don't report expected failures
Another instance where expected failures need to be not reported to the error
reporting service.

(From OE-Core rev: bb1cbb8d5bd7639554edcddf1d2eac4abdbb48c7)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-08-17 10:35:43 +01:00
Ross Burton 9ec7cef3d0 oeqa/selftest: disable report-error class when builds are expected to fail
Some invocations of bitbake are expected to fail, so we don't want to report the
errors to errors.yoctoproject.org.  Also rewrite the messages in
test_invalid_patch so they reflect reality.

[ YOCTO #10052 ]

(From OE-Core rev: 51f74a0d1ce4de9d311becee8e7d7cc7cd703d45)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-08-10 10:46:31 +01:00
Richard Purdie 1bbd580a37 oeqa/selftest/bbtests: Fix failing test after progress changes
The progress patches change the output slightly, update the test to
deal with this.

(From OE-Core rev: 90dbd838fa97c89ace5cb147aa5cff39b94178b1)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-08 09:57:26 +01:00
Daniel Istrate abb5b46ca3 oeqa/selftest/bbtests: Test bbappend order
BitBake should append to recipe in a predictable order.

fix for [YOCTO #9145]
test for [YOCTO #9138]

(From OE-Core rev: 51bef86ce52fdc2455cd1879e3e19d5ccc5c1c9c)

Signed-off-by: Daniel Istrate <daniel.alexandrux.istrate@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-30 12:01:29 +01:00
Daniel Istrate 9cd64edebf oeqa/selftest/bbtests: Test bitbake --setscene-only option
Bitbake option to restore from sstate only within a build
(i.e. execute no real tasks, only setscene)

fix for [YOCTO #8876]

(From OE-Core rev: 3821aee0129b87a859b3a540b82745fdae184721)

Signed-off-by: Daniel Istrate <daniel.alexandrux.istrate@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-18 07:39:30 +00:00
Richard Purdie d585a716d2 oeqa: Update to handle domain specific references in build logs
With the addition of the task name to recipe output, the sanity tests
need updates where they are looking for specific messages.

(From OE-Core rev: 0f2ef4304e6a6f18b4ed13f59000b4a1daa35f6b)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-15 16:28:46 +00:00
Ross Burton 2d008033c5 oeqa/selftest: improve config writing and cleanup
The selftest.inc configuration file is deleted in both tearDown() and setUp() so
there's no need to use addCleanup() to remove statements from it.

Use write_config instead of append_config if the intention is to start from an
empty config file, for clarity.

Finally remove some misleading comments that claim that append_config() writes
to local.conf when it doesn't.

(From OE-Core rev: ad33259b507914bfc8de92d1df12e0974157900e)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-10-21 22:56:07 +01:00
Daniel Istrate 94decbce39 oeqa/selftest/bbtests: Updated bitbake TCs
- Added new TC test_force_task_1 (1354);
check that do_package_write_rpm() re-executes
upon changes in package image.

- Updated TC test_force_task_2 (163);
changed test recipe to zlib and added
do_package() to the task execution list.

- Removed unnecessary imports.

Fix for bug [YOCTO #5875].

(From OE-Core rev: 86a7e68803aca58feca9b6cf479552d14b00996c)

Signed-off-by: Daniel Istrate <daniel.alexandrux.istrate@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-10-21 22:56:06 +01:00
Ross Burton 322c3248e9 oeqa/selftest/bbtests: clean up local DL_DIR/SSTATE_DIR safely
Now that the selftests clean selftest.inc in the tearDown using bitbake in a
function passed to addCleanup (which happens after teardown) will use the
default configuration and not the customised one.

As the intention was to clean away the temporary DL_DIR and SSTATE_DIR, simple
use track_for_cleanup to delete the entire directory instead.

(From OE-Core rev: aa3b859524bb693d3b150a8873a2ee24c78eac0a)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-10-21 22:56:06 +01:00
Costin Constantin 2606cd2774 oeqa/bbtests: add useful failure messages for all test cases
(From OE-Core rev: a6b9cbb08f6052fbfad53444cd0e9eba8599d7b1)

Signed-off-by: Costin Constantin <costin.c.constantin@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-16 15:09:24 +01:00
Richard Purdie 0aaa67a790 oeqa/bbtests: Fix to ensure DL_DIR is set
write_config overwrites the config rather than appends to it, so
ensure we write both variables in one go.

(From OE-Core rev: 50a37ee563d9003e21bfb9280f184cd81c62e9bf)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-08 13:09:28 +01:00
Richard Purdie a24fdcae86 oeqa/bbtests: Fix race over DL_DIR and SSTATE_DIR
Running "-c cleanall" on shared DL_DIR and SSTATE_DIR is antisocial.
It leads to hard to debug races where we wonder why files disappear
and reappear from those directories.

Fix this by using a specific set of directories for these tests. This
avoids a long standing bug on the autobuilder where aspell and man
sources would disappear.

[YOCTO #6276]

(From OE-Core rev: 6b089c4a79dc3aae00c8a6e7ab0f6ba4b4b5f138)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-07 13:33:27 +01:00
Richard Purdie 3360f9343d oeqa/bbtests: Show useful failure message for gplv3 test
If this test fails the current output tells us nothing about what happened.
Show the exit status and output to aid debugging.

(From OE-Core rev: e54437b7e7c5f1b8c33627b3914f896724207837)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-01 15:39:59 +01:00
Richard Purdie 58ecd46c64 oeqa/bbtests: Improve error handling
Currently if a test fails the cleanup isn't performed, leading to cascade
failures in other tests. We can do better.

(From OE-Core rev: db01d6ba2ce87e1586dc8804dd823a0a038d5794)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-01 15:39:59 +01:00
Paul Eggleton a8c98c399f oe-selftest: use spaces to indent python code, not tabs
Indenting-only change.

(From OE-Core rev: b3005576a9e0bcaa47c41f25451d1fe7ef30c7fe)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-03-21 09:42:36 +00:00
Lucian Musat ddc8222443 oeqa/selftest: Added a testcase to verify INCOMPATIBLE_LICENSE option.
Fixes bug 6933

(From OE-Core rev: 984884cbc177300a4fefd56cb027d50b2ff4510c)

Signed-off-by: Lucian Musat <george.l.musat@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-03-10 10:47:44 +00:00
Paul Eggleton 93ba8e0f91 oeqa/selftest: fix test_force_task so it doesn't taint the entire build
Using -C sets a taint on the specified task, and m4-native is in the
dependency chain for just about everything, which means that everything
gets forced to rebuild. We don't need the influence of this test to
extend outside of the test itself, so ensure that the taint gets cleared
at the end of the test.

(From OE-Core rev: 24b41a1e3f2496e38a6ae9c47dfe85887f3d7d6e)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-12-25 08:27:13 +00:00
Paul Eggleton 8bb5dccedb oeqa/selftest: fix test_event_handler for changes in bitbake output
(From OE-Core rev: 3b3c6aa09301c0c19e7fc8720cd3d1b036c5b103)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-12-25 08:27:12 +00:00
Tyler Hall 92785a1a96 lib/oeqa/selftest: Don't match log level in output
To facilitate changing the log level of the "Fetcher failure" message,
search only for the message without the "Error:" prefix.

(From OE-Core rev: 947e6f9005abc71f499f23a4dd3a5a9f8386a369)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-08-11 12:30:52 +01:00
Lucian Musat d8364371b2 oeqs/selftest: Added test case decorators for the rest of selftest testcases except toaster
(From OE-Core rev: 3c820db4624a062644a26de1866123a10a78a006)

Signed-off-by: Lucian Musat <georgex.l.musat@intel.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-07-10 17:38:35 +01:00
Lucian Musat 7279de72c6 Added new test cases for bitbake modes (-e -n -p -r -R -c -k)
(From OE-Core rev: eec4976e3b22efe73e823ad4876d78ad933113f4)

Signed-off-by: Lucian Musat <georgex.l.musat@intel.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-06-06 09:26:04 +01:00
Corneliu Stoicescu 604dd7374b selftest/bbtests.py: Fixed regex and added bitbake output to test_warnings_errors
The test failed when more than 1 error or 1 warning is present.
Also pasting the bitbake output when the test fails.

(From OE-Core rev: abc691026592b406e69f8bf9e4fffe2e6a17fffc)

Signed-off-by: Corneliu Stoicescu <corneliux.stoicescu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-03-27 09:42:07 +00:00
Paul Eggleton 91c372f287 core-image-basic / packagegroup-core-basic: rename to *-full-cmdline
This more accurately represents what this image and packagegroup are
intended to provide (a more complete command-line environment similar
to what you would find on a traditional Linux system), and avoids
confusion with the similarly named core-image-base and
packagegroup-base.

Fixes [YOCTO #2424].

(From OE-Core rev: 959396d5319df6d6599cc74077e85182768b92af)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-03-07 14:45:21 +00:00
Alexandru Palalau f683023b05 oe-selftest: add test for image manifest file creation
(From OE-Core rev: 1542470fa66884d0f7bf67d92916675a8a9820cc)

Signed-off-by: Alexandru Palalau <alexandrux.palalau@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-03-05 15:50:22 +00:00
Corneliu Stoicescu 1611b474b6 lib/oeqa/selftest: add test modules for expected bitbake output and bitbake-layers
Tests for bitbake-layers and expected output for some bitbake options.

(From OE-Core rev: 8408a7700cd9cab4559ddae0bbe57f0d7fae5c37)

Signed-off-by: Corneliu Stoicescu <corneliux.stoicescu@intel.com>
Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-12-03 17:45:51 +00:00