ref-manual, dev-manual: Edits to running automated tests section.
Applied a second round of review edits from Paul Eggleton for the new "Performing Automated Runtime Testing" section. I did some reorganization and some minor wording changes. (From yocto-docs rev: fa8f8e5f0f6c1377a4fcafcd3d933af15ac01ff3) Signed-off-by: Scott Rifenbark <scott.m.rifenbark@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This commit is contained in:
parent
b8bdd92ae6
commit
eb36aaa07f
|
@ -4257,10 +4257,10 @@
|
||||||
|
|
||||||
<para>
|
<para>
|
||||||
The OpenEmbedded build system makes available a series of automated
|
The OpenEmbedded build system makes available a series of automated
|
||||||
tests for images.
|
tests for images to verify runtime functionality.
|
||||||
<note>
|
<note>
|
||||||
The current release of Yocto Project supports these tests
|
Currently, there is only support for running these tests
|
||||||
for QEMU images only.
|
under QEMU.
|
||||||
</note>
|
</note>
|
||||||
These tests are written in Python making use of the
|
These tests are written in Python making use of the
|
||||||
<filename>unittest</filename> module, and the majority of them
|
<filename>unittest</filename> module, and the majority of them
|
||||||
|
@ -4321,37 +4321,6 @@
|
||||||
must accept incoming connections from 192.168.7.0/24,
|
must accept incoming connections from 192.168.7.0/24,
|
||||||
which is the default IP range used for tap devices
|
which is the default IP range used for tap devices
|
||||||
by <filename>runqemu</filename>.</para></listitem>
|
by <filename>runqemu</filename>.</para></listitem>
|
||||||
<listitem><para><emphasis>Have your image built:</emphasis>
|
|
||||||
The image on which you want to run tests needs to
|
|
||||||
be built.
|
|
||||||
For example, the following command would build the
|
|
||||||
<filename>core-image-sato</filename> image when
|
|
||||||
<filename>MACHINE</filename> is set to "qemu".
|
|
||||||
<literallayout class='monospaced'>
|
|
||||||
bitbake core-image-sato
|
|
||||||
</literallayout></para></listitem>
|
|
||||||
<listitem><para><emphasis>Run the tests:</emphasis>
|
|
||||||
You can start the tests automatically or manually.
|
|
||||||
To have the tests start automatically after the
|
|
||||||
OpenEmbedded build system successfully creates a QEMU
|
|
||||||
image, set the
|
|
||||||
<ulink url='&YOCTO_DOCS_REF_URL;#var-TEST_IMAGE'><filename>TEST_IMAGE</filename></ulink>
|
|
||||||
variable to "1" in your
|
|
||||||
<filename>local.conf</filename> file in the
|
|
||||||
<ulink url='&YOCTO_DOCS_DEV_URL;#build-directory'>Build Directory</ulink>.
|
|
||||||
To manually run the tests, globally inherit
|
|
||||||
<ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-testimage'><filename>testimage.class</filename>:</ulink>
|
|
||||||
by editing your <filename>local.conf</filename> file as
|
|
||||||
follows:
|
|
||||||
<literallayout class='monospaced'>
|
|
||||||
INHERIT += "testimage"
|
|
||||||
</literallayout>
|
|
||||||
After editing your
|
|
||||||
<filename>local.conf</filename> file, use BitBake to
|
|
||||||
run the tests:
|
|
||||||
<literallayout class='monospaced'>
|
|
||||||
bitbake -c testimage <qemu-image>
|
|
||||||
</literallayout></para></listitem>
|
|
||||||
</itemizedlist>
|
</itemizedlist>
|
||||||
</para>
|
</para>
|
||||||
|
|
||||||
|
@ -4367,17 +4336,42 @@
|
||||||
<title>Running Tests</title>
|
<title>Running Tests</title>
|
||||||
|
|
||||||
<para>
|
<para>
|
||||||
After setting up to run the tests, use BitBake to start
|
You can start the tests automatically or manually:
|
||||||
the standard suite of tests manually or, if you are running
|
<itemizedlist>
|
||||||
them automatically, build your image:
|
<listitem><para><emphasis>Automatically Running Tests:</emphasis>
|
||||||
<literallayout class='monospaced'>
|
To run the tests automatically after the
|
||||||
bitbake core-image-sato -c testimage
|
OpenEmbedded build system successfully creates an image,
|
||||||
|
first set the
|
||||||
|
<ulink url='&YOCTO_DOCS_REF_URL;#var-TEST_IMAGE'><filename>TEST_IMAGE</filename></ulink>
|
||||||
|
variable to "1" in your <filename>local.conf</filename>
|
||||||
|
file in the
|
||||||
|
<ulink url='&YOCTO_DOCS_DEV_URL;#build-directory'>Build Directory</ulink>:
|
||||||
|
<literallayout class='monospaced'>
|
||||||
|
TEST_IMAGE = "1"
|
||||||
|
</literallayout>
|
||||||
|
Next, simply build your image.
|
||||||
|
If the image successfully builds, the tests will be
|
||||||
|
run:
|
||||||
|
<literallayout class='monospaced'>
|
||||||
bitbake core-image-sato
|
bitbake core-image-sato
|
||||||
</literallayout>
|
</literallayout></para></listitem>
|
||||||
|
<listitem><para><emphasis>Manually Running Tests:</emphasis>
|
||||||
|
To manually run the tests, first globally inherit
|
||||||
|
<ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-testimage'><filename>testimage.class</filename></ulink>
|
||||||
|
by editing your <filename>local.conf</filename> file:
|
||||||
|
<literallayout class='monospaced'>
|
||||||
|
INHERIT += "testimage"
|
||||||
|
</literallayout>
|
||||||
|
Next, use BitBake to run the tests:
|
||||||
|
<literallayout class='monospaced'>
|
||||||
|
bitbake -c testimage <image>
|
||||||
|
</literallayout></para></listitem>
|
||||||
|
</itemizedlist>
|
||||||
</para>
|
</para>
|
||||||
|
|
||||||
<para>
|
<para>
|
||||||
Once the tests start, the following happens:
|
Regardless of how you run the tests, once they start, the
|
||||||
|
following happens:
|
||||||
<itemizedlist>
|
<itemizedlist>
|
||||||
<listitem><para>A copy of the root filesystem is written
|
<listitem><para>A copy of the root filesystem is written
|
||||||
to <filename>${WORKDIR}/testimage</filename>.
|
to <filename>${WORKDIR}/testimage</filename>.
|
||||||
|
@ -4415,11 +4409,10 @@
|
||||||
All test files reside in
|
All test files reside in
|
||||||
<filename>meta/lib/oeqa/runtime</filename> in the
|
<filename>meta/lib/oeqa/runtime</filename> in the
|
||||||
<link linkend='source-directory'>Source Directory</link>.
|
<link linkend='source-directory'>Source Directory</link>.
|
||||||
A test name (sometimes referred to as a module) maps to a
|
A test name maps directly to a Python module.
|
||||||
method name within a class within a module.
|
Each test module may contain a number of individual tests.
|
||||||
Tests can be used within multiple classes and the tests
|
Tests are usually grouped together by the area
|
||||||
are usually grouped together by the area tested (e.g tests for
|
tested (e.g tests for <filename>systemd</filename> reside in
|
||||||
<filename>systemd</filename> reside in
|
|
||||||
<filename>meta/lib/oeqa/runtime/systemd.py</filename>).
|
<filename>meta/lib/oeqa/runtime/systemd.py</filename>).
|
||||||
</para>
|
</para>
|
||||||
|
|
||||||
|
@ -4443,12 +4436,13 @@
|
||||||
variable in <filename>local.conf</filename>.
|
variable in <filename>local.conf</filename>.
|
||||||
Each name in <filename>TEST_SUITES</filename> represents a
|
Each name in <filename>TEST_SUITES</filename> represents a
|
||||||
required test for the image.
|
required test for the image.
|
||||||
Module skipping is not allowed even if a test is not suitable
|
Test modules named within <filename>TEST_SUITES</filename>
|
||||||
for an image (e.g. running the <filename>rpm</filename> tests on
|
cannot be skipped even if a test is not suitable for an image
|
||||||
an image without <filename>rpm</filename>).
|
(e.g. running the rpm tests on an image without
|
||||||
|
<filename>rpm</filename>).
|
||||||
Appending "auto" to <filename>TEST_SUITES</filename> causes the
|
Appending "auto" to <filename>TEST_SUITES</filename> causes the
|
||||||
build system to try to run all tests that are suitable for the
|
build system to try to run all tests that are suitable for the
|
||||||
image (i.e. the build system decides whether to run each test).
|
image (i.e. each test module may elect to skip itself).
|
||||||
</para>
|
</para>
|
||||||
|
|
||||||
<para>
|
<para>
|
||||||
|
@ -4469,13 +4463,8 @@
|
||||||
</para>
|
</para>
|
||||||
|
|
||||||
<para>
|
<para>
|
||||||
The following list summarizes how to run the tests:
|
Here are some things to keep in mind when running tests:
|
||||||
<itemizedlist>
|
<itemizedlist>
|
||||||
<listitem><para>Run the default set of tests simply by
|
|
||||||
running the following:
|
|
||||||
<literallayout class='monospaced'>
|
|
||||||
bitbake <image> -c testimage
|
|
||||||
</literallayout></para></listitem>
|
|
||||||
<listitem><para>The default tests for the image are defined
|
<listitem><para>The default tests for the image are defined
|
||||||
as:
|
as:
|
||||||
<literallayout class='monospaced'>
|
<literallayout class='monospaced'>
|
||||||
|
@ -4509,7 +4498,7 @@
|
||||||
long as
|
long as
|
||||||
<ulink url='&YOCTO_DOCS_REF_URL;#var-BBPATH'><filename>BBPATH</filename></ulink>
|
<ulink url='&YOCTO_DOCS_REF_URL;#var-BBPATH'><filename>BBPATH</filename></ulink>
|
||||||
is extended in the layer's
|
is extended in the layer's
|
||||||
<filename>layer.conf</filename> file as normal.
|
<filename>layer.conf</filename> file as normal).
|
||||||
Just remember that filenames need to map directly to test
|
Just remember that filenames need to map directly to test
|
||||||
(module) names and that you do not use module names that
|
(module) names and that you do not use module names that
|
||||||
collide with existing core tests.
|
collide with existing core tests.
|
||||||
|
|
|
@ -943,8 +943,8 @@
|
||||||
for images.
|
for images.
|
||||||
The class handles loading the tests and starting the image.
|
The class handles loading the tests and starting the image.
|
||||||
<note>
|
<note>
|
||||||
The current release of Yocto Project supports these tests
|
Currently, there is only support for running these tests
|
||||||
for QEMU images only.
|
under QEMU.
|
||||||
</note>
|
</note>
|
||||||
</para>
|
</para>
|
||||||
|
|
||||||
|
|
|
@ -5266,8 +5266,8 @@ PARALLEL_MAKEINST with the description ".
|
||||||
Automatically runs the series of automated tests for
|
Automatically runs the series of automated tests for
|
||||||
images when an image is successfully built.
|
images when an image is successfully built.
|
||||||
<note>
|
<note>
|
||||||
The current release of Yocto Project supports these tests
|
Currently, there is only support for running these tests
|
||||||
for QEMU images only.
|
under QEMU.
|
||||||
</note>
|
</note>
|
||||||
These tests are written in Python making use of the
|
These tests are written in Python making use of the
|
||||||
<filename>unittest</filename> module, and the majority of
|
<filename>unittest</filename> module, and the majority of
|
||||||
|
@ -5322,8 +5322,8 @@ PARALLEL_MAKEINST with the description ".
|
||||||
The OpenEmbedded build system provides a core set of tests
|
The OpenEmbedded build system provides a core set of tests
|
||||||
that can be used against images.
|
that can be used against images.
|
||||||
<note>
|
<note>
|
||||||
The current release of Yocto Project supports these
|
Currently, there is only support for running these tests
|
||||||
tests for QEMU images only.
|
under QEMU.
|
||||||
</note>
|
</note>
|
||||||
Tests include <filename>ping</filename>,
|
Tests include <filename>ping</filename>,
|
||||||
<filename>ssh</filename>, <filename>df</filename> among
|
<filename>ssh</filename>, <filename>df</filename> among
|
||||||
|
|
Loading…
Reference in New Issue