Many error situations in U-Boot print the message:
### ERROR ### Please RESET the board ###
Add this to the list of bad patterns the test system detects. One
practical advantage of this change is to detect the case where sandbox
is told to use a particular DTB file, and the file cannot be opened.
Signed-off-by: Stephen Warren <swarren@nvidia.com>
Acked-by: Simon Glass <sjg@chromium.org>
Currently, bad patterns are only honored when executing a shell command.
Other cases, such as the initial boot-up of U-Boot or when interacting
with command output rather than gathering all output prior to the shell
prompt, do not currently look for bad patterns in console output. This
patch makes sure that bad patterns are honored everywhere.
One benefit of this change is that if U-Boot sandbox fails to start up,
the error message it emits can be caught immediately, rather than relying
on a (long) timeout when waiting for the expected signon message and/or
command prompt.
Signed-off-by: Stephen Warren <swarren@nvidia.com>
Acked-by: Simon Glass <sjg@chromium.org>
A future patch will use the bad_patterns array in multiple places. Rather
than duplicating the code to calculate it, or even sharing it in a
function and simply calling it redundantly when nothing has changed, only
re-calculate the list when some change is made to it. This reduces work.
Signed-off-by: Stephen Warren <swarren@nvidia.com>
Acked-by: Simon Glass <sjg@chromium.org>
Python's coding style docs indicate to use " not ' for docstrings.
test/py has other violations of the coding style docs, since the docs
specify a stranger style than I would expect, but nobody has complained
about those yet:-)
Signed-off-by: Stephen Warren <swarren@nvidia.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
test/py contains logic to detect the target crashing and rebooting by
searching the console output for a U-Boot signon message, which will
presumably be emitted when the system boots after the crash/reset.
Currently, this logic only searches for the exact signon message that
was printed by the U-Boot version under test, upon the assumption that
binary is written into flash, and hence will be the version booted after
any reset. However, this is not a valid assumption; some test setups
download the U-Boot-under-test into RAM and boot it from there, and in
such a scenario an arbitrary U-Boot version may be located in flash and
hence run after any reset.
Fix the reset detection logic to match any U-Boot signon message. This
prevents false negatives.
Signed-off-by: Stephen Warren <swarren@nvidia.com>
Acked-by: Simon Glass <sjg@chromium.org>
find_ram_base() is a shared utility function, not a core part of the
U-Boot console interaction.
Signed-off-by: Stephen Warren <swarren@nvidia.com>
Acked-by: Simon Glass <sjg@chromium.org>
Add various common utility functions. These will be used by a forthcoming
re-written UMS test, and a brand-new DFU test.
Signed-off-by: Stephen Warren <swarren@nvidia.com>
Acked-by: Simon Glass <sjg@chromium.org>
Write a note to the log file when a test sends CTRL-C to U-Boot. This
makes it easier to follow what's happening in the logs, especially since
U-Boot doesn't echo the character back to its output, so there's no other
signal of what's going on.
Signed-off-by: Stephen Warren <swarren@nvidia.com>
Acked-by: Simon Glass <sjg@chromium.org>
Tests may fail for a number of reasons, and in particular for reasons
other than a timeout waiting for U-Boot to print expected data. If the
last operation that a failed test performs is not waiting for U-Boot to
print something, then any trailing output from U-Boot during that test's
operation will not be logged as part of that test, but rather either
along with the next test, or even thrown away, potentiall hiding clues
re: the test failure reason.
Solve this by explicitly draining (and hence logging) the U-Boot output
in the case of failed tests.
Signed-off-by: Stephen Warren <swarren@nvidia.com>
Acked-by: Simon Glass <sjg@chromium.org>
Prior to this change, U-Boot was lazilly (re-)spawned if/when a test
attempted to interact with it, and no active connection existed. This
approach was simple, yet had the disadvantage that U-Boot might be
spawned in the middle of a test function, e.g. after the test had already
performed actions such as creating data files, etc. In that case, this
could cause the log to contain the sequence (1) some test logs, (2)
U-Boot's boot process, (3) the rest of that test's logs. This isn't
optimally readable. This issue will affect the upcoming DFU and enhanced
UMS tests.
This change converts u_boot_console to be a function-scoped fixture, so
that pytest attempts to re-create the object for each test invocation.
This allows the fixture factory function to ensure that U-Boot is spawned
prior to every test. In practice, the same object is returned each time
so there is essentially no additional overhead due to this change.
This allows us to remove:
- The explicit ensure_spawned() call from test_sleep, since the core now
ensures that the spawn happens before the test code is executed.
- The laxy calls to ensure_spawned() in the u_boot_console_*
implementations.
The one downside is that test_env's "state_ttest_env" fixture must be
converted to a function-scoped fixture too, since a module-scoped fixture
cannot use a function-scoped fixture. To avoid overhead, we use the same
trick of returning the same object each time.
Signed-off-by: Stephen Warren <swarren@nvidia.com>
Acked-by: Simon Glass <sjg@chromium.org>
This tool aims to test U-Boot by executing U-Boot shell commands using the
console interface. A single top-level script exists to execute or attach
to the U-Boot console, run the entire script of tests against it, and
summarize the results. Advantages of this approach are:
- Testing is performed in the same way a user or script would interact
with U-Boot; there can be no disconnect.
- There is no need to write or embed test-related code into U-Boot itself.
It is asserted that writing test-related code in Python is simpler and
more flexible that writing it all in C.
- It is reasonably simple to interact with U-Boot in this way.
A few simple tests are provided as examples. Soon, we should convert as
many as possible of the other tests in test/* and test/cmd_ut.c too.
The hook scripts, relay control utilities, and udev rules I use for my
own HW setup are published at https://github.com/swarren/uboot-test-hooks.
See README.md for more details!
Signed-off-by: Stephen Warren <swarren@wwwdotorg.org>
Signed-off-by: Stephen Warren <swarren@nvidia.com>
Tested-by: Michal Simek <michal.simek@xilinx.com>
Tested-by: Simon Glass <sjg@chromium.org>
Acked-by: Simon Glass <sjg@chromium.org> #v3
Stephen Warren
Stephen Warren