test/py/README.md

1 # U-Boot pytest suite
5 This tool aims to test U-Boot by executing U-Boot shell commands using the
7 U-Boot console, run the entire script of tests against it, and summarize the
11   U-Boot; there can be no disconnect.
12 - There is no need to write or embed test-related code into U-Boot itself.
15 - It is reasonably simple to interact with U-Boot in this way.
19 The test suite is implemented using pytest. Interaction with the U-Boot console
40 - Executing a sandbox port of U-Boot on the local machine as a sub-process,
42 - Executing an external "hook" scripts to flash a U-Boot binary onto a
64 To run the testsuite on the sandbox port (U-Boot built as a native user-space
75 The `--build` option tells U-Boot to compile U-Boot. Alternatively, you may
76 omit this option and build U-Boot yourself, in whatever way you choose, before
79 The test script will attach to U-Boot, execute all valid tests for the board,
90 - Instead of running U-Boot directly, it will be run under gdbserver, with
94 - All timeouts in tests are disabled, allowing U-Boot an arbitrary amount of
95   time to execute commands. This is useful if U-Boot is stopped at a breakpoint
132 - `--build` indicates that the test script should compile U-Boot itself
136 - `--build-dir` sets the directory containing the compiled U-Boot binaries.
146 details. Execute `py.test --version` for a brief summary. Note that U-Boot's
162   U-Boot's console log in real time on pytest's stdout.
172 unrelated to U-Boot features.
183 - `UBOOT_SOURCE_DIR` the U-Boot source directory.
185 - `UBOOT_BUILD_DIR` the U-Boot build directory.
191 This script provides access to the U-Boot console. The script's stdin/stdout
198 If you are able to run U-Boot under a hardware simulator such as qemu, then
201 cause U-Boot to start execution from scratch each time. Hopefully your
209 so that the board executes the particular U-Boot binary to be tested. Often,
210 this involves writing the U-Boot binary to the board's flash ROM. The test
215 cause the newly flashed U-Boot binary to be executed.
220 - Some other process has already written the desired U-Boot binary into the
222 - The board allows U-Boot to be downloaded directly into RAM, and executed
224   may be preferable if available, and if cold boot testing of U-Boot is not
240 this script again to restore U-Boot to an operational state before running the
246 The semantics of this script require that when it is executed, U-Boot will
247 start running from scratch. If the U-Boot binary to be tested has been written
251 to download the U-Boot binary directly into RAM and execute it. This would
252 avoid the need for `u-boot-test-flash` to actually write U-Boot to flash, thus
259 your platform. These scripts are not considered part of U-Boot itself.
270 contained in these scripts must be purely derived from U-Boot source code.
271 Hence, these configuration files are part of the U-Boot source tree too.
276 implemented by a particular board's configuration of U-Boot. For example, a
277 U-Boot configuration may support USB device mode and USB Mass Storage, but this
287 U-Boot source tree, and should be installed outside of the source tree. Users
298 ### U-Boot `.config` feature usage
300 The test scripts rely on various U-Boot `.config` features, either directly in
302 the running U-Boot instance in order to test other features.
320 If U-Boot has already been built:
328 If you want the test script to compile U-Boot for you too, then you likely
342 Details specific to the U-Boot test suite are described below.
345 provides the means to interact with the U-Boot console, and retrieve board and
349 U-Boot console, and returns all output from that command. This allows
351 that certain strings are not seen on the U-Boot console. These include shell
352 error messages and the U-Boot sign-on message (in order to detect unexpected