1f6349c3cSSimon GlassTesting in U-Boot 2f6349c3cSSimon Glass================= 3f6349c3cSSimon Glass 4f6349c3cSSimon GlassU-Boot has a large amount of code. This file describes how this code is 5f6349c3cSSimon Glasstested and what tests you should write when adding a new feature. 6f6349c3cSSimon Glass 7f6349c3cSSimon Glass 807f4eadcSSimon GlassRunning tests 907f4eadcSSimon Glass------------- 1007f4eadcSSimon Glass 1107f4eadcSSimon GlassTo run most tests on sandbox, type this: 1207f4eadcSSimon Glass 13*499fde5cSSimon Glass make check 1407f4eadcSSimon Glass 1507f4eadcSSimon Glassin the U-Boot directory. Note that only the pytest suite is run using this 16bcbd0c8fSSimon Glasscommand. 1707f4eadcSSimon Glass 18*499fde5cSSimon GlassSome tests take ages to run. To run just the quick ones, type this: 19*499fde5cSSimon Glass 20*499fde5cSSimon Glass make qcheck 21*499fde5cSSimon Glass 2207f4eadcSSimon Glass 23f6349c3cSSimon GlassSandbox 24f6349c3cSSimon Glass------- 25f6349c3cSSimon GlassU-Boot can be built as a user-space application (e.g. for Linux). This 26f6349c3cSSimon Glassallows test to be executed without needing target hardware. The 'sandbox' 27f6349c3cSSimon Glasstarget provides this feature and it is widely used in tests. 28f6349c3cSSimon Glass 29f6349c3cSSimon Glass 30f6349c3cSSimon GlassPytest Suite 31f6349c3cSSimon Glass------------ 32f6349c3cSSimon Glass 33f6349c3cSSimon GlassMany tests are available using the pytest suite, in test/py. This can run 34f6349c3cSSimon Glasseither on sandbox or on real hardware. It relies on the U-Boot console to 35f6349c3cSSimon Glassinject test commands and check the result. It is slower to run than C code, 36bcbd0c8fSSimon Glassbut provides the ability to unify lots of tests and summarise their results. 37f6349c3cSSimon Glass 38f6349c3cSSimon GlassYou can run the tests on sandbox with: 39f6349c3cSSimon Glass 40f6349c3cSSimon Glass ./test/py/test.py --bd sandbox --build 41f6349c3cSSimon Glass 42f6349c3cSSimon GlassThis will produce HTML output in build-sandbox/test-log.html 43f6349c3cSSimon Glass 44f6349c3cSSimon GlassSee test/py/README.md for more information about the pytest suite. 45f6349c3cSSimon Glass 46f6349c3cSSimon Glass 47f6349c3cSSimon Glasstbot 48f6349c3cSSimon Glass---- 49f6349c3cSSimon Glass 50f6349c3cSSimon GlassTbot provides a way to execute tests on target hardware. It is intended for 51f6349c3cSSimon Glasstrying out both U-Boot and Linux (and potentially other software) on a 52f6349c3cSSimon Glassnumber of boards automatically. It can be used to create a continuous test 53630dfedeSHeiko Schocherenvironment. See http://www.tbot.tools for more information. 54f6349c3cSSimon Glass 55f6349c3cSSimon Glass 56f6349c3cSSimon GlassAd-hoc tests 57f6349c3cSSimon Glass------------ 58f6349c3cSSimon Glass 59f6349c3cSSimon GlassThere are several ad-hoc tests which run outside the pytest environment: 60f6349c3cSSimon Glass 61f6349c3cSSimon Glass test/fs - File system test (shell script) 62bcbd0c8fSSimon Glass test/image - FIT and legacy image tests (shell script and Python) 63f6349c3cSSimon Glass test/stdint - A test that stdint.h can be used in U-Boot (shell script) 64f6349c3cSSimon Glass trace - Test for the tracing feature (shell script) 65f6349c3cSSimon Glass 66bcbd0c8fSSimon GlassTODO: Move these into pytest. 67f6349c3cSSimon Glass 68f6349c3cSSimon Glass 69f6349c3cSSimon GlassWhen to write tests 70f6349c3cSSimon Glass------------------- 71f6349c3cSSimon Glass 72f6349c3cSSimon GlassIf you add code to U-Boot without a test you are taking a risk. Even if you 73f6349c3cSSimon Glassperform thorough manual testing at the time of submission, it may break when 74f6349c3cSSimon Glassfuture changes are made to U-Boot. It may even break when applied to mainline, 75f6349c3cSSimon Glassif other changes interact with it. A good mindset is that untested code 76f6349c3cSSimon Glassprobably doesn't work and should be deleted. 77f6349c3cSSimon Glass 78f6349c3cSSimon GlassYou can assume that the Pytest suite will be run before patches are accepted 79f6349c3cSSimon Glassto mainline, so this provides protection against future breakage. 80f6349c3cSSimon Glass 81f6349c3cSSimon GlassOn the other hand there is quite a bit of code that is not covered with tests, 82f6349c3cSSimon Glassor is covered sparingly. So here are some suggestions: 83f6349c3cSSimon Glass 84f6349c3cSSimon Glass- If you are adding a new uclass, add a sandbox driver and a test that uses it 85f6349c3cSSimon Glass- If you are modifying code covered by an existing test, add a new test case 86f6349c3cSSimon Glass to cover your changes 87f6349c3cSSimon Glass- If the code you are modifying has not tests, consider writing one. Even a 88f6349c3cSSimon Glass very basic test is useful, and may be picked up and enhanced by others. It 89f6349c3cSSimon Glass is much easier to add onto a test - writing a new large test can seem 90f6349c3cSSimon Glass daunting to most contributors. 91f6349c3cSSimon Glass 92f6349c3cSSimon Glass 93f6349c3cSSimon GlassFuture work 94f6349c3cSSimon Glass----------- 95f6349c3cSSimon Glass 96f6349c3cSSimon GlassConverting existing shell scripts into pytest tests. 97