1f6349c3cSSimon GlassTesting in U-Boot 2f6349c3cSSimon Glass================= 3f6349c3cSSimon Glass 4f6349c3cSSimon GlassU-Boot has a large amount of code. This file describes how this code is 5f6349c3cSSimon Glasstested and what tests you should write when adding a new feature. 6f6349c3cSSimon Glass 7f6349c3cSSimon Glass 807f4eadcSSimon GlassRunning tests 907f4eadcSSimon Glass------------- 1007f4eadcSSimon Glass 1107f4eadcSSimon GlassTo run most tests on sandbox, type this: 1207f4eadcSSimon Glass 1307f4eadcSSimon Glass test/run 1407f4eadcSSimon Glass 1507f4eadcSSimon Glassin the U-Boot directory. Note that only the pytest suite is run using this 16bcbd0c8fSSimon Glasscommand. 1707f4eadcSSimon Glass 1807f4eadcSSimon Glass 19f6349c3cSSimon GlassSandbox 20f6349c3cSSimon Glass------- 21f6349c3cSSimon GlassU-Boot can be built as a user-space application (e.g. for Linux). This 22f6349c3cSSimon Glassallows test to be executed without needing target hardware. The 'sandbox' 23f6349c3cSSimon Glasstarget provides this feature and it is widely used in tests. 24f6349c3cSSimon Glass 25f6349c3cSSimon Glass 26f6349c3cSSimon GlassPytest Suite 27f6349c3cSSimon Glass------------ 28f6349c3cSSimon Glass 29f6349c3cSSimon GlassMany tests are available using the pytest suite, in test/py. This can run 30f6349c3cSSimon Glasseither on sandbox or on real hardware. It relies on the U-Boot console to 31f6349c3cSSimon Glassinject test commands and check the result. It is slower to run than C code, 32bcbd0c8fSSimon Glassbut provides the ability to unify lots of tests and summarise their results. 33f6349c3cSSimon Glass 34f6349c3cSSimon GlassYou can run the tests on sandbox with: 35f6349c3cSSimon Glass 36f6349c3cSSimon Glass ./test/py/test.py --bd sandbox --build 37f6349c3cSSimon Glass 38f6349c3cSSimon GlassThis will produce HTML output in build-sandbox/test-log.html 39f6349c3cSSimon Glass 40f6349c3cSSimon GlassSee test/py/README.md for more information about the pytest suite. 41f6349c3cSSimon Glass 42f6349c3cSSimon Glass 43f6349c3cSSimon Glasstbot 44f6349c3cSSimon Glass---- 45f6349c3cSSimon Glass 46f6349c3cSSimon GlassTbot provides a way to execute tests on target hardware. It is intended for 47f6349c3cSSimon Glasstrying out both U-Boot and Linux (and potentially other software) on a 48f6349c3cSSimon Glassnumber of boards automatically. It can be used to create a continuous test 49*630dfedeSHeiko Schocherenvironment. See http://www.tbot.tools for more information. 50f6349c3cSSimon Glass 51f6349c3cSSimon Glass 52f6349c3cSSimon GlassAd-hoc tests 53f6349c3cSSimon Glass------------ 54f6349c3cSSimon Glass 55f6349c3cSSimon GlassThere are several ad-hoc tests which run outside the pytest environment: 56f6349c3cSSimon Glass 57f6349c3cSSimon Glass test/fs - File system test (shell script) 58bcbd0c8fSSimon Glass test/image - FIT and legacy image tests (shell script and Python) 59f6349c3cSSimon Glass test/stdint - A test that stdint.h can be used in U-Boot (shell script) 60f6349c3cSSimon Glass trace - Test for the tracing feature (shell script) 61f6349c3cSSimon Glass 62bcbd0c8fSSimon GlassTODO: Move these into pytest. 63f6349c3cSSimon Glass 64f6349c3cSSimon Glass 65f6349c3cSSimon GlassWhen to write tests 66f6349c3cSSimon Glass------------------- 67f6349c3cSSimon Glass 68f6349c3cSSimon GlassIf you add code to U-Boot without a test you are taking a risk. Even if you 69f6349c3cSSimon Glassperform thorough manual testing at the time of submission, it may break when 70f6349c3cSSimon Glassfuture changes are made to U-Boot. It may even break when applied to mainline, 71f6349c3cSSimon Glassif other changes interact with it. A good mindset is that untested code 72f6349c3cSSimon Glassprobably doesn't work and should be deleted. 73f6349c3cSSimon Glass 74f6349c3cSSimon GlassYou can assume that the Pytest suite will be run before patches are accepted 75f6349c3cSSimon Glassto mainline, so this provides protection against future breakage. 76f6349c3cSSimon Glass 77f6349c3cSSimon GlassOn the other hand there is quite a bit of code that is not covered with tests, 78f6349c3cSSimon Glassor is covered sparingly. So here are some suggestions: 79f6349c3cSSimon Glass 80f6349c3cSSimon Glass- If you are adding a new uclass, add a sandbox driver and a test that uses it 81f6349c3cSSimon Glass- If you are modifying code covered by an existing test, add a new test case 82f6349c3cSSimon Glass to cover your changes 83f6349c3cSSimon Glass- If the code you are modifying has not tests, consider writing one. Even a 84f6349c3cSSimon Glass very basic test is useful, and may be picked up and enhanced by others. It 85f6349c3cSSimon Glass is much easier to add onto a test - writing a new large test can seem 86f6349c3cSSimon Glass daunting to most contributors. 87f6349c3cSSimon Glass 88f6349c3cSSimon Glass 89f6349c3cSSimon GlassFuture work 90f6349c3cSSimon Glass----------- 91f6349c3cSSimon Glass 92f6349c3cSSimon GlassConverting existing shell scripts into pytest tests. 93