1f6349c3cSSimon GlassTesting in U-Boot 2f6349c3cSSimon Glass================= 3f6349c3cSSimon Glass 4f6349c3cSSimon GlassU-Boot has a large amount of code. This file describes how this code is 5f6349c3cSSimon Glasstested and what tests you should write when adding a new feature. 6f6349c3cSSimon Glass 7f6349c3cSSimon Glass 8*07f4eadcSSimon GlassRunning tests 9*07f4eadcSSimon Glass------------- 10*07f4eadcSSimon Glass 11*07f4eadcSSimon GlassTo run most tests on sandbox, type this: 12*07f4eadcSSimon Glass 13*07f4eadcSSimon Glass test/run 14*07f4eadcSSimon Glass 15*07f4eadcSSimon Glassin the U-Boot directory. Note that only the pytest suite is run using this 16*07f4eadcSSimon Glasscomment. 17*07f4eadcSSimon Glass 18*07f4eadcSSimon Glass 19f6349c3cSSimon GlassSandbox 20f6349c3cSSimon Glass------- 21f6349c3cSSimon GlassU-Boot can be built as a user-space application (e.g. for Linux). This 22f6349c3cSSimon Glassallows test to be executed without needing target hardware. The 'sandbox' 23f6349c3cSSimon Glasstarget provides this feature and it is widely used in tests. 24f6349c3cSSimon Glass 25f6349c3cSSimon Glass 26f6349c3cSSimon GlassPytest Suite 27f6349c3cSSimon Glass------------ 28f6349c3cSSimon Glass 29f6349c3cSSimon GlassMany tests are available using the pytest suite, in test/py. This can run 30f6349c3cSSimon Glasseither on sandbox or on real hardware. It relies on the U-Boot console to 31f6349c3cSSimon Glassinject test commands and check the result. It is slower to run than C code, 32f6349c3cSSimon Glassbut provides the ability to unify lots of test and summarise their results. 33f6349c3cSSimon Glass 34f6349c3cSSimon GlassYou can run the tests on sandbox with: 35f6349c3cSSimon Glass 36f6349c3cSSimon Glass ./test/py/test.py --bd sandbox --build 37f6349c3cSSimon Glass 38f6349c3cSSimon GlassThis will produce HTML output in build-sandbox/test-log.html 39f6349c3cSSimon Glass 40f6349c3cSSimon GlassSee test/py/README.md for more information about the pytest suite. 41f6349c3cSSimon Glass 42f6349c3cSSimon Glass 43f6349c3cSSimon Glasstbot 44f6349c3cSSimon Glass---- 45f6349c3cSSimon Glass 46f6349c3cSSimon GlassTbot provides a way to execute tests on target hardware. It is intended for 47f6349c3cSSimon Glasstrying out both U-Boot and Linux (and potentially other software) on a 48f6349c3cSSimon Glassnumber of boards automatically. It can be used to create a continuous test 49f6349c3cSSimon Glassenvironment. See tools/tbot/README for more information. 50f6349c3cSSimon Glass 51f6349c3cSSimon Glass 52f6349c3cSSimon GlassAd-hoc tests 53f6349c3cSSimon Glass------------ 54f6349c3cSSimon Glass 55f6349c3cSSimon GlassThere are several ad-hoc tests which run outside the pytest environment: 56f6349c3cSSimon Glass 57f6349c3cSSimon Glass test/fs - File system test (shell script) 58f6349c3cSSimon Glass test/image - FIT and lagacy image tests (shell script and Python) 59f6349c3cSSimon Glass test/stdint - A test that stdint.h can be used in U-Boot (shell script) 60f6349c3cSSimon Glass trace - Test for the tracing feature (shell script) 61f6349c3cSSimon Glass vboot - Test for verified boot (shell script) 62f6349c3cSSimon Glass 63f6349c3cSSimon GlassThe above should be converted to run as part of the pytest suite. 64f6349c3cSSimon Glass 65f6349c3cSSimon Glass 66f6349c3cSSimon GlassWhen to write tests 67f6349c3cSSimon Glass------------------- 68f6349c3cSSimon Glass 69f6349c3cSSimon GlassIf you add code to U-Boot without a test you are taking a risk. Even if you 70f6349c3cSSimon Glassperform thorough manual testing at the time of submission, it may break when 71f6349c3cSSimon Glassfuture changes are made to U-Boot. It may even break when applied to mainline, 72f6349c3cSSimon Glassif other changes interact with it. A good mindset is that untested code 73f6349c3cSSimon Glassprobably doesn't work and should be deleted. 74f6349c3cSSimon Glass 75f6349c3cSSimon GlassYou can assume that the Pytest suite will be run before patches are accepted 76f6349c3cSSimon Glassto mainline, so this provides protection against future breakage. 77f6349c3cSSimon Glass 78f6349c3cSSimon GlassOn the other hand there is quite a bit of code that is not covered with tests, 79f6349c3cSSimon Glassor is covered sparingly. So here are some suggestions: 80f6349c3cSSimon Glass 81f6349c3cSSimon Glass- If you are adding a new uclass, add a sandbox driver and a test that uses it 82f6349c3cSSimon Glass- If you are modifying code covered by an existing test, add a new test case 83f6349c3cSSimon Glass to cover your changes 84f6349c3cSSimon Glass- If the code you are modifying has not tests, consider writing one. Even a 85f6349c3cSSimon Glass very basic test is useful, and may be picked up and enhanced by others. It 86f6349c3cSSimon Glass is much easier to add onto a test - writing a new large test can seem 87f6349c3cSSimon Glass daunting to most contributors. 88f6349c3cSSimon Glass 89f6349c3cSSimon Glass 90f6349c3cSSimon GlassFuture work 91f6349c3cSSimon Glass----------- 92f6349c3cSSimon Glass 93f6349c3cSSimon GlassConverting existing shell scripts into pytest tests. 94