1c23a283bSBrendan Higgins.. SPDX-License-Identifier: GPL-2.0 2c23a283bSBrendan Higgins 3c23a283bSBrendan Higgins========================== 4c23a283bSBrendan HigginsFrequently Asked Questions 5c23a283bSBrendan Higgins========================== 6c23a283bSBrendan Higgins 7b3606442SHarinder SinghHow is this different from Autotest, kselftest, and so on? 8b3606442SHarinder Singh========================================================== 9c23a283bSBrendan HigginsKUnit is a unit testing framework. Autotest, kselftest (and some others) are 10c23a283bSBrendan Higginsnot. 11c23a283bSBrendan Higgins 12c23a283bSBrendan HigginsA `unit test <https://martinfowler.com/bliki/UnitTest.html>`_ is supposed to 13b3606442SHarinder Singhtest a single unit of code in isolation and hence the name *unit test*. A unit 14b3606442SHarinder Singhtest should be the finest granularity of testing and should allow all possible 15b3606442SHarinder Singhcode paths to be tested in the code under test. This is only possible if the 16b3606442SHarinder Singhcode under test is small and does not have any external dependencies outside of 17c23a283bSBrendan Higginsthe test's control like hardware. 18c23a283bSBrendan Higgins 19c23a283bSBrendan HigginsThere are no testing frameworks currently available for the kernel that do not 20b3606442SHarinder Singhrequire installing the kernel on a test machine or in a virtual machine. All 21b3606442SHarinder Singhtesting frameworks require tests to be written in userspace and run on the 22b3606442SHarinder Singhkernel under test. This is true for Autotest, kselftest, and some others, 23b3606442SHarinder Singhdisqualifying any of them from being considered unit testing frameworks. 24c23a283bSBrendan Higgins 25c23a283bSBrendan HigginsDoes KUnit support running on architectures other than UML? 26c23a283bSBrendan Higgins=========================================================== 27c23a283bSBrendan Higgins 28b3606442SHarinder SinghYes, mostly. 29c23a283bSBrendan Higgins 30b3606442SHarinder SinghFor the most part, the KUnit core framework (what we use to write the tests) 31b3606442SHarinder Singhcan compile to any architecture. It compiles like just another part of the 326ae2bfd3SAlan Maguirekernel and runs when the kernel boots, or when built as a module, when the 33b3606442SHarinder Singhmodule is loaded. However, there is infrastructure, like the KUnit Wrapper 3443ca52a9STales Aparecida(``tools/testing/kunit/kunit.py``) that might not support some architectures 3543ca52a9STales Aparecida(see :ref:`kunit-on-qemu`). 36c23a283bSBrendan Higgins 37b3606442SHarinder SinghIn short, yes, you can run KUnit on other architectures, but it might require 38b3606442SHarinder Singhmore work than using KUnit on UML. 39c23a283bSBrendan Higgins 40c23a283bSBrendan HigginsFor more information, see :ref:`kunit-on-non-uml`. 41c23a283bSBrendan Higgins 42*3920c725STales Aparecida.. _kinds-of-tests: 43*3920c725STales Aparecida 44b3606442SHarinder SinghWhat is the difference between a unit test and other kinds of tests? 45b3606442SHarinder Singh==================================================================== 46c23a283bSBrendan HigginsMost existing tests for the Linux kernel would be categorized as an integration 47c23a283bSBrendan Higginstest, or an end-to-end test. 48c23a283bSBrendan Higgins 49b3606442SHarinder Singh- A unit test is supposed to test a single unit of code in isolation. A unit 50b3606442SHarinder Singh test should be the finest granularity of testing and, as such, allows all 51b3606442SHarinder Singh possible code paths to be tested in the code under test. This is only possible 52b3606442SHarinder Singh if the code under test is small and does not have any external dependencies 53b3606442SHarinder Singh outside of the test's control like hardware. 54c23a283bSBrendan Higgins- An integration test tests the interaction between a minimal set of components, 55c23a283bSBrendan Higgins usually just two or three. For example, someone might write an integration 56c23a283bSBrendan Higgins test to test the interaction between a driver and a piece of hardware, or to 57c23a283bSBrendan Higgins test the interaction between the userspace libraries the kernel provides and 58b3606442SHarinder Singh the kernel itself. However, one of these tests would probably not test the 59c23a283bSBrendan Higgins entire kernel along with hardware interactions and interactions with the 60c23a283bSBrendan Higgins userspace. 61c23a283bSBrendan Higgins- An end-to-end test usually tests the entire system from the perspective of the 62c23a283bSBrendan Higgins code under test. For example, someone might write an end-to-end test for the 63c23a283bSBrendan Higgins kernel by installing a production configuration of the kernel on production 64c23a283bSBrendan Higgins hardware with a production userspace and then trying to exercise some behavior 65c23a283bSBrendan Higgins that depends on interactions between the hardware, the kernel, and userspace. 66c63d2dd7SDavid Gow 67b3606442SHarinder SinghKUnit is not working, what should I do? 68b3606442SHarinder Singh======================================= 69c63d2dd7SDavid Gow 70c63d2dd7SDavid GowUnfortunately, there are a number of things which can break, but here are some 71c63d2dd7SDavid Gowthings to try. 72c63d2dd7SDavid Gow 73b3606442SHarinder Singh1. Run ``./tools/testing/kunit/kunit.py run`` with the ``--raw_output`` 74c63d2dd7SDavid Gow parameter. This might show details or error messages hidden by the kunit_tool 75c63d2dd7SDavid Gow parser. 76c63d2dd7SDavid Gow2. Instead of running ``kunit.py run``, try running ``kunit.py config``, 77c63d2dd7SDavid Gow ``kunit.py build``, and ``kunit.py exec`` independently. This can help track 78c63d2dd7SDavid Gow down where an issue is occurring. (If you think the parser is at fault, you 79b3606442SHarinder Singh can run it manually against ``stdin`` or a file with ``kunit.py parse``.) 80b3606442SHarinder Singh3. Running the UML kernel directly can often reveal issues or error messages, 81b3606442SHarinder Singh ``kunit_tool`` ignores. This should be as simple as running ``./vmlinux`` 82b3606442SHarinder Singh after building the UML kernel (for example, by using ``kunit.py build``). 83b3606442SHarinder Singh Note that UML has some unusual requirements (such as the host having a tmpfs 84b3606442SHarinder Singh filesystem mounted), and has had issues in the past when built statically and 85b3606442SHarinder Singh the host has KASLR enabled. (On older host kernels, you may need to run 86b3606442SHarinder Singh ``setarch `uname -m` -R ./vmlinux`` to disable KASLR.) 87c63d2dd7SDavid Gow4. Make sure the kernel .config has ``CONFIG_KUNIT=y`` and at least one test 88c63d2dd7SDavid Gow (e.g. ``CONFIG_KUNIT_EXAMPLE_TEST=y``). kunit_tool will keep its .config 89c63d2dd7SDavid Gow around, so you can see what config was used after running ``kunit.py run``. 90c63d2dd7SDavid Gow It also preserves any config changes you might make, so you can 91c63d2dd7SDavid Gow enable/disable things with ``make ARCH=um menuconfig`` or similar, and then 92c63d2dd7SDavid Gow re-run kunit_tool. 93c63d2dd7SDavid Gow5. Try to run ``make ARCH=um defconfig`` before running ``kunit.py run``. This 94c63d2dd7SDavid Gow may help clean up any residual config items which could be causing problems. 95f7766424SRandy Dunlap6. Finally, try running KUnit outside UML. KUnit and KUnit tests can be 96c63d2dd7SDavid Gow built into any kernel, or can be built as a module and loaded at runtime. 97c63d2dd7SDavid Gow Doing so should allow you to determine if UML is causing the issue you're 98c63d2dd7SDavid Gow seeing. When tests are built-in, they will execute when the kernel boots, and 99c63d2dd7SDavid Gow modules will automatically execute associated tests when loaded. Test results 100c63d2dd7SDavid Gow can be collected from ``/sys/kernel/debug/kunit/<test suite>/results``, and 10143ca52a9STales Aparecida can be parsed with ``kunit.py parse``. For more details, see :ref:`kunit-on-qemu`. 102c63d2dd7SDavid Gow 103c63d2dd7SDavid GowIf none of the above tricks help, you are always welcome to email any issues to 104c63d2dd7SDavid Gowkunit-dev@googlegroups.com. 105