1c23a283bSBrendan Higgins.. SPDX-License-Identifier: GPL-2.0 2c23a283bSBrendan Higgins 3c23a283bSBrendan Higgins========================== 4c23a283bSBrendan HigginsFrequently Asked Questions 5c23a283bSBrendan Higgins========================== 6c23a283bSBrendan Higgins 7c23a283bSBrendan HigginsHow is this different from Autotest, kselftest, etc? 8c23a283bSBrendan Higgins==================================================== 9c23a283bSBrendan HigginsKUnit is a unit testing framework. Autotest, kselftest (and some others) are 10c23a283bSBrendan Higginsnot. 11c23a283bSBrendan Higgins 12c23a283bSBrendan HigginsA `unit test <https://martinfowler.com/bliki/UnitTest.html>`_ is supposed to 13c23a283bSBrendan Higginstest a single unit of code in isolation, hence the name. A unit test should be 14c23a283bSBrendan Higginsthe finest granularity of testing and as such should allow all possible code 15c23a283bSBrendan Higginspaths to be tested in the code under test; this is only possible if the code 16c23a283bSBrendan Higginsunder test is very small and does not have any external dependencies outside of 17c23a283bSBrendan Higginsthe test's control like hardware. 18c23a283bSBrendan Higgins 19c23a283bSBrendan HigginsThere are no testing frameworks currently available for the kernel that do not 20c23a283bSBrendan Higginsrequire installing the kernel on a test machine or in a VM and all require 21c23a283bSBrendan Higginstests to be written in userspace and run on the kernel under test; this is true 22c23a283bSBrendan Higginsfor Autotest, kselftest, and some others, disqualifying any of them from being 23c23a283bSBrendan Higginsconsidered unit testing frameworks. 24c23a283bSBrendan Higgins 25c23a283bSBrendan HigginsDoes KUnit support running on architectures other than UML? 26c23a283bSBrendan Higgins=========================================================== 27c23a283bSBrendan Higgins 28c23a283bSBrendan HigginsYes, well, mostly. 29c23a283bSBrendan Higgins 30c23a283bSBrendan HigginsFor the most part, the KUnit core framework (what you use to write the tests) 31c23a283bSBrendan Higginscan compile to any architecture; it compiles like just another part of the 326ae2bfd3SAlan Maguirekernel and runs when the kernel boots, or when built as a module, when the 336ae2bfd3SAlan Maguiremodule is loaded. However, there is some infrastructure, 34c23a283bSBrendan Higginslike the KUnit Wrapper (``tools/testing/kunit/kunit.py``) that does not support 35c23a283bSBrendan Higginsother architectures. 36c23a283bSBrendan Higgins 37c23a283bSBrendan HigginsIn short, this means that, yes, you can run KUnit on other architectures, but 38c23a283bSBrendan Higginsit might require more work than using KUnit on UML. 39c23a283bSBrendan Higgins 40c23a283bSBrendan HigginsFor more information, see :ref:`kunit-on-non-uml`. 41c23a283bSBrendan Higgins 42c23a283bSBrendan HigginsWhat is the difference between a unit test and these other kinds of tests? 43c23a283bSBrendan Higgins========================================================================== 44c23a283bSBrendan HigginsMost existing tests for the Linux kernel would be categorized as an integration 45c23a283bSBrendan Higginstest, or an end-to-end test. 46c23a283bSBrendan Higgins 47c23a283bSBrendan Higgins- A unit test is supposed to test a single unit of code in isolation, hence the 48c23a283bSBrendan Higgins name. A unit test should be the finest granularity of testing and as such 49c23a283bSBrendan Higgins should allow all possible code paths to be tested in the code under test; this 50c23a283bSBrendan Higgins is only possible if the code under test is very small and does not have any 51c23a283bSBrendan Higgins external dependencies outside of the test's control like hardware. 52c23a283bSBrendan Higgins- An integration test tests the interaction between a minimal set of components, 53c23a283bSBrendan Higgins usually just two or three. For example, someone might write an integration 54c23a283bSBrendan Higgins test to test the interaction between a driver and a piece of hardware, or to 55c23a283bSBrendan Higgins test the interaction between the userspace libraries the kernel provides and 56c23a283bSBrendan Higgins the kernel itself; however, one of these tests would probably not test the 57c23a283bSBrendan Higgins entire kernel along with hardware interactions and interactions with the 58c23a283bSBrendan Higgins userspace. 59c23a283bSBrendan Higgins- An end-to-end test usually tests the entire system from the perspective of the 60c23a283bSBrendan Higgins code under test. For example, someone might write an end-to-end test for the 61c23a283bSBrendan Higgins kernel by installing a production configuration of the kernel on production 62c23a283bSBrendan Higgins hardware with a production userspace and then trying to exercise some behavior 63c23a283bSBrendan Higgins that depends on interactions between the hardware, the kernel, and userspace. 64c63d2dd7SDavid Gow 65c63d2dd7SDavid GowKUnit isn't working, what should I do? 66c63d2dd7SDavid Gow====================================== 67c63d2dd7SDavid Gow 68c63d2dd7SDavid GowUnfortunately, there are a number of things which can break, but here are some 69c63d2dd7SDavid Gowthings to try. 70c63d2dd7SDavid Gow 71c63d2dd7SDavid Gow1. Try running ``./tools/testing/kunit/kunit.py run`` with the ``--raw_output`` 72c63d2dd7SDavid Gow parameter. This might show details or error messages hidden by the kunit_tool 73c63d2dd7SDavid Gow parser. 74c63d2dd7SDavid Gow2. Instead of running ``kunit.py run``, try running ``kunit.py config``, 75c63d2dd7SDavid Gow ``kunit.py build``, and ``kunit.py exec`` independently. This can help track 76c63d2dd7SDavid Gow down where an issue is occurring. (If you think the parser is at fault, you 77c63d2dd7SDavid Gow can run it manually against stdin or a file with ``kunit.py parse``.) 78c63d2dd7SDavid Gow3. Running the UML kernel directly can often reveal issues or error messages 79c63d2dd7SDavid Gow kunit_tool ignores. This should be as simple as running ``./vmlinux`` after 80c63d2dd7SDavid Gow building the UML kernel (e.g., by using ``kunit.py build``). Note that UML 81c63d2dd7SDavid Gow has some unusual requirements (such as the host having a tmpfs filesystem 82c63d2dd7SDavid Gow mounted), and has had issues in the past when built statically and the host 83c63d2dd7SDavid Gow has KASLR enabled. (On older host kernels, you may need to run ``setarch 84c63d2dd7SDavid Gow `uname -m` -R ./vmlinux`` to disable KASLR.) 85c63d2dd7SDavid Gow4. Make sure the kernel .config has ``CONFIG_KUNIT=y`` and at least one test 86c63d2dd7SDavid Gow (e.g. ``CONFIG_KUNIT_EXAMPLE_TEST=y``). kunit_tool will keep its .config 87c63d2dd7SDavid Gow around, so you can see what config was used after running ``kunit.py run``. 88c63d2dd7SDavid Gow It also preserves any config changes you might make, so you can 89c63d2dd7SDavid Gow enable/disable things with ``make ARCH=um menuconfig`` or similar, and then 90c63d2dd7SDavid Gow re-run kunit_tool. 91c63d2dd7SDavid Gow5. Try to run ``make ARCH=um defconfig`` before running ``kunit.py run``. This 92c63d2dd7SDavid Gow may help clean up any residual config items which could be causing problems. 93c63d2dd7SDavid Gow6. Finally, try running KUnit outside UML. KUnit and KUnit tests can run be 94c63d2dd7SDavid Gow built into any kernel, or can be built as a module and loaded at runtime. 95c63d2dd7SDavid Gow Doing so should allow you to determine if UML is causing the issue you're 96c63d2dd7SDavid Gow seeing. When tests are built-in, they will execute when the kernel boots, and 97c63d2dd7SDavid Gow modules will automatically execute associated tests when loaded. Test results 98c63d2dd7SDavid Gow can be collected from ``/sys/kernel/debug/kunit/<test suite>/results``, and 99c63d2dd7SDavid Gow can be parsed with ``kunit.py parse``. For more details, see "KUnit on 100c63d2dd7SDavid Gow non-UML architectures" in :doc:`usage`. 101c63d2dd7SDavid Gow 102c63d2dd7SDavid GowIf none of the above tricks help, you are always welcome to email any issues to 103c63d2dd7SDavid Gowkunit-dev@googlegroups.com. 104