1.. SPDX-License-Identifier: GPL-2.0
2
3==================
4KUnit Architecture
5==================
6
7The KUnit architecture can be divided into two parts:
8
9- `In-Kernel Testing Framework`_
10- `kunit_tool (Command Line Test Harness)`_
11
12In-Kernel Testing Framework
13===========================
14
15The kernel testing library supports KUnit tests written in C using
16KUnit. KUnit tests are kernel code. KUnit does several things:
17
18- Organizes tests
19- Reports test results
20- Provides test utilities
21
22Test Cases
23----------
24
25The fundamental unit in KUnit is the test case. The KUnit test cases are
26grouped into KUnit suites. A KUnit test case is a function with type
27signature ``void (*)(struct kunit *test)``.
28These test case functions are wrapped in a struct called
29struct kunit_case.
30
31.. note:
32	``generate_params`` is optional for non-parameterized tests.
33
34Each KUnit test case gets a ``struct kunit`` context
35object passed to it that tracks a running test. The KUnit assertion
36macros and other KUnit utilities use the ``struct kunit`` context
37object. As an exception, there are two fields:
38
39- ``->priv``: The setup functions can use it to store arbitrary test
40  user data.
41
42- ``->param_value``: It contains the parameter value which can be
43  retrieved in the parameterized tests.
44
45Test Suites
46-----------
47
48A KUnit suite includes a collection of test cases. The KUnit suites
49are represented by the ``struct kunit_suite``. For example:
50
51.. code-block:: c
52
53	static struct kunit_case example_test_cases[] = {
54		KUNIT_CASE(example_test_foo),
55		KUNIT_CASE(example_test_bar),
56		KUNIT_CASE(example_test_baz),
57		{}
58	};
59
60	static struct kunit_suite example_test_suite = {
61		.name = "example",
62		.init = example_test_init,
63		.exit = example_test_exit,
64		.test_cases = example_test_cases,
65	};
66	kunit_test_suite(example_test_suite);
67
68In the above example, the test suite ``example_test_suite``, runs the
69test cases ``example_test_foo``, ``example_test_bar``, and
70``example_test_baz``. Before running the test, the ``example_test_init``
71is called and after running the test, ``example_test_exit`` is called.
72The ``kunit_test_suite(example_test_suite)`` registers the test suite
73with the KUnit test framework.
74
75Executor
76--------
77
78The KUnit executor can list and run built-in KUnit tests on boot.
79The Test suites are stored in a linker section
80called ``.kunit_test_suites``. For code, see:
81https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/include/asm-generic/vmlinux.lds.h?h=v5.15#n945.
82The linker section consists of an array of pointers to
83``struct kunit_suite``, and is populated by the ``kunit_test_suites()``
84macro. To run all tests compiled into the kernel, the KUnit executor
85iterates over the linker section array.
86
87.. kernel-figure:: kunit_suitememorydiagram.svg
88	:alt:	KUnit Suite Memory
89
90	KUnit Suite Memory Diagram
91
92On the kernel boot, the KUnit executor uses the start and end addresses
93of this section to iterate over and run all tests. For code, see:
94https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/lib/kunit/executor.c
95
96When built as a module, the ``kunit_test_suites()`` macro defines a
97``module_init()`` function, which runs all the tests in the compilation
98unit instead of utilizing the executor.
99
100In KUnit tests, some error classes do not affect other tests
101or parts of the kernel, each KUnit case executes in a separate thread
102context. For code, see:
103https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/lib/kunit/try-catch.c?h=v5.15#n58
104
105Assertion Macros
106----------------
107
108KUnit tests verify state using expectations/assertions.
109All expectations/assertions are formatted as:
110``KUNIT_{EXPECT|ASSERT}_<op>[_MSG](kunit, property[, message])``
111
112- ``{EXPECT|ASSERT}`` determines whether the check is an assertion or an
113  expectation.
114
115	- For an expectation, if the check fails, marks the test as failed
116	  and logs the failure.
117
118	- An assertion, on failure, causes the test case to terminate
119	  immediately.
120
121		- Assertions call function:
122		  ``void __noreturn kunit_abort(struct kunit *)``.
123
124		- ``kunit_abort`` calls function:
125		  ``void __noreturn kunit_try_catch_throw(struct kunit_try_catch *try_catch)``.
126
127		- ``kunit_try_catch_throw`` calls function:
128		  ``void kthread_complete_and_exit(struct completion *, long) __noreturn;``
129		  and terminates the special thread context.
130
131- ``<op>`` denotes a check with options: ``TRUE`` (supplied property
132  has the boolean value “true”), ``EQ`` (two supplied properties are
133  equal), ``NOT_ERR_OR_NULL`` (supplied pointer is not null and does not
134  contain an “err” value).
135
136- ``[_MSG]`` prints a custom message on failure.
137
138Test Result Reporting
139---------------------
140KUnit prints test results in KTAP format. KTAP is based on TAP14, see:
141https://github.com/isaacs/testanything.github.io/blob/tap14/tap-version-14-specification.md.
142KTAP (yet to be standardized format) works with KUnit and Kselftest.
143The KUnit executor prints KTAP results to dmesg, and debugfs
144(if configured).
145
146Parameterized Tests
147-------------------
148
149Each KUnit parameterized test is associated with a collection of
150parameters. The test is invoked multiple times, once for each parameter
151value and the parameter is stored in the ``param_value`` field.
152The test case includes a KUNIT_CASE_PARAM() macro that accepts a
153generator function.
154The generator function is passed the previous parameter and returns the next
155parameter. It also provides a macro to generate common-case generators based on
156arrays.
157
158kunit_tool (Command Line Test Harness)
159======================================
160
161kunit_tool is a Python script ``(tools/testing/kunit/kunit.py)``
162that can be used to configure, build, exec, parse and run (runs other
163commands in order) test results. You can either run KUnit tests using
164kunit_tool or can include KUnit in kernel and parse manually.
165
166- ``configure`` command generates the kernel ``.config`` from a
167  ``.kunitconfig`` file (and any architecture-specific options).
168  For some architectures, additional config options are specified in the
169  ``qemu_config`` Python script
170  (For example: ``tools/testing/kunit/qemu_configs/powerpc.py``).
171  It parses both the existing ``.config`` and the ``.kunitconfig`` files
172  and ensures that ``.config`` is a superset of ``.kunitconfig``.
173  If this is not the case, it will combine the two and run
174  ``make olddefconfig`` to regenerate the ``.config`` file. It then
175  verifies that ``.config`` is now a superset. This checks if all
176  Kconfig dependencies are correctly specified in ``.kunitconfig``.
177  ``kunit_config.py`` includes the parsing Kconfigs code. The code which
178  runs ``make olddefconfig`` is a part of ``kunit_kernel.py``. You can
179  invoke this command via: ``./tools/testing/kunit/kunit.py config`` and
180  generate a ``.config`` file.
181- ``build`` runs ``make`` on the kernel tree with required options
182  (depends on the architecture and some options, for example: build_dir)
183  and reports any errors.
184  To build a KUnit kernel from the current ``.config``, you can use the
185  ``build`` argument: ``./tools/testing/kunit/kunit.py build``.
186- ``exec`` command executes kernel results either directly (using
187  User-mode Linux configuration), or via an emulator such
188  as QEMU. It reads results from the log via standard
189  output (stdout), and passes them to ``parse`` to be parsed.
190  If you already have built a kernel with built-in KUnit tests,
191  you can run the kernel and display the test results with the ``exec``
192  argument: ``./tools/testing/kunit/kunit.py exec``.
193- ``parse`` extracts the KTAP output from a kernel log, parses
194  the test results, and prints a summary. For failed tests, any
195  diagnostic output will be included.
196