1.. SPDX-License-Identifier: GPL-2.0
2
3==================
4KUnit Architecture
5==================
6
7The KUnit architecture can be divided into two parts:
8
9- Kernel testing library
10- kunit_tool (Command line test harness)
11
12In-Kernel Testing Framework
13===========================
14
15The kernel testing library supports KUnit tests written in C using
16KUnit. KUnit tests are kernel code. KUnit does several things:
17
18- Organizes tests
19- Reports test results
20- Provides test utilities
21
22Test Cases
23----------
24
25The fundamental unit in KUnit is the test case. The KUnit test cases are
26grouped into KUnit suites. A KUnit test case is a function with type
27signature ``void (*)(struct kunit *test)``.
28These test case functions are wrapped in a struct called
29``struct kunit_case``. For code, see:
30
31.. kernel-doc:: include/kunit/test.h
32	:identifiers: kunit_case
33
34.. note:
35	``generate_params`` is optional for non-parameterized tests.
36
37Each KUnit test case gets a ``struct kunit`` context
38object passed to it that tracks a running test. The KUnit assertion
39macros and other KUnit utilities use the ``struct kunit`` context
40object. As an exception, there are two fields:
41
42- ``->priv``: The setup functions can use it to store arbitrary test
43  user data.
44
45- ``->param_value``: It contains the parameter value which can be
46  retrieved in the parameterized tests.
47
48Test Suites
49-----------
50
51A KUnit suite includes a collection of test cases. The KUnit suites
52are represented by the ``struct kunit_suite``. For example:
53
54.. code-block:: c
55
56	static struct kunit_case example_test_cases[] = {
57		KUNIT_CASE(example_test_foo),
58		KUNIT_CASE(example_test_bar),
59		KUNIT_CASE(example_test_baz),
60		{}
61	};
62
63	static struct kunit_suite example_test_suite = {
64		.name = "example",
65		.init = example_test_init,
66		.exit = example_test_exit,
67		.test_cases = example_test_cases,
68	};
69	kunit_test_suite(example_test_suite);
70
71In the above example, the test suite ``example_test_suite``, runs the
72test cases ``example_test_foo``, ``example_test_bar``, and
73``example_test_baz``. Before running the test, the ``example_test_init``
74is called and after running the test, ``example_test_exit`` is called.
75The ``kunit_test_suite(example_test_suite)`` registers the test suite
76with the KUnit test framework.
77
78Executor
79--------
80
81The KUnit executor can list and run built-in KUnit tests on boot.
82The Test suites are stored in a linker section
83called ``.kunit_test_suites``. For code, see:
84https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/include/asm-generic/vmlinux.lds.h?h=v5.15#n945.
85The linker section consists of an array of pointers to
86``struct kunit_suite``, and is populated by the ``kunit_test_suites()``
87macro. To run all tests compiled into the kernel, the KUnit executor
88iterates over the linker section array.
89
90.. kernel-figure:: kunit_suitememorydiagram.svg
91	:alt:	KUnit Suite Memory
92
93	KUnit Suite Memory Diagram
94
95On the kernel boot, the KUnit executor uses the start and end addresses
96of this section to iterate over and run all tests. For code, see:
97https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/lib/kunit/executor.c
98
99When built as a module, the ``kunit_test_suites()`` macro defines a
100``module_init()`` function, which runs all the tests in the compilation
101unit instead of utilizing the executor.
102
103In KUnit tests, some error classes do not affect other tests
104or parts of the kernel, each KUnit case executes in a separate thread
105context. For code, see:
106https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/lib/kunit/try-catch.c?h=v5.15#n58
107
108Assertion Macros
109----------------
110
111KUnit tests verify state using expectations/assertions.
112All expectations/assertions are formatted as:
113``KUNIT_{EXPECT|ASSERT}_<op>[_MSG](kunit, property[, message])``
114
115- ``{EXPECT|ASSERT}`` determines whether the check is an assertion or an
116  expectation.
117
118	- For an expectation, if the check fails, marks the test as failed
119	  and logs the failure.
120
121	- An assertion, on failure, causes the test case to terminate
122	  immediately.
123
124		- Assertions call function:
125		  ``void __noreturn kunit_abort(struct kunit *)``.
126
127		- ``kunit_abort`` calls function:
128		  ``void __noreturn kunit_try_catch_throw(struct kunit_try_catch *try_catch)``.
129
130		- ``kunit_try_catch_throw`` calls function:
131		  ``void complete_and_exit(struct completion *, long) __noreturn;``
132		  and terminates the special thread context.
133
134- ``<op>`` denotes a check with options: ``TRUE`` (supplied property
135  has the boolean value “true”), ``EQ`` (two supplied properties are
136  equal), ``NOT_ERR_OR_NULL`` (supplied pointer is not null and does not
137  contain an “err” value).
138
139- ``[_MSG]`` prints a custom message on failure.
140
141Test Result Reporting
142---------------------
143KUnit prints test results in KTAP format. KTAP is based on TAP14, see:
144https://github.com/isaacs/testanything.github.io/blob/tap14/tap-version-14-specification.md.
145KTAP (yet to be standardized format) works with KUnit and Kselftest.
146The KUnit executor prints KTAP results to dmesg, and debugfs
147(if configured).
148
149Parameterized Tests
150-------------------
151
152Each KUnit parameterized test is associated with a collection of
153parameters. The test is invoked multiple times, once for each parameter
154value and the parameter is stored in the ``param_value`` field.
155The test case includes a ``KUNIT_CASE_PARAM()`` macro that accepts a
156generator function.
157The generator function is passed the previous parameter and returns the next
158parameter. It also provides a macro to generate common-case generators based on
159arrays.
160
161For code, see:
162
163.. kernel-doc:: include/kunit/test.h
164	:identifiers: KUNIT_ARRAY_PARAM
165
166
167kunit_tool (Command Line Test Harness)
168======================================
169
170kunit_tool is a Python script ``(tools/testing/kunit/kunit.py)``
171that can be used to configure, build, exec, parse and run (runs other
172commands in order) test results. You can either run KUnit tests using
173kunit_tool or can include KUnit in kernel and parse manually.
174
175- ``configure`` command generates the kernel ``.config`` from a
176  ``.kunitconfig`` file (and any architecture-specific options).
177  For some architectures, additional config options are specified in the
178  ``qemu_config`` Python script
179  (For example: ``tools/testing/kunit/qemu_configs/powerpc.py``).
180  It parses both the existing ``.config`` and the ``.kunitconfig`` files
181  and ensures that ``.config`` is a superset of ``.kunitconfig``.
182  If this is not the case, it will combine the two and run
183  ``make olddefconfig`` to regenerate the ``.config`` file. It then
184  verifies that ``.config`` is now a superset. This checks if all
185  Kconfig dependencies are correctly specified in ``.kunitconfig``.
186  ``kunit_config.py`` includes the parsing Kconfigs code. The code which
187  runs ``make olddefconfig`` is a part of ``kunit_kernel.py``. You can
188  invoke this command via: ``./tools/testing/kunit/kunit.py config`` and
189  generate a ``.config`` file.
190- ``build`` runs ``make`` on the kernel tree with required options
191  (depends on the architecture and some options, for example: build_dir)
192  and reports any errors.
193  To build a KUnit kernel from the current ``.config``, you can use the
194  ``build`` argument: ``./tools/testing/kunit/kunit.py build``.
195- ``exec`` command executes kernel results either directly (using
196  User-mode Linux configuration), or via an emulator such
197  as QEMU. It reads results from the log via standard
198  output (stdout), and passes them to ``parse`` to be parsed.
199  If you already have built a kernel with built-in KUnit tests,
200  you can run the kernel and display the test results with the ``exec``
201  argument: ``./tools/testing/kunit/kunit.py exec``.
202- ``parse`` extracts the KTAP output from a kernel log, parses
203  the test results, and prints a summary. For failed tests, any
204  diagnostic output will be included.
205