1======================
2Linux Kernel Selftests
3======================
4
5The kernel contains a set of "self tests" under the tools/testing/selftests/
6directory. These are intended to be small tests to exercise individual code
7paths in the kernel. Tests are intended to be run after building, installing
8and booting a kernel.
9
10You can find additional information on Kselftest framework, how to
11write new tests using the framework on Kselftest wiki:
12
13https://kselftest.wiki.kernel.org/
14
15On some systems, hot-plug tests could hang forever waiting for cpu and
16memory to be ready to be offlined. A special hot-plug target is created
17to run the full range of hot-plug tests. In default mode, hot-plug tests run
18in safe mode with a limited scope. In limited mode, cpu-hotplug test is
19run on a single cpu as opposed to all hotplug capable cpus, and memory
20hotplug test is run on 2% of hotplug capable memory instead of 10%.
21
22kselftest runs as a userspace process.  Tests that can be written/run in
23userspace may wish to use the `Test Harness`_.  Tests that need to be
24run in kernel space may wish to use a `Test Module`_.
25
26Running the selftests (hotplug tests are run in limited mode)
27=============================================================
28
29To build the tests::
30
31  $ make -C tools/testing/selftests
32
33To run the tests::
34
35  $ make -C tools/testing/selftests run_tests
36
37To build and run the tests with a single command, use::
38
39  $ make kselftest
40
41Note that some tests will require root privileges.
42
43Kselftest supports saving output files in a separate directory and then
44running tests. To locate output files in a separate directory two syntaxes
45are supported. In both cases the working directory must be the root of the
46kernel src. This is applicable to "Running a subset of selftests" section
47below.
48
49To build, save output files in a separate directory with O= ::
50
51  $ make O=/tmp/kselftest kselftest
52
53To build, save output files in a separate directory with KBUILD_OUTPUT ::
54
55  $ export KBUILD_OUTPUT=/tmp/kselftest; make kselftest
56
57The O= assignment takes precedence over the KBUILD_OUTPUT environment
58variable.
59
60The above commands by default run the tests and print full pass/fail report.
61Kselftest supports "summary" option to make it easier to understand the test
62results. Please find the detailed individual test results for each test in
63/tmp/testname file(s) when summary option is specified. This is applicable
64to "Running a subset of selftests" section below.
65
66To run kselftest with summary option enabled ::
67
68  $ make summary=1 kselftest
69
70Running a subset of selftests
71=============================
72
73You can use the "TARGETS" variable on the make command line to specify
74single test to run, or a list of tests to run.
75
76To run only tests targeted for a single subsystem::
77
78  $ make -C tools/testing/selftests TARGETS=ptrace run_tests
79
80You can specify multiple tests to build and run::
81
82  $  make TARGETS="size timers" kselftest
83
84To build, save output files in a separate directory with O= ::
85
86  $ make O=/tmp/kselftest TARGETS="size timers" kselftest
87
88To build, save output files in a separate directory with KBUILD_OUTPUT ::
89
90  $ export KBUILD_OUTPUT=/tmp/kselftest; make TARGETS="size timers" kselftest
91
92Additionally you can use the "SKIP_TARGETS" variable on the make command
93line to specify one or more targets to exclude from the TARGETS list.
94
95To run all tests but a single subsystem::
96
97  $ make -C tools/testing/selftests SKIP_TARGETS=ptrace run_tests
98
99You can specify multiple tests to skip::
100
101  $  make SKIP_TARGETS="size timers" kselftest
102
103You can also specify a restricted list of tests to run together with a
104dedicated skiplist::
105
106  $  make TARGETS="bpf breakpoints size timers" SKIP_TARGETS=bpf kselftest
107
108See the top-level tools/testing/selftests/Makefile for the list of all
109possible targets.
110
111Running the full range hotplug selftests
112========================================
113
114To build the hotplug tests::
115
116  $ make -C tools/testing/selftests hotplug
117
118To run the hotplug tests::
119
120  $ make -C tools/testing/selftests run_hotplug
121
122Note that some tests will require root privileges.
123
124
125Install selftests
126=================
127
128You can use the "install" target of "make" (which calls the `kselftest_install.sh`
129tool) to install selftests in the default location (`tools/testing/selftests/kselftest_install`),
130or in a user specified location via the `INSTALL_PATH` "make" variable.
131
132To install selftests in default location::
133
134   $ make -C tools/testing/selftests install
135
136To install selftests in a user specified location::
137
138   $ make -C tools/testing/selftests install INSTALL_PATH=/some/other/path
139
140Running installed selftests
141===========================
142
143Found in the install directory, as well as in the Kselftest tarball,
144is a script named `run_kselftest.sh` to run the tests.
145
146You can simply do the following to run the installed Kselftests. Please
147note some tests will require root privileges::
148
149   $ cd kselftest_install
150   $ ./run_kselftest.sh
151
152To see the list of available tests, the `-l` option can be used::
153
154   $ ./run_kselftest.sh -l
155
156The `-c` option can be used to run all the tests from a test collection, or
157the `-t` option for specific single tests. Either can be used multiple times::
158
159   $ ./run_kselftest.sh -c bpf -c seccomp -t timers:posix_timers -t timer:nanosleep
160
161For other features see the script usage output, seen with the `-h` option.
162
163Packaging selftests
164===================
165
166In some cases packaging is desired, such as when tests need to run on a
167different system. To package selftests, run::
168
169   $ make -C tools/testing/selftests gen_tar
170
171This generates a tarball in the `INSTALL_PATH/kselftest-packages` directory. By
172default, `.gz` format is used. The tar compression format can be overridden by
173specifying a `FORMAT` make variable. Any value recognized by `tar's auto-compress`_
174option is supported, such as::
175
176    $ make -C tools/testing/selftests gen_tar FORMAT=.xz
177
178`make gen_tar` invokes `make install` so you can use it to package a subset of
179tests by using variables specified in `Running a subset of selftests`_
180section::
181
182    $ make -C tools/testing/selftests gen_tar TARGETS="bpf" FORMAT=.xz
183
184.. _tar's auto-compress: https://www.gnu.org/software/tar/manual/html_node/gzip.html#auto_002dcompress
185
186Contributing new tests
187======================
188
189In general, the rules for selftests are
190
191 * Do as much as you can if you're not root;
192
193 * Don't take too long;
194
195 * Don't break the build on any architecture, and
196
197 * Don't cause the top-level "make run_tests" to fail if your feature is
198   unconfigured.
199
200Contributing new tests (details)
201================================
202
203 * Use TEST_GEN_XXX if such binaries or files are generated during
204   compiling.
205
206   TEST_PROGS, TEST_GEN_PROGS mean it is the executable tested by
207   default.
208
209   TEST_CUSTOM_PROGS should be used by tests that require custom build
210   rules and prevent common build rule use.
211
212   TEST_PROGS are for test shell scripts. Please ensure shell script has
213   its exec bit set. Otherwise, lib.mk run_tests will generate a warning.
214
215   TEST_CUSTOM_PROGS and TEST_PROGS will be run by common run_tests.
216
217   TEST_PROGS_EXTENDED, TEST_GEN_PROGS_EXTENDED mean it is the
218   executable which is not tested by default.
219   TEST_FILES, TEST_GEN_FILES mean it is the file which is used by
220   test.
221
222 * First use the headers inside the kernel source and/or git repo, and then the
223   system headers.  Headers for the kernel release as opposed to headers
224   installed by the distro on the system should be the primary focus to be able
225   to find regressions.
226
227 * If a test needs specific kernel config options enabled, add a config file in
228   the test directory to enable them.
229
230   e.g: tools/testing/selftests/android/config
231
232Test Module
233===========
234
235Kselftest tests the kernel from userspace.  Sometimes things need
236testing from within the kernel, one method of doing this is to create a
237test module.  We can tie the module into the kselftest framework by
238using a shell script test runner.  ``kselftest/module.sh`` is designed
239to facilitate this process.  There is also a header file provided to
240assist writing kernel modules that are for use with kselftest:
241
242- ``tools/testing/selftests/kselftest_module.h``
243- ``tools/testing/selftests/kselftest/module.sh``
244
245How to use
246----------
247
248Here we show the typical steps to create a test module and tie it into
249kselftest.  We use kselftests for lib/ as an example.
250
2511. Create the test module
252
2532. Create the test script that will run (load/unload) the module
254   e.g. ``tools/testing/selftests/lib/printf.sh``
255
2563. Add line to config file e.g. ``tools/testing/selftests/lib/config``
257
2584. Add test script to makefile  e.g. ``tools/testing/selftests/lib/Makefile``
259
2605. Verify it works:
261
262.. code-block:: sh
263
264   # Assumes you have booted a fresh build of this kernel tree
265   cd /path/to/linux/tree
266   make kselftest-merge
267   make modules
268   sudo make modules_install
269   make TARGETS=lib kselftest
270
271Example Module
272--------------
273
274A bare bones test module might look like this:
275
276.. code-block:: c
277
278   // SPDX-License-Identifier: GPL-2.0+
279
280   #define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
281
282   #include "../tools/testing/selftests/kselftest/module.h"
283
284   KSTM_MODULE_GLOBALS();
285
286   /*
287    * Kernel module for testing the foobinator
288    */
289
290   static int __init test_function()
291   {
292           ...
293   }
294
295   static void __init selftest(void)
296   {
297           KSTM_CHECK_ZERO(do_test_case("", 0));
298   }
299
300   KSTM_MODULE_LOADERS(test_foo);
301   MODULE_AUTHOR("John Developer <jd@fooman.org>");
302   MODULE_LICENSE("GPL");
303
304Example test script
305-------------------
306
307.. code-block:: sh
308
309    #!/bin/bash
310    # SPDX-License-Identifier: GPL-2.0+
311    $(dirname $0)/../kselftest/module.sh "foo" test_foo
312
313
314Test Harness
315============
316
317The kselftest_harness.h file contains useful helpers to build tests.  The
318test harness is for userspace testing, for kernel space testing see `Test
319Module`_ above.
320
321The tests from tools/testing/selftests/seccomp/seccomp_bpf.c can be used as
322example.
323
324Example
325-------
326
327.. kernel-doc:: tools/testing/selftests/kselftest_harness.h
328    :doc: example
329
330
331Helpers
332-------
333
334.. kernel-doc:: tools/testing/selftests/kselftest_harness.h
335    :functions: TH_LOG TEST TEST_SIGNAL FIXTURE FIXTURE_DATA FIXTURE_SETUP
336                FIXTURE_TEARDOWN TEST_F TEST_HARNESS_MAIN FIXTURE_VARIANT
337                FIXTURE_VARIANT_ADD
338
339Operators
340---------
341
342.. kernel-doc:: tools/testing/selftests/kselftest_harness.h
343    :doc: operators
344
345.. kernel-doc:: tools/testing/selftests/kselftest_harness.h
346    :functions: ASSERT_EQ ASSERT_NE ASSERT_LT ASSERT_LE ASSERT_GT ASSERT_GE
347                ASSERT_NULL ASSERT_TRUE ASSERT_NULL ASSERT_TRUE ASSERT_FALSE
348                ASSERT_STREQ ASSERT_STRNE EXPECT_EQ EXPECT_NE EXPECT_LT
349                EXPECT_LE EXPECT_GT EXPECT_GE EXPECT_NULL EXPECT_TRUE
350                EXPECT_FALSE EXPECT_STREQ EXPECT_STRNE
351