1======================
2Linux Kernel Selftests
3======================
4
5The kernel contains a set of "self tests" under the tools/testing/selftests/
6directory. These are intended to be small tests to exercise individual code
7paths in the kernel. Tests are intended to be run after building, installing
8and booting a kernel.
9
10Kselftest from mainline can be run on older stable kernels. Running tests
11from mainline offers the best coverage. Several test rings run mainline
12kselftest suite on stable releases. The reason is that when a new test
13gets added to test existing code to regression test a bug, we should be
14able to run that test on an older kernel. Hence, it is important to keep
15code that can still test an older kernel and make sure it skips the test
16gracefully on newer releases.
17
18You can find additional information on Kselftest framework, how to
19write new tests using the framework on Kselftest wiki:
20
21https://kselftest.wiki.kernel.org/
22
23On some systems, hot-plug tests could hang forever waiting for cpu and
24memory to be ready to be offlined. A special hot-plug target is created
25to run the full range of hot-plug tests. In default mode, hot-plug tests run
26in safe mode with a limited scope. In limited mode, cpu-hotplug test is
27run on a single cpu as opposed to all hotplug capable cpus, and memory
28hotplug test is run on 2% of hotplug capable memory instead of 10%.
29
30kselftest runs as a userspace process.  Tests that can be written/run in
31userspace may wish to use the `Test Harness`_.  Tests that need to be
32run in kernel space may wish to use a `Test Module`_.
33
34Running the selftests (hotplug tests are run in limited mode)
35=============================================================
36
37To build the tests::
38
39  $ make -C tools/testing/selftests
40
41To run the tests::
42
43  $ make -C tools/testing/selftests run_tests
44
45To build and run the tests with a single command, use::
46
47  $ make kselftest
48
49Note that some tests will require root privileges.
50
51Kselftest supports saving output files in a separate directory and then
52running tests. To locate output files in a separate directory two syntaxes
53are supported. In both cases the working directory must be the root of the
54kernel src. This is applicable to "Running a subset of selftests" section
55below.
56
57To build, save output files in a separate directory with O= ::
58
59  $ make O=/tmp/kselftest kselftest
60
61To build, save output files in a separate directory with KBUILD_OUTPUT ::
62
63  $ export KBUILD_OUTPUT=/tmp/kselftest; make kselftest
64
65The O= assignment takes precedence over the KBUILD_OUTPUT environment
66variable.
67
68The above commands by default run the tests and print full pass/fail report.
69Kselftest supports "summary" option to make it easier to understand the test
70results. Please find the detailed individual test results for each test in
71/tmp/testname file(s) when summary option is specified. This is applicable
72to "Running a subset of selftests" section below.
73
74To run kselftest with summary option enabled ::
75
76  $ make summary=1 kselftest
77
78Running a subset of selftests
79=============================
80
81You can use the "TARGETS" variable on the make command line to specify
82single test to run, or a list of tests to run.
83
84To run only tests targeted for a single subsystem::
85
86  $ make -C tools/testing/selftests TARGETS=ptrace run_tests
87
88You can specify multiple tests to build and run::
89
90  $  make TARGETS="size timers" kselftest
91
92To build, save output files in a separate directory with O= ::
93
94  $ make O=/tmp/kselftest TARGETS="size timers" kselftest
95
96To build, save output files in a separate directory with KBUILD_OUTPUT ::
97
98  $ export KBUILD_OUTPUT=/tmp/kselftest; make TARGETS="size timers" kselftest
99
100Additionally you can use the "SKIP_TARGETS" variable on the make command
101line to specify one or more targets to exclude from the TARGETS list.
102
103To run all tests but a single subsystem::
104
105  $ make -C tools/testing/selftests SKIP_TARGETS=ptrace run_tests
106
107You can specify multiple tests to skip::
108
109  $  make SKIP_TARGETS="size timers" kselftest
110
111You can also specify a restricted list of tests to run together with a
112dedicated skiplist::
113
114  $  make TARGETS="bpf breakpoints size timers" SKIP_TARGETS=bpf kselftest
115
116See the top-level tools/testing/selftests/Makefile for the list of all
117possible targets.
118
119Running the full range hotplug selftests
120========================================
121
122To build the hotplug tests::
123
124  $ make -C tools/testing/selftests hotplug
125
126To run the hotplug tests::
127
128  $ make -C tools/testing/selftests run_hotplug
129
130Note that some tests will require root privileges.
131
132
133Install selftests
134=================
135
136You can use the "install" target of "make" (which calls the `kselftest_install.sh`
137tool) to install selftests in the default location (`tools/testing/selftests/kselftest_install`),
138or in a user specified location via the `INSTALL_PATH` "make" variable.
139
140To install selftests in default location::
141
142   $ make -C tools/testing/selftests install
143
144To install selftests in a user specified location::
145
146   $ make -C tools/testing/selftests install INSTALL_PATH=/some/other/path
147
148Running installed selftests
149===========================
150
151Found in the install directory, as well as in the Kselftest tarball,
152is a script named `run_kselftest.sh` to run the tests.
153
154You can simply do the following to run the installed Kselftests. Please
155note some tests will require root privileges::
156
157   $ cd kselftest_install
158   $ ./run_kselftest.sh
159
160To see the list of available tests, the `-l` option can be used::
161
162   $ ./run_kselftest.sh -l
163
164The `-c` option can be used to run all the tests from a test collection, or
165the `-t` option for specific single tests. Either can be used multiple times::
166
167   $ ./run_kselftest.sh -c bpf -c seccomp -t timers:posix_timers -t timer:nanosleep
168
169For other features see the script usage output, seen with the `-h` option.
170
171Packaging selftests
172===================
173
174In some cases packaging is desired, such as when tests need to run on a
175different system. To package selftests, run::
176
177   $ make -C tools/testing/selftests gen_tar
178
179This generates a tarball in the `INSTALL_PATH/kselftest-packages` directory. By
180default, `.gz` format is used. The tar compression format can be overridden by
181specifying a `FORMAT` make variable. Any value recognized by `tar's auto-compress`_
182option is supported, such as::
183
184    $ make -C tools/testing/selftests gen_tar FORMAT=.xz
185
186`make gen_tar` invokes `make install` so you can use it to package a subset of
187tests by using variables specified in `Running a subset of selftests`_
188section::
189
190    $ make -C tools/testing/selftests gen_tar TARGETS="bpf" FORMAT=.xz
191
192.. _tar's auto-compress: https://www.gnu.org/software/tar/manual/html_node/gzip.html#auto_002dcompress
193
194Contributing new tests
195======================
196
197In general, the rules for selftests are
198
199 * Do as much as you can if you're not root;
200
201 * Don't take too long;
202
203 * Don't break the build on any architecture, and
204
205 * Don't cause the top-level "make run_tests" to fail if your feature is
206   unconfigured.
207
208Contributing new tests (details)
209================================
210
211 * In your Makefile, use facilities from lib.mk by including it instead of
212   reinventing the wheel. Specify flags and binaries generation flags on
213   need basis before including lib.mk. ::
214
215    CFLAGS = $(KHDR_INCLUDES)
216    TEST_GEN_PROGS := close_range_test
217    include ../lib.mk
218
219 * Use TEST_GEN_XXX if such binaries or files are generated during
220   compiling.
221
222   TEST_PROGS, TEST_GEN_PROGS mean it is the executable tested by
223   default.
224
225   TEST_CUSTOM_PROGS should be used by tests that require custom build
226   rules and prevent common build rule use.
227
228   TEST_PROGS are for test shell scripts. Please ensure shell script has
229   its exec bit set. Otherwise, lib.mk run_tests will generate a warning.
230
231   TEST_CUSTOM_PROGS and TEST_PROGS will be run by common run_tests.
232
233   TEST_PROGS_EXTENDED, TEST_GEN_PROGS_EXTENDED mean it is the
234   executable which is not tested by default.
235   TEST_FILES, TEST_GEN_FILES mean it is the file which is used by
236   test.
237
238 * First use the headers inside the kernel source and/or git repo, and then the
239   system headers.  Headers for the kernel release as opposed to headers
240   installed by the distro on the system should be the primary focus to be able
241   to find regressions. Use KHDR_INCLUDES in Makefile to include headers from
242   the kernel source.
243
244 * If a test needs specific kernel config options enabled, add a config file in
245   the test directory to enable them.
246
247   e.g: tools/testing/selftests/android/config
248
249 * Create a .gitignore file inside test directory and add all generated objects
250   in it.
251
252 * Add new test name in TARGETS in selftests/Makefile::
253
254    TARGETS += android
255
256 * All changes should pass::
257
258    kselftest-{all,install,clean,gen_tar}
259    kselftest-{all,install,clean,gen_tar} O=abo_path
260    kselftest-{all,install,clean,gen_tar} O=rel_path
261    make -C tools/testing/selftests {all,install,clean,gen_tar}
262    make -C tools/testing/selftests {all,install,clean,gen_tar} O=abs_path
263    make -C tools/testing/selftests {all,install,clean,gen_tar} O=rel_path
264
265Test Module
266===========
267
268Kselftest tests the kernel from userspace.  Sometimes things need
269testing from within the kernel, one method of doing this is to create a
270test module.  We can tie the module into the kselftest framework by
271using a shell script test runner.  ``kselftest/module.sh`` is designed
272to facilitate this process.  There is also a header file provided to
273assist writing kernel modules that are for use with kselftest:
274
275- ``tools/testing/selftests/kselftest_module.h``
276- ``tools/testing/selftests/kselftest/module.sh``
277
278Note that test modules should taint the kernel with TAINT_TEST. This will
279happen automatically for modules which are in the ``tools/testing/``
280directory, or for modules which use the ``kselftest_module.h`` header above.
281Otherwise, you'll need to add ``MODULE_INFO(test, "Y")`` to your module
282source. selftests which do not load modules typically should not taint the
283kernel, but in cases where a non-test module is loaded, TEST_TAINT can be
284applied from userspace by writing to ``/proc/sys/kernel/tainted``.
285
286How to use
287----------
288
289Here we show the typical steps to create a test module and tie it into
290kselftest.  We use kselftests for lib/ as an example.
291
2921. Create the test module
293
2942. Create the test script that will run (load/unload) the module
295   e.g. ``tools/testing/selftests/lib/printf.sh``
296
2973. Add line to config file e.g. ``tools/testing/selftests/lib/config``
298
2994. Add test script to makefile  e.g. ``tools/testing/selftests/lib/Makefile``
300
3015. Verify it works:
302
303.. code-block:: sh
304
305   # Assumes you have booted a fresh build of this kernel tree
306   cd /path/to/linux/tree
307   make kselftest-merge
308   make modules
309   sudo make modules_install
310   make TARGETS=lib kselftest
311
312Example Module
313--------------
314
315A bare bones test module might look like this:
316
317.. code-block:: c
318
319   // SPDX-License-Identifier: GPL-2.0+
320
321   #define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
322
323   #include "../tools/testing/selftests/kselftest/module.h"
324
325   KSTM_MODULE_GLOBALS();
326
327   /*
328    * Kernel module for testing the foobinator
329    */
330
331   static int __init test_function()
332   {
333           ...
334   }
335
336   static void __init selftest(void)
337   {
338           KSTM_CHECK_ZERO(do_test_case("", 0));
339   }
340
341   KSTM_MODULE_LOADERS(test_foo);
342   MODULE_AUTHOR("John Developer <jd@fooman.org>");
343   MODULE_LICENSE("GPL");
344   MODULE_INFO(test, "Y");
345
346Example test script
347-------------------
348
349.. code-block:: sh
350
351    #!/bin/bash
352    # SPDX-License-Identifier: GPL-2.0+
353    $(dirname $0)/../kselftest/module.sh "foo" test_foo
354
355
356Test Harness
357============
358
359The kselftest_harness.h file contains useful helpers to build tests.  The
360test harness is for userspace testing, for kernel space testing see `Test
361Module`_ above.
362
363The tests from tools/testing/selftests/seccomp/seccomp_bpf.c can be used as
364example.
365
366Example
367-------
368
369.. kernel-doc:: tools/testing/selftests/kselftest_harness.h
370    :doc: example
371
372
373Helpers
374-------
375
376.. kernel-doc:: tools/testing/selftests/kselftest_harness.h
377    :functions: TH_LOG TEST TEST_SIGNAL FIXTURE FIXTURE_DATA FIXTURE_SETUP
378                FIXTURE_TEARDOWN TEST_F TEST_HARNESS_MAIN FIXTURE_VARIANT
379                FIXTURE_VARIANT_ADD
380
381Operators
382---------
383
384.. kernel-doc:: tools/testing/selftests/kselftest_harness.h
385    :doc: operators
386
387.. kernel-doc:: tools/testing/selftests/kselftest_harness.h
388    :functions: ASSERT_EQ ASSERT_NE ASSERT_LT ASSERT_LE ASSERT_GT ASSERT_GE
389                ASSERT_NULL ASSERT_TRUE ASSERT_NULL ASSERT_TRUE ASSERT_FALSE
390                ASSERT_STREQ ASSERT_STRNE EXPECT_EQ EXPECT_NE EXPECT_LT
391                EXPECT_LE EXPECT_GT EXPECT_GE EXPECT_NULL EXPECT_TRUE
392                EXPECT_FALSE EXPECT_STREQ EXPECT_STRNE
393