1====================== 2Linux Kernel Selftests 3====================== 4 5The kernel contains a set of "self tests" under the tools/testing/selftests/ 6directory. These are intended to be small tests to exercise individual code 7paths in the kernel. Tests are intended to be run after building, installing 8and booting a kernel. 9 10Kselftest from mainline can be run on older stable kernels. Running tests 11from mainline offers the best coverage. Several test rings run mainline 12kselftest suite on stable releases. The reason is that when a new test 13gets added to test existing code to regression test a bug, we should be 14able to run that test on an older kernel. Hence, it is important to keep 15code that can still test an older kernel and make sure it skips the test 16gracefully on newer releases. 17 18You can find additional information on Kselftest framework, how to 19write new tests using the framework on Kselftest wiki: 20 21https://kselftest.wiki.kernel.org/ 22 23On some systems, hot-plug tests could hang forever waiting for cpu and 24memory to be ready to be offlined. A special hot-plug target is created 25to run the full range of hot-plug tests. In default mode, hot-plug tests run 26in safe mode with a limited scope. In limited mode, cpu-hotplug test is 27run on a single cpu as opposed to all hotplug capable cpus, and memory 28hotplug test is run on 2% of hotplug capable memory instead of 10%. 29 30kselftest runs as a userspace process. Tests that can be written/run in 31userspace may wish to use the `Test Harness`_. Tests that need to be 32run in kernel space may wish to use a `Test Module`_. 33 34Running the selftests (hotplug tests are run in limited mode) 35============================================================= 36 37To build the tests:: 38 39 $ make headers 40 $ make -C tools/testing/selftests 41 42To run the tests:: 43 44 $ make -C tools/testing/selftests run_tests 45 46To build and run the tests with a single command, use:: 47 48 $ make kselftest 49 50Note that some tests will require root privileges. 51 52Kselftest supports saving output files in a separate directory and then 53running tests. To locate output files in a separate directory two syntaxes 54are supported. In both cases the working directory must be the root of the 55kernel src. This is applicable to "Running a subset of selftests" section 56below. 57 58To build, save output files in a separate directory with O= :: 59 60 $ make O=/tmp/kselftest kselftest 61 62To build, save output files in a separate directory with KBUILD_OUTPUT :: 63 64 $ export KBUILD_OUTPUT=/tmp/kselftest; make kselftest 65 66The O= assignment takes precedence over the KBUILD_OUTPUT environment 67variable. 68 69The above commands by default run the tests and print full pass/fail report. 70Kselftest supports "summary" option to make it easier to understand the test 71results. Please find the detailed individual test results for each test in 72/tmp/testname file(s) when summary option is specified. This is applicable 73to "Running a subset of selftests" section below. 74 75To run kselftest with summary option enabled :: 76 77 $ make summary=1 kselftest 78 79Running a subset of selftests 80============================= 81 82You can use the "TARGETS" variable on the make command line to specify 83single test to run, or a list of tests to run. 84 85To run only tests targeted for a single subsystem:: 86 87 $ make -C tools/testing/selftests TARGETS=ptrace run_tests 88 89You can specify multiple tests to build and run:: 90 91 $ make TARGETS="size timers" kselftest 92 93To build, save output files in a separate directory with O= :: 94 95 $ make O=/tmp/kselftest TARGETS="size timers" kselftest 96 97To build, save output files in a separate directory with KBUILD_OUTPUT :: 98 99 $ export KBUILD_OUTPUT=/tmp/kselftest; make TARGETS="size timers" kselftest 100 101Additionally you can use the "SKIP_TARGETS" variable on the make command 102line to specify one or more targets to exclude from the TARGETS list. 103 104To run all tests but a single subsystem:: 105 106 $ make -C tools/testing/selftests SKIP_TARGETS=ptrace run_tests 107 108You can specify multiple tests to skip:: 109 110 $ make SKIP_TARGETS="size timers" kselftest 111 112You can also specify a restricted list of tests to run together with a 113dedicated skiplist:: 114 115 $ make TARGETS="bpf breakpoints size timers" SKIP_TARGETS=bpf kselftest 116 117See the top-level tools/testing/selftests/Makefile for the list of all 118possible targets. 119 120Running the full range hotplug selftests 121======================================== 122 123To build the hotplug tests:: 124 125 $ make -C tools/testing/selftests hotplug 126 127To run the hotplug tests:: 128 129 $ make -C tools/testing/selftests run_hotplug 130 131Note that some tests will require root privileges. 132 133 134Install selftests 135================= 136 137You can use the "install" target of "make" (which calls the `kselftest_install.sh` 138tool) to install selftests in the default location (`tools/testing/selftests/kselftest_install`), 139or in a user specified location via the `INSTALL_PATH` "make" variable. 140 141To install selftests in default location:: 142 143 $ make -C tools/testing/selftests install 144 145To install selftests in a user specified location:: 146 147 $ make -C tools/testing/selftests install INSTALL_PATH=/some/other/path 148 149Running installed selftests 150=========================== 151 152Found in the install directory, as well as in the Kselftest tarball, 153is a script named `run_kselftest.sh` to run the tests. 154 155You can simply do the following to run the installed Kselftests. Please 156note some tests will require root privileges:: 157 158 $ cd kselftest_install 159 $ ./run_kselftest.sh 160 161To see the list of available tests, the `-l` option can be used:: 162 163 $ ./run_kselftest.sh -l 164 165The `-c` option can be used to run all the tests from a test collection, or 166the `-t` option for specific single tests. Either can be used multiple times:: 167 168 $ ./run_kselftest.sh -c bpf -c seccomp -t timers:posix_timers -t timer:nanosleep 169 170For other features see the script usage output, seen with the `-h` option. 171 172Packaging selftests 173=================== 174 175In some cases packaging is desired, such as when tests need to run on a 176different system. To package selftests, run:: 177 178 $ make -C tools/testing/selftests gen_tar 179 180This generates a tarball in the `INSTALL_PATH/kselftest-packages` directory. By 181default, `.gz` format is used. The tar compression format can be overridden by 182specifying a `FORMAT` make variable. Any value recognized by `tar's auto-compress`_ 183option is supported, such as:: 184 185 $ make -C tools/testing/selftests gen_tar FORMAT=.xz 186 187`make gen_tar` invokes `make install` so you can use it to package a subset of 188tests by using variables specified in `Running a subset of selftests`_ 189section:: 190 191 $ make -C tools/testing/selftests gen_tar TARGETS="bpf" FORMAT=.xz 192 193.. _tar's auto-compress: https://www.gnu.org/software/tar/manual/html_node/gzip.html#auto_002dcompress 194 195Contributing new tests 196====================== 197 198In general, the rules for selftests are 199 200 * Do as much as you can if you're not root; 201 202 * Don't take too long; 203 204 * Don't break the build on any architecture, and 205 206 * Don't cause the top-level "make run_tests" to fail if your feature is 207 unconfigured. 208 209Contributing new tests (details) 210================================ 211 212 * In your Makefile, use facilities from lib.mk by including it instead of 213 reinventing the wheel. Specify flags and binaries generation flags on 214 need basis before including lib.mk. :: 215 216 CFLAGS = $(KHDR_INCLUDES) 217 TEST_GEN_PROGS := close_range_test 218 include ../lib.mk 219 220 * Use TEST_GEN_XXX if such binaries or files are generated during 221 compiling. 222 223 TEST_PROGS, TEST_GEN_PROGS mean it is the executable tested by 224 default. 225 226 TEST_CUSTOM_PROGS should be used by tests that require custom build 227 rules and prevent common build rule use. 228 229 TEST_PROGS are for test shell scripts. Please ensure shell script has 230 its exec bit set. Otherwise, lib.mk run_tests will generate a warning. 231 232 TEST_CUSTOM_PROGS and TEST_PROGS will be run by common run_tests. 233 234 TEST_PROGS_EXTENDED, TEST_GEN_PROGS_EXTENDED mean it is the 235 executable which is not tested by default. 236 TEST_FILES, TEST_GEN_FILES mean it is the file which is used by 237 test. 238 239 * First use the headers inside the kernel source and/or git repo, and then the 240 system headers. Headers for the kernel release as opposed to headers 241 installed by the distro on the system should be the primary focus to be able 242 to find regressions. Use KHDR_INCLUDES in Makefile to include headers from 243 the kernel source. 244 245 * If a test needs specific kernel config options enabled, add a config file in 246 the test directory to enable them. 247 248 e.g: tools/testing/selftests/android/config 249 250 * Create a .gitignore file inside test directory and add all generated objects 251 in it. 252 253 * Add new test name in TARGETS in selftests/Makefile:: 254 255 TARGETS += android 256 257 * All changes should pass:: 258 259 kselftest-{all,install,clean,gen_tar} 260 kselftest-{all,install,clean,gen_tar} O=abo_path 261 kselftest-{all,install,clean,gen_tar} O=rel_path 262 make -C tools/testing/selftests {all,install,clean,gen_tar} 263 make -C tools/testing/selftests {all,install,clean,gen_tar} O=abs_path 264 make -C tools/testing/selftests {all,install,clean,gen_tar} O=rel_path 265 266Test Module 267=========== 268 269Kselftest tests the kernel from userspace. Sometimes things need 270testing from within the kernel, one method of doing this is to create a 271test module. We can tie the module into the kselftest framework by 272using a shell script test runner. ``kselftest/module.sh`` is designed 273to facilitate this process. There is also a header file provided to 274assist writing kernel modules that are for use with kselftest: 275 276- ``tools/testing/selftests/kselftest_module.h`` 277- ``tools/testing/selftests/kselftest/module.sh`` 278 279Note that test modules should taint the kernel with TAINT_TEST. This will 280happen automatically for modules which are in the ``tools/testing/`` 281directory, or for modules which use the ``kselftest_module.h`` header above. 282Otherwise, you'll need to add ``MODULE_INFO(test, "Y")`` to your module 283source. selftests which do not load modules typically should not taint the 284kernel, but in cases where a non-test module is loaded, TEST_TAINT can be 285applied from userspace by writing to ``/proc/sys/kernel/tainted``. 286 287How to use 288---------- 289 290Here we show the typical steps to create a test module and tie it into 291kselftest. We use kselftests for lib/ as an example. 292 2931. Create the test module 294 2952. Create the test script that will run (load/unload) the module 296 e.g. ``tools/testing/selftests/lib/printf.sh`` 297 2983. Add line to config file e.g. ``tools/testing/selftests/lib/config`` 299 3004. Add test script to makefile e.g. ``tools/testing/selftests/lib/Makefile`` 301 3025. Verify it works: 303 304.. code-block:: sh 305 306 # Assumes you have booted a fresh build of this kernel tree 307 cd /path/to/linux/tree 308 make kselftest-merge 309 make modules 310 sudo make modules_install 311 make TARGETS=lib kselftest 312 313Example Module 314-------------- 315 316A bare bones test module might look like this: 317 318.. code-block:: c 319 320 // SPDX-License-Identifier: GPL-2.0+ 321 322 #define pr_fmt(fmt) KBUILD_MODNAME ": " fmt 323 324 #include "../tools/testing/selftests/kselftest_module.h" 325 326 KSTM_MODULE_GLOBALS(); 327 328 /* 329 * Kernel module for testing the foobinator 330 */ 331 332 static int __init test_function() 333 { 334 ... 335 } 336 337 static void __init selftest(void) 338 { 339 KSTM_CHECK_ZERO(do_test_case("", 0)); 340 } 341 342 KSTM_MODULE_LOADERS(test_foo); 343 MODULE_AUTHOR("John Developer <jd@fooman.org>"); 344 MODULE_LICENSE("GPL"); 345 MODULE_INFO(test, "Y"); 346 347Example test script 348------------------- 349 350.. code-block:: sh 351 352 #!/bin/bash 353 # SPDX-License-Identifier: GPL-2.0+ 354 $(dirname $0)/../kselftest/module.sh "foo" test_foo 355 356 357Test Harness 358============ 359 360The kselftest_harness.h file contains useful helpers to build tests. The 361test harness is for userspace testing, for kernel space testing see `Test 362Module`_ above. 363 364The tests from tools/testing/selftests/seccomp/seccomp_bpf.c can be used as 365example. 366 367Example 368------- 369 370.. kernel-doc:: tools/testing/selftests/kselftest_harness.h 371 :doc: example 372 373 374Helpers 375------- 376 377.. kernel-doc:: tools/testing/selftests/kselftest_harness.h 378 :functions: TH_LOG TEST TEST_SIGNAL FIXTURE FIXTURE_DATA FIXTURE_SETUP 379 FIXTURE_TEARDOWN TEST_F TEST_HARNESS_MAIN FIXTURE_VARIANT 380 FIXTURE_VARIANT_ADD 381 382Operators 383--------- 384 385.. kernel-doc:: tools/testing/selftests/kselftest_harness.h 386 :doc: operators 387 388.. kernel-doc:: tools/testing/selftests/kselftest_harness.h 389 :functions: ASSERT_EQ ASSERT_NE ASSERT_LT ASSERT_LE ASSERT_GT ASSERT_GE 390 ASSERT_NULL ASSERT_TRUE ASSERT_NULL ASSERT_TRUE ASSERT_FALSE 391 ASSERT_STREQ ASSERT_STRNE EXPECT_EQ EXPECT_NE EXPECT_LT 392 EXPECT_LE EXPECT_GT EXPECT_GE EXPECT_NULL EXPECT_TRUE 393 EXPECT_FALSE EXPECT_STREQ EXPECT_STRNE 394