12757aafaSJonathan CorbetThe Kernel Address Sanitizer (KASAN) 22757aafaSJonathan Corbet==================================== 32757aafaSJonathan Corbet 42757aafaSJonathan CorbetOverview 52757aafaSJonathan Corbet-------- 62757aafaSJonathan Corbet 7625d8673SAndrey KonovalovKernelAddressSANitizer (KASAN) is a dynamic memory safety error detector 8625d8673SAndrey Konovalovdesigned to find out-of-bound and use-after-free bugs. KASAN has three modes: 92757aafaSJonathan Corbet 10948e3253SAndrey Konovalov1. generic KASAN (similar to userspace ASan), 11948e3253SAndrey Konovalov2. software tag-based KASAN (similar to userspace HWASan), 12948e3253SAndrey Konovalov3. hardware tag-based KASAN (based on hardware memory tagging). 132757aafaSJonathan Corbet 14948e3253SAndrey KonovalovSoftware KASAN modes (1 and 2) use compile-time instrumentation to insert 15948e3253SAndrey Konovalovvalidity checks before every memory access, and therefore require a compiler 16948e3253SAndrey Konovalovversion that supports that. 172757aafaSJonathan Corbet 18b3b0e6acSAndrey KonovalovGeneric KASAN is supported in both GCC and Clang. With GCC it requires version 19527f6750SMarco Elver8.3.0 or later. Any supported Clang version is compatible, but detection of 20ac4766beSMarco Elverout-of-bounds accesses for global variables is only supported since Clang 11. 21b3b0e6acSAndrey Konovalov 22527f6750SMarco ElverTag-based KASAN is only supported in Clang. 23b3b0e6acSAndrey Konovalov 2442101571SLinus WalleijCurrently generic KASAN is supported for the x86_64, arm, arm64, xtensa, s390 25948e3253SAndrey Konovalovand riscv architectures, and tag-based KASAN modes are supported only for arm64. 262757aafaSJonathan Corbet 272757aafaSJonathan CorbetUsage 282757aafaSJonathan Corbet----- 292757aafaSJonathan Corbet 302757aafaSJonathan CorbetTo enable KASAN configure kernel with:: 312757aafaSJonathan Corbet 322757aafaSJonathan Corbet CONFIG_KASAN = y 332757aafaSJonathan Corbet 34948e3253SAndrey Konovalovand choose between CONFIG_KASAN_GENERIC (to enable generic KASAN), 35948e3253SAndrey KonovalovCONFIG_KASAN_SW_TAGS (to enable software tag-based KASAN), and 36948e3253SAndrey KonovalovCONFIG_KASAN_HW_TAGS (to enable hardware tag-based KASAN). 372757aafaSJonathan Corbet 38948e3253SAndrey KonovalovFor software modes, you also need to choose between CONFIG_KASAN_OUTLINE and 39948e3253SAndrey KonovalovCONFIG_KASAN_INLINE. Outline and inline are compiler instrumentation types. 40948e3253SAndrey KonovalovThe former produces smaller binary while the latter is 1.1 - 2 times faster. 41b3b0e6acSAndrey Konovalov 42948e3253SAndrey KonovalovBoth software KASAN modes work with both SLUB and SLAB memory allocators, 43625d8673SAndrey Konovalovwhile the hardware tag-based KASAN currently only support SLUB. 44625d8673SAndrey Konovalov 45625d8673SAndrey KonovalovFor better error reports that include stack traces, enable CONFIG_STACKTRACE. 462757aafaSJonathan Corbet 470fe9a448SVlastimil BabkaTo augment reports with last allocation and freeing stack of the physical page, 480fe9a448SVlastimil Babkait is recommended to enable also CONFIG_PAGE_OWNER and boot with page_owner=on. 490fe9a448SVlastimil Babka 502757aafaSJonathan CorbetError reports 512757aafaSJonathan Corbet~~~~~~~~~~~~~ 522757aafaSJonathan Corbet 53b3b0e6acSAndrey KonovalovA typical out-of-bounds access generic KASAN report looks like this:: 542757aafaSJonathan Corbet 552757aafaSJonathan Corbet ================================================================== 56b3b0e6acSAndrey Konovalov BUG: KASAN: slab-out-of-bounds in kmalloc_oob_right+0xa8/0xbc [test_kasan] 57b3b0e6acSAndrey Konovalov Write of size 1 at addr ffff8801f44ec37b by task insmod/2760 582757aafaSJonathan Corbet 59b3b0e6acSAndrey Konovalov CPU: 1 PID: 2760 Comm: insmod Not tainted 4.19.0-rc3+ #698 60b3b0e6acSAndrey Konovalov Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.10.2-1 04/01/2014 612757aafaSJonathan Corbet Call Trace: 62b3b0e6acSAndrey Konovalov dump_stack+0x94/0xd8 63b3b0e6acSAndrey Konovalov print_address_description+0x73/0x280 64b3b0e6acSAndrey Konovalov kasan_report+0x144/0x187 65b3b0e6acSAndrey Konovalov __asan_report_store1_noabort+0x17/0x20 66b3b0e6acSAndrey Konovalov kmalloc_oob_right+0xa8/0xbc [test_kasan] 67b3b0e6acSAndrey Konovalov kmalloc_tests_init+0x16/0x700 [test_kasan] 68b3b0e6acSAndrey Konovalov do_one_initcall+0xa5/0x3ae 69b3b0e6acSAndrey Konovalov do_init_module+0x1b6/0x547 70b3b0e6acSAndrey Konovalov load_module+0x75df/0x8070 71b3b0e6acSAndrey Konovalov __do_sys_init_module+0x1c6/0x200 72b3b0e6acSAndrey Konovalov __x64_sys_init_module+0x6e/0xb0 73b3b0e6acSAndrey Konovalov do_syscall_64+0x9f/0x2c0 74b3b0e6acSAndrey Konovalov entry_SYSCALL_64_after_hwframe+0x44/0xa9 75b3b0e6acSAndrey Konovalov RIP: 0033:0x7f96443109da 76b3b0e6acSAndrey Konovalov RSP: 002b:00007ffcf0b51b08 EFLAGS: 00000202 ORIG_RAX: 00000000000000af 77b3b0e6acSAndrey Konovalov RAX: ffffffffffffffda RBX: 000055dc3ee521a0 RCX: 00007f96443109da 78b3b0e6acSAndrey Konovalov RDX: 00007f96445cff88 RSI: 0000000000057a50 RDI: 00007f9644992000 79b3b0e6acSAndrey Konovalov RBP: 000055dc3ee510b0 R08: 0000000000000003 R09: 0000000000000000 80b3b0e6acSAndrey Konovalov R10: 00007f964430cd0a R11: 0000000000000202 R12: 00007f96445cff88 81b3b0e6acSAndrey Konovalov R13: 000055dc3ee51090 R14: 0000000000000000 R15: 0000000000000000 82b3b0e6acSAndrey Konovalov 83b3b0e6acSAndrey Konovalov Allocated by task 2760: 84b3b0e6acSAndrey Konovalov save_stack+0x43/0xd0 85b3b0e6acSAndrey Konovalov kasan_kmalloc+0xa7/0xd0 86b3b0e6acSAndrey Konovalov kmem_cache_alloc_trace+0xe1/0x1b0 87b3b0e6acSAndrey Konovalov kmalloc_oob_right+0x56/0xbc [test_kasan] 88b3b0e6acSAndrey Konovalov kmalloc_tests_init+0x16/0x700 [test_kasan] 89b3b0e6acSAndrey Konovalov do_one_initcall+0xa5/0x3ae 90b3b0e6acSAndrey Konovalov do_init_module+0x1b6/0x547 91b3b0e6acSAndrey Konovalov load_module+0x75df/0x8070 92b3b0e6acSAndrey Konovalov __do_sys_init_module+0x1c6/0x200 93b3b0e6acSAndrey Konovalov __x64_sys_init_module+0x6e/0xb0 94b3b0e6acSAndrey Konovalov do_syscall_64+0x9f/0x2c0 95b3b0e6acSAndrey Konovalov entry_SYSCALL_64_after_hwframe+0x44/0xa9 96b3b0e6acSAndrey Konovalov 97b3b0e6acSAndrey Konovalov Freed by task 815: 98b3b0e6acSAndrey Konovalov save_stack+0x43/0xd0 99b3b0e6acSAndrey Konovalov __kasan_slab_free+0x135/0x190 100b3b0e6acSAndrey Konovalov kasan_slab_free+0xe/0x10 101b3b0e6acSAndrey Konovalov kfree+0x93/0x1a0 102b3b0e6acSAndrey Konovalov umh_complete+0x6a/0xa0 103b3b0e6acSAndrey Konovalov call_usermodehelper_exec_async+0x4c3/0x640 104b3b0e6acSAndrey Konovalov ret_from_fork+0x35/0x40 105b3b0e6acSAndrey Konovalov 106b3b0e6acSAndrey Konovalov The buggy address belongs to the object at ffff8801f44ec300 107b3b0e6acSAndrey Konovalov which belongs to the cache kmalloc-128 of size 128 108b3b0e6acSAndrey Konovalov The buggy address is located 123 bytes inside of 109b3b0e6acSAndrey Konovalov 128-byte region [ffff8801f44ec300, ffff8801f44ec380) 110b3b0e6acSAndrey Konovalov The buggy address belongs to the page: 111b3b0e6acSAndrey Konovalov page:ffffea0007d13b00 count:1 mapcount:0 mapping:ffff8801f7001640 index:0x0 112b3b0e6acSAndrey Konovalov flags: 0x200000000000100(slab) 113b3b0e6acSAndrey Konovalov raw: 0200000000000100 ffffea0007d11dc0 0000001a0000001a ffff8801f7001640 114b3b0e6acSAndrey Konovalov raw: 0000000000000000 0000000080150015 00000001ffffffff 0000000000000000 115b3b0e6acSAndrey Konovalov page dumped because: kasan: bad access detected 116b3b0e6acSAndrey Konovalov 1172757aafaSJonathan Corbet Memory state around the buggy address: 118b3b0e6acSAndrey Konovalov ffff8801f44ec200: fc fc fc fc fc fc fc fc fb fb fb fb fb fb fb fb 119b3b0e6acSAndrey Konovalov ffff8801f44ec280: fb fb fb fb fb fb fb fb fc fc fc fc fc fc fc fc 120b3b0e6acSAndrey Konovalov >ffff8801f44ec300: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 03 1212757aafaSJonathan Corbet ^ 122b3b0e6acSAndrey Konovalov ffff8801f44ec380: fc fc fc fc fc fc fc fc fb fb fb fb fb fb fb fb 123b3b0e6acSAndrey Konovalov ffff8801f44ec400: fb fb fb fb fb fb fb fb fc fc fc fc fc fc fc fc 1242757aafaSJonathan Corbet ================================================================== 1252757aafaSJonathan Corbet 126b3b0e6acSAndrey KonovalovThe header of the report provides a short summary of what kind of bug happened 127b3b0e6acSAndrey Konovalovand what kind of access caused it. It's followed by a stack trace of the bad 128b3b0e6acSAndrey Konovalovaccess, a stack trace of where the accessed memory was allocated (in case bad 129b3b0e6acSAndrey Konovalovaccess happens on a slab object), and a stack trace of where the object was 130b3b0e6acSAndrey Konovalovfreed (in case of a use-after-free bug report). Next comes a description of 131b3b0e6acSAndrey Konovalovthe accessed slab object and information about the accessed memory page. 1322757aafaSJonathan Corbet 1332757aafaSJonathan CorbetIn the last section the report shows memory state around the accessed address. 134625d8673SAndrey KonovalovInternally KASAN tracks memory state separately for each memory granule, which 135625d8673SAndrey Konovalovis either 8 or 16 aligned bytes depending on KASAN mode. Each number in the 136625d8673SAndrey Konovalovmemory state section of the report shows the state of one of the memory 137625d8673SAndrey Konovalovgranules that surround the accessed address. 1382757aafaSJonathan Corbet 139625d8673SAndrey KonovalovFor generic KASAN the size of each memory granule is 8. The state of each 140625d8673SAndrey Konovalovgranule is encoded in one shadow byte. Those 8 bytes can be accessible, 141625d8673SAndrey Konovalovpartially accessible, freed or be a part of a redzone. KASAN uses the following 142625d8673SAndrey Konovalovencoding for each shadow byte: 0 means that all 8 bytes of the corresponding 143625d8673SAndrey Konovalovmemory region are accessible; number N (1 <= N <= 7) means that the first N 144625d8673SAndrey Konovalovbytes are accessible, and other (8 - N) bytes are not; any negative value 145625d8673SAndrey Konovalovindicates that the entire 8-byte word is inaccessible. KASAN uses different 146625d8673SAndrey Konovalovnegative values to distinguish between different kinds of inaccessible memory 147625d8673SAndrey Konovalovlike redzones or freed memory (see mm/kasan/kasan.h). 1482757aafaSJonathan Corbet 1492757aafaSJonathan CorbetIn the report above the arrows point to the shadow byte 03, which means that 1504062c245SAndrey Konovalovthe accessed address is partially accessible. For tag-based KASAN modes this 1514062c245SAndrey Konovalovlast report section shows the memory tags around the accessed address 1524062c245SAndrey Konovalov(see the `Implementation details`_ section). 153625d8673SAndrey Konovalov 154625d8673SAndrey KonovalovBoot parameters 155625d8673SAndrey Konovalov~~~~~~~~~~~~~~~ 156625d8673SAndrey Konovalov 1574062c245SAndrey KonovalovHardware tag-based KASAN mode (see the section about various modes below) is 1587169487bSAndrey Konovalovintended for use in production as a security mitigation. Therefore, it supports 159625d8673SAndrey Konovalovboot parameters that allow to disable KASAN competely or otherwise control 160625d8673SAndrey Konovalovparticular KASAN features. 161625d8673SAndrey Konovalov 16276bc99e8SAndrey Konovalov- ``kasan=off`` or ``=on`` controls whether KASAN is enabled (default: ``on``). 163625d8673SAndrey Konovalov 1642603f8a7SVincenzo Frascino- ``kasan.mode=sync`` or ``=async`` controls whether KASAN is configured in 1652603f8a7SVincenzo Frascino synchronous or asynchronous mode of execution (default: ``sync``). 1662603f8a7SVincenzo Frascino Synchronous mode: a bad access is detected immediately when a tag 1672603f8a7SVincenzo Frascino check fault occurs. 1682603f8a7SVincenzo Frascino Asynchronous mode: a bad access detection is delayed. When a tag check 1692603f8a7SVincenzo Frascino fault occurs, the information is stored in hardware (in the TFSR_EL1 1702603f8a7SVincenzo Frascino register for arm64). The kernel periodically checks the hardware and 1712603f8a7SVincenzo Frascino only reports tag faults during these checks. 1722603f8a7SVincenzo Frascino 17376bc99e8SAndrey Konovalov- ``kasan.stacktrace=off`` or ``=on`` disables or enables alloc and free stack 1741cc4cdb5SAndrey Konovalov traces collection (default: ``on``). 175625d8673SAndrey Konovalov 17676bc99e8SAndrey Konovalov- ``kasan.fault=report`` or ``=panic`` controls whether to only print a KASAN 1777169487bSAndrey Konovalov report or also panic the kernel (default: ``report``). Note, that tag 1787169487bSAndrey Konovalov checking gets disabled after the first reported bug. 179625d8673SAndrey Konovalov 1802757aafaSJonathan CorbetImplementation details 1812757aafaSJonathan Corbet---------------------- 1822757aafaSJonathan Corbet 183b3b0e6acSAndrey KonovalovGeneric KASAN 184b3b0e6acSAndrey Konovalov~~~~~~~~~~~~~ 185b3b0e6acSAndrey Konovalov 186625d8673SAndrey KonovalovFrom a high level perspective, KASAN's approach to memory error detection is 187625d8673SAndrey Konovalovsimilar to that of kmemcheck: use shadow memory to record whether each byte of 188625d8673SAndrey Konovalovmemory is safe to access, and use compile-time instrumentation to insert checks 189625d8673SAndrey Konovalovof shadow memory on each memory access. 1902757aafaSJonathan Corbet 191b3b0e6acSAndrey KonovalovGeneric KASAN dedicates 1/8th of kernel memory to its shadow memory (e.g. 16TB 192b3b0e6acSAndrey Konovalovto cover 128TB on x86_64) and uses direct mapping with a scale and offset to 193b3b0e6acSAndrey Konovalovtranslate a memory address to its corresponding shadow address. 1942757aafaSJonathan Corbet 1952757aafaSJonathan CorbetHere is the function which translates an address to its corresponding shadow 1962757aafaSJonathan Corbetaddress:: 1972757aafaSJonathan Corbet 1982757aafaSJonathan Corbet static inline void *kasan_mem_to_shadow(const void *addr) 1992757aafaSJonathan Corbet { 2002757aafaSJonathan Corbet return ((unsigned long)addr >> KASAN_SHADOW_SCALE_SHIFT) 2012757aafaSJonathan Corbet + KASAN_SHADOW_OFFSET; 2022757aafaSJonathan Corbet } 2032757aafaSJonathan Corbet 2042757aafaSJonathan Corbetwhere ``KASAN_SHADOW_SCALE_SHIFT = 3``. 2052757aafaSJonathan Corbet 206b3b0e6acSAndrey KonovalovCompile-time instrumentation is used to insert memory access checks. Compiler 207b3b0e6acSAndrey Konovalovinserts function calls (__asan_load*(addr), __asan_store*(addr)) before each 208b3b0e6acSAndrey Konovalovmemory access of size 1, 2, 4, 8 or 16. These functions check whether memory 209b3b0e6acSAndrey Konovalovaccess is valid or not by checking corresponding shadow memory. 2102757aafaSJonathan Corbet 2112757aafaSJonathan CorbetGCC 5.0 has possibility to perform inline instrumentation. Instead of making 2122757aafaSJonathan Corbetfunction calls GCC directly inserts the code to check the shadow memory. 2132757aafaSJonathan CorbetThis option significantly enlarges kernel but it gives x1.1-x2 performance 2142757aafaSJonathan Corbetboost over outline instrumented kernel. 215b3b0e6acSAndrey Konovalov 2164784be28SWalter WuGeneric KASAN also reports the last 2 call stacks to creation of work that 2174784be28SWalter Wupotentially has access to an object. Call stacks for the following are shown: 2184784be28SWalter Wucall_rcu() and workqueue queuing. 2199793b626SWalter Wu 220625d8673SAndrey KonovalovGeneric KASAN is the only mode that delays the reuse of freed object via 221625d8673SAndrey Konovalovquarantine (see mm/kasan/quarantine.c for implementation). 222625d8673SAndrey Konovalov 223b3b0e6acSAndrey KonovalovSoftware tag-based KASAN 224b3b0e6acSAndrey Konovalov~~~~~~~~~~~~~~~~~~~~~~~~ 225b3b0e6acSAndrey Konovalov 226948e3253SAndrey KonovalovSoftware tag-based KASAN requires software memory tagging support in the form 227948e3253SAndrey Konovalovof HWASan-like compiler instrumentation (see HWASan documentation for details). 228948e3253SAndrey Konovalov 229948e3253SAndrey KonovalovSoftware tag-based KASAN is currently only implemented for arm64 architecture. 230948e3253SAndrey Konovalov 231948e3253SAndrey KonovalovSoftware tag-based KASAN uses the Top Byte Ignore (TBI) feature of arm64 CPUs 232948e3253SAndrey Konovalovto store a pointer tag in the top byte of kernel pointers. Like generic KASAN 233948e3253SAndrey Konovalovit uses shadow memory to store memory tags associated with each 16-byte memory 234b3b0e6acSAndrey Konovalovcell (therefore it dedicates 1/16th of the kernel memory for shadow memory). 235b3b0e6acSAndrey Konovalov 236948e3253SAndrey KonovalovOn each memory allocation software tag-based KASAN generates a random tag, tags 237948e3253SAndrey Konovalovthe allocated memory with this tag, and embeds this tag into the returned 238948e3253SAndrey Konovalovpointer. 239948e3253SAndrey Konovalov 240b3b0e6acSAndrey KonovalovSoftware tag-based KASAN uses compile-time instrumentation to insert checks 241b3b0e6acSAndrey Konovalovbefore each memory access. These checks make sure that tag of the memory that 242b3b0e6acSAndrey Konovalovis being accessed is equal to tag of the pointer that is used to access this 243948e3253SAndrey Konovalovmemory. In case of a tag mismatch software tag-based KASAN prints a bug report. 244b3b0e6acSAndrey Konovalov 245b3b0e6acSAndrey KonovalovSoftware tag-based KASAN also has two instrumentation modes (outline, that 246b3b0e6acSAndrey Konovalovemits callbacks to check memory accesses; and inline, that performs the shadow 247b3b0e6acSAndrey Konovalovmemory checks inline). With outline instrumentation mode, a bug report is 248b3b0e6acSAndrey Konovalovsimply printed from the function that performs the access check. With inline 249b3b0e6acSAndrey Konovalovinstrumentation a brk instruction is emitted by the compiler, and a dedicated 250b3b0e6acSAndrey Konovalovbrk handler is used to print bug reports. 251b3b0e6acSAndrey Konovalov 252948e3253SAndrey KonovalovSoftware tag-based KASAN uses 0xFF as a match-all pointer tag (accesses through 253948e3253SAndrey Konovalovpointers with 0xFF pointer tag aren't checked). The value 0xFE is currently 254948e3253SAndrey Konovalovreserved to tag freed memory regions. 255948e3253SAndrey Konovalov 256948e3253SAndrey KonovalovSoftware tag-based KASAN currently only supports tagging of 257948e3253SAndrey Konovalovkmem_cache_alloc/kmalloc and page_alloc memory. 258948e3253SAndrey Konovalov 259948e3253SAndrey KonovalovHardware tag-based KASAN 260948e3253SAndrey Konovalov~~~~~~~~~~~~~~~~~~~~~~~~ 261948e3253SAndrey Konovalov 262948e3253SAndrey KonovalovHardware tag-based KASAN is similar to the software mode in concept, but uses 263948e3253SAndrey Konovalovhardware memory tagging support instead of compiler instrumentation and 264948e3253SAndrey Konovalovshadow memory. 265948e3253SAndrey Konovalov 266948e3253SAndrey KonovalovHardware tag-based KASAN is currently only implemented for arm64 architecture 267948e3253SAndrey Konovalovand based on both arm64 Memory Tagging Extension (MTE) introduced in ARMv8.5 268948e3253SAndrey KonovalovInstruction Set Architecture, and Top Byte Ignore (TBI). 269948e3253SAndrey Konovalov 270948e3253SAndrey KonovalovSpecial arm64 instructions are used to assign memory tags for each allocation. 271948e3253SAndrey KonovalovSame tags are assigned to pointers to those allocations. On every memory 272948e3253SAndrey Konovalovaccess, hardware makes sure that tag of the memory that is being accessed is 273948e3253SAndrey Konovalovequal to tag of the pointer that is used to access this memory. In case of a 274948e3253SAndrey Konovalovtag mismatch a fault is generated and a report is printed. 275948e3253SAndrey Konovalov 276948e3253SAndrey KonovalovHardware tag-based KASAN uses 0xFF as a match-all pointer tag (accesses through 277948e3253SAndrey Konovalovpointers with 0xFF pointer tag aren't checked). The value 0xFE is currently 278948e3253SAndrey Konovalovreserved to tag freed memory regions. 279948e3253SAndrey Konovalov 280948e3253SAndrey KonovalovHardware tag-based KASAN currently only supports tagging of 281948e3253SAndrey Konovalovkmem_cache_alloc/kmalloc and page_alloc memory. 2823c5c3cfbSDaniel Axtens 2834062c245SAndrey KonovalovIf the hardware doesn't support MTE (pre ARMv8.5), hardware tag-based KASAN 2844062c245SAndrey Konovalovwon't be enabled. In this case all boot parameters are ignored. 2854062c245SAndrey Konovalov 2864062c245SAndrey KonovalovNote, that enabling CONFIG_KASAN_HW_TAGS always results in in-kernel TBI being 2874062c245SAndrey Konovalovenabled. Even when kasan.mode=off is provided, or when the hardware doesn't 2884062c245SAndrey Konovalovsupport MTE (but supports TBI). 2894062c245SAndrey Konovalov 2907169487bSAndrey KonovalovHardware tag-based KASAN only reports the first found bug. After that MTE tag 2917169487bSAndrey Konovalovchecking gets disabled. 2927169487bSAndrey Konovalov 293*96d7d141SAndrey KonovalovShadow memory 294*96d7d141SAndrey Konovalov------------- 2953c5c3cfbSDaniel Axtens 2963c5c3cfbSDaniel AxtensThe kernel maps memory in a number of different parts of the address 2973c5c3cfbSDaniel Axtensspace. This poses something of a problem for KASAN, which requires 2983c5c3cfbSDaniel Axtensthat all addresses accessed by instrumented code have a valid shadow 2993c5c3cfbSDaniel Axtensregion. 3003c5c3cfbSDaniel Axtens 3013c5c3cfbSDaniel AxtensThe range of kernel virtual addresses is large: there is not enough 3023c5c3cfbSDaniel Axtensreal memory to support a real shadow region for every address that 3033c5c3cfbSDaniel Axtenscould be accessed by the kernel. 3043c5c3cfbSDaniel Axtens 305*96d7d141SAndrey KonovalovDefault behaviour 306*96d7d141SAndrey Konovalov~~~~~~~~~~~~~~~~~ 3073c5c3cfbSDaniel Axtens 3083c5c3cfbSDaniel AxtensBy default, architectures only map real memory over the shadow region 3093c5c3cfbSDaniel Axtensfor the linear mapping (and potentially other small areas). For all 3103c5c3cfbSDaniel Axtensother areas - such as vmalloc and vmemmap space - a single read-only 3113c5c3cfbSDaniel Axtenspage is mapped over the shadow area. This read-only shadow page 3123c5c3cfbSDaniel Axtensdeclares all memory accesses as permitted. 3133c5c3cfbSDaniel Axtens 3143c5c3cfbSDaniel AxtensThis presents a problem for modules: they do not live in the linear 3153c5c3cfbSDaniel Axtensmapping, but in a dedicated module space. By hooking in to the module 3163c5c3cfbSDaniel Axtensallocator, KASAN can temporarily map real shadow memory to cover 3173c5c3cfbSDaniel Axtensthem. This allows detection of invalid accesses to module globals, for 3183c5c3cfbSDaniel Axtensexample. 3193c5c3cfbSDaniel Axtens 3203c5c3cfbSDaniel AxtensThis also creates an incompatibility with ``VMAP_STACK``: if the stack 3213c5c3cfbSDaniel Axtenslives in vmalloc space, it will be shadowed by the read-only page, and 3223c5c3cfbSDaniel Axtensthe kernel will fault when trying to set up the shadow data for stack 3233c5c3cfbSDaniel Axtensvariables. 3243c5c3cfbSDaniel Axtens 3253c5c3cfbSDaniel AxtensCONFIG_KASAN_VMALLOC 3263c5c3cfbSDaniel Axtens~~~~~~~~~~~~~~~~~~~~ 3273c5c3cfbSDaniel Axtens 3283c5c3cfbSDaniel AxtensWith ``CONFIG_KASAN_VMALLOC``, KASAN can cover vmalloc space at the 3293c5c3cfbSDaniel Axtenscost of greater memory usage. Currently this is only supported on x86. 3303c5c3cfbSDaniel Axtens 3313c5c3cfbSDaniel AxtensThis works by hooking into vmalloc and vmap, and dynamically 3323c5c3cfbSDaniel Axtensallocating real shadow memory to back the mappings. 3333c5c3cfbSDaniel Axtens 3343c5c3cfbSDaniel AxtensMost mappings in vmalloc space are small, requiring less than a full 3353c5c3cfbSDaniel Axtenspage of shadow space. Allocating a full shadow page per mapping would 3363c5c3cfbSDaniel Axtenstherefore be wasteful. Furthermore, to ensure that different mappings 3373c5c3cfbSDaniel Axtensuse different shadow pages, mappings would have to be aligned to 3381f600626SAndrey Konovalov``KASAN_GRANULE_SIZE * PAGE_SIZE``. 3393c5c3cfbSDaniel Axtens 340625d8673SAndrey KonovalovInstead, KASAN shares backing space across multiple mappings. It allocates 3413c5c3cfbSDaniel Axtensa backing page when a mapping in vmalloc space uses a particular page 3423c5c3cfbSDaniel Axtensof the shadow region. This page can be shared by other vmalloc 3433c5c3cfbSDaniel Axtensmappings later on. 3443c5c3cfbSDaniel Axtens 345625d8673SAndrey KonovalovKASAN hooks into the vmap infrastructure to lazily clean up unused shadow 3463c5c3cfbSDaniel Axtensmemory. 3473c5c3cfbSDaniel Axtens 348625d8673SAndrey KonovalovTo avoid the difficulties around swapping mappings around, KASAN expects 3493c5c3cfbSDaniel Axtensthat the part of the shadow region that covers the vmalloc space will 3503c5c3cfbSDaniel Axtensnot be covered by the early shadow page, but will be left 3513c5c3cfbSDaniel Axtensunmapped. This will require changes in arch-specific code. 3523c5c3cfbSDaniel Axtens 3533c5c3cfbSDaniel AxtensThis allows ``VMAP_STACK`` support on x86, and can simplify support of 3543c5c3cfbSDaniel Axtensarchitectures that do not have a fixed module region. 3559ab5be97SPatricia Alfonso 356*96d7d141SAndrey KonovalovFor developers 357*96d7d141SAndrey Konovalov-------------- 358*96d7d141SAndrey Konovalov 359*96d7d141SAndrey KonovalovIgnoring accesses 360*96d7d141SAndrey Konovalov~~~~~~~~~~~~~~~~~ 361*96d7d141SAndrey Konovalov 362*96d7d141SAndrey KonovalovSoftware KASAN modes use compiler instrumentation to insert validity checks. 363*96d7d141SAndrey KonovalovSuch instrumentation might be incompatible with some part of the kernel, and 364*96d7d141SAndrey Konovalovtherefore needs to be disabled. To disable instrumentation for specific files 365*96d7d141SAndrey Konovalovor directories, add a line similar to the following to the respective kernel 366*96d7d141SAndrey KonovalovMakefile: 367*96d7d141SAndrey Konovalov 368*96d7d141SAndrey Konovalov- For a single file (e.g. main.o):: 369*96d7d141SAndrey Konovalov 370*96d7d141SAndrey Konovalov KASAN_SANITIZE_main.o := n 371*96d7d141SAndrey Konovalov 372*96d7d141SAndrey Konovalov- For all files in one directory:: 373*96d7d141SAndrey Konovalov 374*96d7d141SAndrey Konovalov KASAN_SANITIZE := n 375*96d7d141SAndrey Konovalov 376*96d7d141SAndrey Konovalov 377*96d7d141SAndrey KonovalovTests 378*96d7d141SAndrey Konovalov~~~~~ 3799ab5be97SPatricia Alfonso 3805d92bdffSAndrey KonovalovKASAN tests consist of two parts: 3819ab5be97SPatricia Alfonso 382625d8673SAndrey Konovalov1. Tests that are integrated with the KUnit Test Framework. Enabled with 383625d8673SAndrey Konovalov``CONFIG_KASAN_KUNIT_TEST``. These tests can be run and partially verified 384625d8673SAndrey Konovalovautomatically in a few different ways, see the instructions below. 3859ab5be97SPatricia Alfonso 386625d8673SAndrey Konovalov2. Tests that are currently incompatible with KUnit. Enabled with 3875d92bdffSAndrey Konovalov``CONFIG_KASAN_MODULE_TEST`` and can only be run as a module. These tests can 388625d8673SAndrey Konovalovonly be verified manually, by loading the kernel module and inspecting the 389625d8673SAndrey Konovalovkernel log for KASAN reports. 390625d8673SAndrey Konovalov 391625d8673SAndrey KonovalovEach KUnit-compatible KASAN test prints a KASAN report if an error is detected. 392625d8673SAndrey KonovalovThen the test prints its number and status. 393625d8673SAndrey Konovalov 394625d8673SAndrey KonovalovWhen a test passes:: 3959ab5be97SPatricia Alfonso 3969ab5be97SPatricia Alfonso ok 28 - kmalloc_double_kzfree 39732519c03SMauro Carvalho Chehab 398625d8673SAndrey KonovalovWhen a test fails due to a failed ``kmalloc``:: 3999ab5be97SPatricia Alfonso 4009ab5be97SPatricia Alfonso # kmalloc_large_oob_right: ASSERTION FAILED at lib/test_kasan.c:163 4019ab5be97SPatricia Alfonso Expected ptr is not null, but is 4029ab5be97SPatricia Alfonso not ok 4 - kmalloc_large_oob_right 40332519c03SMauro Carvalho Chehab 404625d8673SAndrey KonovalovWhen a test fails due to a missing KASAN report:: 4059ab5be97SPatricia Alfonso 4069ab5be97SPatricia Alfonso # kmalloc_double_kzfree: EXPECTATION FAILED at lib/test_kasan.c:629 4079ab5be97SPatricia Alfonso Expected kasan_data->report_expected == kasan_data->report_found, but 4089ab5be97SPatricia Alfonso kasan_data->report_expected == 1 4099ab5be97SPatricia Alfonso kasan_data->report_found == 0 4109ab5be97SPatricia Alfonso not ok 28 - kmalloc_double_kzfree 4119ab5be97SPatricia Alfonso 412625d8673SAndrey KonovalovAt the end the cumulative status of all KASAN tests is printed. On success:: 4139ab5be97SPatricia Alfonso 4149ab5be97SPatricia Alfonso ok 1 - kasan 4159ab5be97SPatricia Alfonso 416625d8673SAndrey KonovalovOr, if one of the tests failed:: 4179ab5be97SPatricia Alfonso 4189ab5be97SPatricia Alfonso not ok 1 - kasan 4199ab5be97SPatricia Alfonso 420625d8673SAndrey Konovalov 421625d8673SAndrey KonovalovThere are a few ways to run KUnit-compatible KASAN tests. 422625d8673SAndrey Konovalov 423625d8673SAndrey Konovalov1. Loadable module 4249ab5be97SPatricia Alfonso 4259ab5be97SPatricia AlfonsoWith ``CONFIG_KUNIT`` enabled, ``CONFIG_KASAN_KUNIT_TEST`` can be built as 426625d8673SAndrey Konovalova loadable module and run on any architecture that supports KASAN by loading 427625d8673SAndrey Konovalovthe module with insmod or modprobe. The module is called ``test_kasan``. 4289ab5be97SPatricia Alfonso 429625d8673SAndrey Konovalov2. Built-In 4309ab5be97SPatricia Alfonso 4319ab5be97SPatricia AlfonsoWith ``CONFIG_KUNIT`` built-in, ``CONFIG_KASAN_KUNIT_TEST`` can be built-in 432625d8673SAndrey Konovalovon any architecure that supports KASAN. These and any other KUnit tests enabled 433625d8673SAndrey Konovalovwill run and print the results at boot as a late-init call. 4349ab5be97SPatricia Alfonso 435625d8673SAndrey Konovalov3. Using kunit_tool 4369ab5be97SPatricia Alfonso 437625d8673SAndrey KonovalovWith ``CONFIG_KUNIT`` and ``CONFIG_KASAN_KUNIT_TEST`` built-in, it's also 438625d8673SAndrey Konovalovpossible use ``kunit_tool`` to see the results of these and other KUnit tests 439625d8673SAndrey Konovalovin a more readable way. This will not print the KASAN reports of the tests that 440625d8673SAndrey Konovalovpassed. Use `KUnit documentation <https://www.kernel.org/doc/html/latest/dev-tools/kunit/index.html>`_ 441625d8673SAndrey Konovalovfor more up-to-date information on ``kunit_tool``. 4429ab5be97SPatricia Alfonso 4439ab5be97SPatricia Alfonso.. _KUnit: https://www.kernel.org/doc/html/latest/dev-tools/kunit/index.html 444