12757aafaSJonathan CorbetThe Kernel Address Sanitizer (KASAN) 22757aafaSJonathan Corbet==================================== 32757aafaSJonathan Corbet 42757aafaSJonathan CorbetOverview 52757aafaSJonathan Corbet-------- 62757aafaSJonathan Corbet 7625d8673SAndrey KonovalovKernelAddressSANitizer (KASAN) is a dynamic memory safety error detector 8625d8673SAndrey Konovalovdesigned to find out-of-bound and use-after-free bugs. KASAN has three modes: 92757aafaSJonathan Corbet 10948e3253SAndrey Konovalov1. generic KASAN (similar to userspace ASan), 11948e3253SAndrey Konovalov2. software tag-based KASAN (similar to userspace HWASan), 12948e3253SAndrey Konovalov3. hardware tag-based KASAN (based on hardware memory tagging). 132757aafaSJonathan Corbet 14*3cbc37dcSAndrey KonovalovGeneric KASAN is mainly used for debugging due to a large memory overhead. 15*3cbc37dcSAndrey KonovalovSoftware tag-based KASAN can be used for dogfood testing as it has a lower 16*3cbc37dcSAndrey Konovalovmemory overhead that allows using it with real workloads. Hardware tag-based 17*3cbc37dcSAndrey KonovalovKASAN comes with low memory and performance overheads and, therefore, can be 18*3cbc37dcSAndrey Konovalovused in production. Either as an in-field memory bug detector or as a security 19*3cbc37dcSAndrey Konovalovmitigation. 20*3cbc37dcSAndrey Konovalov 21*3cbc37dcSAndrey KonovalovSoftware KASAN modes (#1 and #2) use compile-time instrumentation to insert 22*3cbc37dcSAndrey Konovalovvalidity checks before every memory access and, therefore, require a compiler 23948e3253SAndrey Konovalovversion that supports that. 242757aafaSJonathan Corbet 25*3cbc37dcSAndrey KonovalovGeneric KASAN is supported in GCC and Clang. With GCC, it requires version 26527f6750SMarco Elver8.3.0 or later. Any supported Clang version is compatible, but detection of 27ac4766beSMarco Elverout-of-bounds accesses for global variables is only supported since Clang 11. 28b3b0e6acSAndrey Konovalov 29*3cbc37dcSAndrey KonovalovSoftware tag-based KASAN mode is only supported in Clang. 30b3b0e6acSAndrey Konovalov 31*3cbc37dcSAndrey KonovalovThe hardware KASAN mode (#3) relies on hardware to perform the checks but 32*3cbc37dcSAndrey Konovalovstill requires a compiler version that supports memory tagging instructions. 33*3cbc37dcSAndrey KonovalovThis mode is supported in GCC 10+ and Clang 11+. 34*3cbc37dcSAndrey Konovalov 35*3cbc37dcSAndrey KonovalovBoth software KASAN modes work with SLUB and SLAB memory allocators, 36*3cbc37dcSAndrey Konovalovwhile the hardware tag-based KASAN currently only supports SLUB. 37*3cbc37dcSAndrey Konovalov 38*3cbc37dcSAndrey KonovalovCurrently, generic KASAN is supported for the x86_64, arm, arm64, xtensa, s390, 39948e3253SAndrey Konovalovand riscv architectures, and tag-based KASAN modes are supported only for arm64. 402757aafaSJonathan Corbet 412757aafaSJonathan CorbetUsage 422757aafaSJonathan Corbet----- 432757aafaSJonathan Corbet 442757aafaSJonathan CorbetTo enable KASAN configure kernel with:: 452757aafaSJonathan Corbet 462757aafaSJonathan Corbet CONFIG_KASAN = y 472757aafaSJonathan Corbet 48948e3253SAndrey Konovalovand choose between CONFIG_KASAN_GENERIC (to enable generic KASAN), 49948e3253SAndrey KonovalovCONFIG_KASAN_SW_TAGS (to enable software tag-based KASAN), and 50948e3253SAndrey KonovalovCONFIG_KASAN_HW_TAGS (to enable hardware tag-based KASAN). 512757aafaSJonathan Corbet 52948e3253SAndrey KonovalovFor software modes, you also need to choose between CONFIG_KASAN_OUTLINE and 53948e3253SAndrey KonovalovCONFIG_KASAN_INLINE. Outline and inline are compiler instrumentation types. 54948e3253SAndrey KonovalovThe former produces smaller binary while the latter is 1.1 - 2 times faster. 55b3b0e6acSAndrey Konovalov 56625d8673SAndrey KonovalovFor better error reports that include stack traces, enable CONFIG_STACKTRACE. 572757aafaSJonathan Corbet 580fe9a448SVlastimil BabkaTo augment reports with last allocation and freeing stack of the physical page, 590fe9a448SVlastimil Babkait is recommended to enable also CONFIG_PAGE_OWNER and boot with page_owner=on. 600fe9a448SVlastimil Babka 612757aafaSJonathan CorbetError reports 622757aafaSJonathan Corbet~~~~~~~~~~~~~ 632757aafaSJonathan Corbet 64b3b0e6acSAndrey KonovalovA typical out-of-bounds access generic KASAN report looks like this:: 652757aafaSJonathan Corbet 662757aafaSJonathan Corbet ================================================================== 67b3b0e6acSAndrey Konovalov BUG: KASAN: slab-out-of-bounds in kmalloc_oob_right+0xa8/0xbc [test_kasan] 68b3b0e6acSAndrey Konovalov Write of size 1 at addr ffff8801f44ec37b by task insmod/2760 692757aafaSJonathan Corbet 70b3b0e6acSAndrey Konovalov CPU: 1 PID: 2760 Comm: insmod Not tainted 4.19.0-rc3+ #698 71b3b0e6acSAndrey Konovalov Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.10.2-1 04/01/2014 722757aafaSJonathan Corbet Call Trace: 73b3b0e6acSAndrey Konovalov dump_stack+0x94/0xd8 74b3b0e6acSAndrey Konovalov print_address_description+0x73/0x280 75b3b0e6acSAndrey Konovalov kasan_report+0x144/0x187 76b3b0e6acSAndrey Konovalov __asan_report_store1_noabort+0x17/0x20 77b3b0e6acSAndrey Konovalov kmalloc_oob_right+0xa8/0xbc [test_kasan] 78b3b0e6acSAndrey Konovalov kmalloc_tests_init+0x16/0x700 [test_kasan] 79b3b0e6acSAndrey Konovalov do_one_initcall+0xa5/0x3ae 80b3b0e6acSAndrey Konovalov do_init_module+0x1b6/0x547 81b3b0e6acSAndrey Konovalov load_module+0x75df/0x8070 82b3b0e6acSAndrey Konovalov __do_sys_init_module+0x1c6/0x200 83b3b0e6acSAndrey Konovalov __x64_sys_init_module+0x6e/0xb0 84b3b0e6acSAndrey Konovalov do_syscall_64+0x9f/0x2c0 85b3b0e6acSAndrey Konovalov entry_SYSCALL_64_after_hwframe+0x44/0xa9 86b3b0e6acSAndrey Konovalov RIP: 0033:0x7f96443109da 87b3b0e6acSAndrey Konovalov RSP: 002b:00007ffcf0b51b08 EFLAGS: 00000202 ORIG_RAX: 00000000000000af 88b3b0e6acSAndrey Konovalov RAX: ffffffffffffffda RBX: 000055dc3ee521a0 RCX: 00007f96443109da 89b3b0e6acSAndrey Konovalov RDX: 00007f96445cff88 RSI: 0000000000057a50 RDI: 00007f9644992000 90b3b0e6acSAndrey Konovalov RBP: 000055dc3ee510b0 R08: 0000000000000003 R09: 0000000000000000 91b3b0e6acSAndrey Konovalov R10: 00007f964430cd0a R11: 0000000000000202 R12: 00007f96445cff88 92b3b0e6acSAndrey Konovalov R13: 000055dc3ee51090 R14: 0000000000000000 R15: 0000000000000000 93b3b0e6acSAndrey Konovalov 94b3b0e6acSAndrey Konovalov Allocated by task 2760: 95b3b0e6acSAndrey Konovalov save_stack+0x43/0xd0 96b3b0e6acSAndrey Konovalov kasan_kmalloc+0xa7/0xd0 97b3b0e6acSAndrey Konovalov kmem_cache_alloc_trace+0xe1/0x1b0 98b3b0e6acSAndrey Konovalov kmalloc_oob_right+0x56/0xbc [test_kasan] 99b3b0e6acSAndrey Konovalov kmalloc_tests_init+0x16/0x700 [test_kasan] 100b3b0e6acSAndrey Konovalov do_one_initcall+0xa5/0x3ae 101b3b0e6acSAndrey Konovalov do_init_module+0x1b6/0x547 102b3b0e6acSAndrey Konovalov load_module+0x75df/0x8070 103b3b0e6acSAndrey Konovalov __do_sys_init_module+0x1c6/0x200 104b3b0e6acSAndrey Konovalov __x64_sys_init_module+0x6e/0xb0 105b3b0e6acSAndrey Konovalov do_syscall_64+0x9f/0x2c0 106b3b0e6acSAndrey Konovalov entry_SYSCALL_64_after_hwframe+0x44/0xa9 107b3b0e6acSAndrey Konovalov 108b3b0e6acSAndrey Konovalov Freed by task 815: 109b3b0e6acSAndrey Konovalov save_stack+0x43/0xd0 110b3b0e6acSAndrey Konovalov __kasan_slab_free+0x135/0x190 111b3b0e6acSAndrey Konovalov kasan_slab_free+0xe/0x10 112b3b0e6acSAndrey Konovalov kfree+0x93/0x1a0 113b3b0e6acSAndrey Konovalov umh_complete+0x6a/0xa0 114b3b0e6acSAndrey Konovalov call_usermodehelper_exec_async+0x4c3/0x640 115b3b0e6acSAndrey Konovalov ret_from_fork+0x35/0x40 116b3b0e6acSAndrey Konovalov 117b3b0e6acSAndrey Konovalov The buggy address belongs to the object at ffff8801f44ec300 118b3b0e6acSAndrey Konovalov which belongs to the cache kmalloc-128 of size 128 119b3b0e6acSAndrey Konovalov The buggy address is located 123 bytes inside of 120b3b0e6acSAndrey Konovalov 128-byte region [ffff8801f44ec300, ffff8801f44ec380) 121b3b0e6acSAndrey Konovalov The buggy address belongs to the page: 122b3b0e6acSAndrey Konovalov page:ffffea0007d13b00 count:1 mapcount:0 mapping:ffff8801f7001640 index:0x0 123b3b0e6acSAndrey Konovalov flags: 0x200000000000100(slab) 124b3b0e6acSAndrey Konovalov raw: 0200000000000100 ffffea0007d11dc0 0000001a0000001a ffff8801f7001640 125b3b0e6acSAndrey Konovalov raw: 0000000000000000 0000000080150015 00000001ffffffff 0000000000000000 126b3b0e6acSAndrey Konovalov page dumped because: kasan: bad access detected 127b3b0e6acSAndrey Konovalov 1282757aafaSJonathan Corbet Memory state around the buggy address: 129b3b0e6acSAndrey Konovalov ffff8801f44ec200: fc fc fc fc fc fc fc fc fb fb fb fb fb fb fb fb 130b3b0e6acSAndrey Konovalov ffff8801f44ec280: fb fb fb fb fb fb fb fb fc fc fc fc fc fc fc fc 131b3b0e6acSAndrey Konovalov >ffff8801f44ec300: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 03 1322757aafaSJonathan Corbet ^ 133b3b0e6acSAndrey Konovalov ffff8801f44ec380: fc fc fc fc fc fc fc fc fb fb fb fb fb fb fb fb 134b3b0e6acSAndrey Konovalov ffff8801f44ec400: fb fb fb fb fb fb fb fb fc fc fc fc fc fc fc fc 1352757aafaSJonathan Corbet ================================================================== 1362757aafaSJonathan Corbet 137b3b0e6acSAndrey KonovalovThe header of the report provides a short summary of what kind of bug happened 138b3b0e6acSAndrey Konovalovand what kind of access caused it. It's followed by a stack trace of the bad 139b3b0e6acSAndrey Konovalovaccess, a stack trace of where the accessed memory was allocated (in case bad 140b3b0e6acSAndrey Konovalovaccess happens on a slab object), and a stack trace of where the object was 141b3b0e6acSAndrey Konovalovfreed (in case of a use-after-free bug report). Next comes a description of 142b3b0e6acSAndrey Konovalovthe accessed slab object and information about the accessed memory page. 1432757aafaSJonathan Corbet 1442757aafaSJonathan CorbetIn the last section the report shows memory state around the accessed address. 145625d8673SAndrey KonovalovInternally KASAN tracks memory state separately for each memory granule, which 146625d8673SAndrey Konovalovis either 8 or 16 aligned bytes depending on KASAN mode. Each number in the 147625d8673SAndrey Konovalovmemory state section of the report shows the state of one of the memory 148625d8673SAndrey Konovalovgranules that surround the accessed address. 1492757aafaSJonathan Corbet 150625d8673SAndrey KonovalovFor generic KASAN the size of each memory granule is 8. The state of each 151625d8673SAndrey Konovalovgranule is encoded in one shadow byte. Those 8 bytes can be accessible, 152625d8673SAndrey Konovalovpartially accessible, freed or be a part of a redzone. KASAN uses the following 153625d8673SAndrey Konovalovencoding for each shadow byte: 0 means that all 8 bytes of the corresponding 154625d8673SAndrey Konovalovmemory region are accessible; number N (1 <= N <= 7) means that the first N 155625d8673SAndrey Konovalovbytes are accessible, and other (8 - N) bytes are not; any negative value 156625d8673SAndrey Konovalovindicates that the entire 8-byte word is inaccessible. KASAN uses different 157625d8673SAndrey Konovalovnegative values to distinguish between different kinds of inaccessible memory 158625d8673SAndrey Konovalovlike redzones or freed memory (see mm/kasan/kasan.h). 1592757aafaSJonathan Corbet 1602757aafaSJonathan CorbetIn the report above the arrows point to the shadow byte 03, which means that 1614062c245SAndrey Konovalovthe accessed address is partially accessible. For tag-based KASAN modes this 1624062c245SAndrey Konovalovlast report section shows the memory tags around the accessed address 1634062c245SAndrey Konovalov(see the `Implementation details`_ section). 164625d8673SAndrey Konovalov 165625d8673SAndrey KonovalovBoot parameters 166625d8673SAndrey Konovalov~~~~~~~~~~~~~~~ 167625d8673SAndrey Konovalov 1684062c245SAndrey KonovalovHardware tag-based KASAN mode (see the section about various modes below) is 1697169487bSAndrey Konovalovintended for use in production as a security mitigation. Therefore, it supports 170625d8673SAndrey Konovalovboot parameters that allow to disable KASAN competely or otherwise control 171625d8673SAndrey Konovalovparticular KASAN features. 172625d8673SAndrey Konovalov 17376bc99e8SAndrey Konovalov- ``kasan=off`` or ``=on`` controls whether KASAN is enabled (default: ``on``). 174625d8673SAndrey Konovalov 1752603f8a7SVincenzo Frascino- ``kasan.mode=sync`` or ``=async`` controls whether KASAN is configured in 1762603f8a7SVincenzo Frascino synchronous or asynchronous mode of execution (default: ``sync``). 1772603f8a7SVincenzo Frascino Synchronous mode: a bad access is detected immediately when a tag 1782603f8a7SVincenzo Frascino check fault occurs. 1792603f8a7SVincenzo Frascino Asynchronous mode: a bad access detection is delayed. When a tag check 1802603f8a7SVincenzo Frascino fault occurs, the information is stored in hardware (in the TFSR_EL1 1812603f8a7SVincenzo Frascino register for arm64). The kernel periodically checks the hardware and 1822603f8a7SVincenzo Frascino only reports tag faults during these checks. 1832603f8a7SVincenzo Frascino 18476bc99e8SAndrey Konovalov- ``kasan.stacktrace=off`` or ``=on`` disables or enables alloc and free stack 1851cc4cdb5SAndrey Konovalov traces collection (default: ``on``). 186625d8673SAndrey Konovalov 18776bc99e8SAndrey Konovalov- ``kasan.fault=report`` or ``=panic`` controls whether to only print a KASAN 1887169487bSAndrey Konovalov report or also panic the kernel (default: ``report``). Note, that tag 1897169487bSAndrey Konovalov checking gets disabled after the first reported bug. 190625d8673SAndrey Konovalov 1912757aafaSJonathan CorbetImplementation details 1922757aafaSJonathan Corbet---------------------- 1932757aafaSJonathan Corbet 194b3b0e6acSAndrey KonovalovGeneric KASAN 195b3b0e6acSAndrey Konovalov~~~~~~~~~~~~~ 196b3b0e6acSAndrey Konovalov 197625d8673SAndrey KonovalovFrom a high level perspective, KASAN's approach to memory error detection is 198625d8673SAndrey Konovalovsimilar to that of kmemcheck: use shadow memory to record whether each byte of 199625d8673SAndrey Konovalovmemory is safe to access, and use compile-time instrumentation to insert checks 200625d8673SAndrey Konovalovof shadow memory on each memory access. 2012757aafaSJonathan Corbet 202b3b0e6acSAndrey KonovalovGeneric KASAN dedicates 1/8th of kernel memory to its shadow memory (e.g. 16TB 203b3b0e6acSAndrey Konovalovto cover 128TB on x86_64) and uses direct mapping with a scale and offset to 204b3b0e6acSAndrey Konovalovtranslate a memory address to its corresponding shadow address. 2052757aafaSJonathan Corbet 2062757aafaSJonathan CorbetHere is the function which translates an address to its corresponding shadow 2072757aafaSJonathan Corbetaddress:: 2082757aafaSJonathan Corbet 2092757aafaSJonathan Corbet static inline void *kasan_mem_to_shadow(const void *addr) 2102757aafaSJonathan Corbet { 2112757aafaSJonathan Corbet return ((unsigned long)addr >> KASAN_SHADOW_SCALE_SHIFT) 2122757aafaSJonathan Corbet + KASAN_SHADOW_OFFSET; 2132757aafaSJonathan Corbet } 2142757aafaSJonathan Corbet 2152757aafaSJonathan Corbetwhere ``KASAN_SHADOW_SCALE_SHIFT = 3``. 2162757aafaSJonathan Corbet 217b3b0e6acSAndrey KonovalovCompile-time instrumentation is used to insert memory access checks. Compiler 218b3b0e6acSAndrey Konovalovinserts function calls (__asan_load*(addr), __asan_store*(addr)) before each 219b3b0e6acSAndrey Konovalovmemory access of size 1, 2, 4, 8 or 16. These functions check whether memory 220b3b0e6acSAndrey Konovalovaccess is valid or not by checking corresponding shadow memory. 2212757aafaSJonathan Corbet 2222757aafaSJonathan CorbetGCC 5.0 has possibility to perform inline instrumentation. Instead of making 2232757aafaSJonathan Corbetfunction calls GCC directly inserts the code to check the shadow memory. 2242757aafaSJonathan CorbetThis option significantly enlarges kernel but it gives x1.1-x2 performance 2252757aafaSJonathan Corbetboost over outline instrumented kernel. 226b3b0e6acSAndrey Konovalov 2274784be28SWalter WuGeneric KASAN also reports the last 2 call stacks to creation of work that 2284784be28SWalter Wupotentially has access to an object. Call stacks for the following are shown: 2294784be28SWalter Wucall_rcu() and workqueue queuing. 2309793b626SWalter Wu 231625d8673SAndrey KonovalovGeneric KASAN is the only mode that delays the reuse of freed object via 232625d8673SAndrey Konovalovquarantine (see mm/kasan/quarantine.c for implementation). 233625d8673SAndrey Konovalov 234b3b0e6acSAndrey KonovalovSoftware tag-based KASAN 235b3b0e6acSAndrey Konovalov~~~~~~~~~~~~~~~~~~~~~~~~ 236b3b0e6acSAndrey Konovalov 237948e3253SAndrey KonovalovSoftware tag-based KASAN requires software memory tagging support in the form 238948e3253SAndrey Konovalovof HWASan-like compiler instrumentation (see HWASan documentation for details). 239948e3253SAndrey Konovalov 240948e3253SAndrey KonovalovSoftware tag-based KASAN is currently only implemented for arm64 architecture. 241948e3253SAndrey Konovalov 242948e3253SAndrey KonovalovSoftware tag-based KASAN uses the Top Byte Ignore (TBI) feature of arm64 CPUs 243948e3253SAndrey Konovalovto store a pointer tag in the top byte of kernel pointers. Like generic KASAN 244948e3253SAndrey Konovalovit uses shadow memory to store memory tags associated with each 16-byte memory 245b3b0e6acSAndrey Konovalovcell (therefore it dedicates 1/16th of the kernel memory for shadow memory). 246b3b0e6acSAndrey Konovalov 247948e3253SAndrey KonovalovOn each memory allocation software tag-based KASAN generates a random tag, tags 248948e3253SAndrey Konovalovthe allocated memory with this tag, and embeds this tag into the returned 249948e3253SAndrey Konovalovpointer. 250948e3253SAndrey Konovalov 251b3b0e6acSAndrey KonovalovSoftware tag-based KASAN uses compile-time instrumentation to insert checks 252b3b0e6acSAndrey Konovalovbefore each memory access. These checks make sure that tag of the memory that 253b3b0e6acSAndrey Konovalovis being accessed is equal to tag of the pointer that is used to access this 254948e3253SAndrey Konovalovmemory. In case of a tag mismatch software tag-based KASAN prints a bug report. 255b3b0e6acSAndrey Konovalov 256b3b0e6acSAndrey KonovalovSoftware tag-based KASAN also has two instrumentation modes (outline, that 257b3b0e6acSAndrey Konovalovemits callbacks to check memory accesses; and inline, that performs the shadow 258b3b0e6acSAndrey Konovalovmemory checks inline). With outline instrumentation mode, a bug report is 259b3b0e6acSAndrey Konovalovsimply printed from the function that performs the access check. With inline 260b3b0e6acSAndrey Konovalovinstrumentation a brk instruction is emitted by the compiler, and a dedicated 261b3b0e6acSAndrey Konovalovbrk handler is used to print bug reports. 262b3b0e6acSAndrey Konovalov 263948e3253SAndrey KonovalovSoftware tag-based KASAN uses 0xFF as a match-all pointer tag (accesses through 264948e3253SAndrey Konovalovpointers with 0xFF pointer tag aren't checked). The value 0xFE is currently 265948e3253SAndrey Konovalovreserved to tag freed memory regions. 266948e3253SAndrey Konovalov 267948e3253SAndrey KonovalovSoftware tag-based KASAN currently only supports tagging of 268948e3253SAndrey Konovalovkmem_cache_alloc/kmalloc and page_alloc memory. 269948e3253SAndrey Konovalov 270948e3253SAndrey KonovalovHardware tag-based KASAN 271948e3253SAndrey Konovalov~~~~~~~~~~~~~~~~~~~~~~~~ 272948e3253SAndrey Konovalov 273948e3253SAndrey KonovalovHardware tag-based KASAN is similar to the software mode in concept, but uses 274948e3253SAndrey Konovalovhardware memory tagging support instead of compiler instrumentation and 275948e3253SAndrey Konovalovshadow memory. 276948e3253SAndrey Konovalov 277948e3253SAndrey KonovalovHardware tag-based KASAN is currently only implemented for arm64 architecture 278948e3253SAndrey Konovalovand based on both arm64 Memory Tagging Extension (MTE) introduced in ARMv8.5 279948e3253SAndrey KonovalovInstruction Set Architecture, and Top Byte Ignore (TBI). 280948e3253SAndrey Konovalov 281948e3253SAndrey KonovalovSpecial arm64 instructions are used to assign memory tags for each allocation. 282948e3253SAndrey KonovalovSame tags are assigned to pointers to those allocations. On every memory 283948e3253SAndrey Konovalovaccess, hardware makes sure that tag of the memory that is being accessed is 284948e3253SAndrey Konovalovequal to tag of the pointer that is used to access this memory. In case of a 285948e3253SAndrey Konovalovtag mismatch a fault is generated and a report is printed. 286948e3253SAndrey Konovalov 287948e3253SAndrey KonovalovHardware tag-based KASAN uses 0xFF as a match-all pointer tag (accesses through 288948e3253SAndrey Konovalovpointers with 0xFF pointer tag aren't checked). The value 0xFE is currently 289948e3253SAndrey Konovalovreserved to tag freed memory regions. 290948e3253SAndrey Konovalov 291948e3253SAndrey KonovalovHardware tag-based KASAN currently only supports tagging of 292948e3253SAndrey Konovalovkmem_cache_alloc/kmalloc and page_alloc memory. 2933c5c3cfbSDaniel Axtens 2944062c245SAndrey KonovalovIf the hardware doesn't support MTE (pre ARMv8.5), hardware tag-based KASAN 2954062c245SAndrey Konovalovwon't be enabled. In this case all boot parameters are ignored. 2964062c245SAndrey Konovalov 2974062c245SAndrey KonovalovNote, that enabling CONFIG_KASAN_HW_TAGS always results in in-kernel TBI being 2984062c245SAndrey Konovalovenabled. Even when kasan.mode=off is provided, or when the hardware doesn't 2994062c245SAndrey Konovalovsupport MTE (but supports TBI). 3004062c245SAndrey Konovalov 3017169487bSAndrey KonovalovHardware tag-based KASAN only reports the first found bug. After that MTE tag 3027169487bSAndrey Konovalovchecking gets disabled. 3037169487bSAndrey Konovalov 30496d7d141SAndrey KonovalovShadow memory 30596d7d141SAndrey Konovalov------------- 3063c5c3cfbSDaniel Axtens 3073c5c3cfbSDaniel AxtensThe kernel maps memory in a number of different parts of the address 3083c5c3cfbSDaniel Axtensspace. This poses something of a problem for KASAN, which requires 3093c5c3cfbSDaniel Axtensthat all addresses accessed by instrumented code have a valid shadow 3103c5c3cfbSDaniel Axtensregion. 3113c5c3cfbSDaniel Axtens 3123c5c3cfbSDaniel AxtensThe range of kernel virtual addresses is large: there is not enough 3133c5c3cfbSDaniel Axtensreal memory to support a real shadow region for every address that 3143c5c3cfbSDaniel Axtenscould be accessed by the kernel. 3153c5c3cfbSDaniel Axtens 31696d7d141SAndrey KonovalovDefault behaviour 31796d7d141SAndrey Konovalov~~~~~~~~~~~~~~~~~ 3183c5c3cfbSDaniel Axtens 3193c5c3cfbSDaniel AxtensBy default, architectures only map real memory over the shadow region 3203c5c3cfbSDaniel Axtensfor the linear mapping (and potentially other small areas). For all 3213c5c3cfbSDaniel Axtensother areas - such as vmalloc and vmemmap space - a single read-only 3223c5c3cfbSDaniel Axtenspage is mapped over the shadow area. This read-only shadow page 3233c5c3cfbSDaniel Axtensdeclares all memory accesses as permitted. 3243c5c3cfbSDaniel Axtens 3253c5c3cfbSDaniel AxtensThis presents a problem for modules: they do not live in the linear 3263c5c3cfbSDaniel Axtensmapping, but in a dedicated module space. By hooking in to the module 3273c5c3cfbSDaniel Axtensallocator, KASAN can temporarily map real shadow memory to cover 3283c5c3cfbSDaniel Axtensthem. This allows detection of invalid accesses to module globals, for 3293c5c3cfbSDaniel Axtensexample. 3303c5c3cfbSDaniel Axtens 3313c5c3cfbSDaniel AxtensThis also creates an incompatibility with ``VMAP_STACK``: if the stack 3323c5c3cfbSDaniel Axtenslives in vmalloc space, it will be shadowed by the read-only page, and 3333c5c3cfbSDaniel Axtensthe kernel will fault when trying to set up the shadow data for stack 3343c5c3cfbSDaniel Axtensvariables. 3353c5c3cfbSDaniel Axtens 3363c5c3cfbSDaniel AxtensCONFIG_KASAN_VMALLOC 3373c5c3cfbSDaniel Axtens~~~~~~~~~~~~~~~~~~~~ 3383c5c3cfbSDaniel Axtens 3393c5c3cfbSDaniel AxtensWith ``CONFIG_KASAN_VMALLOC``, KASAN can cover vmalloc space at the 3403c5c3cfbSDaniel Axtenscost of greater memory usage. Currently this is only supported on x86. 3413c5c3cfbSDaniel Axtens 3423c5c3cfbSDaniel AxtensThis works by hooking into vmalloc and vmap, and dynamically 3433c5c3cfbSDaniel Axtensallocating real shadow memory to back the mappings. 3443c5c3cfbSDaniel Axtens 3453c5c3cfbSDaniel AxtensMost mappings in vmalloc space are small, requiring less than a full 3463c5c3cfbSDaniel Axtenspage of shadow space. Allocating a full shadow page per mapping would 3473c5c3cfbSDaniel Axtenstherefore be wasteful. Furthermore, to ensure that different mappings 3483c5c3cfbSDaniel Axtensuse different shadow pages, mappings would have to be aligned to 3491f600626SAndrey Konovalov``KASAN_GRANULE_SIZE * PAGE_SIZE``. 3503c5c3cfbSDaniel Axtens 351625d8673SAndrey KonovalovInstead, KASAN shares backing space across multiple mappings. It allocates 3523c5c3cfbSDaniel Axtensa backing page when a mapping in vmalloc space uses a particular page 3533c5c3cfbSDaniel Axtensof the shadow region. This page can be shared by other vmalloc 3543c5c3cfbSDaniel Axtensmappings later on. 3553c5c3cfbSDaniel Axtens 356625d8673SAndrey KonovalovKASAN hooks into the vmap infrastructure to lazily clean up unused shadow 3573c5c3cfbSDaniel Axtensmemory. 3583c5c3cfbSDaniel Axtens 359625d8673SAndrey KonovalovTo avoid the difficulties around swapping mappings around, KASAN expects 3603c5c3cfbSDaniel Axtensthat the part of the shadow region that covers the vmalloc space will 3613c5c3cfbSDaniel Axtensnot be covered by the early shadow page, but will be left 3623c5c3cfbSDaniel Axtensunmapped. This will require changes in arch-specific code. 3633c5c3cfbSDaniel Axtens 3643c5c3cfbSDaniel AxtensThis allows ``VMAP_STACK`` support on x86, and can simplify support of 3653c5c3cfbSDaniel Axtensarchitectures that do not have a fixed module region. 3669ab5be97SPatricia Alfonso 36796d7d141SAndrey KonovalovFor developers 36896d7d141SAndrey Konovalov-------------- 36996d7d141SAndrey Konovalov 37096d7d141SAndrey KonovalovIgnoring accesses 37196d7d141SAndrey Konovalov~~~~~~~~~~~~~~~~~ 37296d7d141SAndrey Konovalov 37396d7d141SAndrey KonovalovSoftware KASAN modes use compiler instrumentation to insert validity checks. 37496d7d141SAndrey KonovalovSuch instrumentation might be incompatible with some part of the kernel, and 37596d7d141SAndrey Konovalovtherefore needs to be disabled. To disable instrumentation for specific files 37696d7d141SAndrey Konovalovor directories, add a line similar to the following to the respective kernel 37796d7d141SAndrey KonovalovMakefile: 37896d7d141SAndrey Konovalov 37996d7d141SAndrey Konovalov- For a single file (e.g. main.o):: 38096d7d141SAndrey Konovalov 38196d7d141SAndrey Konovalov KASAN_SANITIZE_main.o := n 38296d7d141SAndrey Konovalov 38396d7d141SAndrey Konovalov- For all files in one directory:: 38496d7d141SAndrey Konovalov 38596d7d141SAndrey Konovalov KASAN_SANITIZE := n 38696d7d141SAndrey Konovalov 38796d7d141SAndrey Konovalov 38896d7d141SAndrey KonovalovTests 38996d7d141SAndrey Konovalov~~~~~ 3909ab5be97SPatricia Alfonso 3915d92bdffSAndrey KonovalovKASAN tests consist of two parts: 3929ab5be97SPatricia Alfonso 393625d8673SAndrey Konovalov1. Tests that are integrated with the KUnit Test Framework. Enabled with 394625d8673SAndrey Konovalov``CONFIG_KASAN_KUNIT_TEST``. These tests can be run and partially verified 395625d8673SAndrey Konovalovautomatically in a few different ways, see the instructions below. 3969ab5be97SPatricia Alfonso 397625d8673SAndrey Konovalov2. Tests that are currently incompatible with KUnit. Enabled with 3985d92bdffSAndrey Konovalov``CONFIG_KASAN_MODULE_TEST`` and can only be run as a module. These tests can 399625d8673SAndrey Konovalovonly be verified manually, by loading the kernel module and inspecting the 400625d8673SAndrey Konovalovkernel log for KASAN reports. 401625d8673SAndrey Konovalov 402625d8673SAndrey KonovalovEach KUnit-compatible KASAN test prints a KASAN report if an error is detected. 403625d8673SAndrey KonovalovThen the test prints its number and status. 404625d8673SAndrey Konovalov 405625d8673SAndrey KonovalovWhen a test passes:: 4069ab5be97SPatricia Alfonso 4079ab5be97SPatricia Alfonso ok 28 - kmalloc_double_kzfree 40832519c03SMauro Carvalho Chehab 409625d8673SAndrey KonovalovWhen a test fails due to a failed ``kmalloc``:: 4109ab5be97SPatricia Alfonso 4119ab5be97SPatricia Alfonso # kmalloc_large_oob_right: ASSERTION FAILED at lib/test_kasan.c:163 4129ab5be97SPatricia Alfonso Expected ptr is not null, but is 4139ab5be97SPatricia Alfonso not ok 4 - kmalloc_large_oob_right 41432519c03SMauro Carvalho Chehab 415625d8673SAndrey KonovalovWhen a test fails due to a missing KASAN report:: 4169ab5be97SPatricia Alfonso 4179ab5be97SPatricia Alfonso # kmalloc_double_kzfree: EXPECTATION FAILED at lib/test_kasan.c:629 4189ab5be97SPatricia Alfonso Expected kasan_data->report_expected == kasan_data->report_found, but 4199ab5be97SPatricia Alfonso kasan_data->report_expected == 1 4209ab5be97SPatricia Alfonso kasan_data->report_found == 0 4219ab5be97SPatricia Alfonso not ok 28 - kmalloc_double_kzfree 4229ab5be97SPatricia Alfonso 423625d8673SAndrey KonovalovAt the end the cumulative status of all KASAN tests is printed. On success:: 4249ab5be97SPatricia Alfonso 4259ab5be97SPatricia Alfonso ok 1 - kasan 4269ab5be97SPatricia Alfonso 427625d8673SAndrey KonovalovOr, if one of the tests failed:: 4289ab5be97SPatricia Alfonso 4299ab5be97SPatricia Alfonso not ok 1 - kasan 4309ab5be97SPatricia Alfonso 431625d8673SAndrey Konovalov 432625d8673SAndrey KonovalovThere are a few ways to run KUnit-compatible KASAN tests. 433625d8673SAndrey Konovalov 434625d8673SAndrey Konovalov1. Loadable module 4359ab5be97SPatricia Alfonso 4369ab5be97SPatricia AlfonsoWith ``CONFIG_KUNIT`` enabled, ``CONFIG_KASAN_KUNIT_TEST`` can be built as 437625d8673SAndrey Konovalova loadable module and run on any architecture that supports KASAN by loading 438625d8673SAndrey Konovalovthe module with insmod or modprobe. The module is called ``test_kasan``. 4399ab5be97SPatricia Alfonso 440625d8673SAndrey Konovalov2. Built-In 4419ab5be97SPatricia Alfonso 4429ab5be97SPatricia AlfonsoWith ``CONFIG_KUNIT`` built-in, ``CONFIG_KASAN_KUNIT_TEST`` can be built-in 443625d8673SAndrey Konovalovon any architecure that supports KASAN. These and any other KUnit tests enabled 444625d8673SAndrey Konovalovwill run and print the results at boot as a late-init call. 4459ab5be97SPatricia Alfonso 446625d8673SAndrey Konovalov3. Using kunit_tool 4479ab5be97SPatricia Alfonso 448625d8673SAndrey KonovalovWith ``CONFIG_KUNIT`` and ``CONFIG_KASAN_KUNIT_TEST`` built-in, it's also 449625d8673SAndrey Konovalovpossible use ``kunit_tool`` to see the results of these and other KUnit tests 450625d8673SAndrey Konovalovin a more readable way. This will not print the KASAN reports of the tests that 451625d8673SAndrey Konovalovpassed. Use `KUnit documentation <https://www.kernel.org/doc/html/latest/dev-tools/kunit/index.html>`_ 452625d8673SAndrey Konovalovfor more up-to-date information on ``kunit_tool``. 4539ab5be97SPatricia Alfonso 4549ab5be97SPatricia Alfonso.. _KUnit: https://www.kernel.org/doc/html/latest/dev-tools/kunit/index.html 455