1Kernel Memory Leak Detector 2=========================== 3 4Kmemleak provides a way of detecting possible kernel memory leaks in a 5way similar to a `tracing garbage collector 6<https://en.wikipedia.org/wiki/Tracing_garbage_collection>`_, 7with the difference that the orphan objects are not freed but only 8reported via /sys/kernel/debug/kmemleak. A similar method is used by the 9Valgrind tool (``memcheck --leak-check``) to detect the memory leaks in 10user-space applications. 11Kmemleak is supported on x86, arm, arm64, powerpc, sparc, sh, microblaze, mips, 12s390, nds32, arc and xtensa. 13 14Usage 15----- 16 17CONFIG_DEBUG_KMEMLEAK in "Kernel hacking" has to be enabled. A kernel 18thread scans the memory every 10 minutes (by default) and prints the 19number of new unreferenced objects found. If the ``debugfs`` isn't already 20mounted, mount with:: 21 22 # mount -t debugfs nodev /sys/kernel/debug/ 23 24To display the details of all the possible scanned memory leaks:: 25 26 # cat /sys/kernel/debug/kmemleak 27 28To trigger an intermediate memory scan:: 29 30 # echo scan > /sys/kernel/debug/kmemleak 31 32To clear the list of all current possible memory leaks:: 33 34 # echo clear > /sys/kernel/debug/kmemleak 35 36New leaks will then come up upon reading ``/sys/kernel/debug/kmemleak`` 37again. 38 39Note that the orphan objects are listed in the order they were allocated 40and one object at the beginning of the list may cause other subsequent 41objects to be reported as orphan. 42 43Memory scanning parameters can be modified at run-time by writing to the 44``/sys/kernel/debug/kmemleak`` file. The following parameters are supported: 45 46- off 47 disable kmemleak (irreversible) 48- stack=on 49 enable the task stacks scanning (default) 50- stack=off 51 disable the tasks stacks scanning 52- scan=on 53 start the automatic memory scanning thread (default) 54- scan=off 55 stop the automatic memory scanning thread 56- scan=<secs> 57 set the automatic memory scanning period in seconds 58 (default 600, 0 to stop the automatic scanning) 59- scan 60 trigger a memory scan 61- clear 62 clear list of current memory leak suspects, done by 63 marking all current reported unreferenced objects grey, 64 or free all kmemleak objects if kmemleak has been disabled. 65- dump=<addr> 66 dump information about the object found at <addr> 67 68Kmemleak can also be disabled at boot-time by passing ``kmemleak=off`` on 69the kernel command line. 70 71Memory may be allocated or freed before kmemleak is initialised and 72these actions are stored in an early log buffer. The size of this buffer 73is configured via the CONFIG_DEBUG_KMEMLEAK_MEM_POOL_SIZE option. 74 75If CONFIG_DEBUG_KMEMLEAK_DEFAULT_OFF are enabled, the kmemleak is 76disabled by default. Passing ``kmemleak=on`` on the kernel command 77line enables the function. 78 79If you are getting errors like "Error while writing to stdout" or "write_loop: 80Invalid argument", make sure kmemleak is properly enabled. 81 82Basic Algorithm 83--------------- 84 85The memory allocations via :c:func:`kmalloc`, :c:func:`vmalloc`, 86:c:func:`kmem_cache_alloc` and 87friends are traced and the pointers, together with additional 88information like size and stack trace, are stored in a rbtree. 89The corresponding freeing function calls are tracked and the pointers 90removed from the kmemleak data structures. 91 92An allocated block of memory is considered orphan if no pointer to its 93start address or to any location inside the block can be found by 94scanning the memory (including saved registers). This means that there 95might be no way for the kernel to pass the address of the allocated 96block to a freeing function and therefore the block is considered a 97memory leak. 98 99The scanning algorithm steps: 100 101 1. mark all objects as white (remaining white objects will later be 102 considered orphan) 103 2. scan the memory starting with the data section and stacks, checking 104 the values against the addresses stored in the rbtree. If 105 a pointer to a white object is found, the object is added to the 106 gray list 107 3. scan the gray objects for matching addresses (some white objects 108 can become gray and added at the end of the gray list) until the 109 gray set is finished 110 4. the remaining white objects are considered orphan and reported via 111 /sys/kernel/debug/kmemleak 112 113Some allocated memory blocks have pointers stored in the kernel's 114internal data structures and they cannot be detected as orphans. To 115avoid this, kmemleak can also store the number of values pointing to an 116address inside the block address range that need to be found so that the 117block is not considered a leak. One example is __vmalloc(). 118 119Testing specific sections with kmemleak 120--------------------------------------- 121 122Upon initial bootup your /sys/kernel/debug/kmemleak output page may be 123quite extensive. This can also be the case if you have very buggy code 124when doing development. To work around these situations you can use the 125'clear' command to clear all reported unreferenced objects from the 126/sys/kernel/debug/kmemleak output. By issuing a 'scan' after a 'clear' 127you can find new unreferenced objects; this should help with testing 128specific sections of code. 129 130To test a critical section on demand with a clean kmemleak do:: 131 132 # echo clear > /sys/kernel/debug/kmemleak 133 ... test your kernel or modules ... 134 # echo scan > /sys/kernel/debug/kmemleak 135 136Then as usual to get your report with:: 137 138 # cat /sys/kernel/debug/kmemleak 139 140Freeing kmemleak internal objects 141--------------------------------- 142 143To allow access to previously found memory leaks after kmemleak has been 144disabled by the user or due to an fatal error, internal kmemleak objects 145won't be freed when kmemleak is disabled, and those objects may occupy 146a large part of physical memory. 147 148In this situation, you may reclaim memory with:: 149 150 # echo clear > /sys/kernel/debug/kmemleak 151 152Kmemleak API 153------------ 154 155See the include/linux/kmemleak.h header for the functions prototype. 156 157- ``kmemleak_init`` - initialize kmemleak 158- ``kmemleak_alloc`` - notify of a memory block allocation 159- ``kmemleak_alloc_percpu`` - notify of a percpu memory block allocation 160- ``kmemleak_vmalloc`` - notify of a vmalloc() memory allocation 161- ``kmemleak_free`` - notify of a memory block freeing 162- ``kmemleak_free_part`` - notify of a partial memory block freeing 163- ``kmemleak_free_percpu`` - notify of a percpu memory block freeing 164- ``kmemleak_update_trace`` - update object allocation stack trace 165- ``kmemleak_not_leak`` - mark an object as not a leak 166- ``kmemleak_ignore`` - do not scan or report an object as leak 167- ``kmemleak_scan_area`` - add scan areas inside a memory block 168- ``kmemleak_no_scan`` - do not scan a memory block 169- ``kmemleak_erase`` - erase an old value in a pointer variable 170- ``kmemleak_alloc_recursive`` - as kmemleak_alloc but checks the recursiveness 171- ``kmemleak_free_recursive`` - as kmemleak_free but checks the recursiveness 172 173The following functions take a physical address as the object pointer 174and only perform the corresponding action if the address has a lowmem 175mapping: 176 177- ``kmemleak_alloc_phys`` 178- ``kmemleak_free_part_phys`` 179- ``kmemleak_not_leak_phys`` 180- ``kmemleak_ignore_phys`` 181 182Dealing with false positives/negatives 183-------------------------------------- 184 185The false negatives are real memory leaks (orphan objects) but not 186reported by kmemleak because values found during the memory scanning 187point to such objects. To reduce the number of false negatives, kmemleak 188provides the kmemleak_ignore, kmemleak_scan_area, kmemleak_no_scan and 189kmemleak_erase functions (see above). The task stacks also increase the 190amount of false negatives and their scanning is not enabled by default. 191 192The false positives are objects wrongly reported as being memory leaks 193(orphan). For objects known not to be leaks, kmemleak provides the 194kmemleak_not_leak function. The kmemleak_ignore could also be used if 195the memory block is known not to contain other pointers and it will no 196longer be scanned. 197 198Some of the reported leaks are only transient, especially on SMP 199systems, because of pointers temporarily stored in CPU registers or 200stacks. Kmemleak defines MSECS_MIN_AGE (defaulting to 1000) representing 201the minimum age of an object to be reported as a memory leak. 202 203Limitations and Drawbacks 204------------------------- 205 206The main drawback is the reduced performance of memory allocation and 207freeing. To avoid other penalties, the memory scanning is only performed 208when the /sys/kernel/debug/kmemleak file is read. Anyway, this tool is 209intended for debugging purposes where the performance might not be the 210most important requirement. 211 212To keep the algorithm simple, kmemleak scans for values pointing to any 213address inside a block's address range. This may lead to an increased 214number of false negatives. However, it is likely that a real memory leak 215will eventually become visible. 216 217Another source of false negatives is the data stored in non-pointer 218values. In a future version, kmemleak could only scan the pointer 219members in the allocated structures. This feature would solve many of 220the false negative cases described above. 221 222The tool can report false positives. These are cases where an allocated 223block doesn't need to be freed (some cases in the init_call functions), 224the pointer is calculated by other methods than the usual container_of 225macro or the pointer is stored in a location not scanned by kmemleak. 226 227Page allocations and ioremap are not tracked. 228 229Testing with kmemleak-test 230-------------------------- 231 232To check if you have all set up to use kmemleak, you can use the kmemleak-test 233module, a module that deliberately leaks memory. Set CONFIG_DEBUG_KMEMLEAK_TEST 234as module (it can't be used as bult-in) and boot the kernel with kmemleak 235enabled. Load the module and perform a scan with:: 236 237 # modprobe kmemleak-test 238 # echo scan > /sys/kernel/debug/kmemleak 239 240Note that the you may not get results instantly or on the first scanning. When 241kmemleak gets results, it'll log ``kmemleak: <count of leaks> new suspected 242memory leaks``. Then read the file to see then:: 243 244 # cat /sys/kernel/debug/kmemleak 245 unreferenced object 0xffff89862ca702e8 (size 32): 246 comm "modprobe", pid 2088, jiffies 4294680594 (age 375.486s) 247 hex dump (first 32 bytes): 248 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk 249 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b a5 kkkkkkkkkkkkkkk. 250 backtrace: 251 [<00000000e0a73ec7>] 0xffffffffc01d2036 252 [<000000000c5d2a46>] do_one_initcall+0x41/0x1df 253 [<0000000046db7e0a>] do_init_module+0x55/0x200 254 [<00000000542b9814>] load_module+0x203c/0x2480 255 [<00000000c2850256>] __do_sys_finit_module+0xba/0xe0 256 [<000000006564e7ef>] do_syscall_64+0x43/0x110 257 [<000000007c873fa6>] entry_SYSCALL_64_after_hwframe+0x44/0xa9 258 ... 259 260Removing the module with ``rmmod kmemleak_test`` should also trigger some 261kmemleak results. 262