1a8e35fecSJosh PoimboeufObjtool 2a8e35fecSJosh Poimboeuf======= 3a8e35fecSJosh Poimboeuf 4a8e35fecSJosh PoimboeufThe kernel CONFIG_OBJTOOL option enables a host tool named 'objtool' 5a8e35fecSJosh Poimboeufwhich runs at compile time. It can do various validations and 6a8e35fecSJosh Poimboeuftransformations on .o files. 7a8e35fecSJosh Poimboeuf 8a8e35fecSJosh PoimboeufObjtool has become an integral part of the x86-64 kernel toolchain. The 9a8e35fecSJosh Poimboeufkernel depends on it for a variety of security and performance features 10a8e35fecSJosh Poimboeuf(and other types of features as well). 11a8e35fecSJosh Poimboeuf 12a8e35fecSJosh Poimboeuf 13a8e35fecSJosh PoimboeufFeatures 14a8e35fecSJosh Poimboeuf-------- 15a8e35fecSJosh Poimboeuf 16a8e35fecSJosh PoimboeufObjtool has the following features: 17a8e35fecSJosh Poimboeuf 18a8e35fecSJosh Poimboeuf- Stack unwinding metadata validation -- useful for helping to ensure 19a8e35fecSJosh Poimboeuf stack traces are reliable for live patching 20a8e35fecSJosh Poimboeuf 21a8e35fecSJosh Poimboeuf- ORC unwinder metadata generation -- a faster and more precise 22a8e35fecSJosh Poimboeuf alternative to frame pointer based unwinding 23a8e35fecSJosh Poimboeuf 24a8e35fecSJosh Poimboeuf- Retpoline validation -- ensures that all indirect calls go through 25a8e35fecSJosh Poimboeuf retpoline thunks, for Spectre v2 mitigations 26a8e35fecSJosh Poimboeuf 27a8e35fecSJosh Poimboeuf- Retpoline call site annotation -- annotates all retpoline thunk call 28a8e35fecSJosh Poimboeuf sites, enabling the kernel to patch them inline, to prevent "thunk 29a8e35fecSJosh Poimboeuf funneling" for both security and performance reasons 30a8e35fecSJosh Poimboeuf 31a8e35fecSJosh Poimboeuf- Non-instrumentation validation -- validates non-instrumentable 32a8e35fecSJosh Poimboeuf ("noinstr") code rules, preventing instrumentation in low-level C 33a8e35fecSJosh Poimboeuf entry code 34a8e35fecSJosh Poimboeuf 35a8e35fecSJosh Poimboeuf- Static call annotation -- annotates static call sites, enabling the 36a8e35fecSJosh Poimboeuf kernel to implement inline static calls, a faster alternative to some 37a8e35fecSJosh Poimboeuf indirect branches 38a8e35fecSJosh Poimboeuf 39a8e35fecSJosh Poimboeuf- Uaccess validation -- validates uaccess rules for a proper 40a8e35fecSJosh Poimboeuf implementation of Supervisor Mode Access Protection (SMAP) 41a8e35fecSJosh Poimboeuf 42a8e35fecSJosh Poimboeuf- Straight Line Speculation validation -- validates certain SLS 43a8e35fecSJosh Poimboeuf mitigations 44a8e35fecSJosh Poimboeuf 45a8e35fecSJosh Poimboeuf- Indirect Branch Tracking validation -- validates Intel CET IBT rules 46a8e35fecSJosh Poimboeuf to ensure that all functions referenced by function pointers have 47a8e35fecSJosh Poimboeuf corresponding ENDBR instructions 48a8e35fecSJosh Poimboeuf 49a8e35fecSJosh Poimboeuf- Indirect Branch Tracking annotation -- annotates unused ENDBR 50a8e35fecSJosh Poimboeuf instruction sites, enabling the kernel to "seal" them (replace them 51a8e35fecSJosh Poimboeuf with NOPs) to further harden IBT 52a8e35fecSJosh Poimboeuf 53a8e35fecSJosh Poimboeuf- Function entry annotation -- annotates function entries, enabling 54a8e35fecSJosh Poimboeuf kernel function tracing 55a8e35fecSJosh Poimboeuf 56a8e35fecSJosh Poimboeuf- Other toolchain hacks which will go unmentioned at this time... 57a8e35fecSJosh Poimboeuf 58a8e35fecSJosh PoimboeufEach feature can be enabled individually or in combination using the 59a8e35fecSJosh Poimboeufobjtool cmdline. 60a8e35fecSJosh Poimboeuf 61a8e35fecSJosh Poimboeuf 62a8e35fecSJosh PoimboeufObjects 63a8e35fecSJosh Poimboeuf------- 64a8e35fecSJosh Poimboeuf 65a8e35fecSJosh PoimboeufTypically, objtool runs on every translation unit (TU, aka ".o file") in 66a8e35fecSJosh Poimboeufthe kernel. If a TU is part of a kernel module, the '--module' option 67a8e35fecSJosh Poimboeufis added. 68a8e35fecSJosh Poimboeuf 69a8e35fecSJosh PoimboeufHowever: 70a8e35fecSJosh Poimboeuf 71a8e35fecSJosh Poimboeuf- If noinstr validation is enabled, it also runs on vmlinux.o, with all 72a8e35fecSJosh Poimboeuf options removed and '--noinstr' added. 73a8e35fecSJosh Poimboeuf 74a8e35fecSJosh Poimboeuf- If IBT or LTO is enabled, it doesn't run on TUs at all. Instead it 75a8e35fecSJosh Poimboeuf runs on vmlinux.o and linked modules, with all options. 76a8e35fecSJosh Poimboeuf 77a8e35fecSJosh PoimboeufIn summary: 78a8e35fecSJosh Poimboeuf 79a8e35fecSJosh Poimboeuf A) Legacy mode: 80a8e35fecSJosh Poimboeuf TU: objtool [--module] <options> 81a8e35fecSJosh Poimboeuf vmlinux: N/A 82a8e35fecSJosh Poimboeuf module: N/A 83a8e35fecSJosh Poimboeuf 84a8e35fecSJosh Poimboeuf B) CONFIG_NOINSTR_VALIDATION=y && !(CONFIG_X86_KERNEL_IBT=y || CONFIG_LTO=y): 85a8e35fecSJosh Poimboeuf TU: objtool [--module] <options> // no --noinstr 86a8e35fecSJosh Poimboeuf vmlinux: objtool --noinstr // other options removed 87a8e35fecSJosh Poimboeuf module: N/A 88a8e35fecSJosh Poimboeuf 89a8e35fecSJosh Poimboeuf C) CONFIG_X86_KERNEL_IBT=y || CONFIG_LTO=y: 90a8e35fecSJosh Poimboeuf TU: N/A 91a8e35fecSJosh Poimboeuf vmlinux: objtool --noinstr <options> 92a8e35fecSJosh Poimboeuf module: objtool --module --noinstr <options> 93a8e35fecSJosh Poimboeuf 94a8e35fecSJosh Poimboeuf 95a8e35fecSJosh PoimboeufStack validation 96a8e35fecSJosh Poimboeuf---------------- 97a8e35fecSJosh Poimboeuf 98a8e35fecSJosh PoimboeufObjtool's stack validation feature analyzes every .o file and ensures 99a8e35fecSJosh Poimboeufthe validity of its stack metadata. It enforces a set of rules on asm 100a8e35fecSJosh Poimboeufcode and C inline assembly code so that stack traces can be reliable. 101a8e35fecSJosh Poimboeuf 102a8e35fecSJosh PoimboeufFor each function, it recursively follows all possible code paths and 103a8e35fecSJosh Poimboeufvalidates the correct frame pointer state at each instruction. 104a8e35fecSJosh Poimboeuf 105a8e35fecSJosh PoimboeufIt also follows code paths involving special sections, like 106a8e35fecSJosh Poimboeuf.altinstructions, __jump_table, and __ex_table, which can add 107a8e35fecSJosh Poimboeufalternative execution paths to a given instruction (or set of 108a8e35fecSJosh Poimboeufinstructions). Similarly, it knows how to follow switch statements, for 109a8e35fecSJosh Poimboeufwhich gcc sometimes uses jump tables. 110a8e35fecSJosh Poimboeuf 111a8e35fecSJosh PoimboeufHere are some of the benefits of validating stack metadata: 112a8e35fecSJosh Poimboeuf 113a8e35fecSJosh Poimboeufa) More reliable stack traces for frame pointer enabled kernels 114a8e35fecSJosh Poimboeuf 115a8e35fecSJosh Poimboeuf Frame pointers are used for debugging purposes. They allow runtime 116a8e35fecSJosh Poimboeuf code and debug tools to be able to walk the stack to determine the 117a8e35fecSJosh Poimboeuf chain of function call sites that led to the currently executing 118a8e35fecSJosh Poimboeuf code. 119a8e35fecSJosh Poimboeuf 120a8e35fecSJosh Poimboeuf For some architectures, frame pointers are enabled by 121a8e35fecSJosh Poimboeuf CONFIG_FRAME_POINTER. For some other architectures they may be 122a8e35fecSJosh Poimboeuf required by the ABI (sometimes referred to as "backchain pointers"). 123a8e35fecSJosh Poimboeuf 124a8e35fecSJosh Poimboeuf For C code, gcc automatically generates instructions for setting up 125a8e35fecSJosh Poimboeuf frame pointers when the -fno-omit-frame-pointer option is used. 126a8e35fecSJosh Poimboeuf 127a8e35fecSJosh Poimboeuf But for asm code, the frame setup instructions have to be written by 128a8e35fecSJosh Poimboeuf hand, which most people don't do. So the end result is that 129a8e35fecSJosh Poimboeuf CONFIG_FRAME_POINTER is honored for C code but not for most asm code. 130a8e35fecSJosh Poimboeuf 131a8e35fecSJosh Poimboeuf For stack traces based on frame pointers to be reliable, all 132a8e35fecSJosh Poimboeuf functions which call other functions must first create a stack frame 133a8e35fecSJosh Poimboeuf and update the frame pointer. If a first function doesn't properly 134a8e35fecSJosh Poimboeuf create a stack frame before calling a second function, the *caller* 135a8e35fecSJosh Poimboeuf of the first function will be skipped on the stack trace. 136a8e35fecSJosh Poimboeuf 137a8e35fecSJosh Poimboeuf For example, consider the following example backtrace with frame 138a8e35fecSJosh Poimboeuf pointers enabled: 139a8e35fecSJosh Poimboeuf 140a8e35fecSJosh Poimboeuf [<ffffffff81812584>] dump_stack+0x4b/0x63 141a8e35fecSJosh Poimboeuf [<ffffffff812d6dc2>] cmdline_proc_show+0x12/0x30 142a8e35fecSJosh Poimboeuf [<ffffffff8127f568>] seq_read+0x108/0x3e0 143a8e35fecSJosh Poimboeuf [<ffffffff812cce62>] proc_reg_read+0x42/0x70 144a8e35fecSJosh Poimboeuf [<ffffffff81256197>] __vfs_read+0x37/0x100 145a8e35fecSJosh Poimboeuf [<ffffffff81256b16>] vfs_read+0x86/0x130 146a8e35fecSJosh Poimboeuf [<ffffffff81257898>] SyS_read+0x58/0xd0 147a8e35fecSJosh Poimboeuf [<ffffffff8181c1f2>] entry_SYSCALL_64_fastpath+0x12/0x76 148a8e35fecSJosh Poimboeuf 149a8e35fecSJosh Poimboeuf It correctly shows that the caller of cmdline_proc_show() is 150a8e35fecSJosh Poimboeuf seq_read(). 151a8e35fecSJosh Poimboeuf 152a8e35fecSJosh Poimboeuf If we remove the frame pointer logic from cmdline_proc_show() by 153a8e35fecSJosh Poimboeuf replacing the frame pointer related instructions with nops, here's 154a8e35fecSJosh Poimboeuf what it looks like instead: 155a8e35fecSJosh Poimboeuf 156a8e35fecSJosh Poimboeuf [<ffffffff81812584>] dump_stack+0x4b/0x63 157a8e35fecSJosh Poimboeuf [<ffffffff812d6dc2>] cmdline_proc_show+0x12/0x30 158a8e35fecSJosh Poimboeuf [<ffffffff812cce62>] proc_reg_read+0x42/0x70 159a8e35fecSJosh Poimboeuf [<ffffffff81256197>] __vfs_read+0x37/0x100 160a8e35fecSJosh Poimboeuf [<ffffffff81256b16>] vfs_read+0x86/0x130 161a8e35fecSJosh Poimboeuf [<ffffffff81257898>] SyS_read+0x58/0xd0 162a8e35fecSJosh Poimboeuf [<ffffffff8181c1f2>] entry_SYSCALL_64_fastpath+0x12/0x76 163a8e35fecSJosh Poimboeuf 164a8e35fecSJosh Poimboeuf Notice that cmdline_proc_show()'s caller, seq_read(), has been 165a8e35fecSJosh Poimboeuf skipped. Instead the stack trace seems to show that 166a8e35fecSJosh Poimboeuf cmdline_proc_show() was called by proc_reg_read(). 167a8e35fecSJosh Poimboeuf 168a8e35fecSJosh Poimboeuf The benefit of objtool here is that because it ensures that *all* 169a8e35fecSJosh Poimboeuf functions honor CONFIG_FRAME_POINTER, no functions will ever[*] be 170a8e35fecSJosh Poimboeuf skipped on a stack trace. 171a8e35fecSJosh Poimboeuf 172a8e35fecSJosh Poimboeuf [*] unless an interrupt or exception has occurred at the very 173a8e35fecSJosh Poimboeuf beginning of a function before the stack frame has been created, 174a8e35fecSJosh Poimboeuf or at the very end of the function after the stack frame has been 175a8e35fecSJosh Poimboeuf destroyed. This is an inherent limitation of frame pointers. 176a8e35fecSJosh Poimboeuf 177a8e35fecSJosh Poimboeufb) ORC (Oops Rewind Capability) unwind table generation 178a8e35fecSJosh Poimboeuf 179a8e35fecSJosh Poimboeuf An alternative to frame pointers and DWARF, ORC unwind data can be 180a8e35fecSJosh Poimboeuf used to walk the stack. Unlike frame pointers, ORC data is out of 181a8e35fecSJosh Poimboeuf band. So it doesn't affect runtime performance and it can be 182a8e35fecSJosh Poimboeuf reliable even when interrupts or exceptions are involved. 183a8e35fecSJosh Poimboeuf 184ff61f079SJonathan Corbet For more details, see Documentation/arch/x86/orc-unwinder.rst. 185a8e35fecSJosh Poimboeuf 186a8e35fecSJosh Poimboeufc) Higher live patching compatibility rate 187a8e35fecSJosh Poimboeuf 188a8e35fecSJosh Poimboeuf Livepatch has an optional "consistency model", which is needed for 189a8e35fecSJosh Poimboeuf more complex patches. In order for the consistency model to work, 190a8e35fecSJosh Poimboeuf stack traces need to be reliable (or an unreliable condition needs to 191a8e35fecSJosh Poimboeuf be detectable). Objtool makes that possible. 192a8e35fecSJosh Poimboeuf 193a8e35fecSJosh Poimboeuf For more details, see the livepatch documentation in the Linux kernel 194a8e35fecSJosh Poimboeuf source tree at Documentation/livepatch/livepatch.rst. 195a8e35fecSJosh Poimboeuf 196a8e35fecSJosh PoimboeufTo achieve the validation, objtool enforces the following rules: 197a8e35fecSJosh Poimboeuf 198a8e35fecSJosh Poimboeuf1. Each callable function must be annotated as such with the ELF 199a8e35fecSJosh Poimboeuf function type. In asm code, this is typically done using the 200a8e35fecSJosh Poimboeuf ENTRY/ENDPROC macros. If objtool finds a return instruction 201a8e35fecSJosh Poimboeuf outside of a function, it flags an error since that usually indicates 202a8e35fecSJosh Poimboeuf callable code which should be annotated accordingly. 203a8e35fecSJosh Poimboeuf 204a8e35fecSJosh Poimboeuf This rule is needed so that objtool can properly identify each 205a8e35fecSJosh Poimboeuf callable function in order to analyze its stack metadata. 206a8e35fecSJosh Poimboeuf 207a8e35fecSJosh Poimboeuf2. Conversely, each section of code which is *not* callable should *not* 208a8e35fecSJosh Poimboeuf be annotated as an ELF function. The ENDPROC macro shouldn't be used 209a8e35fecSJosh Poimboeuf in this case. 210a8e35fecSJosh Poimboeuf 211a8e35fecSJosh Poimboeuf This rule is needed so that objtool can ignore non-callable code. 212a8e35fecSJosh Poimboeuf Such code doesn't have to follow any of the other rules. 213a8e35fecSJosh Poimboeuf 214a8e35fecSJosh Poimboeuf3. Each callable function which calls another function must have the 215a8e35fecSJosh Poimboeuf correct frame pointer logic, if required by CONFIG_FRAME_POINTER or 216a8e35fecSJosh Poimboeuf the architecture's back chain rules. This can by done in asm code 217a8e35fecSJosh Poimboeuf with the FRAME_BEGIN/FRAME_END macros. 218a8e35fecSJosh Poimboeuf 219a8e35fecSJosh Poimboeuf This rule ensures that frame pointer based stack traces will work as 220a8e35fecSJosh Poimboeuf designed. If function A doesn't create a stack frame before calling 221a8e35fecSJosh Poimboeuf function B, the _caller_ of function A will be skipped on the stack 222a8e35fecSJosh Poimboeuf trace. 223a8e35fecSJosh Poimboeuf 224a8e35fecSJosh Poimboeuf4. Dynamic jumps and jumps to undefined symbols are only allowed if: 225a8e35fecSJosh Poimboeuf 226a8e35fecSJosh Poimboeuf a) the jump is part of a switch statement; or 227a8e35fecSJosh Poimboeuf 228a8e35fecSJosh Poimboeuf b) the jump matches sibling call semantics and the frame pointer has 229a8e35fecSJosh Poimboeuf the same value it had on function entry. 230a8e35fecSJosh Poimboeuf 231a8e35fecSJosh Poimboeuf This rule is needed so that objtool can reliably analyze all of a 232a8e35fecSJosh Poimboeuf function's code paths. If a function jumps to code in another file, 233a8e35fecSJosh Poimboeuf and it's not a sibling call, objtool has no way to follow the jump 234a8e35fecSJosh Poimboeuf because it only analyzes a single file at a time. 235a8e35fecSJosh Poimboeuf 236a8e35fecSJosh Poimboeuf5. A callable function may not execute kernel entry/exit instructions. 237a8e35fecSJosh Poimboeuf The only code which needs such instructions is kernel entry code, 238a8e35fecSJosh Poimboeuf which shouldn't be be in callable functions anyway. 239a8e35fecSJosh Poimboeuf 240a8e35fecSJosh Poimboeuf This rule is just a sanity check to ensure that callable functions 241a8e35fecSJosh Poimboeuf return normally. 242a8e35fecSJosh Poimboeuf 243a8e35fecSJosh Poimboeuf 244a8e35fecSJosh PoimboeufObjtool warnings 245a8e35fecSJosh Poimboeuf---------------- 246a8e35fecSJosh Poimboeuf 247ca653464SJosh PoimboeufNOTE: When requesting help with an objtool warning, please recreate with 248ca653464SJosh PoimboeufOBJTOOL_VERBOSE=1 (e.g., "make OBJTOOL_VERBOSE=1") and send the full 249ced23d2eSJosh Poimboeufoutput, including any disassembly or backtrace below the warning, to the 250ced23d2eSJosh Poimboeufobjtool maintainers. 251ca653464SJosh Poimboeuf 252a8e35fecSJosh PoimboeufFor asm files, if you're getting an error which doesn't make sense, 253a8e35fecSJosh Poimboeuffirst make sure that the affected code follows the above rules. 254a8e35fecSJosh Poimboeuf 255a8e35fecSJosh PoimboeufFor C files, the common culprits are inline asm statements and calls to 256a8e35fecSJosh Poimboeuf"noreturn" functions. See below for more details. 257a8e35fecSJosh Poimboeuf 258a8e35fecSJosh PoimboeufAnother possible cause for errors in C code is if the Makefile removes 259a8e35fecSJosh Poimboeuf-fno-omit-frame-pointer or adds -fomit-frame-pointer to the gcc options. 260a8e35fecSJosh Poimboeuf 261a8e35fecSJosh PoimboeufHere are some examples of common warnings reported by objtool, what 262a8e35fecSJosh Poimboeufthey mean, and suggestions for how to fix them. When in doubt, ping 263a8e35fecSJosh Poimboeufthe objtool maintainers. 264a8e35fecSJosh Poimboeuf 265a8e35fecSJosh Poimboeuf 266a8e35fecSJosh Poimboeuf1. file.o: warning: objtool: func()+0x128: call without frame pointer save/setup 267a8e35fecSJosh Poimboeuf 268a8e35fecSJosh Poimboeuf The func() function made a function call without first saving and/or 269a8e35fecSJosh Poimboeuf updating the frame pointer, and CONFIG_FRAME_POINTER is enabled. 270a8e35fecSJosh Poimboeuf 271a8e35fecSJosh Poimboeuf If the error is for an asm file, and func() is indeed a callable 272a8e35fecSJosh Poimboeuf function, add proper frame pointer logic using the FRAME_BEGIN and 273a8e35fecSJosh Poimboeuf FRAME_END macros. Otherwise, if it's not a callable function, remove 274a8e35fecSJosh Poimboeuf its ELF function annotation by changing ENDPROC to END, and instead 275a8e35fecSJosh Poimboeuf use the manual unwind hint macros in asm/unwind_hints.h. 276a8e35fecSJosh Poimboeuf 277a8e35fecSJosh Poimboeuf If it's a GCC-compiled .c file, the error may be because the function 278a8e35fecSJosh Poimboeuf uses an inline asm() statement which has a "call" instruction. An 279a8e35fecSJosh Poimboeuf asm() statement with a call instruction must declare the use of the 280a8e35fecSJosh Poimboeuf stack pointer in its output operand. On x86_64, this means adding 281a8e35fecSJosh Poimboeuf the ASM_CALL_CONSTRAINT as an output constraint: 282a8e35fecSJosh Poimboeuf 283a8e35fecSJosh Poimboeuf asm volatile("call func" : ASM_CALL_CONSTRAINT); 284a8e35fecSJosh Poimboeuf 285a8e35fecSJosh Poimboeuf Otherwise the stack frame may not get created before the call. 286a8e35fecSJosh Poimboeuf 287a8e35fecSJosh Poimboeuf 288a8e35fecSJosh Poimboeuf2. file.o: warning: objtool: .text+0x53: unreachable instruction 289a8e35fecSJosh Poimboeuf 290a8e35fecSJosh Poimboeuf Objtool couldn't find a code path to reach the instruction. 291a8e35fecSJosh Poimboeuf 292a8e35fecSJosh Poimboeuf If the error is for an asm file, and the instruction is inside (or 293a8e35fecSJosh Poimboeuf reachable from) a callable function, the function should be annotated 294a8e35fecSJosh Poimboeuf with the ENTRY/ENDPROC macros (ENDPROC is the important one). 295a8e35fecSJosh Poimboeuf Otherwise, the code should probably be annotated with the unwind hint 296a8e35fecSJosh Poimboeuf macros in asm/unwind_hints.h so objtool and the unwinder can know the 297a8e35fecSJosh Poimboeuf stack state associated with the code. 298a8e35fecSJosh Poimboeuf 299a8e35fecSJosh Poimboeuf If you're 100% sure the code won't affect stack traces, or if you're 300a8e35fecSJosh Poimboeuf a just a bad person, you can tell objtool to ignore it. See the 301a8e35fecSJosh Poimboeuf "Adding exceptions" section below. 302a8e35fecSJosh Poimboeuf 303a8e35fecSJosh Poimboeuf If it's not actually in a callable function (e.g. kernel entry code), 304a8e35fecSJosh Poimboeuf change ENDPROC to END. 305a8e35fecSJosh Poimboeuf 306fedb724cSJosh Poimboeuf3. file.o: warning: objtool: foo+0x48c: bar() is missing a __noreturn annotation 307fedb724cSJosh Poimboeuf 308fedb724cSJosh Poimboeuf The call from foo() to bar() doesn't return, but bar() is missing the 309*6245ce4aSJosh Poimboeuf __noreturn annotation. NOTE: In addition to annotating the function 310*6245ce4aSJosh Poimboeuf with __noreturn, please also add it to tools/objtool/noreturns.h. 311a8e35fecSJosh Poimboeuf 312a8e35fecSJosh Poimboeuf4. file.o: warning: objtool: func(): can't find starting instruction 313a8e35fecSJosh Poimboeuf or 314a8e35fecSJosh Poimboeuf file.o: warning: objtool: func()+0x11dd: can't decode instruction 315a8e35fecSJosh Poimboeuf 316a8e35fecSJosh Poimboeuf Does the file have data in a text section? If so, that can confuse 317a8e35fecSJosh Poimboeuf objtool's instruction decoder. Move the data to a more appropriate 318a8e35fecSJosh Poimboeuf section like .data or .rodata. 319a8e35fecSJosh Poimboeuf 320a8e35fecSJosh Poimboeuf 321a8e35fecSJosh Poimboeuf5. file.o: warning: objtool: func()+0x6: unsupported instruction in callable function 322a8e35fecSJosh Poimboeuf 323a8e35fecSJosh Poimboeuf This is a kernel entry/exit instruction like sysenter or iret. Such 324a8e35fecSJosh Poimboeuf instructions aren't allowed in a callable function, and are most 325a8e35fecSJosh Poimboeuf likely part of the kernel entry code. They should usually not have 326a8e35fecSJosh Poimboeuf the callable function annotation (ENDPROC) and should always be 327a8e35fecSJosh Poimboeuf annotated with the unwind hint macros in asm/unwind_hints.h. 328a8e35fecSJosh Poimboeuf 329a8e35fecSJosh Poimboeuf 330a8e35fecSJosh Poimboeuf6. file.o: warning: objtool: func()+0x26: sibling call from callable instruction with modified stack frame 331a8e35fecSJosh Poimboeuf 332a8e35fecSJosh Poimboeuf This is a dynamic jump or a jump to an undefined symbol. Objtool 333a8e35fecSJosh Poimboeuf assumed it's a sibling call and detected that the frame pointer 334a8e35fecSJosh Poimboeuf wasn't first restored to its original state. 335a8e35fecSJosh Poimboeuf 336a8e35fecSJosh Poimboeuf If it's not really a sibling call, you may need to move the 337a8e35fecSJosh Poimboeuf destination code to the local file. 338a8e35fecSJosh Poimboeuf 339a8e35fecSJosh Poimboeuf If the instruction is not actually in a callable function (e.g. 340a8e35fecSJosh Poimboeuf kernel entry code), change ENDPROC to END and annotate manually with 341a8e35fecSJosh Poimboeuf the unwind hint macros in asm/unwind_hints.h. 342a8e35fecSJosh Poimboeuf 343a8e35fecSJosh Poimboeuf 344a8e35fecSJosh Poimboeuf7. file: warning: objtool: func()+0x5c: stack state mismatch 345a8e35fecSJosh Poimboeuf 346a8e35fecSJosh Poimboeuf The instruction's frame pointer state is inconsistent, depending on 347a8e35fecSJosh Poimboeuf which execution path was taken to reach the instruction. 348a8e35fecSJosh Poimboeuf 349a8e35fecSJosh Poimboeuf Make sure that, when CONFIG_FRAME_POINTER is enabled, the function 350a8e35fecSJosh Poimboeuf pushes and sets up the frame pointer (for x86_64, this means rbp) at 351a8e35fecSJosh Poimboeuf the beginning of the function and pops it at the end of the function. 352a8e35fecSJosh Poimboeuf Also make sure that no other code in the function touches the frame 353a8e35fecSJosh Poimboeuf pointer. 354a8e35fecSJosh Poimboeuf 355a8e35fecSJosh Poimboeuf Another possibility is that the code has some asm or inline asm which 356a8e35fecSJosh Poimboeuf does some unusual things to the stack or the frame pointer. In such 357a8e35fecSJosh Poimboeuf cases it's probably appropriate to use the unwind hint macros in 358a8e35fecSJosh Poimboeuf asm/unwind_hints.h. 359a8e35fecSJosh Poimboeuf 360a8e35fecSJosh Poimboeuf 361a8e35fecSJosh Poimboeuf8. file.o: warning: objtool: funcA() falls through to next function funcB() 362a8e35fecSJosh Poimboeuf 363a8e35fecSJosh Poimboeuf This means that funcA() doesn't end with a return instruction or an 364a8e35fecSJosh Poimboeuf unconditional jump, and that objtool has determined that the function 365a8e35fecSJosh Poimboeuf can fall through into the next function. There could be different 366a8e35fecSJosh Poimboeuf reasons for this: 367a8e35fecSJosh Poimboeuf 368a8e35fecSJosh Poimboeuf 1) funcA()'s last instruction is a call to a "noreturn" function like 369a8e35fecSJosh Poimboeuf panic(). In this case the noreturn function needs to be added to 370a8e35fecSJosh Poimboeuf objtool's hard-coded global_noreturns array. Feel free to bug the 371a8e35fecSJosh Poimboeuf objtool maintainer, or you can submit a patch. 372a8e35fecSJosh Poimboeuf 373a8e35fecSJosh Poimboeuf 2) funcA() uses the unreachable() annotation in a section of code 374a8e35fecSJosh Poimboeuf that is actually reachable. 375a8e35fecSJosh Poimboeuf 376a8e35fecSJosh Poimboeuf 3) If funcA() calls an inline function, the object code for funcA() 377a8e35fecSJosh Poimboeuf might be corrupt due to a gcc bug. For more details, see: 378a8e35fecSJosh Poimboeuf https://gcc.gnu.org/bugzilla/show_bug.cgi?id=70646 379a8e35fecSJosh Poimboeuf 380a8e35fecSJosh Poimboeuf9. file.o: warning: objtool: funcA() call to funcB() with UACCESS enabled 381a8e35fecSJosh Poimboeuf 382a8e35fecSJosh Poimboeuf This means that an unexpected call to a non-whitelisted function exists 383a8e35fecSJosh Poimboeuf outside of arch-specific guards. 384a8e35fecSJosh Poimboeuf X86: SMAP (stac/clac): __uaccess_begin()/__uaccess_end() 385a8e35fecSJosh Poimboeuf ARM: PAN: uaccess_enable()/uaccess_disable() 386a8e35fecSJosh Poimboeuf 387a8e35fecSJosh Poimboeuf These functions should be called to denote a minimal critical section around 388a8e35fecSJosh Poimboeuf access to __user variables. See also: https://lwn.net/Articles/517475/ 389a8e35fecSJosh Poimboeuf 390a8e35fecSJosh Poimboeuf The intention of the warning is to prevent calls to funcB() from eventually 391a8e35fecSJosh Poimboeuf calling schedule(), potentially leaking the AC flags state, and not 392a8e35fecSJosh Poimboeuf restoring them correctly. 393a8e35fecSJosh Poimboeuf 394a8e35fecSJosh Poimboeuf It also helps verify that there are no unexpected calls to funcB() which may 395a8e35fecSJosh Poimboeuf access user space pages with protections against doing so disabled. 396a8e35fecSJosh Poimboeuf 397a8e35fecSJosh Poimboeuf To fix, either: 398a8e35fecSJosh Poimboeuf 1) remove explicit calls to funcB() from funcA(). 399a8e35fecSJosh Poimboeuf 2) add the correct guards before and after calls to low level functions like 400a8e35fecSJosh Poimboeuf __get_user_size()/__put_user_size(). 401a8e35fecSJosh Poimboeuf 3) add funcB to uaccess_safe_builtin whitelist in tools/objtool/check.c, if 402a8e35fecSJosh Poimboeuf funcB obviously does not call schedule(), and is marked notrace (since 403a8e35fecSJosh Poimboeuf function tracing inserts additional calls, which is not obvious from the 404a8e35fecSJosh Poimboeuf sources). 405a8e35fecSJosh Poimboeuf 406a8e35fecSJosh Poimboeuf10. file.o: warning: func()+0x5c: stack layout conflict in alternatives 407a8e35fecSJosh Poimboeuf 408a8e35fecSJosh Poimboeuf This means that in the use of the alternative() or ALTERNATIVE() 409a8e35fecSJosh Poimboeuf macro, the code paths have conflicting modifications to the stack. 410a8e35fecSJosh Poimboeuf The problem is that there is only one ORC unwind table, which means 411a8e35fecSJosh Poimboeuf that the ORC unwind entries must be consistent for all possible 412a8e35fecSJosh Poimboeuf instruction boundaries regardless of which code has been patched. 413a8e35fecSJosh Poimboeuf This limitation can be overcome by massaging the alternatives with 414a8e35fecSJosh Poimboeuf NOPs to shift the stack changes around so they no longer conflict. 415a8e35fecSJosh Poimboeuf 416a8e35fecSJosh Poimboeuf11. file.o: warning: unannotated intra-function call 417a8e35fecSJosh Poimboeuf 418a8e35fecSJosh Poimboeuf This warning means that a direct call is done to a destination which 419a8e35fecSJosh Poimboeuf is not at the beginning of a function. If this is a legit call, you 420a8e35fecSJosh Poimboeuf can remove this warning by putting the ANNOTATE_INTRA_FUNCTION_CALL 421a8e35fecSJosh Poimboeuf directive right before the call. 422a8e35fecSJosh Poimboeuf 42303d7a105SMichal Kubecek12. file.o: warning: func(): not an indirect call target 42403d7a105SMichal Kubecek 42503d7a105SMichal Kubecek This means that objtool is running with --ibt and a function expected 42603d7a105SMichal Kubecek to be an indirect call target is not. In particular, this happens for 42703d7a105SMichal Kubecek init_module() or cleanup_module() if a module relies on these special 42803d7a105SMichal Kubecek names and does not use module_init() / module_exit() macros to create 42903d7a105SMichal Kubecek them. 43003d7a105SMichal Kubecek 431a8e35fecSJosh Poimboeuf 432a8e35fecSJosh PoimboeufIf the error doesn't seem to make sense, it could be a bug in objtool. 433a8e35fecSJosh PoimboeufFeel free to ask the objtool maintainer for help. 434a8e35fecSJosh Poimboeuf 435a8e35fecSJosh Poimboeuf 436a8e35fecSJosh PoimboeufAdding exceptions 437a8e35fecSJosh Poimboeuf----------------- 438a8e35fecSJosh Poimboeuf 439a8e35fecSJosh PoimboeufIf you _really_ need objtool to ignore something, and are 100% sure 440a8e35fecSJosh Poimboeufthat it won't affect kernel stack traces, you can tell objtool to 441a8e35fecSJosh Poimboeufignore it: 442a8e35fecSJosh Poimboeuf 443a8e35fecSJosh Poimboeuf- To skip validation of a function, use the STACK_FRAME_NON_STANDARD 444a8e35fecSJosh Poimboeuf macro. 445a8e35fecSJosh Poimboeuf 446a8e35fecSJosh Poimboeuf- To skip validation of a file, add 447a8e35fecSJosh Poimboeuf 448a8e35fecSJosh Poimboeuf OBJECT_FILES_NON_STANDARD_filename.o := y 449a8e35fecSJosh Poimboeuf 450a8e35fecSJosh Poimboeuf to the Makefile. 451a8e35fecSJosh Poimboeuf 452a8e35fecSJosh Poimboeuf- To skip validation of a directory, add 453a8e35fecSJosh Poimboeuf 454a8e35fecSJosh Poimboeuf OBJECT_FILES_NON_STANDARD := y 455a8e35fecSJosh Poimboeuf 456a8e35fecSJosh Poimboeuf to the Makefile. 457a8e35fecSJosh Poimboeuf 458a8e35fecSJosh PoimboeufNOTE: OBJECT_FILES_NON_STANDARD doesn't work for link time validation of 459a8e35fecSJosh Poimboeufvmlinux.o or a linked module. So it should only be used for files which 460a8e35fecSJosh Poimboeufaren't linked into vmlinux or a module. 461