1==============
2BPF Design Q&A
3==============
4
5BPF extensibility and applicability to networking, tracing, security
6in the linux kernel and several user space implementations of BPF
7virtual machine led to a number of misunderstanding on what BPF actually is.
8This short QA is an attempt to address that and outline a direction
9of where BPF is heading long term.
10
11.. contents::
12    :local:
13    :depth: 3
14
15Questions and Answers
16=====================
17
18Q: Is BPF a generic instruction set similar to x64 and arm64?
19-------------------------------------------------------------
20A: NO.
21
22Q: Is BPF a generic virtual machine ?
23-------------------------------------
24A: NO.
25
26BPF is generic instruction set *with* C calling convention.
27-----------------------------------------------------------
28
29Q: Why C calling convention was chosen?
30~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
31
32A: Because BPF programs are designed to run in the linux kernel
33which is written in C, hence BPF defines instruction set compatible
34with two most used architectures x64 and arm64 (and takes into
35consideration important quirks of other architectures) and
36defines calling convention that is compatible with C calling
37convention of the linux kernel on those architectures.
38
39Q: Can multiple return values be supported in the future?
40~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
41A: NO. BPF allows only register R0 to be used as return value.
42
43Q: Can more than 5 function arguments be supported in the future?
44~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
45A: NO. BPF calling convention only allows registers R1-R5 to be used
46as arguments. BPF is not a standalone instruction set.
47(unlike x64 ISA that allows msft, cdecl and other conventions)
48
49Q: Can BPF programs access instruction pointer or return address?
50-----------------------------------------------------------------
51A: NO.
52
53Q: Can BPF programs access stack pointer ?
54------------------------------------------
55A: NO.
56
57Only frame pointer (register R10) is accessible.
58From compiler point of view it's necessary to have stack pointer.
59For example, LLVM defines register R11 as stack pointer in its
60BPF backend, but it makes sure that generated code never uses it.
61
62Q: Does C-calling convention diminishes possible use cases?
63-----------------------------------------------------------
64A: YES.
65
66BPF design forces addition of major functionality in the form
67of kernel helper functions and kernel objects like BPF maps with
68seamless interoperability between them. It lets kernel call into
69BPF programs and programs call kernel helpers with zero overhead,
70as all of them were native C code. That is particularly the case
71for JITed BPF programs that are indistinguishable from
72native kernel C code.
73
74Q: Does it mean that 'innovative' extensions to BPF code are disallowed?
75------------------------------------------------------------------------
76A: Soft yes.
77
78At least for now, until BPF core has support for
79bpf-to-bpf calls, indirect calls, loops, global variables,
80jump tables, read-only sections, and all other normal constructs
81that C code can produce.
82
83Q: Can loops be supported in a safe way?
84----------------------------------------
85A: It's not clear yet.
86
87BPF developers are trying to find a way to
88support bounded loops where the verifier can guarantee that
89the program terminates in less than 4096 instructions.
90
91Instruction level questions
92---------------------------
93
94Q: LD_ABS and LD_IND instructions vs C code
95~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
96
97Q: How come LD_ABS and LD_IND instruction are present in BPF whereas
98C code cannot express them and has to use builtin intrinsics?
99
100A: This is artifact of compatibility with classic BPF. Modern
101networking code in BPF performs better without them.
102See 'direct packet access'.
103
104Q: BPF instructions mapping not one-to-one to native CPU
105~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
106Q: It seems not all BPF instructions are one-to-one to native CPU.
107For example why BPF_JNE and other compare and jumps are not cpu-like?
108
109A: This was necessary to avoid introducing flags into ISA which are
110impossible to make generic and efficient across CPU architectures.
111
112Q: Why BPF_DIV instruction doesn't map to x64 div?
113~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
114A: Because if we picked one-to-one relationship to x64 it would have made
115it more complicated to support on arm64 and other archs. Also it
116needs div-by-zero runtime check.
117
118Q: Why there is no BPF_SDIV for signed divide operation?
119~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
120A: Because it would be rarely used. llvm errors in such case and
121prints a suggestion to use unsigned divide instead.
122
123Q: Why BPF has implicit prologue and epilogue?
124~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
125A: Because architectures like sparc have register windows and in general
126there are enough subtle differences between architectures, so naive
127store return address into stack won't work. Another reason is BPF has
128to be safe from division by zero (and legacy exception path
129of LD_ABS insn). Those instructions need to invoke epilogue and
130return implicitly.
131
132Q: Why BPF_JLT and BPF_JLE instructions were not introduced in the beginning?
133~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
134A: Because classic BPF didn't have them and BPF authors felt that compiler
135workaround would be acceptable. Turned out that programs lose performance
136due to lack of these compare instructions and they were added.
137These two instructions is a perfect example what kind of new BPF
138instructions are acceptable and can be added in the future.
139These two already had equivalent instructions in native CPUs.
140New instructions that don't have one-to-one mapping to HW instructions
141will not be accepted.
142
143Q: BPF 32-bit subregister requirements
144~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
145Q: BPF 32-bit subregisters have a requirement to zero upper 32-bits of BPF
146registers which makes BPF inefficient virtual machine for 32-bit
147CPU architectures and 32-bit HW accelerators. Can true 32-bit registers
148be added to BPF in the future?
149
150A: NO. The first thing to improve performance on 32-bit archs is to teach
151LLVM to generate code that uses 32-bit subregisters. Then second step
152is to teach verifier to mark operations where zero-ing upper bits
153is unnecessary. Then JITs can take advantage of those markings and
154drastically reduce size of generated code and improve performance.
155
156Q: Does BPF have a stable ABI?
157------------------------------
158A: YES. BPF instructions, arguments to BPF programs, set of helper
159functions and their arguments, recognized return codes are all part
160of ABI. However there is one specific exception to tracing programs
161which are using helpers like bpf_probe_read() to walk kernel internal
162data structures and compile with kernel internal headers. Both of these
163kernel internals are subject to change and can break with newer kernels
164such that the program needs to be adapted accordingly.
165
166Q: How much stack space a BPF program uses?
167-------------------------------------------
168A: Currently all program types are limited to 512 bytes of stack
169space, but the verifier computes the actual amount of stack used
170and both interpreter and most JITed code consume necessary amount.
171
172Q: Can BPF be offloaded to HW?
173------------------------------
174A: YES. BPF HW offload is supported by NFP driver.
175
176Q: Does classic BPF interpreter still exist?
177--------------------------------------------
178A: NO. Classic BPF programs are converted into extend BPF instructions.
179
180Q: Can BPF call arbitrary kernel functions?
181-------------------------------------------
182A: NO. BPF programs can only call a set of helper functions which
183is defined for every program type.
184
185Q: Can BPF overwrite arbitrary kernel memory?
186---------------------------------------------
187A: NO.
188
189Tracing bpf programs can *read* arbitrary memory with bpf_probe_read()
190and bpf_probe_read_str() helpers. Networking programs cannot read
191arbitrary memory, since they don't have access to these helpers.
192Programs can never read or write arbitrary memory directly.
193
194Q: Can BPF overwrite arbitrary user memory?
195-------------------------------------------
196A: Sort-of.
197
198Tracing BPF programs can overwrite the user memory
199of the current task with bpf_probe_write_user(). Every time such
200program is loaded the kernel will print warning message, so
201this helper is only useful for experiments and prototypes.
202Tracing BPF programs are root only.
203
204Q: bpf_trace_printk() helper warning
205------------------------------------
206Q: When bpf_trace_printk() helper is used the kernel prints nasty
207warning message. Why is that?
208
209A: This is done to nudge program authors into better interfaces when
210programs need to pass data to user space. Like bpf_perf_event_output()
211can be used to efficiently stream data via perf ring buffer.
212BPF maps can be used for asynchronous data sharing between kernel
213and user space. bpf_trace_printk() should only be used for debugging.
214
215Q: New functionality via kernel modules?
216----------------------------------------
217Q: Can BPF functionality such as new program or map types, new
218helpers, etc be added out of kernel module code?
219
220A: NO.
221