Blame view
Documentation/bpf/bpf_design_QA.rst
11.3 KB
1a6ac1d59 bpf, doc: convert... |
1 2 3 |
============== BPF Design Q&A ============== |
2e39748a4 bpf: document ans... |
4 5 6 7 8 |
BPF extensibility and applicability to networking, tracing, security in the linux kernel and several user space implementations of BPF virtual machine led to a number of misunderstanding on what BPF actually is. This short QA is an attempt to address that and outline a direction of where BPF is heading long term. |
1a6ac1d59 bpf, doc: convert... |
9 10 11 12 13 14 |
.. contents:: :local: :depth: 3 Questions and Answers ===================== |
2e39748a4 bpf: document ans... |
15 |
Q: Is BPF a generic instruction set similar to x64 and arm64? |
1a6ac1d59 bpf, doc: convert... |
16 |
------------------------------------------------------------- |
2e39748a4 bpf: document ans... |
17 18 19 |
A: NO. Q: Is BPF a generic virtual machine ? |
1a6ac1d59 bpf, doc: convert... |
20 |
------------------------------------- |
2e39748a4 bpf: document ans... |
21 |
A: NO. |
1a6ac1d59 bpf, doc: convert... |
22 23 |
BPF is generic instruction set *with* C calling convention. ----------------------------------------------------------- |
2e39748a4 bpf: document ans... |
24 25 |
Q: Why C calling convention was chosen? |
1a6ac1d59 bpf, doc: convert... |
26 |
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
2e39748a4 bpf: document ans... |
27 |
A: Because BPF programs are designed to run in the linux kernel |
1a6ac1d59 bpf, doc: convert... |
28 29 30 31 32 |
which is written in C, hence BPF defines instruction set compatible with two most used architectures x64 and arm64 (and takes into consideration important quirks of other architectures) and defines calling convention that is compatible with C calling convention of the linux kernel on those architectures. |
2e39748a4 bpf: document ans... |
33 |
|
46604676c docs/bpf: minor c... |
34 |
Q: Can multiple return values be supported in the future? |
1a6ac1d59 bpf, doc: convert... |
35 |
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
2e39748a4 bpf: document ans... |
36 |
A: NO. BPF allows only register R0 to be used as return value. |
46604676c docs/bpf: minor c... |
37 |
Q: Can more than 5 function arguments be supported in the future? |
1a6ac1d59 bpf, doc: convert... |
38 |
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
2e39748a4 bpf: document ans... |
39 |
A: NO. BPF calling convention only allows registers R1-R5 to be used |
1a6ac1d59 bpf, doc: convert... |
40 41 |
as arguments. BPF is not a standalone instruction set. (unlike x64 ISA that allows msft, cdecl and other conventions) |
2e39748a4 bpf: document ans... |
42 |
|
46604676c docs/bpf: minor c... |
43 |
Q: Can BPF programs access instruction pointer or return address? |
1a6ac1d59 bpf, doc: convert... |
44 |
----------------------------------------------------------------- |
2e39748a4 bpf: document ans... |
45 |
A: NO. |
46604676c docs/bpf: minor c... |
46 |
Q: Can BPF programs access stack pointer ? |
1a6ac1d59 bpf, doc: convert... |
47 48 49 50 51 |
------------------------------------------ A: NO. Only frame pointer (register R10) is accessible. From compiler point of view it's necessary to have stack pointer. |
46604676c docs/bpf: minor c... |
52 |
For example, LLVM defines register R11 as stack pointer in its |
1a6ac1d59 bpf, doc: convert... |
53 |
BPF backend, but it makes sure that generated code never uses it. |
2e39748a4 bpf: document ans... |
54 55 |
Q: Does C-calling convention diminishes possible use cases? |
1a6ac1d59 bpf, doc: convert... |
56 57 58 59 60 61 |
----------------------------------------------------------- A: YES. BPF design forces addition of major functionality in the form of kernel helper functions and kernel objects like BPF maps with seamless interoperability between them. It lets kernel call into |
46604676c docs/bpf: minor c... |
62 63 |
BPF programs and programs call kernel helpers with zero overhead, as all of them were native C code. That is particularly the case |
1a6ac1d59 bpf, doc: convert... |
64 65 |
for JITed BPF programs that are indistinguishable from native kernel C code. |
2e39748a4 bpf: document ans... |
66 67 |
Q: Does it mean that 'innovative' extensions to BPF code are disallowed? |
1a6ac1d59 bpf, doc: convert... |
68 69 |
------------------------------------------------------------------------ A: Soft yes. |
46604676c docs/bpf: minor c... |
70 |
At least for now, until BPF core has support for |
1a6ac1d59 bpf, doc: convert... |
71 |
bpf-to-bpf calls, indirect calls, loops, global variables, |
46604676c docs/bpf: minor c... |
72 |
jump tables, read-only sections, and all other normal constructs |
1a6ac1d59 bpf, doc: convert... |
73 |
that C code can produce. |
2e39748a4 bpf: document ans... |
74 75 |
Q: Can loops be supported in a safe way? |
1a6ac1d59 bpf, doc: convert... |
76 77 78 79 |
---------------------------------------- A: It's not clear yet. BPF developers are trying to find a way to |
3b8802446 bpf: document the... |
80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 |
support bounded loops. Q: What are the verifier limits? -------------------------------- A: The only limit known to the user space is BPF_MAXINSNS (4096). It's the maximum number of instructions that the unprivileged bpf program can have. The verifier has various internal limits. Like the maximum number of instructions that can be explored during program analysis. Currently, that limit is set to 1 million. Which essentially means that the largest program can consist of 1 million NOP instructions. There is a limit to the maximum number of subsequent branches, a limit to the number of nested bpf-to-bpf calls, a limit to the number of the verifier states per instruction, a limit to the number of maps used by the program. All these limits can be hit with a sufficiently complex program. There are also non-numerical limits that can cause the program to be rejected. The verifier used to recognize only pointer + constant expressions. Now it can recognize pointer + bounded_register. bpf_lookup_map_elem(key) had a requirement that 'key' must be a pointer to the stack. Now, 'key' can be a pointer to map value. The verifier is steadily getting 'smarter'. The limits are being removed. The only way to know that the program is going to be accepted by the verifier is to try to load it. The bpf development process guarantees that the future kernel versions will accept all bpf programs that were accepted by the earlier versions. |
1a6ac1d59 bpf, doc: convert... |
106 107 108 109 110 111 |
Instruction level questions --------------------------- Q: LD_ABS and LD_IND instructions vs C code ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
2e39748a4 bpf: document ans... |
112 113 |
Q: How come LD_ABS and LD_IND instruction are present in BPF whereas |
1a6ac1d59 bpf, doc: convert... |
114 |
C code cannot express them and has to use builtin intrinsics? |
2e39748a4 bpf: document ans... |
115 |
A: This is artifact of compatibility with classic BPF. Modern |
1a6ac1d59 bpf, doc: convert... |
116 117 |
networking code in BPF performs better without them. See 'direct packet access'. |
2e39748a4 bpf: document ans... |
118 |
|
1a6ac1d59 bpf, doc: convert... |
119 120 |
Q: BPF instructions mapping not one-to-one to native CPU ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
2e39748a4 bpf: document ans... |
121 |
Q: It seems not all BPF instructions are one-to-one to native CPU. |
1a6ac1d59 bpf, doc: convert... |
122 |
For example why BPF_JNE and other compare and jumps are not cpu-like? |
2e39748a4 bpf: document ans... |
123 |
A: This was necessary to avoid introducing flags into ISA which are |
1a6ac1d59 bpf, doc: convert... |
124 |
impossible to make generic and efficient across CPU architectures. |
2e39748a4 bpf: document ans... |
125 |
|
46604676c docs/bpf: minor c... |
126 |
Q: Why BPF_DIV instruction doesn't map to x64 div? |
1a6ac1d59 bpf, doc: convert... |
127 |
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
2e39748a4 bpf: document ans... |
128 |
A: Because if we picked one-to-one relationship to x64 it would have made |
1a6ac1d59 bpf, doc: convert... |
129 130 |
it more complicated to support on arm64 and other archs. Also it needs div-by-zero runtime check. |
2e39748a4 bpf: document ans... |
131 |
|
46604676c docs/bpf: minor c... |
132 |
Q: Why there is no BPF_SDIV for signed divide operation? |
1a6ac1d59 bpf, doc: convert... |
133 |
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
2e39748a4 bpf: document ans... |
134 |
A: Because it would be rarely used. llvm errors in such case and |
46604676c docs/bpf: minor c... |
135 |
prints a suggestion to use unsigned divide instead. |
2e39748a4 bpf: document ans... |
136 137 |
Q: Why BPF has implicit prologue and epilogue? |
1a6ac1d59 bpf, doc: convert... |
138 |
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
2e39748a4 bpf: document ans... |
139 |
A: Because architectures like sparc have register windows and in general |
1a6ac1d59 bpf, doc: convert... |
140 141 142 143 144 |
there are enough subtle differences between architectures, so naive store return address into stack won't work. Another reason is BPF has to be safe from division by zero (and legacy exception path of LD_ABS insn). Those instructions need to invoke epilogue and return implicitly. |
2e39748a4 bpf: document ans... |
145 146 |
Q: Why BPF_JLT and BPF_JLE instructions were not introduced in the beginning? |
1a6ac1d59 bpf, doc: convert... |
147 |
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
2e39748a4 bpf: document ans... |
148 |
A: Because classic BPF didn't have them and BPF authors felt that compiler |
1a6ac1d59 bpf, doc: convert... |
149 150 151 152 153 154 155 156 157 158 |
workaround would be acceptable. Turned out that programs lose performance due to lack of these compare instructions and they were added. These two instructions is a perfect example what kind of new BPF instructions are acceptable and can be added in the future. These two already had equivalent instructions in native CPUs. New instructions that don't have one-to-one mapping to HW instructions will not be accepted. Q: BPF 32-bit subregister requirements ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
2e39748a4 bpf: document ans... |
159 |
Q: BPF 32-bit subregisters have a requirement to zero upper 32-bits of BPF |
1a6ac1d59 bpf, doc: convert... |
160 161 162 |
registers which makes BPF inefficient virtual machine for 32-bit CPU architectures and 32-bit HW accelerators. Can true 32-bit registers be added to BPF in the future? |
c231c22a9 bpf: doc: update ... |
163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 |
A: NO. But some optimizations on zero-ing the upper 32 bits for BPF registers are available, and can be leveraged to improve the performance of JITed BPF programs for 32-bit architectures. Starting with version 7, LLVM is able to generate instructions that operate on 32-bit subregisters, provided the option -mattr=+alu32 is passed for compiling a program. Furthermore, the verifier can now mark the instructions for which zero-ing the upper bits of the destination register is required, and insert an explicit zero-extension (zext) instruction (a mov32 variant). This means that for architectures without zext hardware support, the JIT back-ends do not need to clear the upper bits for subregisters written by alu32 instructions or narrow loads. Instead, the back-ends simply need to support code generation for that mov32 variant, and to overwrite bpf_jit_needs_zext() to make it return "true" (in order to enable zext insertion in the verifier). Note that it is possible for a JIT back-end to have partial hardware support for zext. In that case, if verifier zext insertion is enabled, it could lead to the insertion of unnecessary zext instructions. Such instructions could be removed by creating a simple peephole inside the JIT back-end: if one instruction has hardware support for zext and if the next instruction is an explicit zext, then the latter can be skipped when doing the code generation. |
2e39748a4 bpf: document ans... |
188 189 |
Q: Does BPF have a stable ABI? |
1a6ac1d59 bpf, doc: convert... |
190 |
------------------------------ |
2e39748a4 bpf: document ans... |
191 |
A: YES. BPF instructions, arguments to BPF programs, set of helper |
1a6ac1d59 bpf, doc: convert... |
192 |
functions and their arguments, recognized return codes are all part |
a769fa720 bpf, doc: update ... |
193 194 195 196 197 |
of ABI. However there is one specific exception to tracing programs which are using helpers like bpf_probe_read() to walk kernel internal data structures and compile with kernel internal headers. Both of these kernel internals are subject to change and can break with newer kernels such that the program needs to be adapted accordingly. |
2e39748a4 bpf: document ans... |
198 199 |
Q: How much stack space a BPF program uses? |
1a6ac1d59 bpf, doc: convert... |
200 |
------------------------------------------- |
2e39748a4 bpf: document ans... |
201 |
A: Currently all program types are limited to 512 bytes of stack |
1a6ac1d59 bpf, doc: convert... |
202 203 |
space, but the verifier computes the actual amount of stack used and both interpreter and most JITed code consume necessary amount. |
2e39748a4 bpf: document ans... |
204 205 |
Q: Can BPF be offloaded to HW? |
1a6ac1d59 bpf, doc: convert... |
206 |
------------------------------ |
2e39748a4 bpf: document ans... |
207 208 209 |
A: YES. BPF HW offload is supported by NFP driver. Q: Does classic BPF interpreter still exist? |
1a6ac1d59 bpf, doc: convert... |
210 |
-------------------------------------------- |
2e39748a4 bpf: document ans... |
211 212 213 |
A: NO. Classic BPF programs are converted into extend BPF instructions. Q: Can BPF call arbitrary kernel functions? |
1a6ac1d59 bpf, doc: convert... |
214 |
------------------------------------------- |
2e39748a4 bpf: document ans... |
215 |
A: NO. BPF programs can only call a set of helper functions which |
1a6ac1d59 bpf, doc: convert... |
216 |
is defined for every program type. |
2e39748a4 bpf: document ans... |
217 218 |
Q: Can BPF overwrite arbitrary kernel memory? |
1a6ac1d59 bpf, doc: convert... |
219 220 221 222 223 224 225 |
--------------------------------------------- A: NO. Tracing bpf programs can *read* arbitrary memory with bpf_probe_read() and bpf_probe_read_str() helpers. Networking programs cannot read arbitrary memory, since they don't have access to these helpers. Programs can never read or write arbitrary memory directly. |
2e39748a4 bpf: document ans... |
226 227 |
Q: Can BPF overwrite arbitrary user memory? |
1a6ac1d59 bpf, doc: convert... |
228 229 230 231 232 233 234 235 |
------------------------------------------- A: Sort-of. Tracing BPF programs can overwrite the user memory of the current task with bpf_probe_write_user(). Every time such program is loaded the kernel will print warning message, so this helper is only useful for experiments and prototypes. Tracing BPF programs are root only. |
2e39748a4 bpf: document ans... |
236 |
|
1a6ac1d59 bpf, doc: convert... |
237 238 |
Q: bpf_trace_printk() helper warning ------------------------------------ |
2e39748a4 bpf: document ans... |
239 |
Q: When bpf_trace_printk() helper is used the kernel prints nasty |
1a6ac1d59 bpf, doc: convert... |
240 |
warning message. Why is that? |
2e39748a4 bpf: document ans... |
241 |
A: This is done to nudge program authors into better interfaces when |
1a6ac1d59 bpf, doc: convert... |
242 243 244 245 |
programs need to pass data to user space. Like bpf_perf_event_output() can be used to efficiently stream data via perf ring buffer. BPF maps can be used for asynchronous data sharing between kernel and user space. bpf_trace_printk() should only be used for debugging. |
2e39748a4 bpf: document ans... |
246 |
|
1a6ac1d59 bpf, doc: convert... |
247 248 |
Q: New functionality via kernel modules? ---------------------------------------- |
2e39748a4 bpf: document ans... |
249 |
Q: Can BPF functionality such as new program or map types, new |
1a6ac1d59 bpf, doc: convert... |
250 |
helpers, etc be added out of kernel module code? |
2e39748a4 bpf: document ans... |
251 |
A: NO. |