1==================
2BPF Selftest Notes
3==================
4General instructions on running selftests can be found in
5`Documentation/bpf/bpf_devel_QA.rst`__.
6
7__ /Documentation/bpf/bpf_devel_QA.rst#q-how-to-run-bpf-selftests
8
9=============
10BPF CI System
11=============
12
13BPF employs a continuous integration (CI) system to check patch submission in an
14automated fashion. The system runs selftests for each patch in a series. Results
15are propagated to patchwork, where failures are highlighted similar to
16violations of other checks (such as additional warnings being emitted or a
17``scripts/checkpatch.pl`` reported deficiency):
18
19  https://patchwork.kernel.org/project/netdevbpf/list/?delegate=121173
20
21The CI system executes tests on multiple architectures. It uses a kernel
22configuration derived from both the generic and architecture specific config
23file fragments below ``tools/testing/selftests/bpf/`` (e.g., ``config`` and
24``config.x86_64``).
25
26Denylisting Tests
27=================
28
29It is possible for some architectures to not have support for all BPF features.
30In such a case tests in CI may fail. An example of such a shortcoming is BPF
31trampoline support on IBM's s390x architecture. For cases like this, an in-tree
32deny list file, located at ``tools/testing/selftests/bpf/DENYLIST.<arch>``, can
33be used to prevent the test from running on such an architecture.
34
35In addition to that, the generic ``tools/testing/selftests/bpf/DENYLIST`` is
36honored on every architecture running tests.
37
38These files are organized in three columns. The first column lists the test in
39question. This can be the name of a test suite or of an individual test. The
40remaining two columns provide additional meta data that helps identify and
41classify the entry: column two is a copy and paste of the error being reported
42when running the test in the setting in question. The third column, if
43available, summarizes the underlying problem. A value of ``trampoline``, for
44example, indicates that lack of trampoline support is causing the test to fail.
45This last entry helps identify tests that can be re-enabled once such support is
46added.
47
48=========================
49Running Selftests in a VM
50=========================
51
52It's now possible to run the selftests using ``tools/testing/selftests/bpf/vmtest.sh``.
53The script tries to ensure that the tests are run with the same environment as they
54would be run post-submit in the CI used by the Maintainers, with the exception
55that deny lists are not automatically honored.
56
57This script uses the in-tree kernel configuration and downloads a VM userspace
58image from the system used by the CI. It builds the kernel (without overwriting
59your existing Kconfig), recompiles the bpf selftests, runs them (by default
60``tools/testing/selftests/bpf/test_progs``) and saves the resulting output (by
61default in ``~/.bpf_selftests``).
62
63Script dependencies:
64- clang (preferably built from sources, https://github.com/llvm/llvm-project);
65- pahole (preferably built from sources, https://git.kernel.org/pub/scm/devel/pahole/pahole.git/);
66- qemu;
67- docutils (for ``rst2man``);
68- libcap-devel.
69
70For more information about using the script, run:
71
72.. code-block:: console
73
74  $ tools/testing/selftests/bpf/vmtest.sh -h
75
76In case of linker errors when running selftests, try using static linking:
77
78.. code-block:: console
79
80  $ LDLIBS=-static vmtest.sh
81
82.. note:: Some distros may not support static linking.
83
84.. note:: The script uses pahole and clang based on host environment setting.
85          If you want to change pahole and llvm, you can change `PATH` environment
86          variable in the beginning of script.
87
88.. note:: The script currently only supports x86_64 and s390x architectures.
89
90Additional information about selftest failures are
91documented here.
92
93profiler[23] test failures with clang/llvm <12.0.0
94==================================================
95
96With clang/llvm <12.0.0, the profiler[23] test may fail.
97The symptom looks like
98
99.. code-block:: c
100
101  // r9 is a pointer to map_value
102  // r7 is a scalar
103  17:       bf 96 00 00 00 00 00 00 r6 = r9
104  18:       0f 76 00 00 00 00 00 00 r6 += r7
105  math between map_value pointer and register with unbounded min value is not allowed
106
107  // the instructions below will not be seen in the verifier log
108  19:       a5 07 01 00 01 01 00 00 if r7 < 257 goto +1
109  20:       bf 96 00 00 00 00 00 00 r6 = r9
110  // r6 is used here
111
112The verifier will reject such code with above error.
113At insn 18 the r7 is indeed unbounded. The later insn 19 checks the bounds and
114the insn 20 undoes map_value addition. It is currently impossible for the
115verifier to understand such speculative pointer arithmetic.
116Hence `this patch`__ addresses it on the compiler side. It was committed on llvm 12.
117
118__ https://reviews.llvm.org/D85570
119
120The corresponding C code
121
122.. code-block:: c
123
124  for (int i = 0; i < MAX_CGROUPS_PATH_DEPTH; i++) {
125          filepart_length = bpf_probe_read_str(payload, ...);
126          if (filepart_length <= MAX_PATH) {
127                  barrier_var(filepart_length); // workaround
128                  payload += filepart_length;
129          }
130  }
131
132bpf_iter test failures with clang/llvm 10.0.0
133=============================================
134
135With clang/llvm 10.0.0, the following two bpf_iter tests failed:
136  * ``bpf_iter/ipv6_route``
137  * ``bpf_iter/netlink``
138
139The symptom for ``bpf_iter/ipv6_route`` looks like
140
141.. code-block:: c
142
143  2: (79) r8 = *(u64 *)(r1 +8)
144  ...
145  14: (bf) r2 = r8
146  15: (0f) r2 += r1
147  ; BPF_SEQ_PRINTF(seq, "%pi6 %02x ", &rt->fib6_dst.addr, rt->fib6_dst.plen);
148  16: (7b) *(u64 *)(r8 +64) = r2
149  only read is supported
150
151The symptom for ``bpf_iter/netlink`` looks like
152
153.. code-block:: c
154
155  ; struct netlink_sock *nlk = ctx->sk;
156  2: (79) r7 = *(u64 *)(r1 +8)
157  ...
158  15: (bf) r2 = r7
159  16: (0f) r2 += r1
160  ; BPF_SEQ_PRINTF(seq, "%pK %-3d ", s, s->sk_protocol);
161  17: (7b) *(u64 *)(r7 +0) = r2
162  only read is supported
163
164This is due to a llvm BPF backend bug. `The fix`__
165has been pushed to llvm 10.x release branch and will be
166available in 10.0.1. The patch is available in llvm 11.0.0 trunk.
167
168__  https://reviews.llvm.org/D78466
169
170bpf_verif_scale/loop6.bpf.o test failure with Clang 12
171======================================================
172
173With Clang 12, the following bpf_verif_scale test failed:
174  * ``bpf_verif_scale/loop6.bpf.o``
175
176The verifier output looks like
177
178.. code-block:: c
179
180  R1 type=ctx expected=fp
181  The sequence of 8193 jumps is too complex.
182
183The reason is compiler generating the following code
184
185.. code-block:: c
186
187  ;       for (i = 0; (i < VIRTIO_MAX_SGS) && (i < num); i++) {
188      14:       16 05 40 00 00 00 00 00 if w5 == 0 goto +64 <LBB0_6>
189      15:       bc 51 00 00 00 00 00 00 w1 = w5
190      16:       04 01 00 00 ff ff ff ff w1 += -1
191      17:       67 05 00 00 20 00 00 00 r5 <<= 32
192      18:       77 05 00 00 20 00 00 00 r5 >>= 32
193      19:       a6 01 01 00 05 00 00 00 if w1 < 5 goto +1 <LBB0_4>
194      20:       b7 05 00 00 06 00 00 00 r5 = 6
195  00000000000000a8 <LBB0_4>:
196      21:       b7 02 00 00 00 00 00 00 r2 = 0
197      22:       b7 01 00 00 00 00 00 00 r1 = 0
198  ;       for (i = 0; (i < VIRTIO_MAX_SGS) && (i < num); i++) {
199      23:       7b 1a e0 ff 00 00 00 00 *(u64 *)(r10 - 32) = r1
200      24:       7b 5a c0 ff 00 00 00 00 *(u64 *)(r10 - 64) = r5
201
202Note that insn #15 has w1 = w5 and w1 is refined later but
203r5(w5) is eventually saved on stack at insn #24 for later use.
204This cause later verifier failure. The bug has been `fixed`__ in
205Clang 13.
206
207__  https://reviews.llvm.org/D97479
208
209BPF CO-RE-based tests and Clang version
210=======================================
211
212A set of selftests use BPF target-specific built-ins, which might require
213bleeding-edge Clang versions (Clang 12 nightly at this time).
214
215Few sub-tests of core_reloc test suit (part of test_progs test runner) require
216the following built-ins, listed with corresponding Clang diffs introducing
217them to Clang/LLVM. These sub-tests are going to be skipped if Clang is too
218old to support them, they shouldn't cause build failures or runtime test
219failures:
220
221- __builtin_btf_type_id() [0_, 1_, 2_];
222- __builtin_preserve_type_info(), __builtin_preserve_enum_value() [3_, 4_].
223
224.. _0: https://reviews.llvm.org/D74572
225.. _1: https://reviews.llvm.org/D74668
226.. _2: https://reviews.llvm.org/D85174
227.. _3: https://reviews.llvm.org/D83878
228.. _4: https://reviews.llvm.org/D83242
229
230Floating-point tests and Clang version
231======================================
232
233Certain selftests, e.g. core_reloc, require support for the floating-point
234types, which was introduced in `Clang 13`__. The older Clang versions will
235either crash when compiling these tests, or generate an incorrect BTF.
236
237__  https://reviews.llvm.org/D83289
238
239Kernel function call test and Clang version
240===========================================
241
242Some selftests (e.g. kfunc_call and bpf_tcp_ca) require a LLVM support
243to generate extern function in BTF.  It was introduced in `Clang 13`__.
244
245Without it, the error from compiling bpf selftests looks like:
246
247.. code-block:: console
248
249  libbpf: failed to find BTF for extern 'tcp_slow_start' [25] section: -2
250
251__ https://reviews.llvm.org/D93563
252
253btf_tag test and Clang version
254==============================
255
256The btf_tag selftest requires LLVM support to recognize the btf_decl_tag and
257btf_type_tag attributes. They are introduced in `Clang 14` [0_, 1_].
258The subtests ``btf_type_tag_user_{mod1, mod2, vmlinux}`` also requires
259pahole version ``1.23``.
260
261Without them, the btf_tag selftest will be skipped and you will observe:
262
263.. code-block:: console
264
265  #<test_num> btf_tag:SKIP
266
267.. _0: https://reviews.llvm.org/D111588
268.. _1: https://reviews.llvm.org/D111199
269
270Clang dependencies for static linking tests
271===========================================
272
273linked_vars, linked_maps, and linked_funcs tests depend on `Clang fix`__ to
274generate valid BTF information for weak variables. Please make sure you use
275Clang that contains the fix.
276
277__ https://reviews.llvm.org/D100362
278
279Clang relocation changes
280========================
281
282Clang 13 patch `clang reloc patch`_  made some changes on relocations such
283that existing relocation types are broken into more types and
284each new type corresponds to only one way to resolve relocation.
285See `kernel llvm reloc`_ for more explanation and some examples.
286Using clang 13 to compile old libbpf which has static linker support,
287there will be a compilation failure::
288
289  libbpf: ELF relo #0 in section #6 has unexpected type 2 in .../bpf_tcp_nogpl.bpf.o
290
291Here, ``type 2`` refers to new relocation type ``R_BPF_64_ABS64``.
292To fix this issue, user newer libbpf.
293
294.. Links
295.. _clang reloc patch: https://reviews.llvm.org/D102712
296.. _kernel llvm reloc: /Documentation/bpf/llvm_reloc.rst
297
298Clang dependencies for the u32 spill test (xdpwall)
299===================================================
300The xdpwall selftest requires a change in `Clang 14`__.
301
302Without it, the xdpwall selftest will fail and the error message
303from running test_progs will look like:
304
305.. code-block:: console
306
307  test_xdpwall:FAIL:Does LLVM have https://reviews.llvm.org/D109073? unexpected error: -4007
308
309__ https://reviews.llvm.org/D109073
310