1==================
2BPF Selftest Notes
3==================
4General instructions on running selftests can be found in
5`Documentation/bpf/bpf_devel_QA.rst`__.
6
7__ /Documentation/bpf/bpf_devel_QA.rst#q-how-to-run-bpf-selftests
8
9=========================
10Running Selftests in a VM
11=========================
12
13It's now possible to run the selftests using ``tools/testing/selftests/bpf/vmtest.sh``.
14The script tries to ensure that the tests are run with the same environment as they
15would be run post-submit in the CI used by the Maintainers.
16
17This script downloads a suitable Kconfig and VM userspace image from the system used by
18the CI. It builds the kernel (without overwriting your existing Kconfig), recompiles the
19bpf selftests, runs them (by default ``tools/testing/selftests/bpf/test_progs``) and
20saves the resulting output (by default in ``~/.bpf_selftests``).
21
22Script dependencies:
23- clang (preferably built from sources, https://github.com/llvm/llvm-project);
24- pahole (preferably built from sources, https://git.kernel.org/pub/scm/devel/pahole/pahole.git/);
25- qemu;
26- docutils (for ``rst2man``);
27- libcap-devel.
28
29For more information on about using the script, run:
30
31.. code-block:: console
32
33  $ tools/testing/selftests/bpf/vmtest.sh -h
34
35In case of linker errors when running selftests, try using static linking:
36
37.. code-block:: console
38
39  $ LDLIBS=-static vmtest.sh
40
41.. note:: Some distros may not support static linking.
42
43.. note:: The script uses pahole and clang based on host environment setting.
44          If you want to change pahole and llvm, you can change `PATH` environment
45          variable in the beginning of script.
46
47.. note:: The script currently only supports x86_64 and s390x architectures.
48
49Additional information about selftest failures are
50documented here.
51
52profiler[23] test failures with clang/llvm <12.0.0
53==================================================
54
55With clang/llvm <12.0.0, the profiler[23] test may fail.
56The symptom looks like
57
58.. code-block:: c
59
60  // r9 is a pointer to map_value
61  // r7 is a scalar
62  17:       bf 96 00 00 00 00 00 00 r6 = r9
63  18:       0f 76 00 00 00 00 00 00 r6 += r7
64  math between map_value pointer and register with unbounded min value is not allowed
65
66  // the instructions below will not be seen in the verifier log
67  19:       a5 07 01 00 01 01 00 00 if r7 < 257 goto +1
68  20:       bf 96 00 00 00 00 00 00 r6 = r9
69  // r6 is used here
70
71The verifier will reject such code with above error.
72At insn 18 the r7 is indeed unbounded. The later insn 19 checks the bounds and
73the insn 20 undoes map_value addition. It is currently impossible for the
74verifier to understand such speculative pointer arithmetic.
75Hence `this patch`__ addresses it on the compiler side. It was committed on llvm 12.
76
77__ https://reviews.llvm.org/D85570
78
79The corresponding C code
80
81.. code-block:: c
82
83  for (int i = 0; i < MAX_CGROUPS_PATH_DEPTH; i++) {
84          filepart_length = bpf_probe_read_str(payload, ...);
85          if (filepart_length <= MAX_PATH) {
86                  barrier_var(filepart_length); // workaround
87                  payload += filepart_length;
88          }
89  }
90
91bpf_iter test failures with clang/llvm 10.0.0
92=============================================
93
94With clang/llvm 10.0.0, the following two bpf_iter tests failed:
95  * ``bpf_iter/ipv6_route``
96  * ``bpf_iter/netlink``
97
98The symptom for ``bpf_iter/ipv6_route`` looks like
99
100.. code-block:: c
101
102  2: (79) r8 = *(u64 *)(r1 +8)
103  ...
104  14: (bf) r2 = r8
105  15: (0f) r2 += r1
106  ; BPF_SEQ_PRINTF(seq, "%pi6 %02x ", &rt->fib6_dst.addr, rt->fib6_dst.plen);
107  16: (7b) *(u64 *)(r8 +64) = r2
108  only read is supported
109
110The symptom for ``bpf_iter/netlink`` looks like
111
112.. code-block:: c
113
114  ; struct netlink_sock *nlk = ctx->sk;
115  2: (79) r7 = *(u64 *)(r1 +8)
116  ...
117  15: (bf) r2 = r7
118  16: (0f) r2 += r1
119  ; BPF_SEQ_PRINTF(seq, "%pK %-3d ", s, s->sk_protocol);
120  17: (7b) *(u64 *)(r7 +0) = r2
121  only read is supported
122
123This is due to a llvm BPF backend bug. `The fix`__
124has been pushed to llvm 10.x release branch and will be
125available in 10.0.1. The patch is available in llvm 11.0.0 trunk.
126
127__  https://reviews.llvm.org/D78466
128
129bpf_verif_scale/loop6.o test failure with Clang 12
130==================================================
131
132With Clang 12, the following bpf_verif_scale test failed:
133  * ``bpf_verif_scale/loop6.o``
134
135The verifier output looks like
136
137.. code-block:: c
138
139  R1 type=ctx expected=fp
140  The sequence of 8193 jumps is too complex.
141
142The reason is compiler generating the following code
143
144.. code-block:: c
145
146  ;       for (i = 0; (i < VIRTIO_MAX_SGS) && (i < num); i++) {
147      14:       16 05 40 00 00 00 00 00 if w5 == 0 goto +64 <LBB0_6>
148      15:       bc 51 00 00 00 00 00 00 w1 = w5
149      16:       04 01 00 00 ff ff ff ff w1 += -1
150      17:       67 05 00 00 20 00 00 00 r5 <<= 32
151      18:       77 05 00 00 20 00 00 00 r5 >>= 32
152      19:       a6 01 01 00 05 00 00 00 if w1 < 5 goto +1 <LBB0_4>
153      20:       b7 05 00 00 06 00 00 00 r5 = 6
154  00000000000000a8 <LBB0_4>:
155      21:       b7 02 00 00 00 00 00 00 r2 = 0
156      22:       b7 01 00 00 00 00 00 00 r1 = 0
157  ;       for (i = 0; (i < VIRTIO_MAX_SGS) && (i < num); i++) {
158      23:       7b 1a e0 ff 00 00 00 00 *(u64 *)(r10 - 32) = r1
159      24:       7b 5a c0 ff 00 00 00 00 *(u64 *)(r10 - 64) = r5
160
161Note that insn #15 has w1 = w5 and w1 is refined later but
162r5(w5) is eventually saved on stack at insn #24 for later use.
163This cause later verifier failure. The bug has been `fixed`__ in
164Clang 13.
165
166__  https://reviews.llvm.org/D97479
167
168BPF CO-RE-based tests and Clang version
169=======================================
170
171A set of selftests use BPF target-specific built-ins, which might require
172bleeding-edge Clang versions (Clang 12 nightly at this time).
173
174Few sub-tests of core_reloc test suit (part of test_progs test runner) require
175the following built-ins, listed with corresponding Clang diffs introducing
176them to Clang/LLVM. These sub-tests are going to be skipped if Clang is too
177old to support them, they shouldn't cause build failures or runtime test
178failures:
179
180- __builtin_btf_type_id() [0_, 1_, 2_];
181- __builtin_preserve_type_info(), __builtin_preserve_enum_value() [3_, 4_].
182
183.. _0: https://reviews.llvm.org/D74572
184.. _1: https://reviews.llvm.org/D74668
185.. _2: https://reviews.llvm.org/D85174
186.. _3: https://reviews.llvm.org/D83878
187.. _4: https://reviews.llvm.org/D83242
188
189Floating-point tests and Clang version
190======================================
191
192Certain selftests, e.g. core_reloc, require support for the floating-point
193types, which was introduced in `Clang 13`__. The older Clang versions will
194either crash when compiling these tests, or generate an incorrect BTF.
195
196__  https://reviews.llvm.org/D83289
197
198Kernel function call test and Clang version
199===========================================
200
201Some selftests (e.g. kfunc_call and bpf_tcp_ca) require a LLVM support
202to generate extern function in BTF.  It was introduced in `Clang 13`__.
203
204Without it, the error from compiling bpf selftests looks like:
205
206.. code-block:: console
207
208  libbpf: failed to find BTF for extern 'tcp_slow_start' [25] section: -2
209
210__ https://reviews.llvm.org/D93563
211
212btf_tag test and Clang version
213==============================
214
215The btf_tag selftest requires LLVM support to recognize the btf_decl_tag and
216btf_type_tag attributes. They are introduced in `Clang 14` [0_, 1_].
217The subtests ``btf_type_tag_user_{mod1, mod2, vmlinux}`` also requires
218pahole version ``1.23``.
219
220Without them, the btf_tag selftest will be skipped and you will observe:
221
222.. code-block:: console
223
224  #<test_num> btf_tag:SKIP
225
226.. _0: https://reviews.llvm.org/D111588
227.. _1: https://reviews.llvm.org/D111199
228
229Clang dependencies for static linking tests
230===========================================
231
232linked_vars, linked_maps, and linked_funcs tests depend on `Clang fix`__ to
233generate valid BTF information for weak variables. Please make sure you use
234Clang that contains the fix.
235
236__ https://reviews.llvm.org/D100362
237
238Clang relocation changes
239========================
240
241Clang 13 patch `clang reloc patch`_  made some changes on relocations such
242that existing relocation types are broken into more types and
243each new type corresponds to only one way to resolve relocation.
244See `kernel llvm reloc`_ for more explanation and some examples.
245Using clang 13 to compile old libbpf which has static linker support,
246there will be a compilation failure::
247
248  libbpf: ELF relo #0 in section #6 has unexpected type 2 in .../bpf_tcp_nogpl.o
249
250Here, ``type 2`` refers to new relocation type ``R_BPF_64_ABS64``.
251To fix this issue, user newer libbpf.
252
253.. Links
254.. _clang reloc patch: https://reviews.llvm.org/D102712
255.. _kernel llvm reloc: /Documentation/bpf/llvm_reloc.rst
256
257Clang dependencies for the u32 spill test (xdpwall)
258===================================================
259The xdpwall selftest requires a change in `Clang 14`__.
260
261Without it, the xdpwall selftest will fail and the error message
262from running test_progs will look like:
263
264.. code-block:: console
265
266  test_xdpwall:FAIL:Does LLVM have https://reviews.llvm.org/D109073? unexpected error: -4007
267
268__ https://reviews.llvm.org/D109073
269