1================== 2BPF Selftest Notes 3================== 4General instructions on running selftests can be found in 5`Documentation/bpf/bpf_devel_QA.rst`__. 6 7__ /Documentation/bpf/bpf_devel_QA.rst#q-how-to-run-bpf-selftests 8 9========================= 10Running Selftests in a VM 11========================= 12 13It's now possible to run the selftests using ``tools/testing/selftests/bpf/vmtest.sh``. 14The script tries to ensure that the tests are run with the same environment as they 15would be run post-submit in the CI used by the Maintainers. 16 17This script downloads a suitable Kconfig and VM userspace image from the system used by 18the CI. It builds the kernel (without overwriting your existing Kconfig), recompiles the 19bpf selftests, runs them (by default ``tools/testing/selftests/bpf/test_progs``) and 20saves the resulting output (by default in ``~/.bpf_selftests``). 21 22Script dependencies: 23- clang (preferably built from sources, https://github.com/llvm/llvm-project); 24- pahole (preferably built from sources, https://git.kernel.org/pub/scm/devel/pahole/pahole.git/); 25- qemu; 26- docutils (for ``rst2man``); 27- libcap-devel. 28 29For more information on about using the script, run: 30 31.. code-block:: console 32 33 $ tools/testing/selftests/bpf/vmtest.sh -h 34 35In case of linker errors when running selftests, try using static linking: 36 37.. code-block:: console 38 39 $ LDLIBS=-static vmtest.sh 40 41.. note:: Some distros may not support static linking. 42 43.. note:: The script uses pahole and clang based on host environment setting. 44 If you want to change pahole and llvm, you can change `PATH` environment 45 variable in the beginning of script. 46 47.. note:: The script currently only supports x86_64 and s390x architectures. 48 49Additional information about selftest failures are 50documented here. 51 52profiler[23] test failures with clang/llvm <12.0.0 53================================================== 54 55With clang/llvm <12.0.0, the profiler[23] test may fail. 56The symptom looks like 57 58.. code-block:: c 59 60 // r9 is a pointer to map_value 61 // r7 is a scalar 62 17: bf 96 00 00 00 00 00 00 r6 = r9 63 18: 0f 76 00 00 00 00 00 00 r6 += r7 64 math between map_value pointer and register with unbounded min value is not allowed 65 66 // the instructions below will not be seen in the verifier log 67 19: a5 07 01 00 01 01 00 00 if r7 < 257 goto +1 68 20: bf 96 00 00 00 00 00 00 r6 = r9 69 // r6 is used here 70 71The verifier will reject such code with above error. 72At insn 18 the r7 is indeed unbounded. The later insn 19 checks the bounds and 73the insn 20 undoes map_value addition. It is currently impossible for the 74verifier to understand such speculative pointer arithmetic. 75Hence `this patch`__ addresses it on the compiler side. It was committed on llvm 12. 76 77__ https://reviews.llvm.org/D85570 78 79The corresponding C code 80 81.. code-block:: c 82 83 for (int i = 0; i < MAX_CGROUPS_PATH_DEPTH; i++) { 84 filepart_length = bpf_probe_read_str(payload, ...); 85 if (filepart_length <= MAX_PATH) { 86 barrier_var(filepart_length); // workaround 87 payload += filepart_length; 88 } 89 } 90 91bpf_iter test failures with clang/llvm 10.0.0 92============================================= 93 94With clang/llvm 10.0.0, the following two bpf_iter tests failed: 95 * ``bpf_iter/ipv6_route`` 96 * ``bpf_iter/netlink`` 97 98The symptom for ``bpf_iter/ipv6_route`` looks like 99 100.. code-block:: c 101 102 2: (79) r8 = *(u64 *)(r1 +8) 103 ... 104 14: (bf) r2 = r8 105 15: (0f) r2 += r1 106 ; BPF_SEQ_PRINTF(seq, "%pi6 %02x ", &rt->fib6_dst.addr, rt->fib6_dst.plen); 107 16: (7b) *(u64 *)(r8 +64) = r2 108 only read is supported 109 110The symptom for ``bpf_iter/netlink`` looks like 111 112.. code-block:: c 113 114 ; struct netlink_sock *nlk = ctx->sk; 115 2: (79) r7 = *(u64 *)(r1 +8) 116 ... 117 15: (bf) r2 = r7 118 16: (0f) r2 += r1 119 ; BPF_SEQ_PRINTF(seq, "%pK %-3d ", s, s->sk_protocol); 120 17: (7b) *(u64 *)(r7 +0) = r2 121 only read is supported 122 123This is due to a llvm BPF backend bug. `The fix`__ 124has been pushed to llvm 10.x release branch and will be 125available in 10.0.1. The patch is available in llvm 11.0.0 trunk. 126 127__ https://reviews.llvm.org/D78466 128 129bpf_verif_scale/loop6.o test failure with Clang 12 130================================================== 131 132With Clang 12, the following bpf_verif_scale test failed: 133 * ``bpf_verif_scale/loop6.o`` 134 135The verifier output looks like 136 137.. code-block:: c 138 139 R1 type=ctx expected=fp 140 The sequence of 8193 jumps is too complex. 141 142The reason is compiler generating the following code 143 144.. code-block:: c 145 146 ; for (i = 0; (i < VIRTIO_MAX_SGS) && (i < num); i++) { 147 14: 16 05 40 00 00 00 00 00 if w5 == 0 goto +64 <LBB0_6> 148 15: bc 51 00 00 00 00 00 00 w1 = w5 149 16: 04 01 00 00 ff ff ff ff w1 += -1 150 17: 67 05 00 00 20 00 00 00 r5 <<= 32 151 18: 77 05 00 00 20 00 00 00 r5 >>= 32 152 19: a6 01 01 00 05 00 00 00 if w1 < 5 goto +1 <LBB0_4> 153 20: b7 05 00 00 06 00 00 00 r5 = 6 154 00000000000000a8 <LBB0_4>: 155 21: b7 02 00 00 00 00 00 00 r2 = 0 156 22: b7 01 00 00 00 00 00 00 r1 = 0 157 ; for (i = 0; (i < VIRTIO_MAX_SGS) && (i < num); i++) { 158 23: 7b 1a e0 ff 00 00 00 00 *(u64 *)(r10 - 32) = r1 159 24: 7b 5a c0 ff 00 00 00 00 *(u64 *)(r10 - 64) = r5 160 161Note that insn #15 has w1 = w5 and w1 is refined later but 162r5(w5) is eventually saved on stack at insn #24 for later use. 163This cause later verifier failure. The bug has been `fixed`__ in 164Clang 13. 165 166__ https://reviews.llvm.org/D97479 167 168BPF CO-RE-based tests and Clang version 169======================================= 170 171A set of selftests use BPF target-specific built-ins, which might require 172bleeding-edge Clang versions (Clang 12 nightly at this time). 173 174Few sub-tests of core_reloc test suit (part of test_progs test runner) require 175the following built-ins, listed with corresponding Clang diffs introducing 176them to Clang/LLVM. These sub-tests are going to be skipped if Clang is too 177old to support them, they shouldn't cause build failures or runtime test 178failures: 179 180- __builtin_btf_type_id() [0_, 1_, 2_]; 181- __builtin_preserve_type_info(), __builtin_preserve_enum_value() [3_, 4_]. 182 183.. _0: https://reviews.llvm.org/D74572 184.. _1: https://reviews.llvm.org/D74668 185.. _2: https://reviews.llvm.org/D85174 186.. _3: https://reviews.llvm.org/D83878 187.. _4: https://reviews.llvm.org/D83242 188 189Floating-point tests and Clang version 190====================================== 191 192Certain selftests, e.g. core_reloc, require support for the floating-point 193types, which was introduced in `Clang 13`__. The older Clang versions will 194either crash when compiling these tests, or generate an incorrect BTF. 195 196__ https://reviews.llvm.org/D83289 197 198Kernel function call test and Clang version 199=========================================== 200 201Some selftests (e.g. kfunc_call and bpf_tcp_ca) require a LLVM support 202to generate extern function in BTF. It was introduced in `Clang 13`__. 203 204Without it, the error from compiling bpf selftests looks like: 205 206.. code-block:: console 207 208 libbpf: failed to find BTF for extern 'tcp_slow_start' [25] section: -2 209 210__ https://reviews.llvm.org/D93563 211 212btf_tag test and Clang version 213============================== 214 215The btf_tag selftest requires LLVM support to recognize the btf_decl_tag and 216btf_type_tag attributes. They are introduced in `Clang 14` [0_, 1_]. 217The subtests ``btf_type_tag_user_{mod1, mod2, vmlinux}`` also requires 218pahole version ``1.23``. 219 220Without them, the btf_tag selftest will be skipped and you will observe: 221 222.. code-block:: console 223 224 #<test_num> btf_tag:SKIP 225 226.. _0: https://reviews.llvm.org/D111588 227.. _1: https://reviews.llvm.org/D111199 228 229Clang dependencies for static linking tests 230=========================================== 231 232linked_vars, linked_maps, and linked_funcs tests depend on `Clang fix`__ to 233generate valid BTF information for weak variables. Please make sure you use 234Clang that contains the fix. 235 236__ https://reviews.llvm.org/D100362 237 238Clang relocation changes 239======================== 240 241Clang 13 patch `clang reloc patch`_ made some changes on relocations such 242that existing relocation types are broken into more types and 243each new type corresponds to only one way to resolve relocation. 244See `kernel llvm reloc`_ for more explanation and some examples. 245Using clang 13 to compile old libbpf which has static linker support, 246there will be a compilation failure:: 247 248 libbpf: ELF relo #0 in section #6 has unexpected type 2 in .../bpf_tcp_nogpl.o 249 250Here, ``type 2`` refers to new relocation type ``R_BPF_64_ABS64``. 251To fix this issue, user newer libbpf. 252 253.. Links 254.. _clang reloc patch: https://reviews.llvm.org/D102712 255.. _kernel llvm reloc: /Documentation/bpf/llvm_reloc.rst 256 257Clang dependencies for the u32 spill test (xdpwall) 258=================================================== 259The xdpwall selftest requires a change in `Clang 14`__. 260 261Without it, the xdpwall selftest will fail and the error message 262from running test_progs will look like: 263 264.. code-block:: console 265 266 test_xdpwall:FAIL:Does LLVM have https://reviews.llvm.org/D109073? unexpected error: -4007 267 268__ https://reviews.llvm.org/D109073 269