1================== 2BPF Selftest Notes 3================== 4General instructions on running selftests can be found in 5`Documentation/bpf/bpf_devel_QA.rst`__. 6 7__ /Documentation/bpf/bpf_devel_QA.rst#q-how-to-run-bpf-selftests 8 9========================= 10Running Selftests in a VM 11========================= 12 13It's now possible to run the selftests using ``tools/testing/selftests/bpf/vmtest.sh``. 14The script tries to ensure that the tests are run with the same environment as they 15would be run post-submit in the CI used by the Maintainers. 16 17This script downloads a suitable Kconfig and VM userspace image from the system used by 18the CI. It builds the kernel (without overwriting your existing Kconfig), recompiles the 19bpf selftests, runs them (by default ``tools/testing/selftests/bpf/test_progs``) and 20saves the resulting output (by default in ``~/.bpf_selftests``). 21 22Script dependencies: 23- clang (preferably built from sources, https://github.com/llvm/llvm-project); 24- pahole (preferably built from sources, https://git.kernel.org/pub/scm/devel/pahole/pahole.git/); 25- qemu; 26- docutils (for ``rst2man``); 27- libcap-devel. 28 29For more information on about using the script, run: 30 31.. code-block:: console 32 33 $ tools/testing/selftests/bpf/vmtest.sh -h 34 35.. note:: The script uses pahole and clang based on host environment setting. 36 If you want to change pahole and llvm, you can change `PATH` environment 37 variable in the beginning of script. 38 39.. note:: The script currently only supports x86_64. 40 41Additional information about selftest failures are 42documented here. 43 44profiler[23] test failures with clang/llvm <12.0.0 45================================================== 46 47With clang/llvm <12.0.0, the profiler[23] test may fail. 48The symptom looks like 49 50.. code-block:: c 51 52 // r9 is a pointer to map_value 53 // r7 is a scalar 54 17: bf 96 00 00 00 00 00 00 r6 = r9 55 18: 0f 76 00 00 00 00 00 00 r6 += r7 56 math between map_value pointer and register with unbounded min value is not allowed 57 58 // the instructions below will not be seen in the verifier log 59 19: a5 07 01 00 01 01 00 00 if r7 < 257 goto +1 60 20: bf 96 00 00 00 00 00 00 r6 = r9 61 // r6 is used here 62 63The verifier will reject such code with above error. 64At insn 18 the r7 is indeed unbounded. The later insn 19 checks the bounds and 65the insn 20 undoes map_value addition. It is currently impossible for the 66verifier to understand such speculative pointer arithmetic. 67Hence `this patch`__ addresses it on the compiler side. It was committed on llvm 12. 68 69__ https://reviews.llvm.org/D85570 70 71The corresponding C code 72 73.. code-block:: c 74 75 for (int i = 0; i < MAX_CGROUPS_PATH_DEPTH; i++) { 76 filepart_length = bpf_probe_read_str(payload, ...); 77 if (filepart_length <= MAX_PATH) { 78 barrier_var(filepart_length); // workaround 79 payload += filepart_length; 80 } 81 } 82 83bpf_iter test failures with clang/llvm 10.0.0 84============================================= 85 86With clang/llvm 10.0.0, the following two bpf_iter tests failed: 87 * ``bpf_iter/ipv6_route`` 88 * ``bpf_iter/netlink`` 89 90The symptom for ``bpf_iter/ipv6_route`` looks like 91 92.. code-block:: c 93 94 2: (79) r8 = *(u64 *)(r1 +8) 95 ... 96 14: (bf) r2 = r8 97 15: (0f) r2 += r1 98 ; BPF_SEQ_PRINTF(seq, "%pi6 %02x ", &rt->fib6_dst.addr, rt->fib6_dst.plen); 99 16: (7b) *(u64 *)(r8 +64) = r2 100 only read is supported 101 102The symptom for ``bpf_iter/netlink`` looks like 103 104.. code-block:: c 105 106 ; struct netlink_sock *nlk = ctx->sk; 107 2: (79) r7 = *(u64 *)(r1 +8) 108 ... 109 15: (bf) r2 = r7 110 16: (0f) r2 += r1 111 ; BPF_SEQ_PRINTF(seq, "%pK %-3d ", s, s->sk_protocol); 112 17: (7b) *(u64 *)(r7 +0) = r2 113 only read is supported 114 115This is due to a llvm BPF backend bug. `The fix`__ 116has been pushed to llvm 10.x release branch and will be 117available in 10.0.1. The patch is available in llvm 11.0.0 trunk. 118 119__ https://reviews.llvm.org/D78466 120 121bpf_verif_scale/loop6.o test failure with Clang 12 122================================================== 123 124With Clang 12, the following bpf_verif_scale test failed: 125 * ``bpf_verif_scale/loop6.o`` 126 127The verifier output looks like 128 129.. code-block:: c 130 131 R1 type=ctx expected=fp 132 The sequence of 8193 jumps is too complex. 133 134The reason is compiler generating the following code 135 136.. code-block:: c 137 138 ; for (i = 0; (i < VIRTIO_MAX_SGS) && (i < num); i++) { 139 14: 16 05 40 00 00 00 00 00 if w5 == 0 goto +64 <LBB0_6> 140 15: bc 51 00 00 00 00 00 00 w1 = w5 141 16: 04 01 00 00 ff ff ff ff w1 += -1 142 17: 67 05 00 00 20 00 00 00 r5 <<= 32 143 18: 77 05 00 00 20 00 00 00 r5 >>= 32 144 19: a6 01 01 00 05 00 00 00 if w1 < 5 goto +1 <LBB0_4> 145 20: b7 05 00 00 06 00 00 00 r5 = 6 146 00000000000000a8 <LBB0_4>: 147 21: b7 02 00 00 00 00 00 00 r2 = 0 148 22: b7 01 00 00 00 00 00 00 r1 = 0 149 ; for (i = 0; (i < VIRTIO_MAX_SGS) && (i < num); i++) { 150 23: 7b 1a e0 ff 00 00 00 00 *(u64 *)(r10 - 32) = r1 151 24: 7b 5a c0 ff 00 00 00 00 *(u64 *)(r10 - 64) = r5 152 153Note that insn #15 has w1 = w5 and w1 is refined later but 154r5(w5) is eventually saved on stack at insn #24 for later use. 155This cause later verifier failure. The bug has been `fixed`__ in 156Clang 13. 157 158__ https://reviews.llvm.org/D97479 159 160BPF CO-RE-based tests and Clang version 161======================================= 162 163A set of selftests use BPF target-specific built-ins, which might require 164bleeding-edge Clang versions (Clang 12 nightly at this time). 165 166Few sub-tests of core_reloc test suit (part of test_progs test runner) require 167the following built-ins, listed with corresponding Clang diffs introducing 168them to Clang/LLVM. These sub-tests are going to be skipped if Clang is too 169old to support them, they shouldn't cause build failures or runtime test 170failures: 171 172- __builtin_btf_type_id() [0_, 1_, 2_]; 173- __builtin_preserve_type_info(), __builtin_preserve_enum_value() [3_, 4_]. 174 175.. _0: https://reviews.llvm.org/D74572 176.. _1: https://reviews.llvm.org/D74668 177.. _2: https://reviews.llvm.org/D85174 178.. _3: https://reviews.llvm.org/D83878 179.. _4: https://reviews.llvm.org/D83242 180 181Floating-point tests and Clang version 182====================================== 183 184Certain selftests, e.g. core_reloc, require support for the floating-point 185types, which was introduced in `Clang 13`__. The older Clang versions will 186either crash when compiling these tests, or generate an incorrect BTF. 187 188__ https://reviews.llvm.org/D83289 189 190Kernel function call test and Clang version 191=========================================== 192 193Some selftests (e.g. kfunc_call and bpf_tcp_ca) require a LLVM support 194to generate extern function in BTF. It was introduced in `Clang 13`__. 195 196Without it, the error from compiling bpf selftests looks like: 197 198.. code-block:: console 199 200 libbpf: failed to find BTF for extern 'tcp_slow_start' [25] section: -2 201 202__ https://reviews.llvm.org/D93563 203 204Clang dependencies for static linking tests 205=========================================== 206 207linked_vars, linked_maps, and linked_funcs tests depend on `Clang fix`__ to 208generate valid BTF information for weak variables. Please make sure you use 209Clang that contains the fix. 210 211__ https://reviews.llvm.org/D100362 212 213Clang relocation changes 214======================== 215 216Clang 13 patch `clang reloc patch`_ made some changes on relocations such 217that existing relocation types are broken into more types and 218each new type corresponds to only one way to resolve relocation. 219See `kernel llvm reloc`_ for more explanation and some examples. 220Using clang 13 to compile old libbpf which has static linker support, 221there will be a compilation failure:: 222 223 libbpf: ELF relo #0 in section #6 has unexpected type 2 in .../bpf_tcp_nogpl.o 224 225Here, ``type 2`` refers to new relocation type ``R_BPF_64_ABS64``. 226To fix this issue, user newer libbpf. 227 228.. Links 229.. _clang reloc patch: https://reviews.llvm.org/D102712 230.. _kernel llvm reloc: /Documentation/bpf/llvm_reloc.rst 231