199aaf53eSYonghong Song================== 299aaf53eSYonghong SongBPF Selftest Notes 399aaf53eSYonghong Song================== 46736aa79SAlan MaguireGeneral instructions on running selftests can be found in 505a98d76SAndrei Matei`Documentation/bpf/bpf_devel_QA.rst`__. 605a98d76SAndrei Matei 705a98d76SAndrei Matei__ /Documentation/bpf/bpf_devel_QA.rst#q-how-to-run-bpf-selftests 805a98d76SAndrei Matei 9*81bfcc3fSDaniel Müller============= 10*81bfcc3fSDaniel MüllerBPF CI System 11*81bfcc3fSDaniel Müller============= 12*81bfcc3fSDaniel Müller 13*81bfcc3fSDaniel MüllerBPF employs a continuous integration (CI) system to check patch submission in an 14*81bfcc3fSDaniel Müllerautomated fashion. The system runs selftests for each patch in a series. Results 15*81bfcc3fSDaniel Müllerare propagated to patchwork, where failures are highlighted similar to 16*81bfcc3fSDaniel Müllerviolations of other checks (such as additional warnings being emitted or a 17*81bfcc3fSDaniel Müller``scripts/checkpatch.pl`` reported deficiency): 18*81bfcc3fSDaniel Müller 19*81bfcc3fSDaniel Müller https://patchwork.kernel.org/project/netdevbpf/list/?delegate=121173 20*81bfcc3fSDaniel Müller 21*81bfcc3fSDaniel MüllerThe CI system executes tests on multiple architectures. It uses a kernel 22*81bfcc3fSDaniel Müllerconfiguration derived from both the generic and architecture specific config 23*81bfcc3fSDaniel Müllerfile fragments below ``tools/testing/selftests/bpf/`` (e.g., ``config`` and 24*81bfcc3fSDaniel Müller``config.x86_64``). 25*81bfcc3fSDaniel Müller 26*81bfcc3fSDaniel MüllerDenylisting Tests 27*81bfcc3fSDaniel Müller================= 28*81bfcc3fSDaniel Müller 29*81bfcc3fSDaniel MüllerIt is possible for some architectures to not have support for all BPF features. 30*81bfcc3fSDaniel MüllerIn such a case tests in CI may fail. An example of such a shortcoming is BPF 31*81bfcc3fSDaniel Müllertrampoline support on IBM's s390x architecture. For cases like this, an in-tree 32*81bfcc3fSDaniel Müllerdeny list file, located at ``tools/testing/selftests/bpf/DENYLIST.<arch>``, can 33*81bfcc3fSDaniel Müllerbe used to prevent the test from running on such an architecture. 34*81bfcc3fSDaniel Müller 35*81bfcc3fSDaniel MüllerIn addition to that, the generic ``tools/testing/selftests/bpf/DENYLIST`` is 36*81bfcc3fSDaniel Müllerhonored on every architecture running tests. 37*81bfcc3fSDaniel Müller 38*81bfcc3fSDaniel MüllerThese files are organized in three columns. The first column lists the test in 39*81bfcc3fSDaniel Müllerquestion. This can be the name of a test suite or of an individual test. The 40*81bfcc3fSDaniel Müllerremaining two columns provide additional meta data that helps identify and 41*81bfcc3fSDaniel Müllerclassify the entry: column two is a copy and paste of the error being reported 42*81bfcc3fSDaniel Müllerwhen running the test in the setting in question. The third column, if 43*81bfcc3fSDaniel Mülleravailable, summarizes the underlying problem. A value of ``trampoline``, for 44*81bfcc3fSDaniel Müllerexample, indicates that lack of trampoline support is causing the test to fail. 45*81bfcc3fSDaniel MüllerThis last entry helps identify tests that can be re-enabled once such support is 46*81bfcc3fSDaniel Mülleradded. 47*81bfcc3fSDaniel Müller 48881949f7SKP Singh========================= 49881949f7SKP SinghRunning Selftests in a VM 50881949f7SKP Singh========================= 51881949f7SKP Singh 52881949f7SKP SinghIt's now possible to run the selftests using ``tools/testing/selftests/bpf/vmtest.sh``. 53881949f7SKP SinghThe script tries to ensure that the tests are run with the same environment as they 54*81bfcc3fSDaniel Müllerwould be run post-submit in the CI used by the Maintainers, with the exception 55*81bfcc3fSDaniel Müllerthat deny lists are not automatically honored. 56881949f7SKP Singh 576c4e777fSDaniel MüllerThis script uses the in-tree kernel configuration and downloads a VM userspace 586c4e777fSDaniel Müllerimage from the system used by the CI. It builds the kernel (without overwriting 596c4e777fSDaniel Mülleryour existing Kconfig), recompiles the bpf selftests, runs them (by default 606c4e777fSDaniel Müller``tools/testing/selftests/bpf/test_progs``) and saves the resulting output (by 616c4e777fSDaniel Müllerdefault in ``~/.bpf_selftests``). 62881949f7SKP Singh 637a188442SEvgeniy LitvinenkoScript dependencies: 647a188442SEvgeniy Litvinenko- clang (preferably built from sources, https://github.com/llvm/llvm-project); 657a188442SEvgeniy Litvinenko- pahole (preferably built from sources, https://git.kernel.org/pub/scm/devel/pahole/pahole.git/); 667a188442SEvgeniy Litvinenko- qemu; 677a188442SEvgeniy Litvinenko- docutils (for ``rst2man``); 687a188442SEvgeniy Litvinenko- libcap-devel. 697a188442SEvgeniy Litvinenko 706c4e777fSDaniel MüllerFor more information about using the script, run: 71881949f7SKP Singh 72881949f7SKP Singh.. code-block:: console 73881949f7SKP Singh 74881949f7SKP Singh $ tools/testing/selftests/bpf/vmtest.sh -h 75881949f7SKP Singh 765ad0a415SKP SinghIn case of linker errors when running selftests, try using static linking: 775ad0a415SKP Singh 785ad0a415SKP Singh.. code-block:: console 795ad0a415SKP Singh 805ad0a415SKP Singh $ LDLIBS=-static vmtest.sh 815ad0a415SKP Singh 825ad0a415SKP Singh.. note:: Some distros may not support static linking. 835ad0a415SKP Singh 84881949f7SKP Singh.. note:: The script uses pahole and clang based on host environment setting. 85881949f7SKP Singh If you want to change pahole and llvm, you can change `PATH` environment 86881949f7SKP Singh variable in the beginning of script. 87881949f7SKP Singh 88e878ae2dSKP Singh.. note:: The script currently only supports x86_64 and s390x architectures. 8999aaf53eSYonghong Song 9099aaf53eSYonghong SongAdditional information about selftest failures are 9199aaf53eSYonghong Songdocumented here. 9299aaf53eSYonghong Song 9303d4d13fSAlexei Starovoitovprofiler[23] test failures with clang/llvm <12.0.0 9403d4d13fSAlexei Starovoitov================================================== 9503d4d13fSAlexei Starovoitov 9603d4d13fSAlexei StarovoitovWith clang/llvm <12.0.0, the profiler[23] test may fail. 9703d4d13fSAlexei StarovoitovThe symptom looks like 9803d4d13fSAlexei Starovoitov 9903d4d13fSAlexei Starovoitov.. code-block:: c 10003d4d13fSAlexei Starovoitov 10103d4d13fSAlexei Starovoitov // r9 is a pointer to map_value 10203d4d13fSAlexei Starovoitov // r7 is a scalar 10303d4d13fSAlexei Starovoitov 17: bf 96 00 00 00 00 00 00 r6 = r9 10403d4d13fSAlexei Starovoitov 18: 0f 76 00 00 00 00 00 00 r6 += r7 10503d4d13fSAlexei Starovoitov math between map_value pointer and register with unbounded min value is not allowed 10603d4d13fSAlexei Starovoitov 10703d4d13fSAlexei Starovoitov // the instructions below will not be seen in the verifier log 10803d4d13fSAlexei Starovoitov 19: a5 07 01 00 01 01 00 00 if r7 < 257 goto +1 10903d4d13fSAlexei Starovoitov 20: bf 96 00 00 00 00 00 00 r6 = r9 11003d4d13fSAlexei Starovoitov // r6 is used here 11103d4d13fSAlexei Starovoitov 11203d4d13fSAlexei StarovoitovThe verifier will reject such code with above error. 11303d4d13fSAlexei StarovoitovAt insn 18 the r7 is indeed unbounded. The later insn 19 checks the bounds and 11403d4d13fSAlexei Starovoitovthe insn 20 undoes map_value addition. It is currently impossible for the 11503d4d13fSAlexei Starovoitovverifier to understand such speculative pointer arithmetic. 1161c26ac6aSAndrei MateiHence `this patch`__ addresses it on the compiler side. It was committed on llvm 12. 1171c26ac6aSAndrei Matei 1181c26ac6aSAndrei Matei__ https://reviews.llvm.org/D85570 11903d4d13fSAlexei Starovoitov 12003d4d13fSAlexei StarovoitovThe corresponding C code 1211c26ac6aSAndrei Matei 12203d4d13fSAlexei Starovoitov.. code-block:: c 12303d4d13fSAlexei Starovoitov 12403d4d13fSAlexei Starovoitov for (int i = 0; i < MAX_CGROUPS_PATH_DEPTH; i++) { 12503d4d13fSAlexei Starovoitov filepart_length = bpf_probe_read_str(payload, ...); 12603d4d13fSAlexei Starovoitov if (filepart_length <= MAX_PATH) { 12703d4d13fSAlexei Starovoitov barrier_var(filepart_length); // workaround 12803d4d13fSAlexei Starovoitov payload += filepart_length; 12903d4d13fSAlexei Starovoitov } 13003d4d13fSAlexei Starovoitov } 13103d4d13fSAlexei Starovoitov 13299aaf53eSYonghong Songbpf_iter test failures with clang/llvm 10.0.0 13399aaf53eSYonghong Song============================================= 13499aaf53eSYonghong Song 13599aaf53eSYonghong SongWith clang/llvm 10.0.0, the following two bpf_iter tests failed: 13699aaf53eSYonghong Song * ``bpf_iter/ipv6_route`` 13799aaf53eSYonghong Song * ``bpf_iter/netlink`` 13899aaf53eSYonghong Song 13999aaf53eSYonghong SongThe symptom for ``bpf_iter/ipv6_route`` looks like 14099aaf53eSYonghong Song 14199aaf53eSYonghong Song.. code-block:: c 14299aaf53eSYonghong Song 14399aaf53eSYonghong Song 2: (79) r8 = *(u64 *)(r1 +8) 14499aaf53eSYonghong Song ... 14599aaf53eSYonghong Song 14: (bf) r2 = r8 14699aaf53eSYonghong Song 15: (0f) r2 += r1 14799aaf53eSYonghong Song ; BPF_SEQ_PRINTF(seq, "%pi6 %02x ", &rt->fib6_dst.addr, rt->fib6_dst.plen); 14899aaf53eSYonghong Song 16: (7b) *(u64 *)(r8 +64) = r2 14999aaf53eSYonghong Song only read is supported 15099aaf53eSYonghong Song 15199aaf53eSYonghong SongThe symptom for ``bpf_iter/netlink`` looks like 15299aaf53eSYonghong Song 15399aaf53eSYonghong Song.. code-block:: c 15499aaf53eSYonghong Song 15599aaf53eSYonghong Song ; struct netlink_sock *nlk = ctx->sk; 15699aaf53eSYonghong Song 2: (79) r7 = *(u64 *)(r1 +8) 15799aaf53eSYonghong Song ... 15899aaf53eSYonghong Song 15: (bf) r2 = r7 15999aaf53eSYonghong Song 16: (0f) r2 += r1 16099aaf53eSYonghong Song ; BPF_SEQ_PRINTF(seq, "%pK %-3d ", s, s->sk_protocol); 16199aaf53eSYonghong Song 17: (7b) *(u64 *)(r7 +0) = r2 16299aaf53eSYonghong Song only read is supported 16399aaf53eSYonghong Song 1641c26ac6aSAndrei MateiThis is due to a llvm BPF backend bug. `The fix`__ 16599aaf53eSYonghong Songhas been pushed to llvm 10.x release branch and will be 1661c26ac6aSAndrei Mateiavailable in 10.0.1. The patch is available in llvm 11.0.0 trunk. 1671c26ac6aSAndrei Matei 1681c26ac6aSAndrei Matei__ https://reviews.llvm.org/D78466 169149cb339SAndrii Nakryiko 170afef88e6SDaniel Müllerbpf_verif_scale/loop6.bpf.o test failure with Clang 12 171afef88e6SDaniel Müller====================================================== 17286a35af6SYonghong Song 17386a35af6SYonghong SongWith Clang 12, the following bpf_verif_scale test failed: 174afef88e6SDaniel Müller * ``bpf_verif_scale/loop6.bpf.o`` 17586a35af6SYonghong Song 17686a35af6SYonghong SongThe verifier output looks like 17786a35af6SYonghong Song 17886a35af6SYonghong Song.. code-block:: c 17986a35af6SYonghong Song 18086a35af6SYonghong Song R1 type=ctx expected=fp 18186a35af6SYonghong Song The sequence of 8193 jumps is too complex. 18286a35af6SYonghong Song 18386a35af6SYonghong SongThe reason is compiler generating the following code 18486a35af6SYonghong Song 18586a35af6SYonghong Song.. code-block:: c 18686a35af6SYonghong Song 18786a35af6SYonghong Song ; for (i = 0; (i < VIRTIO_MAX_SGS) && (i < num); i++) { 18886a35af6SYonghong Song 14: 16 05 40 00 00 00 00 00 if w5 == 0 goto +64 <LBB0_6> 18986a35af6SYonghong Song 15: bc 51 00 00 00 00 00 00 w1 = w5 19086a35af6SYonghong Song 16: 04 01 00 00 ff ff ff ff w1 += -1 19186a35af6SYonghong Song 17: 67 05 00 00 20 00 00 00 r5 <<= 32 19286a35af6SYonghong Song 18: 77 05 00 00 20 00 00 00 r5 >>= 32 19386a35af6SYonghong Song 19: a6 01 01 00 05 00 00 00 if w1 < 5 goto +1 <LBB0_4> 19486a35af6SYonghong Song 20: b7 05 00 00 06 00 00 00 r5 = 6 19586a35af6SYonghong Song 00000000000000a8 <LBB0_4>: 19686a35af6SYonghong Song 21: b7 02 00 00 00 00 00 00 r2 = 0 19786a35af6SYonghong Song 22: b7 01 00 00 00 00 00 00 r1 = 0 19886a35af6SYonghong Song ; for (i = 0; (i < VIRTIO_MAX_SGS) && (i < num); i++) { 19986a35af6SYonghong Song 23: 7b 1a e0 ff 00 00 00 00 *(u64 *)(r10 - 32) = r1 20086a35af6SYonghong Song 24: 7b 5a c0 ff 00 00 00 00 *(u64 *)(r10 - 64) = r5 20186a35af6SYonghong Song 20286a35af6SYonghong SongNote that insn #15 has w1 = w5 and w1 is refined later but 20386a35af6SYonghong Songr5(w5) is eventually saved on stack at insn #24 for later use. 20486a35af6SYonghong SongThis cause later verifier failure. The bug has been `fixed`__ in 20586a35af6SYonghong SongClang 13. 20686a35af6SYonghong Song 20786a35af6SYonghong Song__ https://reviews.llvm.org/D97479 20886a35af6SYonghong Song 209149cb339SAndrii NakryikoBPF CO-RE-based tests and Clang version 210149cb339SAndrii Nakryiko======================================= 211149cb339SAndrii Nakryiko 212149cb339SAndrii NakryikoA set of selftests use BPF target-specific built-ins, which might require 213149cb339SAndrii Nakryikobleeding-edge Clang versions (Clang 12 nightly at this time). 214149cb339SAndrii Nakryiko 215149cb339SAndrii NakryikoFew sub-tests of core_reloc test suit (part of test_progs test runner) require 216149cb339SAndrii Nakryikothe following built-ins, listed with corresponding Clang diffs introducing 217149cb339SAndrii Nakryikothem to Clang/LLVM. These sub-tests are going to be skipped if Clang is too 218149cb339SAndrii Nakryikoold to support them, they shouldn't cause build failures or runtime test 219149cb339SAndrii Nakryikofailures: 220149cb339SAndrii Nakryiko 2211c26ac6aSAndrei Matei- __builtin_btf_type_id() [0_, 1_, 2_]; 2221c26ac6aSAndrei Matei- __builtin_preserve_type_info(), __builtin_preserve_enum_value() [3_, 4_]. 223149cb339SAndrii Nakryiko 2241c26ac6aSAndrei Matei.. _0: https://reviews.llvm.org/D74572 2251c26ac6aSAndrei Matei.. _1: https://reviews.llvm.org/D74668 2261c26ac6aSAndrei Matei.. _2: https://reviews.llvm.org/D85174 2271c26ac6aSAndrei Matei.. _3: https://reviews.llvm.org/D83878 2281c26ac6aSAndrei Matei.. _4: https://reviews.llvm.org/D83242 2293fcd50d6SIlya Leoshkevich 2303fcd50d6SIlya LeoshkevichFloating-point tests and Clang version 2313fcd50d6SIlya Leoshkevich====================================== 2323fcd50d6SIlya Leoshkevich 2333fcd50d6SIlya LeoshkevichCertain selftests, e.g. core_reloc, require support for the floating-point 2343fcd50d6SIlya Leoshkevichtypes, which was introduced in `Clang 13`__. The older Clang versions will 2353fcd50d6SIlya Leoshkevicheither crash when compiling these tests, or generate an incorrect BTF. 2363fcd50d6SIlya Leoshkevich 2373fcd50d6SIlya Leoshkevich__ https://reviews.llvm.org/D83289 2382ba4badcSMartin KaFai Lau 2392ba4badcSMartin KaFai LauKernel function call test and Clang version 2402ba4badcSMartin KaFai Lau=========================================== 2412ba4badcSMartin KaFai Lau 2422ba4badcSMartin KaFai LauSome selftests (e.g. kfunc_call and bpf_tcp_ca) require a LLVM support 2432ba4badcSMartin KaFai Lauto generate extern function in BTF. It was introduced in `Clang 13`__. 2442ba4badcSMartin KaFai Lau 2452ba4badcSMartin KaFai LauWithout it, the error from compiling bpf selftests looks like: 2462ba4badcSMartin KaFai Lau 2472ba4badcSMartin KaFai Lau.. code-block:: console 2482ba4badcSMartin KaFai Lau 2492ba4badcSMartin KaFai Lau libbpf: failed to find BTF for extern 'tcp_slow_start' [25] section: -2 2502ba4badcSMartin KaFai Lau 2512ba4badcSMartin KaFai Lau__ https://reviews.llvm.org/D93563 252a9dab4e4SAndrii Nakryiko 2532220ecf5SYonghong Songbtf_tag test and Clang version 2542220ecf5SYonghong Song============================== 2552220ecf5SYonghong Song 2563f1d0dc0SYonghong SongThe btf_tag selftest requires LLVM support to recognize the btf_decl_tag and 2573f1d0dc0SYonghong Songbtf_type_tag attributes. They are introduced in `Clang 14` [0_, 1_]. 25867ef7e1aSYonghong SongThe subtests ``btf_type_tag_user_{mod1, mod2, vmlinux}`` also requires 25967ef7e1aSYonghong Songpahole version ``1.23``. 2602220ecf5SYonghong Song 2613f1d0dc0SYonghong SongWithout them, the btf_tag selftest will be skipped and you will observe: 2622220ecf5SYonghong Song 2632220ecf5SYonghong Song.. code-block:: console 2642220ecf5SYonghong Song 2652220ecf5SYonghong Song #<test_num> btf_tag:SKIP 2662220ecf5SYonghong Song 2673f1d0dc0SYonghong Song.. _0: https://reviews.llvm.org/D111588 2683f1d0dc0SYonghong Song.. _1: https://reviews.llvm.org/D111199 2692220ecf5SYonghong Song 270a9dab4e4SAndrii NakryikoClang dependencies for static linking tests 271a9dab4e4SAndrii Nakryiko=========================================== 272a9dab4e4SAndrii Nakryiko 273a9dab4e4SAndrii Nakryikolinked_vars, linked_maps, and linked_funcs tests depend on `Clang fix`__ to 274a9dab4e4SAndrii Nakryikogenerate valid BTF information for weak variables. Please make sure you use 275a9dab4e4SAndrii NakryikoClang that contains the fix. 276a9dab4e4SAndrii Nakryiko 277a9dab4e4SAndrii Nakryiko__ https://reviews.llvm.org/D100362 278fc8c262eSYonghong Song 279fc8c262eSYonghong SongClang relocation changes 280fc8c262eSYonghong Song======================== 281fc8c262eSYonghong Song 282fc8c262eSYonghong SongClang 13 patch `clang reloc patch`_ made some changes on relocations such 283fc8c262eSYonghong Songthat existing relocation types are broken into more types and 284fc8c262eSYonghong Songeach new type corresponds to only one way to resolve relocation. 285fc8c262eSYonghong SongSee `kernel llvm reloc`_ for more explanation and some examples. 286fc8c262eSYonghong SongUsing clang 13 to compile old libbpf which has static linker support, 287fc8c262eSYonghong Songthere will be a compilation failure:: 288fc8c262eSYonghong Song 289afef88e6SDaniel Müller libbpf: ELF relo #0 in section #6 has unexpected type 2 in .../bpf_tcp_nogpl.bpf.o 290fc8c262eSYonghong Song 291fc8c262eSYonghong SongHere, ``type 2`` refers to new relocation type ``R_BPF_64_ABS64``. 292fc8c262eSYonghong SongTo fix this issue, user newer libbpf. 293fc8c262eSYonghong Song 294fc8c262eSYonghong Song.. Links 295fc8c262eSYonghong Song.. _clang reloc patch: https://reviews.llvm.org/D102712 296fc8c262eSYonghong Song.. _kernel llvm reloc: /Documentation/bpf/llvm_reloc.rst 29754ea6079SMartin KaFai Lau 29854ea6079SMartin KaFai LauClang dependencies for the u32 spill test (xdpwall) 29954ea6079SMartin KaFai Lau=================================================== 30054ea6079SMartin KaFai LauThe xdpwall selftest requires a change in `Clang 14`__. 30154ea6079SMartin KaFai Lau 30254ea6079SMartin KaFai LauWithout it, the xdpwall selftest will fail and the error message 30354ea6079SMartin KaFai Laufrom running test_progs will look like: 30454ea6079SMartin KaFai Lau 30554ea6079SMartin KaFai Lau.. code-block:: console 30654ea6079SMartin KaFai Lau 30754ea6079SMartin KaFai Lau test_xdpwall:FAIL:Does LLVM have https://reviews.llvm.org/D109073? unexpected error: -4007 30854ea6079SMartin KaFai Lau 30954ea6079SMartin KaFai Lau__ https://reviews.llvm.org/D109073 310