1=================== 2Testing BPF on s390 3=================== 4 51. Introduction 6*************** 7 8IBM Z are mainframe computers, which are descendants of IBM System/360 from 9year 1964. They are supported by the Linux kernel under the name "s390". This 10document describes how to test BPF in an s390 QEMU guest. 11 122. One-time setup 13***************** 14 15The following is required to build and run the test suite: 16 17 * s390 GCC 18 * s390 development headers and libraries 19 * Clang with BPF support 20 * QEMU with s390 support 21 * Disk image with s390 rootfs 22 23Debian supports installing compiler and libraries for s390 out of the box. 24Users of other distros may use debootstrap in order to set up a Debian chroot:: 25 26 sudo debootstrap \ 27 --variant=minbase \ 28 --include=sudo \ 29 testing \ 30 ./s390-toolchain 31 sudo mount --rbind /dev ./s390-toolchain/dev 32 sudo mount --rbind /proc ./s390-toolchain/proc 33 sudo mount --rbind /sys ./s390-toolchain/sys 34 sudo chroot ./s390-toolchain 35 36Once on Debian, the build prerequisites can be installed as follows:: 37 38 sudo dpkg --add-architecture s390x 39 sudo apt-get update 40 sudo apt-get install \ 41 bc \ 42 bison \ 43 cmake \ 44 debootstrap \ 45 dwarves \ 46 flex \ 47 g++ \ 48 gcc \ 49 g++-s390x-linux-gnu \ 50 gcc-s390x-linux-gnu \ 51 gdb-multiarch \ 52 git \ 53 make \ 54 python3 \ 55 qemu-system-misc \ 56 qemu-utils \ 57 rsync \ 58 libcap-dev:s390x \ 59 libelf-dev:s390x \ 60 libncurses-dev 61 62Latest Clang targeting BPF can be installed as follows:: 63 64 git clone https://github.com/llvm/llvm-project.git 65 ln -s ../../clang llvm-project/llvm/tools/ 66 mkdir llvm-project-build 67 cd llvm-project-build 68 cmake \ 69 -DLLVM_TARGETS_TO_BUILD=BPF \ 70 -DCMAKE_BUILD_TYPE=Release \ 71 -DCMAKE_INSTALL_PREFIX=/opt/clang-bpf \ 72 ../llvm-project/llvm 73 make 74 sudo make install 75 export PATH=/opt/clang-bpf/bin:$PATH 76 77The disk image can be prepared using a loopback mount and debootstrap:: 78 79 qemu-img create -f raw ./s390.img 1G 80 sudo losetup -f ./s390.img 81 sudo mkfs.ext4 /dev/loopX 82 mkdir ./s390.rootfs 83 sudo mount /dev/loopX ./s390.rootfs 84 sudo debootstrap \ 85 --foreign \ 86 --arch=s390x \ 87 --variant=minbase \ 88 --include=" \ 89 iproute2, \ 90 iputils-ping, \ 91 isc-dhcp-client, \ 92 kmod, \ 93 libcap2, \ 94 libelf1, \ 95 netcat, \ 96 procps" \ 97 testing \ 98 ./s390.rootfs 99 sudo umount ./s390.rootfs 100 sudo losetup -d /dev/loopX 101 1023. Compilation 103************** 104 105In addition to the usual Kconfig options required to run the BPF test suite, it 106is also helpful to select:: 107 108 CONFIG_NET_9P=y 109 CONFIG_9P_FS=y 110 CONFIG_NET_9P_VIRTIO=y 111 CONFIG_VIRTIO_PCI=y 112 113as that would enable a very easy way to share files with the s390 virtual 114machine. 115 116Compiling kernel, modules and testsuite, as well as preparing gdb scripts to 117simplify debugging, can be done using the following commands:: 118 119 make ARCH=s390 CROSS_COMPILE=s390x-linux-gnu- menuconfig 120 make ARCH=s390 CROSS_COMPILE=s390x-linux-gnu- bzImage modules scripts_gdb 121 make ARCH=s390 CROSS_COMPILE=s390x-linux-gnu- \ 122 -C tools/testing/selftests \ 123 TARGETS=bpf \ 124 INSTALL_PATH=$PWD/tools/testing/selftests/kselftest_install \ 125 install 126 1274. Running the test suite 128************************* 129 130The virtual machine can be started as follows:: 131 132 qemu-system-s390x \ 133 -cpu max,zpci=on \ 134 -smp 2 \ 135 -m 4G \ 136 -kernel linux/arch/s390/boot/compressed/vmlinux \ 137 -drive file=./s390.img,if=virtio,format=raw \ 138 -nographic \ 139 -append 'root=/dev/vda rw console=ttyS1' \ 140 -virtfs local,path=./linux,security_model=none,mount_tag=linux \ 141 -object rng-random,filename=/dev/urandom,id=rng0 \ 142 -device virtio-rng-ccw,rng=rng0 \ 143 -netdev user,id=net0 \ 144 -device virtio-net-ccw,netdev=net0 145 146When using this on a real IBM Z, ``-enable-kvm`` may be added for better 147performance. When starting the virtual machine for the first time, disk image 148setup must be finalized using the following command:: 149 150 /debootstrap/debootstrap --second-stage 151 152Directory with the code built on the host as well as ``/proc`` and ``/sys`` 153need to be mounted as follows:: 154 155 mkdir -p /linux 156 mount -t 9p linux /linux 157 mount -t proc proc /proc 158 mount -t sysfs sys /sys 159 160After that, the test suite can be run using the following commands:: 161 162 cd /linux/tools/testing/selftests/kselftest_install 163 ./run_kselftest.sh 164 165As usual, tests can be also run individually:: 166 167 cd /linux/tools/testing/selftests/bpf 168 ./test_verifier 169 1705. Debugging 171************ 172 173It is possible to debug the s390 kernel using QEMU GDB stub, which is activated 174by passing ``-s`` to QEMU. 175 176It is preferable to turn KASLR off, so that gdb would know where to find the 177kernel image in memory, by building the kernel with:: 178 179 RANDOMIZE_BASE=n 180 181GDB can then be attached using the following command:: 182 183 gdb-multiarch -ex 'target remote localhost:1234' ./vmlinux 184 1856. Network 186********** 187 188In case one needs to use the network in the virtual machine in order to e.g. 189install additional packages, it can be configured using:: 190 191 dhclient eth0 192 1937. Links 194******** 195 196This document is a compilation of techniques, whose more comprehensive 197descriptions can be found by following these links: 198 199- `Debootstrap <https://wiki.debian.org/EmDebian/CrossDebootstrap>`_ 200- `Multiarch <https://wiki.debian.org/Multiarch/HOWTO>`_ 201- `Building LLVM <https://llvm.org/docs/CMake.html>`_ 202- `Cross-compiling the kernel <https://wiki.gentoo.org/wiki/Embedded_Handbook/General/Cross-compiling_the_kernel>`_ 203- `QEMU s390x Guest Support <https://wiki.qemu.org/Documentation/Platforms/S390X>`_ 204- `Plan 9 folder sharing over Virtio <https://wiki.qemu.org/Documentation/9psetup>`_ 205- `Using GDB with QEMU <https://wiki.osdev.org/Kernel_Debugging#Use_GDB_with_QEMU>`_ 206