1=============== 2XDP RX Metadata 3=============== 4 5This document describes how an eXpress Data Path (XDP) program can access 6hardware metadata related to a packet using a set of helper functions, 7and how it can pass that metadata on to other consumers. 8 9General Design 10============== 11 12XDP has access to a set of kfuncs to manipulate the metadata in an XDP frame. 13Every device driver that wishes to expose additional packet metadata can 14implement these kfuncs. The set of kfuncs is declared in ``include/net/xdp.h`` 15via ``XDP_METADATA_KFUNC_xxx``. 16 17Currently, the following kfuncs are supported. In the future, as more 18metadata is supported, this set will grow: 19 20.. kernel-doc:: net/core/xdp.c 21 :identifiers: bpf_xdp_metadata_rx_timestamp bpf_xdp_metadata_rx_hash 22 23An XDP program can use these kfuncs to read the metadata into stack 24variables for its own consumption. Or, to pass the metadata on to other 25consumers, an XDP program can store it into the metadata area carried 26ahead of the packet. 27 28Not all kfuncs have to be implemented by the device driver; when not 29implemented, the default ones that return ``-EOPNOTSUPP`` will be used. 30 31Within an XDP frame, the metadata layout (accessed via ``xdp_buff``) is 32as follows:: 33 34 +----------+-----------------+------+ 35 | headroom | custom metadata | data | 36 +----------+-----------------+------+ 37 ^ ^ 38 | | 39 xdp_buff->data_meta xdp_buff->data 40 41An XDP program can store individual metadata items into this ``data_meta`` 42area in whichever format it chooses. Later consumers of the metadata 43will have to agree on the format by some out of band contract (like for 44the AF_XDP use case, see below). 45 46AF_XDP 47====== 48 49:doc:`af_xdp` use-case implies that there is a contract between the BPF 50program that redirects XDP frames into the ``AF_XDP`` socket (``XSK``) and 51the final consumer. Thus the BPF program manually allocates a fixed number of 52bytes out of metadata via ``bpf_xdp_adjust_meta`` and calls a subset 53of kfuncs to populate it. The userspace ``XSK`` consumer computes 54``xsk_umem__get_data() - METADATA_SIZE`` to locate that metadata. 55Note, ``xsk_umem__get_data`` is defined in ``libxdp`` and 56``METADATA_SIZE`` is an application-specific constant (``AF_XDP`` receive 57descriptor does _not_ explicitly carry the size of the metadata). 58 59Here is the ``AF_XDP`` consumer layout (note missing ``data_meta`` pointer):: 60 61 +----------+-----------------+------+ 62 | headroom | custom metadata | data | 63 +----------+-----------------+------+ 64 ^ 65 | 66 rx_desc->address 67 68XDP_PASS 69======== 70 71This is the path where the packets processed by the XDP program are passed 72into the kernel. The kernel creates the ``skb`` out of the ``xdp_buff`` 73contents. Currently, every driver has custom kernel code to parse 74the descriptors and populate ``skb`` metadata when doing this ``xdp_buff->skb`` 75conversion, and the XDP metadata is not used by the kernel when building 76``skbs``. However, TC-BPF programs can access the XDP metadata area using 77the ``data_meta`` pointer. 78 79In the future, we'd like to support a case where an XDP program 80can override some of the metadata used for building ``skbs``. 81 82bpf_redirect_map 83================ 84 85``bpf_redirect_map`` can redirect the frame to a different device. 86Some devices (like virtual ethernet links) support running a second XDP 87program after the redirect. However, the final consumer doesn't have 88access to the original hardware descriptor and can't access any of 89the original metadata. The same applies to XDP programs installed 90into devmaps and cpumaps. 91 92This means that for redirected packets only custom metadata is 93currently supported, which has to be prepared by the initial XDP program 94before redirect. If the frame is eventually passed to the kernel, the 95``skb`` created from such a frame won't have any hardware metadata populated 96in its ``skb``. If such a packet is later redirected into an ``XSK``, 97that will also only have access to the custom metadata. 98 99bpf_tail_call 100============= 101 102Adding programs that access metadata kfuncs to the ``BPF_MAP_TYPE_PROG_ARRAY`` 103is currently not supported. 104 105Example 106======= 107 108See ``tools/testing/selftests/bpf/progs/xdp_metadata.c`` and 109``tools/testing/selftests/bpf/prog_tests/xdp_metadata.c`` for an example of 110BPF program that handles XDP metadata. 111