1.. SPDX-License-Identifier: GPL-2.0-only
2.. Copyright Red Hat
3
4==============================================
5BPF_MAP_TYPE_SOCKMAP and BPF_MAP_TYPE_SOCKHASH
6==============================================
7
8.. note::
9   - ``BPF_MAP_TYPE_SOCKMAP`` was introduced in kernel version 4.14
10   - ``BPF_MAP_TYPE_SOCKHASH`` was introduced in kernel version 4.18
11
12``BPF_MAP_TYPE_SOCKMAP`` and ``BPF_MAP_TYPE_SOCKHASH`` maps can be used to
13redirect skbs between sockets or to apply policy at the socket level based on
14the result of a BPF (verdict) program with the help of the BPF helpers
15``bpf_sk_redirect_map()``, ``bpf_sk_redirect_hash()``,
16``bpf_msg_redirect_map()`` and ``bpf_msg_redirect_hash()``.
17
18``BPF_MAP_TYPE_SOCKMAP`` is backed by an array that uses an integer key as the
19index to look up a reference to a ``struct sock``. The map values are socket
20descriptors. Similarly, ``BPF_MAP_TYPE_SOCKHASH`` is a hash backed BPF map that
21holds references to sockets via their socket descriptors.
22
23.. note::
24    The value type is either __u32 or __u64; the latter (__u64) is to support
25    returning socket cookies to userspace. Returning the ``struct sock *`` that
26    the map holds to user-space is neither safe nor useful.
27
28These maps may have BPF programs attached to them, specifically a parser program
29and a verdict program. The parser program determines how much data has been
30parsed and therefore how much data needs to be queued to come to a verdict. The
31verdict program is essentially the redirect program and can return a verdict
32of ``__SK_DROP``, ``__SK_PASS``, or ``__SK_REDIRECT``.
33
34When a socket is inserted into one of these maps, its socket callbacks are
35replaced and a ``struct sk_psock`` is attached to it. Additionally, this
36``sk_psock`` inherits the programs that are attached to the map.
37
38A sock object may be in multiple maps, but can only inherit a single
39parse or verdict program. If adding a sock object to a map would result
40in having multiple parser programs the update will return an EBUSY error.
41
42The supported programs to attach to these maps are:
43
44.. code-block:: c
45
46	struct sk_psock_progs {
47		struct bpf_prog *msg_parser;
48		struct bpf_prog *stream_parser;
49		struct bpf_prog *stream_verdict;
50		struct bpf_prog	*skb_verdict;
51	};
52
53.. note::
54    Users are not allowed to attach ``stream_verdict`` and ``skb_verdict``
55    programs to the same map.
56
57The attach types for the map programs are:
58
59- ``msg_parser`` program - ``BPF_SK_MSG_VERDICT``.
60- ``stream_parser`` program - ``BPF_SK_SKB_STREAM_PARSER``.
61- ``stream_verdict`` program - ``BPF_SK_SKB_STREAM_VERDICT``.
62- ``skb_verdict`` program - ``BPF_SK_SKB_VERDICT``.
63
64There are additional helpers available to use with the parser and verdict
65programs: ``bpf_msg_apply_bytes()`` and ``bpf_msg_cork_bytes()``. With
66``bpf_msg_apply_bytes()`` BPF programs can tell the infrastructure how many
67bytes the given verdict should apply to. The helper ``bpf_msg_cork_bytes()``
68handles a different case where a BPF program cannot reach a verdict on a msg
69until it receives more bytes AND the program doesn't want to forward the packet
70until it is known to be good.
71
72Finally, the helpers ``bpf_msg_pull_data()`` and ``bpf_msg_push_data()`` are
73available to ``BPF_PROG_TYPE_SK_MSG`` BPF programs to pull in data and set the
74start and end pointers to given values or to add metadata to the ``struct
75sk_msg_buff *msg``.
76
77All these helpers will be described in more detail below.
78
79Usage
80=====
81Kernel BPF
82----------
83bpf_msg_redirect_map()
84^^^^^^^^^^^^^^^^^^^^^^
85.. code-block:: c
86
87	long bpf_msg_redirect_map(struct sk_msg_buff *msg, struct bpf_map *map, u32 key, u64 flags)
88
89This helper is used in programs implementing policies at the socket level. If
90the message ``msg`` is allowed to pass (i.e., if the verdict BPF program
91returns ``SK_PASS``), redirect it to the socket referenced by ``map`` (of type
92``BPF_MAP_TYPE_SOCKMAP``) at index ``key``. Both ingress and egress interfaces
93can be used for redirection. The ``BPF_F_INGRESS`` value in ``flags`` is used
94to select the ingress path otherwise the egress path is selected. This is the
95only flag supported for now.
96
97Returns ``SK_PASS`` on success, or ``SK_DROP`` on error.
98
99bpf_sk_redirect_map()
100^^^^^^^^^^^^^^^^^^^^^
101.. code-block:: c
102
103    long bpf_sk_redirect_map(struct sk_buff *skb, struct bpf_map *map, u32 key u64 flags)
104
105Redirect the packet to the socket referenced by ``map`` (of type
106``BPF_MAP_TYPE_SOCKMAP``) at index ``key``. Both ingress and egress interfaces
107can be used for redirection. The ``BPF_F_INGRESS`` value in ``flags`` is used
108to select the ingress path otherwise the egress path is selected. This is the
109only flag supported for now.
110
111Returns ``SK_PASS`` on success, or ``SK_DROP`` on error.
112
113bpf_map_lookup_elem()
114^^^^^^^^^^^^^^^^^^^^^
115.. code-block:: c
116
117    void *bpf_map_lookup_elem(struct bpf_map *map, const void *key)
118
119socket entries of type ``struct sock *`` can be retrieved using the
120``bpf_map_lookup_elem()`` helper.
121
122bpf_sock_map_update()
123^^^^^^^^^^^^^^^^^^^^^
124.. code-block:: c
125
126    long bpf_sock_map_update(struct bpf_sock_ops *skops, struct bpf_map *map, void *key, u64 flags)
127
128Add an entry to, or update a ``map`` referencing sockets. The ``skops`` is used
129as a new value for the entry associated to ``key``. The ``flags`` argument can
130be one of the following:
131
132- ``BPF_ANY``: Create a new element or update an existing element.
133- ``BPF_NOEXIST``: Create a new element only if it did not exist.
134- ``BPF_EXIST``: Update an existing element.
135
136If the ``map`` has BPF programs (parser and verdict), those will be inherited
137by the socket being added. If the socket is already attached to BPF programs,
138this results in an error.
139
140Returns 0 on success, or a negative error in case of failure.
141
142bpf_sock_hash_update()
143^^^^^^^^^^^^^^^^^^^^^^
144.. code-block:: c
145
146    long bpf_sock_hash_update(struct bpf_sock_ops *skops, struct bpf_map *map, void *key, u64 flags)
147
148Add an entry to, or update a sockhash ``map`` referencing sockets. The ``skops``
149is used as a new value for the entry associated to ``key``.
150
151The ``flags`` argument can be one of the following:
152
153- ``BPF_ANY``: Create a new element or update an existing element.
154- ``BPF_NOEXIST``: Create a new element only if it did not exist.
155- ``BPF_EXIST``: Update an existing element.
156
157If the ``map`` has BPF programs (parser and verdict), those will be inherited
158by the socket being added. If the socket is already attached to BPF programs,
159this results in an error.
160
161Returns 0 on success, or a negative error in case of failure.
162
163bpf_msg_redirect_hash()
164^^^^^^^^^^^^^^^^^^^^^^^
165.. code-block:: c
166
167    long bpf_msg_redirect_hash(struct sk_msg_buff *msg, struct bpf_map *map, void *key, u64 flags)
168
169This helper is used in programs implementing policies at the socket level. If
170the message ``msg`` is allowed to pass (i.e., if the verdict BPF program returns
171``SK_PASS``), redirect it to the socket referenced by ``map`` (of type
172``BPF_MAP_TYPE_SOCKHASH``) using hash ``key``. Both ingress and egress
173interfaces can be used for redirection. The ``BPF_F_INGRESS`` value in
174``flags`` is used to select the ingress path otherwise the egress path is
175selected. This is the only flag supported for now.
176
177Returns ``SK_PASS`` on success, or ``SK_DROP`` on error.
178
179bpf_sk_redirect_hash()
180^^^^^^^^^^^^^^^^^^^^^^
181.. code-block:: c
182
183    long bpf_sk_redirect_hash(struct sk_buff *skb, struct bpf_map *map, void *key, u64 flags)
184
185This helper is used in programs implementing policies at the skb socket level.
186If the sk_buff ``skb`` is allowed to pass (i.e., if the verdict BPF program
187returns ``SK_PASS``), redirect it to the socket referenced by ``map`` (of type
188``BPF_MAP_TYPE_SOCKHASH``) using hash ``key``. Both ingress and egress
189interfaces can be used for redirection. The ``BPF_F_INGRESS`` value in
190``flags`` is used to select the ingress path otherwise the egress path is
191selected. This is the only flag supported for now.
192
193Returns ``SK_PASS`` on success, or ``SK_DROP`` on error.
194
195bpf_msg_apply_bytes()
196^^^^^^^^^^^^^^^^^^^^^^
197.. code-block:: c
198
199    long bpf_msg_apply_bytes(struct sk_msg_buff *msg, u32 bytes)
200
201For socket policies, apply the verdict of the BPF program to the next (number
202of ``bytes``) of message ``msg``. For example, this helper can be used in the
203following cases:
204
205- A single ``sendmsg()`` or ``sendfile()`` system call contains multiple
206  logical messages that the BPF program is supposed to read and for which it
207  should apply a verdict.
208- A BPF program only cares to read the first ``bytes`` of a ``msg``. If the
209  message has a large payload, then setting up and calling the BPF program
210  repeatedly for all bytes, even though the verdict is already known, would
211  create unnecessary overhead.
212
213Returns 0
214
215bpf_msg_cork_bytes()
216^^^^^^^^^^^^^^^^^^^^^^
217.. code-block:: c
218
219    long bpf_msg_cork_bytes(struct sk_msg_buff *msg, u32 bytes)
220
221For socket policies, prevent the execution of the verdict BPF program for
222message ``msg`` until the number of ``bytes`` have been accumulated.
223
224This can be used when one needs a specific number of bytes before a verdict can
225be assigned, even if the data spans multiple ``sendmsg()`` or ``sendfile()``
226calls.
227
228Returns 0
229
230bpf_msg_pull_data()
231^^^^^^^^^^^^^^^^^^^^^^
232.. code-block:: c
233
234    long bpf_msg_pull_data(struct sk_msg_buff *msg, u32 start, u32 end, u64 flags)
235
236For socket policies, pull in non-linear data from user space for ``msg`` and set
237pointers ``msg->data`` and ``msg->data_end`` to ``start`` and ``end`` bytes
238offsets into ``msg``, respectively.
239
240If a program of type ``BPF_PROG_TYPE_SK_MSG`` is run on a ``msg`` it can only
241parse data that the (``data``, ``data_end``) pointers have already consumed.
242For ``sendmsg()`` hooks this is likely the first scatterlist element. But for
243calls relying on the ``sendpage`` handler (e.g., ``sendfile()``) this will be
244the range (**0**, **0**) because the data is shared with user space and by
245default the objective is to avoid allowing user space to modify data while (or
246after) BPF verdict is being decided. This helper can be used to pull in data
247and to set the start and end pointers to given values. Data will be copied if
248necessary (i.e., if data was not linear and if start and end pointers do not
249point to the same chunk).
250
251A call to this helper is susceptible to change the underlying packet buffer.
252Therefore, at load time, all checks on pointers previously done by the verifier
253are invalidated and must be performed again, if the helper is used in
254combination with direct packet access.
255
256All values for ``flags`` are reserved for future usage, and must be left at
257zero.
258
259Returns 0 on success, or a negative error in case of failure.
260
261bpf_map_lookup_elem()
262^^^^^^^^^^^^^^^^^^^^^
263
264.. code-block:: c
265
266	void *bpf_map_lookup_elem(struct bpf_map *map, const void *key)
267
268Look up a socket entry in the sockmap or sockhash map.
269
270Returns the socket entry associated to ``key``, or NULL if no entry was found.
271
272bpf_map_update_elem()
273^^^^^^^^^^^^^^^^^^^^^
274.. code-block:: c
275
276	long bpf_map_update_elem(struct bpf_map *map, const void *key, const void *value, u64 flags)
277
278Add or update a socket entry in a sockmap or sockhash.
279
280The flags argument can be one of the following:
281
282- BPF_ANY: Create a new element or update an existing element.
283- BPF_NOEXIST: Create a new element only if it did not exist.
284- BPF_EXIST: Update an existing element.
285
286Returns 0 on success, or a negative error in case of failure.
287
288bpf_map_delete_elem()
289^^^^^^^^^^^^^^^^^^^^^^
290.. code-block:: c
291
292    long bpf_map_delete_elem(struct bpf_map *map, const void *key)
293
294Delete a socket entry from a sockmap or a sockhash.
295
296Returns	0 on success, or a negative error in case of failure.
297
298User space
299----------
300bpf_map_update_elem()
301^^^^^^^^^^^^^^^^^^^^^
302.. code-block:: c
303
304	int bpf_map_update_elem(int fd, const void *key, const void *value, __u64 flags)
305
306Sockmap entries can be added or updated using the ``bpf_map_update_elem()``
307function. The ``key`` parameter is the index value of the sockmap array. And the
308``value`` parameter is the FD value of that socket.
309
310Under the hood, the sockmap update function uses the socket FD value to
311retrieve the associated socket and its attached psock.
312
313The flags argument can be one of the following:
314
315- BPF_ANY: Create a new element or update an existing element.
316- BPF_NOEXIST: Create a new element only if it did not exist.
317- BPF_EXIST: Update an existing element.
318
319bpf_map_lookup_elem()
320^^^^^^^^^^^^^^^^^^^^^
321.. code-block:: c
322
323    int bpf_map_lookup_elem(int fd, const void *key, void *value)
324
325Sockmap entries can be retrieved using the ``bpf_map_lookup_elem()`` function.
326
327.. note::
328	The entry returned is a socket cookie rather than a socket itself.
329
330bpf_map_delete_elem()
331^^^^^^^^^^^^^^^^^^^^^
332.. code-block:: c
333
334    int bpf_map_delete_elem(int fd, const void *key)
335
336Sockmap entries can be deleted using the ``bpf_map_delete_elem()``
337function.
338
339Returns 0 on success, or negative error in case of failure.
340
341Examples
342========
343
344Kernel BPF
345----------
346Several examples of the use of sockmap APIs can be found in:
347
348- `tools/testing/selftests/bpf/progs/test_sockmap_kern.h`_
349- `tools/testing/selftests/bpf/progs/sockmap_parse_prog.c`_
350- `tools/testing/selftests/bpf/progs/sockmap_verdict_prog.c`_
351- `tools/testing/selftests/bpf/progs/test_sockmap_listen.c`_
352- `tools/testing/selftests/bpf/progs/test_sockmap_update.c`_
353
354The following code snippet shows how to declare a sockmap.
355
356.. code-block:: c
357
358	struct {
359		__uint(type, BPF_MAP_TYPE_SOCKMAP);
360		__uint(max_entries, 1);
361		__type(key, __u32);
362		__type(value, __u64);
363	} sock_map_rx SEC(".maps");
364
365The following code snippet shows a sample parser program.
366
367.. code-block:: c
368
369	SEC("sk_skb/stream_parser")
370	int bpf_prog_parser(struct __sk_buff *skb)
371	{
372		return skb->len;
373	}
374
375The following code snippet shows a simple verdict program that interacts with a
376sockmap to redirect traffic to another socket based on the local port.
377
378.. code-block:: c
379
380	SEC("sk_skb/stream_verdict")
381	int bpf_prog_verdict(struct __sk_buff *skb)
382	{
383		__u32 lport = skb->local_port;
384		__u32 idx = 0;
385
386		if (lport == 10000)
387			return bpf_sk_redirect_map(skb, &sock_map_rx, idx, 0);
388
389		return SK_PASS;
390	}
391
392The following code snippet shows how to declare a sockhash map.
393
394.. code-block:: c
395
396	struct socket_key {
397		__u32 src_ip;
398		__u32 dst_ip;
399		__u32 src_port;
400		__u32 dst_port;
401	};
402
403	struct {
404		__uint(type, BPF_MAP_TYPE_SOCKHASH);
405		__uint(max_entries, 1);
406		__type(key, struct socket_key);
407		__type(value, __u64);
408	} sock_hash_rx SEC(".maps");
409
410The following code snippet shows a simple verdict program that interacts with a
411sockhash to redirect traffic to another socket based on a hash of some of the
412skb parameters.
413
414.. code-block:: c
415
416	static inline
417	void extract_socket_key(struct __sk_buff *skb, struct socket_key *key)
418	{
419		key->src_ip = skb->remote_ip4;
420		key->dst_ip = skb->local_ip4;
421		key->src_port = skb->remote_port >> 16;
422		key->dst_port = (bpf_htonl(skb->local_port)) >> 16;
423	}
424
425	SEC("sk_skb/stream_verdict")
426	int bpf_prog_verdict(struct __sk_buff *skb)
427	{
428		struct socket_key key;
429
430		extract_socket_key(skb, &key);
431
432		return bpf_sk_redirect_hash(skb, &sock_hash_rx, &key, 0);
433	}
434
435User space
436----------
437Several examples of the use of sockmap APIs can be found in:
438
439- `tools/testing/selftests/bpf/prog_tests/sockmap_basic.c`_
440- `tools/testing/selftests/bpf/test_sockmap.c`_
441- `tools/testing/selftests/bpf/test_maps.c`_
442
443The following code sample shows how to create a sockmap, attach a parser and
444verdict program, as well as add a socket entry.
445
446.. code-block:: c
447
448	int create_sample_sockmap(int sock, int parse_prog_fd, int verdict_prog_fd)
449	{
450		int index = 0;
451		int map, err;
452
453		map = bpf_map_create(BPF_MAP_TYPE_SOCKMAP, NULL, sizeof(int), sizeof(int), 1, NULL);
454		if (map < 0) {
455			fprintf(stderr, "Failed to create sockmap: %s\n", strerror(errno));
456			return -1;
457		}
458
459		err = bpf_prog_attach(parse_prog_fd, map, BPF_SK_SKB_STREAM_PARSER, 0);
460		if (err){
461			fprintf(stderr, "Failed to attach_parser_prog_to_map: %s\n", strerror(errno));
462			goto out;
463		}
464
465		err = bpf_prog_attach(verdict_prog_fd, map, BPF_SK_SKB_STREAM_VERDICT, 0);
466		if (err){
467			fprintf(stderr, "Failed to attach_verdict_prog_to_map: %s\n", strerror(errno));
468			goto out;
469		}
470
471		err = bpf_map_update_elem(map, &index, &sock, BPF_NOEXIST);
472		if (err) {
473			fprintf(stderr, "Failed to update sockmap: %s\n", strerror(errno));
474			goto out;
475		}
476
477	out:
478		close(map);
479		return err;
480	}
481
482References
483===========
484
485- https://github.com/jrfastab/linux-kernel-xdp/commit/c89fd73cb9d2d7f3c716c3e00836f07b1aeb261f
486- https://lwn.net/Articles/731133/
487- http://vger.kernel.org/lpc_net2018_talks/ktls_bpf_paper.pdf
488- https://lwn.net/Articles/748628/
489- https://lore.kernel.org/bpf/20200218171023.844439-7-jakub@cloudflare.com/
490
491.. _`tools/testing/selftests/bpf/progs/test_sockmap_kern.h`: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/tools/testing/selftests/bpf/progs/test_sockmap_kern.h
492.. _`tools/testing/selftests/bpf/progs/sockmap_parse_prog.c`: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/tools/testing/selftests/bpf/progs/sockmap_parse_prog.c
493.. _`tools/testing/selftests/bpf/progs/sockmap_verdict_prog.c`: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/tools/testing/selftests/bpf/progs/sockmap_verdict_prog.c
494.. _`tools/testing/selftests/bpf/prog_tests/sockmap_basic.c`: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/tools/testing/selftests/bpf/prog_tests/sockmap_basic.c
495.. _`tools/testing/selftests/bpf/test_sockmap.c`: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/tools/testing/selftests/bpf/test_sockmap.c
496.. _`tools/testing/selftests/bpf/test_maps.c`: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/tools/testing/selftests/bpf/test_maps.c
497.. _`tools/testing/selftests/bpf/progs/test_sockmap_listen.c`: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/tools/testing/selftests/bpf/progs/test_sockmap_listen.c
498.. _`tools/testing/selftests/bpf/progs/test_sockmap_update.c`: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/tools/testing/selftests/bpf/progs/test_sockmap_update.c
499