Merge drm/drm-next into drm-intel-next

Catch-up with 5.17-rc2 and trying to align with drm-intel-gt-next
for a possible topic branch for merging the split of i915_regs...

Signed-off-by: Rodrigo Viv

Merge drm/drm-next into drm-intel-next

Catch-up with 5.17-rc2 and trying to align with drm-intel-gt-next
for a possible topic branch for merging the split of i915_regs...

Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>

show more ...

Merge drm/drm-fixes into drm-misc-fixes

Backmerging drm/drm-fixes into drm-misc-fixes for v5.17-rc1.

Signed-off-by: Thomas Zimmermann <tzimmermann@suse.de>

Merge remote-tracking branch 'torvalds/master' into perf/core

To pick up fixes and get in line with other trees, powerpc kernel
mostly this time, but BPF as well.

Signed-off-by: Arnaldo Carvalho de

Merge remote-tracking branch 'torvalds/master' into perf/core

To pick up fixes and get in line with other trees, powerpc kernel
mostly this time, but BPF as well.

Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>

show more ...

Merge tag '5.17-net-next' of git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net-next

Pull networking updates from Jakub Kicinski:
"Core
----

- Defer freeing TCP skbs to the BH handler,

Merge tag '5.17-net-next' of git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net-next

Pull networking updates from Jakub Kicinski:
"Core
----

- Defer freeing TCP skbs to the BH handler, whenever possible, or at
least perform the freeing outside of the socket lock section to
decrease cross-CPU allocator work and improve latency.

- Add netdevice refcount tracking to locate sources of netdevice and
net namespace refcount leaks.

- Make Tx watchdog less intrusive - avoid pausing Tx and restarting
all queues from a single CPU removing latency spikes.

- Various small optimizations throughout the stack from Eric Dumazet.

- Make netdev->dev_addr[] constant, force modifications to go via
appropriate helpers to allow us to keep addresses in ordered data
structures.

- Replace unix_table_lock with per-hash locks, improving performance
of bind() calls.

- Extend skb drop tracepoint with a drop reason.

- Allow SO_MARK and SO_PRIORITY setsockopt under CAP_NET_RAW.

BPF
---

- New helpers:
- bpf_find_vma(), find and inspect VMAs for profiling use cases
- bpf_loop(), runtime-bounded loop helper trading some execution
time for much faster (if at all converging) verification
- bpf_strncmp(), improve performance, avoid compiler flakiness
- bpf_get_func_arg(), bpf_get_func_ret(), bpf_get_func_arg_cnt()
for tracing programs, all inlined by the verifier

- Support BPF relocations (CO-RE) in the kernel loader.

- Further the support for BTF_TYPE_TAG annotations.

- Allow access to local storage in sleepable helpers.

- Convert verifier argument types to a composable form with different
attributes which can be shared across types (ro, maybe-null).

- Prepare libbpf for upcoming v1.0 release by cleaning up APIs,
creating new, extensible ones where missing and deprecating those
to be removed.

Protocols
---------

- WiFi (mac80211/cfg80211):
- notify user space about long "come back in N" AP responses,
allow it to react to such temporary rejections
- allow non-standard VHT MCS 10/11 rates
- use coarse time in airtime fairness code to save CPU cycles

- Bluetooth:
- rework of HCI command execution serialization to use a common
queue and work struct, and improve handling errors reported in
the middle of a batch of commands
- rework HCI event handling to use skb_pull_data, avoiding packet
parsing pitfalls
- support AOSP Bluetooth Quality Report

- SMC:
- support net namespaces, following the RDMA model
- improve connection establishment latency by pre-clearing buffers
- introduce TCP ULP for automatic redirection to SMC

- Multi-Path TCP:
- support ioctls: SIOCINQ, OUTQ, and OUTQNSD
- support socket options: IP_TOS, IP_FREEBIND, IP_TRANSPARENT,
IPV6_FREEBIND, and IPV6_TRANSPARENT, TCP_CORK and TCP_NODELAY
- support cmsgs: TCP_INQ
- improvements in the data scheduler (assigning data to subflows)
- support fastclose option (quick shutdown of the full MPTCP
connection, similar to TCP RST in regular TCP)

- MCTP (Management Component Transport) over serial, as defined by
DMTF spec DSP0253 - "MCTP Serial Transport Binding".

Driver API
----------

- Support timestamping on bond interfaces in active/passive mode.

- Introduce generic phylink link mode validation for drivers which
don't have any quirks and where MAC capability bits fully express
what's supported. Allow PCS layer to participate in the validation.
Convert a number of drivers.

- Add support to set/get size of buffers on the Rx rings and size of
the tx copybreak buffer via ethtool.

- Support offloading TC actions as first-class citizens rather than
only as attributes of filters, improve sharing and device resource
utilization.

- WiFi (mac80211/cfg80211):
- support forwarding offload (ndo_fill_forward_path)
- support for background radar detection hardware
- SA Query Procedures offload on the AP side

New hardware / drivers
----------------------

- tsnep - FPGA based TSN endpoint Ethernet MAC used in PLCs with
real-time requirements for isochronous communication with protocols
like OPC UA Pub/Sub.

- Qualcomm BAM-DMUX WWAN - driver for data channels of modems
integrated into many older Qualcomm SoCs, e.g. MSM8916 or MSM8974
(qcom_bam_dmux).

- Microchip LAN966x multi-port Gigabit AVB/TSN Ethernet Switch driver
with support for bridging, VLANs and multicast forwarding
(lan966x).

- iwlmei driver for co-operating between Intel's WiFi driver and
Intel's Active Management Technology (AMT) devices.

- mse102x - Vertexcom MSE102x Homeplug GreenPHY chips

- Bluetooth:
- MediaTek MT7921 SDIO devices
- Foxconn MT7922A
- Realtek RTL8852AE

Drivers
-------

- Significantly improve performance in the datapaths of: lan78xx,
ax88179_178a, lantiq_xrx200, bnxt.

- Intel Ethernet NICs:
- igb: support PTP/time PEROUT and EXTTS SDP functions on
82580/i354/i350 adapters
- ixgbevf: new PF -> VF mailbox API which avoids the risk of
mailbox corruption with ESXi
- iavf: support configuration of VLAN features of finer
granularity, stacked tags and filtering
- ice: PTP support for new E822 devices with sub-ns precision
- ice: support firmware activation without reboot

- Mellanox Ethernet NICs (mlx5):
- expose control over IRQ coalescing mode (CQE vs EQE) via ethtool
- support TC forwarding when tunnel encap and decap happen between
two ports of the same NIC
- dynamically size and allow disabling various features to save
resources for running in embedded / SmartNIC scenarios

- Broadcom Ethernet NICs (bnxt):
- use page frag allocator to improve Rx performance
- expose control over IRQ coalescing mode (CQE vs EQE) via ethtool

- Other Ethernet NICs:
- amd-xgbe: add Ryzen 6000 (Yellow Carp) Ethernet support

- Microsoft cloud/virtual NIC (mana):
- add XDP support (PASS, DROP, TX)

- Mellanox Ethernet switches (mlxsw):
- initial support for Spectrum-4 ASICs
- VxLAN with IPv6 underlay

- Marvell Ethernet switches (prestera):
- support flower flow templates
- add basic IP forwarding support

- NXP embedded Ethernet switches (ocelot & felix):
- support Per-Stream Filtering and Policing (PSFP)
- enable cut-through forwarding between ports by default
- support FDMA to improve packet Rx/Tx to CPU

- Other embedded switches:
- hellcreek: improve trapping management (STP and PTP) packets
- qca8k: support link aggregation and port mirroring

- Qualcomm 802.11ax WiFi (ath11k):
- qca6390, wcn6855: enable 802.11 power save mode in station mode
- BSS color change support
- WCN6855 hw2.1 support
- 11d scan offload support
- scan MAC address randomization support
- full monitor mode, only supported on QCN9074
- qca6390/wcn6855: report signal and tx bitrate
- qca6390: rfkill support
- qca6390/wcn6855: regdb.bin support

- Intel WiFi (iwlwifi):
- support SAR GEO Offset Mapping (SGOM) and Time-Aware-SAR (TAS)
in cooperation with the BIOS
- support for Optimized Connectivity Experience (OCE) scan
- support firmware API version 68
- lots of preparatory work for the upcoming Bz device family

- MediaTek WiFi (mt76):
- Specific Absorption Rate (SAR) support
- mt7921: 160 MHz channel support

- RealTek WiFi (rtw88):
- Specific Absorption Rate (SAR) support
- scan offload

- Other WiFi NICs
- ath10k: support fetching (pre-)calibration data from nvmem
- brcmfmac: configure keep-alive packet on suspend
- wcn36xx: beacon filter support"

* tag '5.17-net-next' of git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net-next: (2048 commits)
tcp: tcp_send_challenge_ack delete useless param `skb`
net/qla3xxx: Remove useless DMA-32 fallback configuration
rocker: Remove useless DMA-32 fallback configuration
hinic: Remove useless DMA-32 fallback configuration
lan743x: Remove useless DMA-32 fallback configuration
net: enetc: Remove useless DMA-32 fallback configuration
cxgb4vf: Remove useless DMA-32 fallback configuration
cxgb4: Remove useless DMA-32 fallback configuration
cxgb3: Remove useless DMA-32 fallback configuration
bnx2x: Remove useless DMA-32 fallback configuration
et131x: Remove useless DMA-32 fallback configuration
be2net: Remove useless DMA-32 fallback configuration
vmxnet3: Remove useless DMA-32 fallback configuration
bna: Simplify DMA setting
net: alteon: Simplify DMA setting
myri10ge: Simplify DMA setting
qlcnic: Simplify DMA setting
net: allwinner: Fix print format
page_pool: remove spinlock in page_pool_refill_alloc_cache()
amt: fix wrong return type of amt_send_membership_update()
...

show more ...

Merge https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next

Andrii Nakryiko says:

====================
bpf-next 2021-12-10 v2

We've added 115 non-merge commits during the last 26 day(s) whi

Merge https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next

Andrii Nakryiko says:

====================
bpf-next 2021-12-10 v2

We've added 115 non-merge commits during the last 26 day(s) which contain
a total of 182 files changed, 5747 insertions(+), 2564 deletions(-).

The main changes are:

1) Various samples fixes, from Alexander Lobakin.

2) BPF CO-RE support in kernel and light skeleton, from Alexei Starovoitov.

3) A batch of new unified APIs for libbpf, logging improvements, version
querying, etc. Also a batch of old deprecations for old APIs and various
bug fixes, in preparation for libbpf 1.0, from Andrii Nakryiko.

4) BPF documentation reorganization and improvements, from Christoph Hellwig
and Dave Tucker.

5) Support for declarative initialization of BPF_MAP_TYPE_PROG_ARRAY in
libbpf, from Hengqi Chen.

6) Verifier log fixes, from Hou Tao.

7) Runtime-bounded loops support with bpf_loop() helper, from Joanne Koong.

8) Extend branch record capturing to all platforms that support it,
from Kajol Jain.

9) Light skeleton codegen improvements, from Kumar Kartikeya Dwivedi.

10) bpftool doc-generating script improvements, from Quentin Monnet.

11) Two libbpf v0.6 bug fixes, from Shuyi Cheng and Vincent Minet.

12) Deprecation warning fix for perf/bpf_counter, from Song Liu.

13) MAX_TAIL_CALL_CNT unification and MIPS build fix for libbpf,
from Tiezhu Yang.

14) BTF_KING_TYPE_TAG follow-up fixes, from Yonghong Song.

15) Selftests fixes and improvements, from Ilya Leoshkevich, Jean-Philippe
Brucker, Jiri Olsa, Maxim Mikityanskiy, Tirthendu Sarkar, Yucong Sun,
and others.

* https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next: (115 commits)
libbpf: Add "bool skipped" to struct bpf_map
libbpf: Fix typo in btf__dedup@LIBBPF_0.0.2 definition
bpftool: Switch bpf_object__load_xattr() to bpf_object__load()
selftests/bpf: Remove the only use of deprecated bpf_object__load_xattr()
selftests/bpf: Add test for libbpf's custom log_buf behavior
selftests/bpf: Replace all uses of bpf_load_btf() with bpf_btf_load()
libbpf: Deprecate bpf_object__load_xattr()
libbpf: Add per-program log buffer setter and getter
libbpf: Preserve kernel error code and remove kprobe prog type guessing
libbpf: Improve logging around BPF program loading
libbpf: Allow passing user log setting through bpf_object_open_opts
libbpf: Allow passing preallocated log_buf when loading BTF into kernel
libbpf: Add OPTS-based bpf_btf_load() API
libbpf: Fix bpf_prog_load() log_buf logic for log_level 0
samples/bpf: Remove unneeded variable
bpf: Remove redundant assignment to pointer t
selftests/bpf: Fix a compilation warning
perf/bpf_counter: Use bpf_map_create instead of bpf_create_map
samples: bpf: Fix 'unknown warning group' build warning on Clang
samples: bpf: Fix xdp_sample_user.o linking with Clang
...
====================

Link: https://lore.kernel.org/r/20211210234746.2100561-1-andrii@kernel.org
Signed-off-by: Jakub Kicinski <kuba@kernel.org>

show more ...

Merge branch 'Deprecate bpf_prog_load_xattr() API'

Andrii Nakryiko says:

====================

Few lines in the last patch to mark bpf_prog_load_xattr() deprecated required
a decent amount of clean

Merge branch 'Deprecate bpf_prog_load_xattr() API'

Andrii Nakryiko says:

====================

Few lines in the last patch to mark bpf_prog_load_xattr() deprecated required
a decent amount of clean ups in all the other patches. samples/bpf is big part
of the clean up.

This patch set also bumps libbpf version to 0.7, as libbpf v0.6 release will
be cut shortly.
====================

Signed-off-by: Alexei Starovoitov <ast@kernel.org>

show more ...

samples/bpf: Get rid of deprecated libbpf API uses

Replace deprecated APIs with new ones. Also mute source code using
deprecated AF_XDP (xsk.h). Figuring out what to do with all the AF_XDP
stuff is

samples/bpf: Get rid of deprecated libbpf API uses

Replace deprecated APIs with new ones. Also mute source code using
deprecated AF_XDP (xsk.h). Figuring out what to do with all the AF_XDP
stuff is a separate problem that should be solved with its own set of
changes.

Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://lore.kernel.org/bpf/20211201232824.3166325-9-andrii@kernel.org

show more ...

Merge remote-tracking branch 'net/master'

Merge branch 'next' into for-linus

Prepare input updates for 5.12 merge window.

Merge tag 'v5.11' into next

Merge with mainline to get latest APIs and device tree bindings.

Merge tag 'amba-make-remove-return-void' of https://git.pengutronix.de/git/ukl/linux into devel-stable

Tag for adaptions to struct amba_driver::remove changing prototype

Merge branch 'cpuidle-fix' into fixes

Merge drm/drm-next into drm-intel-gt-next

Backmerging to get a common base for merging topic branches between
drm-intel-next and drm-intel-gt-next.

Signed-off-by: Joonas Lahtinen <joonas.lahtinen@l

Merge drm/drm-next into drm-intel-gt-next

Backmerging to get a common base for merging topic branches between
drm-intel-next and drm-intel-gt-next.

Signed-off-by: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>

show more ...

Merge drm/drm-next into drm-intel-next

sync-up to not fall too much behind.

Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>

Merge branch 'for-linus' into for-next

Back-merge of 5.11-devel branch for syncing the result changes.

Signed-off-by: Takashi Iwai <tiwai@suse.de>

Merge tag 'asoc-fix-v5.11-rc2' of https://git.kernel.org/pub/scm/linux/kernel/git/broonie/sound into for-linus

ASoC: Fixes for v5.11

A collection of mostly driver specific fixes, plus a maintainers

Merge tag 'asoc-fix-v5.11-rc2' of https://git.kernel.org/pub/scm/linux/kernel/git/broonie/sound into for-linus

ASoC: Fixes for v5.11

A collection of mostly driver specific fixes, plus a maintainership
update for TI and a fix for DAPM driver removal paths.

show more ...

Merge drm/drm-next into drm-misc-next

Staying in sync to drm-next, and to be able to pull ttm fixes.

Signed-off-by: Maarten Lankhorst <maarten.lankhorst@linux.intel.com>

Merge tag 'v5.11-rc1' into spi-5.11

Linux 5.11-rc1

Merge tag 'v5.11-rc1' into regulator-5.11

Linux 5.11-rc1

Merge tag 'v5.11-rc1' into asoc-5.11

Linux 5.11-rc1

Merge remote-tracking branch 'torvalds/master' into perf/core

To pick up fixes and check what UAPI headers need to be synched.

Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>

Merge tag 'net-next-5.11' of git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net-next

Pull networking updates from Jakub Kicinski:
"Core:

- support "prefer busy polling" NAPI operation mod

Merge tag 'net-next-5.11' of git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net-next

Pull networking updates from Jakub Kicinski:
"Core:

- support "prefer busy polling" NAPI operation mode, where we defer
softirq for some time expecting applications to periodically busy
poll

- AF_XDP: improve efficiency by more batching and hindering the
adjacency cache prefetcher

- af_packet: make packet_fanout.arr size configurable up to 64K

- tcp: optimize TCP zero copy receive in presence of partial or
unaligned reads making zero copy a performance win for much smaller
messages

- XDP: add bulk APIs for returning / freeing frames

- sched: support fragmenting IP packets as they come out of conntrack

- net: allow virtual netdevs to forward UDP L4 and fraglist GSO skbs

BPF:

- BPF switch from crude rlimit-based to memcg-based memory accounting

- BPF type format information for kernel modules and related tracing
enhancements

- BPF implement task local storage for BPF LSM

- allow the FENTRY/FEXIT/RAW_TP tracing programs to use
bpf_sk_storage

Protocols:

- mptcp: improve multiple xmit streams support, memory accounting and
many smaller improvements

- TLS: support CHACHA20-POLY1305 cipher

- seg6: add support for SRv6 End.DT4/DT6 behavior

- sctp: Implement RFC 6951: UDP Encapsulation of SCTP

- ppp_generic: add ability to bridge channels directly

- bridge: Connectivity Fault Management (CFM) support as is defined
in IEEE 802.1Q section 12.14.

Drivers:

- mlx5: make use of the new auxiliary bus to organize the driver
internals

- mlx5: more accurate port TX timestamping support

- mlxsw:
- improve the efficiency of offloaded next hop updates by using
the new nexthop object API
- support blackhole nexthops
- support IEEE 802.1ad (Q-in-Q) bridging

- rtw88: major bluetooth co-existance improvements

- iwlwifi: support new 6 GHz frequency band

- ath11k: Fast Initial Link Setup (FILS)

- mt7915: dual band concurrent (DBDC) support

- net: ipa: add basic support for IPA v4.5

Refactor:

- a few pieces of in_interrupt() cleanup work from Sebastian Andrzej
Siewior

- phy: add support for shared interrupts; get rid of multiple driver
APIs and have the drivers write a full IRQ handler, slight growth
of driver code should be compensated by the simpler API which also
allows shared IRQs

- add common code for handling netdev per-cpu counters

- move TX packet re-allocation from Ethernet switch tag drivers to a
central place

- improve efficiency and rename nla_strlcpy

- number of W=1 warning cleanups as we now catch those in a patchwork
build bot

Old code removal:

- wan: delete the DLCI / SDLA drivers

- wimax: move to staging

- wifi: remove old WDS wifi bridging support"

* tag 'net-next-5.11' of git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net-next: (1922 commits)
net: hns3: fix expression that is currently always true
net: fix proc_fs init handling in af_packet and tls
nfc: pn533: convert comma to semicolon
af_vsock: Assign the vsock transport considering the vsock address flags
af_vsock: Set VMADDR_FLAG_TO_HOST flag on the receive path
vsock_addr: Check for supported flag values
vm_sockets: Add VMADDR_FLAG_TO_HOST vsock flag
vm_sockets: Add flags field in the vsock address data structure
net: Disable NETIF_F_HW_TLS_TX when HW_CSUM is disabled
tcp: Add logic to check for SYN w/ data in tcp_simple_retransmit
net: mscc: ocelot: install MAC addresses in .ndo_set_rx_mode from process context
nfc: s3fwrn5: Release the nfc firmware
net: vxget: clean up sparse warnings
mlxsw: spectrum_router: Use eXtended mezzanine to offload IPv4 router
mlxsw: spectrum: Set KVH XLT cache mode for Spectrum2/3
mlxsw: spectrum_router_xm: Introduce basic XM cache flushing
mlxsw: reg: Add Router LPM Cache Enable Register
mlxsw: reg: Add Router LPM Cache ML Delete Register
mlxsw: spectrum_router_xm: Implement L-value tracking for M-index
mlxsw: reg: Add XM Router M Table Register
...

show more ...

Merge https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next

Alexei Starovoitov says:

====================
pull-request: bpf-next 2020-12-03

The main changes are:

1) Support BTF in kernel m

Merge https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next

Alexei Starovoitov says:

====================
pull-request: bpf-next 2020-12-03

The main changes are:

1) Support BTF in kernel modules, from Andrii.

2) Introduce preferred busy-polling, from Björn.

3) bpf_ima_inode_hash() and bpf_bprm_opts_set() helpers, from KP Singh.

4) Memcg-based memory accounting for bpf objects, from Roman.

5) Allow bpf_{s,g}etsockopt from cgroup bind{4,6} hooks, from Stanislav.

* https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next: (118 commits)
selftests/bpf: Fix invalid use of strncat in test_sockmap
libbpf: Use memcpy instead of strncpy to please GCC
selftests/bpf: Add fentry/fexit/fmod_ret selftest for kernel module
selftests/bpf: Add tp_btf CO-RE reloc test for modules
libbpf: Support attachment of BPF tracing programs to kernel modules
libbpf: Factor out low-level BPF program loading helper
bpf: Allow to specify kernel module BTFs when attaching BPF programs
bpf: Remove hard-coded btf_vmlinux assumption from BPF verifier
selftests/bpf: Add CO-RE relocs selftest relying on kernel module BTF
selftests/bpf: Add support for marking sub-tests as skipped
selftests/bpf: Add bpf_testmod kernel module for testing
libbpf: Add kernel module BTF support for CO-RE relocations
libbpf: Refactor CO-RE relocs to not assume a single BTF object
libbpf: Add internal helper to load BTF data by FD
bpf: Keep module's btf_data_size intact after load
bpf: Fix bpf_put_raw_tracepoint()'s use of __module_address()
selftests/bpf: Add Userspace tests for TCP_WINDOW_CLAMP
bpf: Adds support for setting window clamp
samples/bpf: Fix spelling mistake "recieving" -> "receiving"
bpf: Fix cold build of test_progs-no_alu32
...
====================

Link: https://lore.kernel.org/r/20201204021936.85653-1-alexei.starovoitov@gmail.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>

show more ...

Merge branch 'switch to memcg-based memory accounting'

Roman Gushchin says:

====================

Currently bpf is using the memlock rlimit for the memory accounting.
This approach has its downside

Merge branch 'switch to memcg-based memory accounting'

Roman Gushchin says:

====================

Currently bpf is using the memlock rlimit for the memory accounting.
This approach has its downsides and over time has created a significant
amount of problems:

1) The limit is per-user, but because most bpf operations are performed
as root, the limit has a little value.

2) It's hard to come up with a specific maximum value. Especially because
the counter is shared with non-bpf use cases (e.g. memlock()).
Any specific value is either too low and creates false failures
or is too high and useless.

3) Charging is not connected to the actual memory allocation. Bpf code
should manually calculate the estimated cost and charge the counter,
and then take care of uncharging, including all fail paths.
It adds to the code complexity and makes it easy to leak a charge.

4) There is no simple way of getting the current value of the counter.
We've used drgn for it, but it's far from being convenient.

5) Cryptic -EPERM is returned on exceeding the limit. Libbpf even had
a function to "explain" this case for users.

6) rlimits are generally considered as (at least partially) obsolete.
They do not provide a comprehensive system for the control of physical
resources: memory, cpu, io etc. All resource control developments
in the recent years were related to cgroups.

In order to overcome these problems let's switch to the memory cgroup-based
memory accounting of bpf objects. With the recent addition of the percpu
memory accounting, now it's possible to provide a comprehensive accounting
of the memory used by bpf programs and maps.

This approach has the following advantages:
1) The limit is per-cgroup and hierarchical. It's way more flexible and allows
a better control over memory usage by different workloads.

2) The actual memory consumption is taken into account. It happens automatically
on the allocation time if __GFP_ACCOUNT flags is passed. Uncharging is also
performed automatically on releasing the memory. So the code on the bpf side
becomes simpler and safer.

3) There is a simple way to get the current value and statistics.

Cgroup-based accounting adds new requirements:
1) The kernel config should have CONFIG_CGROUPS and CONFIG_MEMCG_KMEM enabled.
These options are usually enabled, maybe excluding tiny builds for embedded
devices.
2) The system should have a configured cgroup hierarchy, including reasonable
memory limits and/or guarantees. Modern systems usually delegate this task
to systemd or similar task managers.

Without meeting these requirements there are no limits on how much memory bpf
can use and a non-root user is able to hurt the system by allocating too much.
But because per-user rlimits do not provide a functional system to protect
and manage physical resources anyway, anyone who seriously depends on it,
should use cgroups.

When a bpf map is created, the memory cgroup of the process which creates
the map is recorded. Subsequently all memory allocation related to the bpf map
are charged to the same cgroup. It includes allocations made from interrupts
and by any processes. Bpf program memory is charged to the memory cgroup of
a process which loads the program.

The patchset consists of the following parts:
1) 4 mm patches are required on the mm side, otherwise vmallocs cannot be mapped
to userspace
2) memcg-based accounting for various bpf objects: progs and maps
3) removal of the rlimit-based accounting
4) removal of rlimit adjustments in userspace samples

v9:
- always charge the saved memory cgroup, by Daniel, Toke and Alexei
- added bpf_map_kzalloc()
- rebase and minor fixes

v8:
- extended the cover letter to be more clear on new requirements, by Daniel
- an approximate value is provided by map memlock info, by Alexei

v7:
- introduced bpf_map_kmalloc_node() and bpf_map_alloc_percpu(), by Alexei
- switched allocations made from an interrupt context to new helpers,
by Daniel
- rebase and minor fixes

v6:
- rebased to the latest version of the remote charging API
- fixed signatures, added acks

v5:
- rebased to the latest version of the remote charging API
- implemented kmem accounting from an interrupt context, by Shakeel
- rebased to latest changes in mm allowed to map vmallocs to userspace
- fixed a build issue in kselftests, by Alexei
- fixed a use-after-free bug in bpf_map_free_deferred()
- added bpf line info coverage, by Shakeel
- split bpf map charging preparations into a separate patch

v4:
- covered allocations made from an interrupt context, by Daniel
- added some clarifications to the cover letter

v3:
- droped the userspace part for further discussions/refinements,
by Andrii and Song

v2:
- fixed build issue, caused by the remaining rlimit-based accounting
for sockhash maps
====================

Signed-off-by: Alexei Starovoitov <ast@kernel.org>

show more ...

bpf: samples: Do not touch RLIMIT_MEMLOCK

Since bpf is not using rlimit memlock for the memory accounting
and control, do not change the limit in sample applications.

Signed-off-by: Roman Gushchin

bpf: samples: Do not touch RLIMIT_MEMLOCK

Since bpf is not using rlimit memlock for the memory accounting
and control, do not change the limit in sample applications.

Signed-off-by: Roman Gushchin <guro@fb.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Acked-by: Song Liu <songliubraving@fb.com>
Link: https://lore.kernel.org/bpf/20201201215900.3569844-35-guro@fb.com

show more ...