Merge drm/drm-fixes into drm-misc-fixes

Forwarding to v6.6-rc1.

Signed-off-by: Thomas Zimmermann <tzimmermann@suse.de>

Merge tag 'docs-6.6' of git://git.lwn.net/linux

Pull documentation updates from Jonathan Corbet:
"Documentation work keeps chugging along; this includes:

- Work from Carlos Bilbao to integrate

Merge tag 'docs-6.6' of git://git.lwn.net/linux

Pull documentation updates from Jonathan Corbet:
"Documentation work keeps chugging along; this includes:

- Work from Carlos Bilbao to integrate rustdoc output into the
generated HTML documentation. This took some work to figure out how
to do it without slowing the docs build and without creating people
who don't have Rust installed, but Carlos got there

- Move the loongarch and mips architecture documentation under
Documentation/arch/

- Some more maintainer documentation from Jakub

... plus the usual assortment of updates, translations, and fixes"

* tag 'docs-6.6' of git://git.lwn.net/linux: (56 commits)
Docu: genericirq.rst: fix irq-example
input: docs: pxrc: remove reference to phoenix-sim
Documentation: serial-console: Fix literal block marker
docs/mm: remove references to hmm_mirror ops and clean typos
docs/zh_CN: correct regi_chg(),regi_add() to region_chg(),region_add()
Documentation: Fix typos
Documentation/ABI: Fix typos
scripts: kernel-doc: fix macro handling in enums
scripts: kernel-doc: parse DEFINE_DMA_UNMAP_[ADDR|LEN]
Documentation: riscv: Update boot image header since EFI stub is supported
Documentation: riscv: Add early boot document
Documentation: arm: Add bootargs to the table of added DT parameters
docs: kernel-parameters: Refer to the correct bitmap function
doc: update params of memhp_default_state=
docs: Add book to process/kernel-docs.rst
docs: sparse: fix invalid link addresses
docs: vfs: clean up after the iterate() removal
docs: Add a section on surveys to the researcher guidelines
docs: move mips under arch
docs: move loongarch under arch
...

show more ...

Documentation: Fix typos

Fix typos in Documentation.

Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Link: https://lore.kernel.org/r/20230814212822.193684-4-helgaas@kernel.org
Signed-off-by: Jon

Documentation: Fix typos

Fix typos in Documentation.

Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Link: https://lore.kernel.org/r/20230814212822.193684-4-helgaas@kernel.org
Signed-off-by: Jonathan Corbet <corbet@lwn.net>

show more ...

Merge branch 'next' into for-linus

Prepare input updates for 6.4 merge window.

Merge drm/drm-next into drm-intel-gt-next

Need to pull in commit from drm-next (earlier in drm-intel-next):

1eca0778f4b3 ("drm/i915: add struct i915_dsm to wrap dsm members together")

In order to

Merge drm/drm-next into drm-intel-gt-next

Need to pull in commit from drm-next (earlier in drm-intel-next):

1eca0778f4b3 ("drm/i915: add struct i915_dsm to wrap dsm members together")

In order to merge following patch to drm-intel-gt-next:

https://patchwork.freedesktop.org/patch/530942/?series=114925&rev=6

Signed-off-by: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>

show more ...

Merge drm/drm-next into drm-intel-next

Catch up with 6.3-rc cycle...

Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>

Merge remote-tracking branch 'drm/drm-next' into msm-next

Merge drm-next into msm-next to pick up external clk and PM dependencies
for improved a6xx GPU reset sequence.

Signed-off-by: Rob Clark <ro

Merge remote-tracking branch 'drm/drm-next' into msm-next

Merge drm-next into msm-next to pick up external clk and PM dependencies
for improved a6xx GPU reset sequence.

Signed-off-by: Rob Clark <robdclark@chromium.org>

show more ...

Merge tag 'v6.3-rc2' into next

Merge with mainline to get of_property_present() and other newer APIs.

Merge drm/drm-fixes into drm-misc-fixes

Backmerging to get latest upstream.

Signed-off-by: Thomas Zimmermann <tzimmermann@suse.de>

Merge drm/drm-next into drm-misc-next

Backmerging to get v6.3-rc1 and sync with the other DRM trees.

Signed-off-by: Thomas Zimmermann <tzimmermann@suse.de>

Merge remote-tracking branch 'acme/perf-tools' into perf-tools-next

To pick up perf-tools fixes just merged upstream.

Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>

Merge tag 'net-next-6.3' of git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net-next

Pull networking updates from Jakub Kicinski:
"Core:

- Add dedicated kmem_cache for typical/small skb->h

Merge tag 'net-next-6.3' of git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net-next

Pull networking updates from Jakub Kicinski:
"Core:

- Add dedicated kmem_cache for typical/small skb->head, avoid having
to access struct page at kfree time, and improve memory use.

- Introduce sysctl to set default RPS configuration for new netdevs.

- Define Netlink protocol specification format which can be used to
describe messages used by each family and auto-generate parsers.
Add tools for generating kernel data structures and uAPI headers.

- Expose all net/core sysctls inside netns.

- Remove 4s sleep in netpoll if carrier is instantly detected on
boot.

- Add configurable limit of MDB entries per port, and port-vlan.

- Continue populating drop reasons throughout the stack.

- Retire a handful of legacy Qdiscs and classifiers.

Protocols:

- Support IPv4 big TCP (TSO frames larger than 64kB).

- Add IP_LOCAL_PORT_RANGE socket option, to control local port range
on socket by socket basis.

- Track and report in procfs number of MPTCP sockets used.

- Support mixing IPv4 and IPv6 flows in the in-kernel MPTCP path
manager.

- IPv6: don't check net.ipv6.route.max_size and rely on garbage
collection to free memory (similarly to IPv4).

- Support Penultimate Segment Pop (PSP) flavor in SRv6 (RFC8986).

- ICMP: add per-rate limit counters.

- Add support for user scanning requests in ieee802154.

- Remove static WEP support.

- Support minimal Wi-Fi 7 Extremely High Throughput (EHT) rate
reporting.

- WiFi 7 EHT channel puncturing support (client & AP).

BPF:

- Add a rbtree data structure following the "next-gen data structure"
precedent set by recently added linked list, that is, by using
kfunc + kptr instead of adding a new BPF map type.

- Expose XDP hints via kfuncs with initial support for RX hash and
timestamp metadata.

- Add BPF_F_NO_TUNNEL_KEY extension to bpf_skb_set_tunnel_key to
better support decap on GRE tunnel devices not operating in collect
metadata.

- Improve x86 JIT's codegen for PROBE_MEM runtime error checks.

- Remove the need for trace_printk_lock for bpf_trace_printk and
bpf_trace_vprintk helpers.

- Extend libbpf's bpf_tracing.h support for tracing arguments of
kprobes/uprobes and syscall as a special case.

- Significantly reduce the search time for module symbols by
livepatch and BPF.

- Enable cpumasks to be used as kptrs, which is useful for tracing
programs tracking which tasks end up running on which CPUs in
different time intervals.

- Add support for BPF trampoline on s390x and riscv64.

- Add capability to export the XDP features supported by the NIC.

- Add __bpf_kfunc tag for marking kernel functions as kfuncs.

- Add cgroup.memory=nobpf kernel parameter option to disable BPF
memory accounting for container environments.

Netfilter:

- Remove the CLUSTERIP target. It has been marked as obsolete for
years, and we still have WARN splats wrt races of the out-of-band
/proc interface installed by this target.

- Add 'destroy' commands to nf_tables. They are identical to the
existing 'delete' commands, but do not return an error if the
referenced object (set, chain, rule...) did not exist.

Driver API:

- Improve cpumask_local_spread() locality to help NICs set the right
IRQ affinity on AMD platforms.

- Separate C22 and C45 MDIO bus transactions more clearly.

- Introduce new DCB table to control DSCP rewrite on egress.

- Support configuration of Physical Layer Collision Avoidance (PLCA)
Reconciliation Sublayer (RS) (802.3cg-2019). Modern version of
shared medium Ethernet.

- Support for MAC Merge layer (IEEE 802.3-2018 clause 99). Allowing
preemption of low priority frames by high priority frames.

- Add support for controlling MACSec offload using netlink SET.

- Rework devlink instance refcounts to allow registration and
de-registration under the instance lock. Split the code into
multiple files, drop some of the unnecessarily granular locks and
factor out common parts of netlink operation handling.

- Add TX frame aggregation parameters (for USB drivers).

- Add a new attr TCA_EXT_WARN_MSG to report TC (offload) warning
messages with notifications for debug.

- Allow offloading of UDP NEW connections via act_ct.

- Add support for per action HW stats in TC.

- Support hardware miss to TC action (continue processing in SW from
a specific point in the action chain).

- Warn if old Wireless Extension user space interface is used with
modern cfg80211/mac80211 drivers. Do not support Wireless
Extensions for Wi-Fi 7 devices at all. Everyone should switch to
using nl80211 interface instead.

- Improve the CAN bit timing configuration. Use extack to return
error messages directly to user space, update the SJW handling,
including the definition of a new default value that will benefit
CAN-FD controllers, by increasing their oscillator tolerance.

New hardware / drivers:

- Ethernet:
- nVidia BlueField-3 support (control traffic driver)
- Ethernet support for imx93 SoCs
- Motorcomm yt8531 gigabit Ethernet PHY
- onsemi NCN26000 10BASE-T1S PHY (with support for PLCA)
- Microchip LAN8841 PHY (incl. cable diagnostics and PTP)
- Amlogic gxl MDIO mux

- WiFi:
- RealTek RTL8188EU (rtl8xxxu)
- Qualcomm Wi-Fi 7 devices (ath12k)

- CAN:
- Renesas R-Car V4H

Drivers:

- Bluetooth:
- Set Per Platform Antenna Gain (PPAG) for Intel controllers.

- Ethernet NICs:
- Intel (1G, igc):
- support TSN / Qbv / packet scheduling features of i226 model
- Intel (100G, ice):
- use GNSS subsystem instead of TTY
- multi-buffer XDP support
- extend support for GPIO pins to E823 devices
- nVidia/Mellanox:
- update the shared buffer configuration on PFC commands
- implement PTP adjphase function for HW offset control
- TC support for Geneve and GRE with VF tunnel offload
- more efficient crypto key management method
- multi-port eswitch support
- Netronome/Corigine:
- add DCB IEEE support
- support IPsec offloading for NFP3800
- Freescale/NXP (enetc):
- support XDP_REDIRECT for XDP non-linear buffers
- improve reconfig, avoid link flap and waiting for idle
- support MAC Merge layer
- Other NICs:
- sfc/ef100: add basic devlink support for ef100
- ionic: rx_push mode operation (writing descriptors via MMIO)
- bnxt: use the auxiliary bus abstraction for RDMA
- r8169: disable ASPM and reset bus in case of tx timeout
- cpsw: support QSGMII mode for J721e CPSW9G
- cpts: support pulse-per-second output
- ngbe: add an mdio bus driver
- usbnet: optimize usbnet_bh() by avoiding unnecessary queuing
- r8152: handle devices with FW with NCM support
- amd-xgbe: support 10Mbps, 2.5GbE speeds and rx-adaptation
- virtio-net: support multi buffer XDP
- virtio/vsock: replace virtio_vsock_pkt with sk_buff
- tsnep: XDP support

- Ethernet high-speed switches:
- nVidia/Mellanox (mlxsw):
- add support for latency TLV (in FW control messages)
- Microchip (sparx5):
- separate explicit and implicit traffic forwarding rules, make
the implicit rules always active
- add support for egress DSCP rewrite
- IS0 VCAP support (Ingress Classification)
- IS2 VCAP filters (protos, L3 addrs, L4 ports, flags, ToS
etc.)
- ES2 VCAP support (Egress Access Control)
- support for Per-Stream Filtering and Policing (802.1Q,
8.6.5.1)

- Ethernet embedded switches:
- Marvell (mv88e6xxx):
- add MAB (port auth) offload support
- enable PTP receive for mv88e6390
- NXP (ocelot):
- support MAC Merge layer
- support for the the vsc7512 internal copper phys
- Microchip:
- lan9303: convert to PHYLINK
- lan966x: support TC flower filter statistics
- lan937x: PTP support for KSZ9563/KSZ8563 and LAN937x
- lan937x: support Credit Based Shaper configuration
- ksz9477: support Energy Efficient Ethernet
- other:
- qca8k: convert to regmap read/write API, use bulk operations
- rswitch: Improve TX timestamp accuracy

- Intel WiFi (iwlwifi):
- EHT (Wi-Fi 7) rate reporting
- STEP equalizer support: transfer some STEP (connection to radio
on platforms with integrated wifi) related parameters from the
BIOS to the firmware.

- Qualcomm 802.11ax WiFi (ath11k):
- IPQ5018 support
- Fine Timing Measurement (FTM) responder role support
- channel 177 support

- MediaTek WiFi (mt76):
- per-PHY LED support
- mt7996: EHT (Wi-Fi 7) support
- Wireless Ethernet Dispatch (WED) reset support
- switch to using page pool allocator

- RealTek WiFi (rtw89):
- support new version of Bluetooth co-existance

- Mobile:
- rmnet: support TX aggregation"

* tag 'net-next-6.3' of git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net-next: (1872 commits)
page_pool: add a comment explaining the fragment counter usage
net: ethtool: fix __ethtool_dev_mm_supported() implementation
ethtool: pse-pd: Fix double word in comments
xsk: add linux/vmalloc.h to xsk.c
sefltests: netdevsim: wait for devlink instance after netns removal
selftest: fib_tests: Always cleanup before exit
net/mlx5e: Align IPsec ASO result memory to be as required by hardware
net/mlx5e: TC, Set CT miss to the specific ct action instance
net/mlx5e: Rename CHAIN_TO_REG to MAPPED_OBJ_TO_REG
net/mlx5: Refactor tc miss handling to a single function
net/mlx5: Kconfig: Make tc offload depend on tc skb extension
net/sched: flower: Support hardware miss to tc action
net/sched: flower: Move filter handle initialization earlier
net/sched: cls_api: Support hardware miss to tc action
net/sched: Rename user cookie and act cookie
sfc: fix builds without CONFIG_RTC_LIB
sfc: clean up some inconsistent indentings
net/mlx4_en: Introduce flexible array to silence overflow warning
net: lan966x: Fix possible deadlock inside PTP
net/ulp: Remove redundant ->clone() test in inet_clone_ulp().
...

show more ...

Merge tag 'for-netdev' of https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next

Daniel Borkmann says:

====================
pull-request: bpf-next 2023-02-17

We've added 64 non-merge commits

Merge tag 'for-netdev' of https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next

Daniel Borkmann says:

====================
pull-request: bpf-next 2023-02-17

We've added 64 non-merge commits during the last 7 day(s) which contain
a total of 158 files changed, 4190 insertions(+), 988 deletions(-).

The main changes are:

1) Add a rbtree data structure following the "next-gen data structure"
precedent set by recently-added linked-list, that is, by using
kfunc + kptr instead of adding a new BPF map type, from Dave Marchevsky.

2) Add a new benchmark for hashmap lookups to BPF selftests,
from Anton Protopopov.

3) Fix bpf_fib_lookup to only return valid neighbors and add an option
to skip the neigh table lookup, from Martin KaFai Lau.

4) Add cgroup.memory=nobpf kernel parameter option to disable BPF memory
accouting for container environments, from Yafang Shao.

5) Batch of ice multi-buffer and driver performance fixes,
from Alexander Lobakin.

6) Fix a bug in determining whether global subprog's argument is
PTR_TO_CTX, which is based on type names which breaks kprobe progs,
from Andrii Nakryiko.

7) Prep work for future -mcpu=v4 LLVM option which includes usage of
BPF_ST insn. Thus improve BPF_ST-related value tracking in verifier,
from Eduard Zingerman.

8) More prep work for later building selftests with Memory Sanitizer
in order to detect usages of undefined memory, from Ilya Leoshkevich.

9) Fix xsk sockets to check IFF_UP earlier to avoid a NULL pointer
dereference via sendmsg(), from Maciej Fijalkowski.

10) Implement BPF trampoline for RV64 JIT compiler, from Pu Lehui.

11) Fix BPF memory allocator in combination with BPF hashtab where it could
corrupt special fields e.g. used in bpf_spin_lock, from Hou Tao.

12) Fix LoongArch BPF JIT to always use 4 instructions for function
address so that instruction sequences don't change between passes,
from Hengqi Chen.

* tag 'for-netdev' of https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next: (64 commits)
selftests/bpf: Add bpf_fib_lookup test
bpf: Add BPF_FIB_LOOKUP_SKIP_NEIGH for bpf_fib_lookup
riscv, bpf: Add bpf trampoline support for RV64
riscv, bpf: Add bpf_arch_text_poke support for RV64
riscv, bpf: Factor out emit_call for kernel and bpf context
riscv: Extend patch_text for multiple instructions
Revert "bpf, test_run: fix &xdp_frame misplacement for LIVE_FRAMES"
selftests/bpf: Add global subprog context passing tests
selftests/bpf: Convert test_global_funcs test to test_loader framework
bpf: Fix global subprog context argument resolution logic
LoongArch, bpf: Use 4 instructions for function address in JIT
bpf: bpf_fib_lookup should not return neigh in NUD_FAILED state
bpf: Disable bh in bpf_test_run for xdp and tc prog
xsk: check IFF_UP earlier in Tx path
Fix typos in selftest/bpf files
selftests/bpf: Use bpf_{btf,link,map,prog}_get_info_by_fd()
samples/bpf: Use bpf_{btf,link,map,prog}_get_info_by_fd()
bpftool: Use bpf_{btf,link,map,prog}_get_info_by_fd()
libbpf: Use bpf_{btf,link,map,prog}_get_info_by_fd()
libbpf: Introduce bpf_{btf,link,map,prog}_get_info_by_fd()
...
====================

Link: https://lore.kernel.org/r/20230217221737.31122-1-daniel@iogearbox.net
Signed-off-by: Jakub Kicinski <kuba@kernel.org>

show more ...

Documentation: bpf: Add missing line break separator in node_data struct code block

Stephen Rothwell reported htmldocs warning when merging bpf-next tree,
which was the same warning as reported by k

Documentation: bpf: Add missing line break separator in node_data struct code block

Stephen Rothwell reported htmldocs warning when merging bpf-next tree,
which was the same warning as reported by kernel test robot:

Documentation/bpf/graph_ds_impl.rst:62: ERROR: Error in "code-block" directive:
maximum 1 argument(s) allowed, 12 supplied.

The error is due to Sphinx confuses node_data struct declaration with
code-block directive option.

Fix the warning by separating the code-block marker with node_data struct
declaration.

Link: https://lore.kernel.org/linux-next/20230215144505.4751d823@canb.auug.org.au/
Link: https://lore.kernel.org/linux-doc/202302151123.wUE5FYFx-lkp@intel.com/
Fixes: c31315c3aa0929 ("bpf, documentation: Add graph documentation for non-owning refs")
Reported-by: Stephen Rothwell <sfr@canb.auug.org.au>
Reported-by: kernel test robot <lkp@intel.com>
Signed-off-by: Bagas Sanjaya <bagasdotme@gmail.com>
Link: https://lore.kernel.org/r/20230215123253.41552-3-bagasdotme@gmail.com
Signed-off-by: Alexei Starovoitov <ast@kernel.org>

show more ...

Merge branch 'BPF rbtree next-gen datastructure'

Dave Marchevsky says:

====================
This series adds a rbtree datastructure following the "next-gen
datastructure" precedent set by recently-

Merge branch 'BPF rbtree next-gen datastructure'

Dave Marchevsky says:

====================
This series adds a rbtree datastructure following the "next-gen
datastructure" precedent set by recently-added linked-list [0]. This is
a reimplementation of previous rbtree RFC [1] to use kfunc + kptr
instead of adding a new map type. This series adds a smaller set of API
functions than that RFC - just the minimum needed to support current
cgfifo example scheduler in ongoing sched_ext effort [2], namely:

bpf_rbtree_add
bpf_rbtree_remove
bpf_rbtree_first

The meat of this series is bugfixes and verifier infra work to support
these API functions. Adding more rbtree kfuncs in future patches should
be straightforward as a result.

First, the series refactors and extends linked_list's release_on_unlock
logic. The concept of "reference to node that was added to data
structure" is formalized as "non-owning reference". From linked_list's
perspective this non-owning reference after
linked_list_push_{front,back} has same semantics as release_on_unlock,
with the addition of writes to such references being valid in the
critical section. Such references are no longer marked PTR_UNTRUSTED.
Patches 2 and 13 go into more detail.

The series then adds rbtree API kfuncs and necessary verifier support
for them - namely support for callback args to kfuncs and some
non-owning reference interactions that linked_list didn't need.

BPF rbtree uses struct rb_root_cached + existing rbtree lib under the
hood. From the BPF program writer's perspective, a BPF rbtree is very
similar to existing linked list. Consider the following example:

struct node_data {
long key;
long data;
struct bpf_rb_node node;
}

static bool less(struct bpf_rb_node *a, const struct bpf_rb_node *b)
{
struct node_data *node_a;
struct node_data *node_b;

node_a = container_of(a, struct node_data, node);
node_b = container_of(b, struct node_data, node);

return node_a->key < node_b->key;
}

private(A) struct bpf_spin_lock glock;
private(A) struct bpf_rb_root groot __contains(node_data, node);

/* ... in BPF program */
struct node_data *n, *m;
struct bpf_rb_node *res;

n = bpf_obj_new(typeof(*n));
if (!n)
/* skip */
n->key = 5;
n->data = 10;

bpf_spin_lock(&glock);
bpf_rbtree_add(&groot, &n->node, less);
bpf_spin_unlock(&glock);

bpf_spin_lock(&glock);
res = bpf_rbtree_first(&groot);
if (!res)
/* skip */
res = bpf_rbtree_remove(&groot, res);
if (!res)
/* skip */
bpf_spin_unlock(&glock);

m = container_of(res, struct node_data, node);
bpf_obj_drop(m);

Some obvious similarities:

* Special bpf_rb_root and bpf_rb_node types have same semantics
as bpf_list_head and bpf_list_node, respectively
* __contains is used to associated node type with root
* The spin_lock associated with a rbtree must be held when using
rbtree API kfuncs
* Nodes are allocated via bpf_obj_new and dropped via bpf_obj_drop
* Rbtree takes ownership of node lifetime when a node is added.
Removing a node gives ownership back to the program, requiring a
bpf_obj_drop before program exit

Some new additions as well:

* Support for callbacks in kfunc args is added to enable 'less'
callback use above
* bpf_rbtree_first is the first graph API function to return a
non-owning reference instead of convering an arg from own->non-own
* Because all references to nodes already added to the rbtree are
non-owning, bpf_rbtree_remove must accept such a reference in order
to remove it from the tree

Summary of patches:
Patches 1 - 5 implement the meat of rbtree-specific support in this
series, gradually building up to implemented kfuncs that verify as
expected.

Patch 6 adds the bpf_rbtree_{add,first,remove} to bpf_experimental.h.

Patch 7 adds tests, Patch 9 adds documentation.

[0]: lore.kernel.org/bpf/20221118015614.2013203-1-memxor@gmail.com
[1]: lore.kernel.org/bpf/20220830172759.4069786-1-davemarchevsky@fb.com
[2]: lore.kernel.org/bpf/20221130082313.3241517-1-tj@kernel.org

Changelog:

v5 -> v6: lore.kernel.org/bpf/20230212092715.1422619-1-davemarchevsky@fb.com/

Patch #'s below refer to the patch's number in v5 unless otherwise stated.

* General / Patch 1
* Rebase onto latest bpf-next: "bpf: Migrate release_on_unlock logic to non-owning ref semantics"
* This was Patch 1 of v4, was applied, not included in v6

* Patch 3 - "bpf: Add bpf_rbtree_{add,remove,first} kfuncs"
* Use bpf_callback_t instead of plain-C fn ptr for bpf_rbtree_add. This
necessitated having bpf_rbtree_add duplicate rbtree_add's functionality.
Wrapper function was used w/ internal __bpf_rbtree_add helper so that
bpf_experimental.h proto could continue to use plain-C fn ptr so BPF progs
could benefit from typechecking (Alexei)

v4 -> v5: lore.kernel.org/bpf/20230209174144.3280955-1-davemarchevsky@fb.com/

Patch #'s below refer to the patch's number in v4 unless otherwise stated.

* General
* Rebase onto latest bpf-next: "Merge branch 'bpf, mm: introduce cgroup.memory=nobpf'"

* Patches 1-3 are squashed into "bpf: Migrate release_on_unlock logic to non-owning ref semantics".
* Added type_is_non_owning_ref helper (Alexei)
* Use a NON_OWN_REF type flag instead of separate bool (Alexei)

* Patch 8 - "bpf: Special verifier handling for bpf_rbtree_{remove, first}"
* When doing btf_parse_fields, reject structs with both bpf_list_node and
bpf_rb_node fields. This is a temporary measure that can be removed after
"collection identity" followup. See comment added in btf_parse_fields for
more detail (Kumar, Alexei)
* Add linked_list BTF test exercising check added to btf_parse_fields
* Minor changes and moving around of some reg type checks due to NON_OWN_REF type flag
introduction

* Patch 10 - "selftests/bpf: Add rbtree selftests"
* Migrate failure tests to RUN_TESTS, __failure, __msg() framework (Alexei)

v3 -> v4: lore.kernel.org/bpf/20230131180016.3368305-1-davemarchevsky@fb.com/

Patch #'s below refer to the patch's number in v3 unless otherwise stated.

* General
* Don't base this series on "bpf: Refactor release_regno searching logic",
which was submitted separately as a refactor.
* Rebase onto latest bpf-next: "samples/bpf: Add openat2() enter/exit tracepoint to syscall_tp sample"

* Patch 2 - "bpf: Improve bpf_reg_state space usage for non-owning ref lock"
* print_verifier_state change was adding redundant comma after "non_own_ref",
fix it to put comma in correct place
* invalidate_non_owning_refs no longer needs to take bpf_active_lock param,
since any non-owning ref reg in env's cur_state is assumed to use that
state's active_lock (Alexei)
* invalidate_non_owning_refs' reg loop should check that the reg being
inspected is a PTR_TO_BTF_ID before checking reg->non_owning_ref_lock,
since that field is part of a union and may be filled w/ meaningless bytes
if reg != PTR_TO_BTF_ID (Alexei)

* Patch 3 - "selftests/bpf: Update linked_list tests for non-owning ref semantics"
* Change the string searched for by the following tests:
* linked_list/incorrect_node_off1
* linked_list/double_push_front
* linked_list/double_push_back

necessary due to rebase / dropping of "release_regno searching logic" patch
(see "General" changes)

* Patch 8 - "bpf: Special verifier handling for bpf_rbtree_{remove, first}"
* Just call invalidate_non_owning_refs w/ env instead of env, lock. (see
Patch 2 changes)

* Patch 11 - "bpf, documentation: Add graph documentation for non-owning refs"
* Fix documentation formatting and improve content (David)
* v3's version of patch 11 was missing some changes, v4's patch 11 is still
addressing David's feedback from v2

v2 -> v3: lore.kernel.org/bpf/20221217082506.1570898-1-davemarchevsky@fb.com/

Patch #'s below refer to the patch's number in v2 unless otherwise stated.

* Patch 1 - "bpf: Support multiple arg regs w/ ref_obj_id for kfuncs"
* No longer needed as v3 doesn't have multiple ref_obj_id arg regs
* The refactoring pieces were submitted separately
(https://lore.kernel.org/bpf/20230121002417.1684602-1-davemarchevsky@fb.com/)

* Patch 2 - "bpf: Migrate release_on_unlock logic to non-owning ref semantics"
* Remove KF_RELEASE_NON_OWN flag from list API push methods, just match
against specific kfuncs for now (Alexei, David)
* Separate "release non owning reference" logic from KF_RELEASE logic
(Alexei, David)
* reg_find_field_offset now correctly tests 'rec' instead of 'reg' after
calling reg_btf_record (Dan Carpenter)

* New patch added after Patch 2 - "bpf: Improve bpf_reg_state space usage for non-owning ref lock"
* Eliminates extra bpf_reg_state memory usage by using a bool instead of
copying lock identity

* Patch 4 - "bpf: rename list_head -> graph_root in field info types"
* v2's version was applied to bpf-next, not including in respins

* Patch 6 - "bpf: Add bpf_rbtree_{add,remove,first} kfuncs"
* Remove KF_RELEASE_NON_OWN flag from rbtree_add, just add it to specific
kfunc matching (Alexei, David)

* Patch 9 - "bpf: Special verifier handling for bpf_rbtree_{remove, first}"
* Remove KF_INVALIDATE_NON_OWN kfunc flag, just match against specific kfunc
for now (Alexei, David)

* Patch 11 - "libbpf: Make BTF mandatory if program BTF has spin_lock or alloc_obj type"
* Drop for now, will submit separately

* Patch 12 - "selftests/bpf: Add rbtree selftests"
* Some expected-failure tests have different error messages due to "release
non-owning reference logic" being separated from KF_RELEASE logic in Patch
2 changes

* Patch 13 - "bpf, documentation: Add graph documentation for non-owning refs"
* Fix documentation formatting and improve content (David)

v1 -> v2: lore.kernel.org/bpf/20221206231000.3180914-1-davemarchevsky@fb.com/

Series-wide changes:
* Rename datastructure_{head,node,api} -> graph_{root,node,api} (Alexei)
* "graph datastructure" in patch summaries to refer to linked_list + rbtree
instead of "next-gen datastructure" (Alexei)
* Move from hacky marking of non-owning references as PTR_UNTRUSTED to
cleaner implementation (Alexei)
* Add invalidation of non-owning refs to rbtree_remove (Kumar, Alexei)

Patch #'s below refer to the patch's number in v1 unless otherwise stated.

Note that in v1 most of the meaty verifier changes were in the latter half
of the series. Here, about half of that complexity has been moved to
"bpf: Migrate release_on_unlock logic to non-owning ref semantics" - was Patch
3 in v1.

* Patch 1 - "bpf: Loosen alloc obj test in verifier's reg_btf_record"
* Was applied, dropped from further iterations

* Patch 2 - "bpf: map_check_btf should fail if btf_parse_fields fails"
* Dropped in favor of verifier check-on-use: when some normal verifier
checking expects the map to have btf_fields correctly parsed, it won't
find any and verification will fail

* New patch added before Patch 3 - "bpf: Support multiple arg regs w/ ref_obj_id for kfuncs"
* Addition of KF_RELEASE_NON_OWN flag, which requires KF_RELEASE, and tagging
of bpf_list_push_{front,back} KF_RELEASE | KF_RELEASE_NON_OWN, means that
list-in-list push_{front,back} will trigger "only one ref_obj_id arg reg"
logic. This is because "head" arg to those functions can be a list-in-list,
which itself can be an owning reference with ref_obj_id. So need to
support multiple ref_obj_id for release kfuncs.

* Patch 3 - "bpf: Minor refactor of ref_set_release_on_unlock"
* Now a major refactor w/ a rename to reflect this
* "bpf: Migrate release_on_unlock logic to non-owning ref semantics"
* Replaces release_on_unlock with active_lock logic as discussed in v1

* New patch added after Patch 3 - "selftests/bpf: Update linked_list tests for non_owning_ref logic"
* Removes "write after push" linked_list failure tests - no longer failure
scenarios.

* Patch 4 - "bpf: rename list_head -> datastructure_head in field info types"
* rename to graph_root instead. Similar renamings across the series - see
series-wide changes.

* Patch 5 - "bpf: Add basic bpf_rb_{root,node} support"
* OWNER_FIELD_MASK -> GRAPH_ROOT_MASK, OWNEE_FIELD_MASK -> GRAPH_NODE_MASK,
and change of "owner"/"ownee" in big btf_check_and_fixup_fields comment to
"root"/"node" (Alexei)

* Patch 6 - "bpf: Add bpf_rbtree_{add,remove,first} kfuncs"
* bpf_rbtree_remove can no longer return NULL. v2 continues v1's "use type
system to prevent remove of node that isn't in a datastructure" approach,
so rbtree_remove should never have been able to return NULL

* Patch 7 - "bpf: Add support for bpf_rb_root and bpf_rb_node in kfunc args"
* is_bpf_datastructure_api_kfunc -> is_bpf_graph_api_kfunc (Alexei)

* Patch 8 - "bpf: Add callback validation to kfunc verifier logic"
* Explicitly disallow rbtree_remove in rbtree callback
* Explicitly disallow bpf_spin_{lock,unlock} call in rbtree callback,
preventing possibility of "unbalanced" unlock (Alexei)

* Patch 10 - "bpf, x86: BPF_PROBE_MEM handling for insn->off < 0"
* Now that non-owning refs aren't marked PTR_UNTRUSTED it's not necessary to
include this patch as part of the series
* After conversation w/ Alexei, did another pass and submitted as an
independent series (lore.kernel.org/bpf/20221213182726.325137-1-davemarchevsky@fb.com/)

* Patch 13 - "selftests/bpf: Add rbtree selftests"
* Since bpf_rbtree_remove can no longer return null, remove null checks
* Remove test confirming that rbtree_first isn't allowed in callback. We want
this to be possible
* Add failure test confirming that rbtree_remove's new non-owning reference
invalidation behavior behaves as expected
* Add SEC("license") to rbtree_btf_fail__* progs. They were previously
failing due to lack of this section. Now they're failing for correct
reasons.
* rbtree_btf_fail__add_wrong_type.c - add locking around rbtree_add, rename
the bpf prog to something reasonable

* New patch added after patch 13 - "bpf, documentation: Add graph documentation for non-owning refs"
* Summarizes details of owning and non-owning refs which we hashed out in
v1
====================

Signed-off-by: Alexei Starovoitov <ast@kernel.org>

show more ...

bpf, documentation: Add graph documentation for non-owning refs

It is difficult to intuit the semantics of owning and non-owning
references from verifier code. In order to keep the high-level detail

bpf, documentation: Add graph documentation for non-owning refs

It is difficult to intuit the semantics of owning and non-owning
references from verifier code. In order to keep the high-level details
from being lost in the mailing list, this patch adds documentation
explaining semantics and details.

The target audience of doc added in this patch is folks working on BPF
internals, as there's focus on "what should the verifier do here". Via
reorganization or copy-and-paste, much of the content can probably be
repurposed for BPF program writer audience as well.

Signed-off-by: Dave Marchevsky <davemarchevsky@fb.com>
Link: https://lore.kernel.org/r/20230214004017.2534011-9-davemarchevsky@fb.com
Signed-off-by: Alexei Starovoitov <ast@kernel.org>

show more ...