132c0f0beSMauro Carvalho Chehab.. SPDX-License-Identifier: GPL-2.0 232c0f0beSMauro Carvalho Chehab.. include:: <isonum.txt> 332c0f0beSMauro Carvalho Chehab 432c0f0beSMauro Carvalho Chehab=============================================== 532c0f0beSMauro Carvalho ChehabEthernet switch device driver model (switchdev) 632c0f0beSMauro Carvalho Chehab=============================================== 732c0f0beSMauro Carvalho Chehab 832c0f0beSMauro Carvalho ChehabCopyright |copy| 2014 Jiri Pirko <jiri@resnulli.us> 932c0f0beSMauro Carvalho Chehab 1032c0f0beSMauro Carvalho ChehabCopyright |copy| 2014-2015 Scott Feldman <sfeldma@gmail.com> 1132c0f0beSMauro Carvalho Chehab 1232c0f0beSMauro Carvalho Chehab 1332c0f0beSMauro Carvalho ChehabThe Ethernet switch device driver model (switchdev) is an in-kernel driver 1432c0f0beSMauro Carvalho Chehabmodel for switch devices which offload the forwarding (data) plane from the 1532c0f0beSMauro Carvalho Chehabkernel. 1632c0f0beSMauro Carvalho Chehab 1732c0f0beSMauro Carvalho ChehabFigure 1 is a block diagram showing the components of the switchdev model for 1832c0f0beSMauro Carvalho Chehaban example setup using a data-center-class switch ASIC chip. Other setups 1932c0f0beSMauro Carvalho Chehabwith SR-IOV or soft switches, such as OVS, are possible. 2032c0f0beSMauro Carvalho Chehab 2132c0f0beSMauro Carvalho Chehab:: 2232c0f0beSMauro Carvalho Chehab 2332c0f0beSMauro Carvalho Chehab 2432c0f0beSMauro Carvalho Chehab User-space tools 2532c0f0beSMauro Carvalho Chehab 2632c0f0beSMauro Carvalho Chehab user space | 2732c0f0beSMauro Carvalho Chehab +-------------------------------------------------------------------+ 2832c0f0beSMauro Carvalho Chehab kernel | Netlink 2932c0f0beSMauro Carvalho Chehab | 3032c0f0beSMauro Carvalho Chehab +--------------+-------------------------------+ 3132c0f0beSMauro Carvalho Chehab | Network stack | 3232c0f0beSMauro Carvalho Chehab | (Linux) | 3332c0f0beSMauro Carvalho Chehab | | 3432c0f0beSMauro Carvalho Chehab +----------------------------------------------+ 3532c0f0beSMauro Carvalho Chehab 3632c0f0beSMauro Carvalho Chehab sw1p2 sw1p4 sw1p6 3732c0f0beSMauro Carvalho Chehab sw1p1 + sw1p3 + sw1p5 + eth1 3832c0f0beSMauro Carvalho Chehab + | + | + | + 3932c0f0beSMauro Carvalho Chehab | | | | | | | 4032c0f0beSMauro Carvalho Chehab +--+----+----+----+----+----+---+ +-----+-----+ 4132c0f0beSMauro Carvalho Chehab | Switch driver | | mgmt | 4232c0f0beSMauro Carvalho Chehab | (this document) | | driver | 4332c0f0beSMauro Carvalho Chehab | | | | 4432c0f0beSMauro Carvalho Chehab +--------------+----------------+ +-----------+ 4532c0f0beSMauro Carvalho Chehab | 4632c0f0beSMauro Carvalho Chehab kernel | HW bus (eg PCI) 4732c0f0beSMauro Carvalho Chehab +-------------------------------------------------------------------+ 4832c0f0beSMauro Carvalho Chehab hardware | 4932c0f0beSMauro Carvalho Chehab +--------------+----------------+ 5032c0f0beSMauro Carvalho Chehab | Switch device (sw1) | 5132c0f0beSMauro Carvalho Chehab | +----+ +--------+ 5232c0f0beSMauro Carvalho Chehab | | v offloaded data path | mgmt port 5332c0f0beSMauro Carvalho Chehab | | | | 5432c0f0beSMauro Carvalho Chehab +--|----|----+----+----+----+---+ 5532c0f0beSMauro Carvalho Chehab | | | | | | 5632c0f0beSMauro Carvalho Chehab + + + + + + 5732c0f0beSMauro Carvalho Chehab p1 p2 p3 p4 p5 p6 5832c0f0beSMauro Carvalho Chehab 5932c0f0beSMauro Carvalho Chehab front-panel ports 6032c0f0beSMauro Carvalho Chehab 6132c0f0beSMauro Carvalho Chehab 6232c0f0beSMauro Carvalho Chehab Fig 1. 6332c0f0beSMauro Carvalho Chehab 6432c0f0beSMauro Carvalho Chehab 6532c0f0beSMauro Carvalho ChehabInclude Files 6632c0f0beSMauro Carvalho Chehab------------- 6732c0f0beSMauro Carvalho Chehab 6832c0f0beSMauro Carvalho Chehab:: 6932c0f0beSMauro Carvalho Chehab 7032c0f0beSMauro Carvalho Chehab #include <linux/netdevice.h> 7132c0f0beSMauro Carvalho Chehab #include <net/switchdev.h> 7232c0f0beSMauro Carvalho Chehab 7332c0f0beSMauro Carvalho Chehab 7432c0f0beSMauro Carvalho ChehabConfiguration 7532c0f0beSMauro Carvalho Chehab------------- 7632c0f0beSMauro Carvalho Chehab 7732c0f0beSMauro Carvalho ChehabUse "depends NET_SWITCHDEV" in driver's Kconfig to ensure switchdev model 7832c0f0beSMauro Carvalho Chehabsupport is built for driver. 7932c0f0beSMauro Carvalho Chehab 8032c0f0beSMauro Carvalho Chehab 8132c0f0beSMauro Carvalho ChehabSwitch Ports 8232c0f0beSMauro Carvalho Chehab------------ 8332c0f0beSMauro Carvalho Chehab 8432c0f0beSMauro Carvalho ChehabOn switchdev driver initialization, the driver will allocate and register a 8532c0f0beSMauro Carvalho Chehabstruct net_device (using register_netdev()) for each enumerated physical switch 8632c0f0beSMauro Carvalho Chehabport, called the port netdev. A port netdev is the software representation of 8732c0f0beSMauro Carvalho Chehabthe physical port and provides a conduit for control traffic to/from the 8832c0f0beSMauro Carvalho Chehabcontroller (the kernel) and the network, as well as an anchor point for higher 8932c0f0beSMauro Carvalho Chehablevel constructs such as bridges, bonds, VLANs, tunnels, and L3 routers. Using 9032c0f0beSMauro Carvalho Chehabstandard netdev tools (iproute2, ethtool, etc), the port netdev can also 9132c0f0beSMauro Carvalho Chehabprovide to the user access to the physical properties of the switch port such 9232c0f0beSMauro Carvalho Chehabas PHY link state and I/O statistics. 9332c0f0beSMauro Carvalho Chehab 9432c0f0beSMauro Carvalho ChehabThere is (currently) no higher-level kernel object for the switch beyond the 9532c0f0beSMauro Carvalho Chehabport netdevs. All of the switchdev driver ops are netdev ops or switchdev ops. 9632c0f0beSMauro Carvalho Chehab 9732c0f0beSMauro Carvalho ChehabA switch management port is outside the scope of the switchdev driver model. 9832c0f0beSMauro Carvalho ChehabTypically, the management port is not participating in offloaded data plane and 9932c0f0beSMauro Carvalho Chehabis loaded with a different driver, such as a NIC driver, on the management port 10032c0f0beSMauro Carvalho Chehabdevice. 10132c0f0beSMauro Carvalho Chehab 10232c0f0beSMauro Carvalho ChehabSwitch ID 10332c0f0beSMauro Carvalho Chehab^^^^^^^^^ 10432c0f0beSMauro Carvalho Chehab 10532c0f0beSMauro Carvalho ChehabThe switchdev driver must implement the net_device operation 10632c0f0beSMauro Carvalho Chehabndo_get_port_parent_id for each port netdev, returning the same physical ID for 10732c0f0beSMauro Carvalho Chehabeach port of a switch. The ID must be unique between switches on the same 10832c0f0beSMauro Carvalho Chehabsystem. The ID does not need to be unique between switches on different 10932c0f0beSMauro Carvalho Chehabsystems. 11032c0f0beSMauro Carvalho Chehab 11132c0f0beSMauro Carvalho ChehabThe switch ID is used to locate ports on a switch and to know if aggregated 11232c0f0beSMauro Carvalho Chehabports belong to the same switch. 11332c0f0beSMauro Carvalho Chehab 11432c0f0beSMauro Carvalho ChehabPort Netdev Naming 11532c0f0beSMauro Carvalho Chehab^^^^^^^^^^^^^^^^^^ 11632c0f0beSMauro Carvalho Chehab 11732c0f0beSMauro Carvalho ChehabUdev rules should be used for port netdev naming, using some unique attribute 11832c0f0beSMauro Carvalho Chehabof the port as a key, for example the port MAC address or the port PHYS name. 11932c0f0beSMauro Carvalho ChehabHard-coding of kernel netdev names within the driver is discouraged; let the 12032c0f0beSMauro Carvalho Chehabkernel pick the default netdev name, and let udev set the final name based on a 12132c0f0beSMauro Carvalho Chehabport attribute. 12232c0f0beSMauro Carvalho Chehab 12332c0f0beSMauro Carvalho ChehabUsing port PHYS name (ndo_get_phys_port_name) for the key is particularly 12432c0f0beSMauro Carvalho Chehabuseful for dynamically-named ports where the device names its ports based on 12532c0f0beSMauro Carvalho Chehabexternal configuration. For example, if a physical 40G port is split logically 12632c0f0beSMauro Carvalho Chehabinto 4 10G ports, resulting in 4 port netdevs, the device can give a unique 12732c0f0beSMauro Carvalho Chehabname for each port using port PHYS name. The udev rule would be:: 12832c0f0beSMauro Carvalho Chehab 12932c0f0beSMauro Carvalho Chehab SUBSYSTEM=="net", ACTION=="add", ATTR{phys_switch_id}=="<phys_switch_id>", \ 13032c0f0beSMauro Carvalho Chehab ATTR{phys_port_name}!="", NAME="swX$attr{phys_port_name}" 13132c0f0beSMauro Carvalho Chehab 13232c0f0beSMauro Carvalho ChehabSuggested naming convention is "swXpYsZ", where X is the switch name or ID, Y 13332c0f0beSMauro Carvalho Chehabis the port name or ID, and Z is the sub-port name or ID. For example, sw1p1s0 13432c0f0beSMauro Carvalho Chehabwould be sub-port 0 on port 1 on switch 1. 13532c0f0beSMauro Carvalho Chehab 13632c0f0beSMauro Carvalho ChehabPort Features 13732c0f0beSMauro Carvalho Chehab^^^^^^^^^^^^^ 13832c0f0beSMauro Carvalho Chehab 13932c0f0beSMauro Carvalho ChehabNETIF_F_NETNS_LOCAL 14032c0f0beSMauro Carvalho Chehab 14132c0f0beSMauro Carvalho ChehabIf the switchdev driver (and device) only supports offloading of the default 14232c0f0beSMauro Carvalho Chehabnetwork namespace (netns), the driver should set this feature flag to prevent 14332c0f0beSMauro Carvalho Chehabthe port netdev from being moved out of the default netns. A netns-aware 14432c0f0beSMauro Carvalho Chehabdriver/device would not set this flag and be responsible for partitioning 14532c0f0beSMauro Carvalho Chehabhardware to preserve netns containment. This means hardware cannot forward 14632c0f0beSMauro Carvalho Chehabtraffic from a port in one namespace to another port in another namespace. 14732c0f0beSMauro Carvalho Chehab 14832c0f0beSMauro Carvalho ChehabPort Topology 14932c0f0beSMauro Carvalho Chehab^^^^^^^^^^^^^ 15032c0f0beSMauro Carvalho Chehab 15132c0f0beSMauro Carvalho ChehabThe port netdevs representing the physical switch ports can be organized into 15232c0f0beSMauro Carvalho Chehabhigher-level switching constructs. The default construct is a standalone 15332c0f0beSMauro Carvalho Chehabrouter port, used to offload L3 forwarding. Two or more ports can be bonded 15432c0f0beSMauro Carvalho Chehabtogether to form a LAG. Two or more ports (or LAGs) can be bridged to bridge 15532c0f0beSMauro Carvalho ChehabL2 networks. VLANs can be applied to sub-divide L2 networks. L2-over-L3 15632c0f0beSMauro Carvalho Chehabtunnels can be built on ports. These constructs are built using standard Linux 15732c0f0beSMauro Carvalho Chehabtools such as the bridge driver, the bonding/team drivers, and netlink-based 15832c0f0beSMauro Carvalho Chehabtools such as iproute2. 15932c0f0beSMauro Carvalho Chehab 16032c0f0beSMauro Carvalho ChehabThe switchdev driver can know a particular port's position in the topology by 16132c0f0beSMauro Carvalho Chehabmonitoring NETDEV_CHANGEUPPER notifications. For example, a port moved into a 16232c0f0beSMauro Carvalho Chehabbond will see it's upper master change. If that bond is moved into a bridge, 16332c0f0beSMauro Carvalho Chehabthe bond's upper master will change. And so on. The driver will track such 16432c0f0beSMauro Carvalho Chehabmovements to know what position a port is in in the overall topology by 16532c0f0beSMauro Carvalho Chehabregistering for netdevice events and acting on NETDEV_CHANGEUPPER. 16632c0f0beSMauro Carvalho Chehab 16732c0f0beSMauro Carvalho ChehabL2 Forwarding Offload 16832c0f0beSMauro Carvalho Chehab--------------------- 16932c0f0beSMauro Carvalho Chehab 17032c0f0beSMauro Carvalho ChehabThe idea is to offload the L2 data forwarding (switching) path from the kernel 17132c0f0beSMauro Carvalho Chehabto the switchdev device by mirroring bridge FDB entries down to the device. An 17232c0f0beSMauro Carvalho ChehabFDB entry is the {port, MAC, VLAN} tuple forwarding destination. 17332c0f0beSMauro Carvalho Chehab 17432c0f0beSMauro Carvalho ChehabTo offloading L2 bridging, the switchdev driver/device should support: 17532c0f0beSMauro Carvalho Chehab 17632c0f0beSMauro Carvalho Chehab - Static FDB entries installed on a bridge port 17732c0f0beSMauro Carvalho Chehab - Notification of learned/forgotten src mac/vlans from device 17832c0f0beSMauro Carvalho Chehab - STP state changes on the port 17932c0f0beSMauro Carvalho Chehab - VLAN flooding of multicast/broadcast and unknown unicast packets 18032c0f0beSMauro Carvalho Chehab 18132c0f0beSMauro Carvalho ChehabStatic FDB Entries 18232c0f0beSMauro Carvalho Chehab^^^^^^^^^^^^^^^^^^ 18332c0f0beSMauro Carvalho Chehab 18432c0f0beSMauro Carvalho ChehabThe switchdev driver should implement ndo_fdb_add, ndo_fdb_del and ndo_fdb_dump 18532c0f0beSMauro Carvalho Chehabto support static FDB entries installed to the device. Static bridge FDB 18632c0f0beSMauro Carvalho Chehabentries are installed, for example, using iproute2 bridge cmd:: 18732c0f0beSMauro Carvalho Chehab 18832c0f0beSMauro Carvalho Chehab bridge fdb add ADDR dev DEV [vlan VID] [self] 18932c0f0beSMauro Carvalho Chehab 19032c0f0beSMauro Carvalho ChehabThe driver should use the helper switchdev_port_fdb_xxx ops for ndo_fdb_xxx 19132c0f0beSMauro Carvalho Chehabops, and handle add/delete/dump of SWITCHDEV_OBJ_ID_PORT_FDB object using 19232c0f0beSMauro Carvalho Chehabswitchdev_port_obj_xxx ops. 19332c0f0beSMauro Carvalho Chehab 19432c0f0beSMauro Carvalho ChehabXXX: what should be done if offloading this rule to hardware fails (for 19532c0f0beSMauro Carvalho Chehabexample, due to full capacity in hardware tables) ? 19632c0f0beSMauro Carvalho Chehab 19732c0f0beSMauro Carvalho ChehabNote: by default, the bridge does not filter on VLAN and only bridges untagged 19832c0f0beSMauro Carvalho Chehabtraffic. To enable VLAN support, turn on VLAN filtering:: 19932c0f0beSMauro Carvalho Chehab 20032c0f0beSMauro Carvalho Chehab echo 1 >/sys/class/net/<bridge>/bridge/vlan_filtering 20132c0f0beSMauro Carvalho Chehab 20232c0f0beSMauro Carvalho ChehabNotification of Learned/Forgotten Source MAC/VLANs 20332c0f0beSMauro Carvalho Chehab^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 20432c0f0beSMauro Carvalho Chehab 20532c0f0beSMauro Carvalho ChehabThe switch device will learn/forget source MAC address/VLAN on ingress packets 20632c0f0beSMauro Carvalho Chehaband notify the switch driver of the mac/vlan/port tuples. The switch driver, 20732c0f0beSMauro Carvalho Chehabin turn, will notify the bridge driver using the switchdev notifier call:: 20832c0f0beSMauro Carvalho Chehab 20932c0f0beSMauro Carvalho Chehab err = call_switchdev_notifiers(val, dev, info, extack); 21032c0f0beSMauro Carvalho Chehab 21132c0f0beSMauro Carvalho ChehabWhere val is SWITCHDEV_FDB_ADD when learning and SWITCHDEV_FDB_DEL when 21232c0f0beSMauro Carvalho Chehabforgetting, and info points to a struct switchdev_notifier_fdb_info. On 21332c0f0beSMauro Carvalho ChehabSWITCHDEV_FDB_ADD, the bridge driver will install the FDB entry into the 21432c0f0beSMauro Carvalho Chehabbridge's FDB and mark the entry as NTF_EXT_LEARNED. The iproute2 bridge 21532c0f0beSMauro Carvalho Chehabcommand will label these entries "offload":: 21632c0f0beSMauro Carvalho Chehab 21732c0f0beSMauro Carvalho Chehab $ bridge fdb 21832c0f0beSMauro Carvalho Chehab 52:54:00:12:35:01 dev sw1p1 master br0 permanent 21932c0f0beSMauro Carvalho Chehab 00:02:00:00:02:00 dev sw1p1 master br0 offload 22032c0f0beSMauro Carvalho Chehab 00:02:00:00:02:00 dev sw1p1 self 22132c0f0beSMauro Carvalho Chehab 52:54:00:12:35:02 dev sw1p2 master br0 permanent 22232c0f0beSMauro Carvalho Chehab 00:02:00:00:03:00 dev sw1p2 master br0 offload 22332c0f0beSMauro Carvalho Chehab 00:02:00:00:03:00 dev sw1p2 self 22432c0f0beSMauro Carvalho Chehab 33:33:00:00:00:01 dev eth0 self permanent 22532c0f0beSMauro Carvalho Chehab 01:00:5e:00:00:01 dev eth0 self permanent 22632c0f0beSMauro Carvalho Chehab 33:33:ff:00:00:00 dev eth0 self permanent 22732c0f0beSMauro Carvalho Chehab 01:80:c2:00:00:0e dev eth0 self permanent 22832c0f0beSMauro Carvalho Chehab 33:33:00:00:00:01 dev br0 self permanent 22932c0f0beSMauro Carvalho Chehab 01:00:5e:00:00:01 dev br0 self permanent 23032c0f0beSMauro Carvalho Chehab 33:33:ff:12:35:01 dev br0 self permanent 23132c0f0beSMauro Carvalho Chehab 23232c0f0beSMauro Carvalho ChehabLearning on the port should be disabled on the bridge using the bridge command:: 23332c0f0beSMauro Carvalho Chehab 23432c0f0beSMauro Carvalho Chehab bridge link set dev DEV learning off 23532c0f0beSMauro Carvalho Chehab 23632c0f0beSMauro Carvalho ChehabLearning on the device port should be enabled, as well as learning_sync:: 23732c0f0beSMauro Carvalho Chehab 23832c0f0beSMauro Carvalho Chehab bridge link set dev DEV learning on self 23932c0f0beSMauro Carvalho Chehab bridge link set dev DEV learning_sync on self 24032c0f0beSMauro Carvalho Chehab 24132c0f0beSMauro Carvalho ChehabLearning_sync attribute enables syncing of the learned/forgotten FDB entry to 24232c0f0beSMauro Carvalho Chehabthe bridge's FDB. It's possible, but not optimal, to enable learning on the 24332c0f0beSMauro Carvalho Chehabdevice port and on the bridge port, and disable learning_sync. 24432c0f0beSMauro Carvalho Chehab 24532c0f0beSMauro Carvalho ChehabTo support learning, the driver implements switchdev op 24632c0f0beSMauro Carvalho Chehabswitchdev_port_attr_set for SWITCHDEV_ATTR_PORT_ID_{PRE}_BRIDGE_FLAGS. 24732c0f0beSMauro Carvalho Chehab 24832c0f0beSMauro Carvalho ChehabFDB Ageing 24932c0f0beSMauro Carvalho Chehab^^^^^^^^^^ 25032c0f0beSMauro Carvalho Chehab 25132c0f0beSMauro Carvalho ChehabThe bridge will skip ageing FDB entries marked with NTF_EXT_LEARNED and it is 25232c0f0beSMauro Carvalho Chehabthe responsibility of the port driver/device to age out these entries. If the 25332c0f0beSMauro Carvalho Chehabport device supports ageing, when the FDB entry expires, it will notify the 25432c0f0beSMauro Carvalho Chehabdriver which in turn will notify the bridge with SWITCHDEV_FDB_DEL. If the 25532c0f0beSMauro Carvalho Chehabdevice does not support ageing, the driver can simulate ageing using a 25632c0f0beSMauro Carvalho Chehabgarbage collection timer to monitor FDB entries. Expired entries will be 25732c0f0beSMauro Carvalho Chehabnotified to the bridge using SWITCHDEV_FDB_DEL. See rocker driver for 25832c0f0beSMauro Carvalho Chehabexample of driver running ageing timer. 25932c0f0beSMauro Carvalho Chehab 26032c0f0beSMauro Carvalho ChehabTo keep an NTF_EXT_LEARNED entry "alive", the driver should refresh the FDB 26132c0f0beSMauro Carvalho Chehabentry by calling call_switchdev_notifiers(SWITCHDEV_FDB_ADD, ...). The 26232c0f0beSMauro Carvalho Chehabnotification will reset the FDB entry's last-used time to now. The driver 26332c0f0beSMauro Carvalho Chehabshould rate limit refresh notifications, for example, no more than once a 26432c0f0beSMauro Carvalho Chehabsecond. (The last-used time is visible using the bridge -s fdb option). 26532c0f0beSMauro Carvalho Chehab 26632c0f0beSMauro Carvalho ChehabSTP State Change on Port 26732c0f0beSMauro Carvalho Chehab^^^^^^^^^^^^^^^^^^^^^^^^ 26832c0f0beSMauro Carvalho Chehab 26932c0f0beSMauro Carvalho ChehabInternally or with a third-party STP protocol implementation (e.g. mstpd), the 27032c0f0beSMauro Carvalho Chehabbridge driver maintains the STP state for ports, and will notify the switch 27132c0f0beSMauro Carvalho Chehabdriver of STP state change on a port using the switchdev op 27232c0f0beSMauro Carvalho Chehabswitchdev_attr_port_set for SWITCHDEV_ATTR_PORT_ID_STP_UPDATE. 27332c0f0beSMauro Carvalho Chehab 27432c0f0beSMauro Carvalho ChehabState is one of BR_STATE_*. The switch driver can use STP state updates to 27532c0f0beSMauro Carvalho Chehabupdate ingress packet filter list for the port. For example, if port is 27632c0f0beSMauro Carvalho ChehabDISABLED, no packets should pass, but if port moves to BLOCKED, then STP BPDUs 27732c0f0beSMauro Carvalho Chehaband other IEEE 01:80:c2:xx:xx:xx link-local multicast packets can pass. 27832c0f0beSMauro Carvalho Chehab 27932c0f0beSMauro Carvalho ChehabNote that STP BDPUs are untagged and STP state applies to all VLANs on the port 28032c0f0beSMauro Carvalho Chehabso packet filters should be applied consistently across untagged and tagged 28132c0f0beSMauro Carvalho ChehabVLANs on the port. 28232c0f0beSMauro Carvalho Chehab 28332c0f0beSMauro Carvalho ChehabFlooding L2 domain 28432c0f0beSMauro Carvalho Chehab^^^^^^^^^^^^^^^^^^ 28532c0f0beSMauro Carvalho Chehab 28632c0f0beSMauro Carvalho ChehabFor a given L2 VLAN domain, the switch device should flood multicast/broadcast 28732c0f0beSMauro Carvalho Chehaband unknown unicast packets to all ports in domain, if allowed by port's 28832c0f0beSMauro Carvalho Chehabcurrent STP state. The switch driver, knowing which ports are within which 28932c0f0beSMauro Carvalho Chehabvlan L2 domain, can program the switch device for flooding. The packet may 29032c0f0beSMauro Carvalho Chehabbe sent to the port netdev for processing by the bridge driver. The 29132c0f0beSMauro Carvalho Chehabbridge should not reflood the packet to the same ports the device flooded, 29232c0f0beSMauro Carvalho Chehabotherwise there will be duplicate packets on the wire. 29332c0f0beSMauro Carvalho Chehab 29432c0f0beSMauro Carvalho ChehabTo avoid duplicate packets, the switch driver should mark a packet as already 29532c0f0beSMauro Carvalho Chehabforwarded by setting the skb->offload_fwd_mark bit. The bridge driver will mark 29632c0f0beSMauro Carvalho Chehabthe skb using the ingress bridge port's mark and prevent it from being forwarded 29732c0f0beSMauro Carvalho Chehabthrough any bridge port with the same mark. 29832c0f0beSMauro Carvalho Chehab 29932c0f0beSMauro Carvalho ChehabIt is possible for the switch device to not handle flooding and push the 30032c0f0beSMauro Carvalho Chehabpackets up to the bridge driver for flooding. This is not ideal as the number 30132c0f0beSMauro Carvalho Chehabof ports scale in the L2 domain as the device is much more efficient at 30232c0f0beSMauro Carvalho Chehabflooding packets that software. 30332c0f0beSMauro Carvalho Chehab 30432c0f0beSMauro Carvalho ChehabIf supported by the device, flood control can be offloaded to it, preventing 30532c0f0beSMauro Carvalho Chehabcertain netdevs from flooding unicast traffic for which there is no FDB entry. 30632c0f0beSMauro Carvalho Chehab 30732c0f0beSMauro Carvalho ChehabIGMP Snooping 30832c0f0beSMauro Carvalho Chehab^^^^^^^^^^^^^ 30932c0f0beSMauro Carvalho Chehab 31032c0f0beSMauro Carvalho ChehabIn order to support IGMP snooping, the port netdevs should trap to the bridge 31132c0f0beSMauro Carvalho Chehabdriver all IGMP join and leave messages. 31232c0f0beSMauro Carvalho ChehabThe bridge multicast module will notify port netdevs on every multicast group 31332c0f0beSMauro Carvalho Chehabchanged whether it is static configured or dynamically joined/leave. 31432c0f0beSMauro Carvalho ChehabThe hardware implementation should be forwarding all registered multicast 31532c0f0beSMauro Carvalho Chehabtraffic groups only to the configured ports. 31632c0f0beSMauro Carvalho Chehab 31732c0f0beSMauro Carvalho ChehabL3 Routing Offload 31832c0f0beSMauro Carvalho Chehab------------------ 31932c0f0beSMauro Carvalho Chehab 32032c0f0beSMauro Carvalho ChehabOffloading L3 routing requires that device be programmed with FIB entries from 32132c0f0beSMauro Carvalho Chehabthe kernel, with the device doing the FIB lookup and forwarding. The device 32232c0f0beSMauro Carvalho Chehabdoes a longest prefix match (LPM) on FIB entries matching route prefix and 32332c0f0beSMauro Carvalho Chehabforwards the packet to the matching FIB entry's nexthop(s) egress ports. 32432c0f0beSMauro Carvalho Chehab 32532c0f0beSMauro Carvalho ChehabTo program the device, the driver has to register a FIB notifier handler 32632c0f0beSMauro Carvalho Chehabusing register_fib_notifier. The following events are available: 32732c0f0beSMauro Carvalho Chehab 32832c0f0beSMauro Carvalho Chehab=================== =================================================== 32932c0f0beSMauro Carvalho ChehabFIB_EVENT_ENTRY_ADD used for both adding a new FIB entry to the device, 33032c0f0beSMauro Carvalho Chehab or modifying an existing entry on the device. 33132c0f0beSMauro Carvalho ChehabFIB_EVENT_ENTRY_DEL used for removing a FIB entry 33232c0f0beSMauro Carvalho ChehabFIB_EVENT_RULE_ADD, 33332c0f0beSMauro Carvalho ChehabFIB_EVENT_RULE_DEL used to propagate FIB rule changes 33432c0f0beSMauro Carvalho Chehab=================== =================================================== 33532c0f0beSMauro Carvalho Chehab 33632c0f0beSMauro Carvalho ChehabFIB_EVENT_ENTRY_ADD and FIB_EVENT_ENTRY_DEL events pass:: 33732c0f0beSMauro Carvalho Chehab 33832c0f0beSMauro Carvalho Chehab struct fib_entry_notifier_info { 33932c0f0beSMauro Carvalho Chehab struct fib_notifier_info info; /* must be first */ 34032c0f0beSMauro Carvalho Chehab u32 dst; 34132c0f0beSMauro Carvalho Chehab int dst_len; 34232c0f0beSMauro Carvalho Chehab struct fib_info *fi; 34332c0f0beSMauro Carvalho Chehab u8 tos; 34432c0f0beSMauro Carvalho Chehab u8 type; 34532c0f0beSMauro Carvalho Chehab u32 tb_id; 34632c0f0beSMauro Carvalho Chehab u32 nlflags; 34732c0f0beSMauro Carvalho Chehab }; 34832c0f0beSMauro Carvalho Chehab 34932c0f0beSMauro Carvalho Chehabto add/modify/delete IPv4 dst/dest_len prefix on table tb_id. The ``*fi`` 35032c0f0beSMauro Carvalho Chehabstructure holds details on the route and route's nexthops. ``*dev`` is one 35132c0f0beSMauro Carvalho Chehabof the port netdevs mentioned in the route's next hop list. 35232c0f0beSMauro Carvalho Chehab 35332c0f0beSMauro Carvalho ChehabRoutes offloaded to the device are labeled with "offload" in the ip route 35432c0f0beSMauro Carvalho Chehablisting:: 35532c0f0beSMauro Carvalho Chehab 35632c0f0beSMauro Carvalho Chehab $ ip route show 35732c0f0beSMauro Carvalho Chehab default via 192.168.0.2 dev eth0 35832c0f0beSMauro Carvalho Chehab 11.0.0.0/30 dev sw1p1 proto kernel scope link src 11.0.0.2 offload 35932c0f0beSMauro Carvalho Chehab 11.0.0.4/30 via 11.0.0.1 dev sw1p1 proto zebra metric 20 offload 36032c0f0beSMauro Carvalho Chehab 11.0.0.8/30 dev sw1p2 proto kernel scope link src 11.0.0.10 offload 36132c0f0beSMauro Carvalho Chehab 11.0.0.12/30 via 11.0.0.9 dev sw1p2 proto zebra metric 20 offload 36232c0f0beSMauro Carvalho Chehab 12.0.0.2 proto zebra metric 30 offload 36332c0f0beSMauro Carvalho Chehab nexthop via 11.0.0.1 dev sw1p1 weight 1 36432c0f0beSMauro Carvalho Chehab nexthop via 11.0.0.9 dev sw1p2 weight 1 36532c0f0beSMauro Carvalho Chehab 12.0.0.3 via 11.0.0.1 dev sw1p1 proto zebra metric 20 offload 36632c0f0beSMauro Carvalho Chehab 12.0.0.4 via 11.0.0.9 dev sw1p2 proto zebra metric 20 offload 36732c0f0beSMauro Carvalho Chehab 192.168.0.0/24 dev eth0 proto kernel scope link src 192.168.0.15 36832c0f0beSMauro Carvalho Chehab 36932c0f0beSMauro Carvalho ChehabThe "offload" flag is set in case at least one device offloads the FIB entry. 37032c0f0beSMauro Carvalho Chehab 37132c0f0beSMauro Carvalho ChehabXXX: add/mod/del IPv6 FIB API 37232c0f0beSMauro Carvalho Chehab 37332c0f0beSMauro Carvalho ChehabNexthop Resolution 37432c0f0beSMauro Carvalho Chehab^^^^^^^^^^^^^^^^^^ 37532c0f0beSMauro Carvalho Chehab 37632c0f0beSMauro Carvalho ChehabThe FIB entry's nexthop list contains the nexthop tuple (gateway, dev), but for 37732c0f0beSMauro Carvalho Chehabthe switch device to forward the packet with the correct dst mac address, the 37832c0f0beSMauro Carvalho Chehabnexthop gateways must be resolved to the neighbor's mac address. Neighbor mac 37932c0f0beSMauro Carvalho Chehabaddress discovery comes via the ARP (or ND) process and is available via the 38032c0f0beSMauro Carvalho Chehabarp_tbl neighbor table. To resolve the routes nexthop gateways, the driver 38132c0f0beSMauro Carvalho Chehabshould trigger the kernel's neighbor resolution process. See the rocker 38232c0f0beSMauro Carvalho Chehabdriver's rocker_port_ipv4_resolve() for an example. 38332c0f0beSMauro Carvalho Chehab 38432c0f0beSMauro Carvalho ChehabThe driver can monitor for updates to arp_tbl using the netevent notifier 38532c0f0beSMauro Carvalho ChehabNETEVENT_NEIGH_UPDATE. The device can be programmed with resolved nexthops 38632c0f0beSMauro Carvalho Chehabfor the routes as arp_tbl updates. The driver implements ndo_neigh_destroy 38732c0f0beSMauro Carvalho Chehabto know when arp_tbl neighbor entries are purged from the port. 388