13c91d114SIoana Ciornei============ 23c91d114SIoana CiorneiArchitecture 33c91d114SIoana Ciornei============ 43c91d114SIoana Ciornei 53c91d114SIoana CiorneiThis document describes the **Distributed Switch Architecture (DSA)** subsystem 63c91d114SIoana Ciorneidesign principles, limitations, interactions with other subsystems, and how to 73c91d114SIoana Ciorneidevelop drivers for this subsystem as well as a TODO for developers interested 83c91d114SIoana Ciorneiin joining the effort. 93c91d114SIoana Ciornei 103c91d114SIoana CiorneiDesign principles 113c91d114SIoana Ciornei================= 123c91d114SIoana Ciornei 135a48b743SBjorn HelgaasThe Distributed Switch Architecture subsystem was primarily designed to 145a48b743SBjorn Helgaassupport Marvell Ethernet switches (MV88E6xxx, a.k.a. Link Street product 155a48b743SBjorn Helgaasline) using Linux, but has since evolved to support other vendors as well. 163c91d114SIoana Ciornei 173c91d114SIoana CiorneiThe original philosophy behind this design was to be able to use unmodified 183c91d114SIoana CiorneiLinux tools such as bridge, iproute2, ifconfig to work transparently whether 193c91d114SIoana Ciorneithey configured/queried a switch port network device or a regular network 203c91d114SIoana Ciorneidevice. 213c91d114SIoana Ciornei 225a48b743SBjorn HelgaasAn Ethernet switch typically comprises multiple front-panel ports and one 235a48b743SBjorn Helgaasor more CPU or management ports. The DSA subsystem currently relies on the 243c91d114SIoana Ciorneipresence of a management port connected to an Ethernet controller capable of 253c91d114SIoana Ciorneireceiving Ethernet frames from the switch. This is a very common setup for all 263c91d114SIoana Ciorneikinds of Ethernet switches found in Small Home and Office products: routers, 275a48b743SBjorn Helgaasgateways, or even top-of-rack switches. This host Ethernet controller will 283c91d114SIoana Ciorneibe later referred to as "master" and "cpu" in DSA terminology and code. 293c91d114SIoana Ciornei 303c91d114SIoana CiorneiThe D in DSA stands for Distributed, because the subsystem has been designed 313c91d114SIoana Ciorneiwith the ability to configure and manage cascaded switches on top of each other 323c91d114SIoana Ciorneiusing upstream and downstream Ethernet links between switches. These specific 333c91d114SIoana Ciorneiports are referred to as "dsa" ports in DSA terminology and code. A collection 343c91d114SIoana Ciorneiof multiple switches connected to each other is called a "switch tree". 353c91d114SIoana Ciornei 365a48b743SBjorn HelgaasFor each front-panel port, DSA creates specialized network devices which are 373c91d114SIoana Ciorneiused as controlling and data-flowing endpoints for use by the Linux networking 383c91d114SIoana Ciorneistack. These specialized network interfaces are referred to as "slave" network 393c91d114SIoana Ciorneiinterfaces in DSA terminology and code. 403c91d114SIoana Ciornei 413c91d114SIoana CiorneiThe ideal case for using DSA is when an Ethernet switch supports a "switch tag" 423c91d114SIoana Ciorneiwhich is a hardware feature making the switch insert a specific tag for each 435a48b743SBjorn HelgaasEthernet frame it receives to/from specific ports to help the management 443c91d114SIoana Ciorneiinterface figure out: 453c91d114SIoana Ciornei 463c91d114SIoana Ciornei- what port is this frame coming from 473c91d114SIoana Ciornei- what was the reason why this frame got forwarded 483c91d114SIoana Ciornei- how to send CPU originated traffic to specific ports 493c91d114SIoana Ciornei 503c91d114SIoana CiorneiThe subsystem does support switches not capable of inserting/stripping tags, but 513c91d114SIoana Ciorneithe features might be slightly limited in that case (traffic separation relies 523c91d114SIoana Ciorneion Port-based VLAN IDs). 533c91d114SIoana Ciornei 543c91d114SIoana CiorneiNote that DSA does not currently create network interfaces for the "cpu" and 553c91d114SIoana Ciornei"dsa" ports because: 563c91d114SIoana Ciornei 573c91d114SIoana Ciornei- the "cpu" port is the Ethernet switch facing side of the management 583c91d114SIoana Ciornei controller, and as such, would create a duplication of feature, since you 593c91d114SIoana Ciornei would get two interfaces for the same conduit: master netdev, and "cpu" netdev 603c91d114SIoana Ciornei 613c91d114SIoana Ciornei- the "dsa" port(s) are just conduits between two or more switches, and as such 623c91d114SIoana Ciornei cannot really be used as proper network interfaces either, only the 633c91d114SIoana Ciornei downstream, or the top-most upstream interface makes sense with that model 643c91d114SIoana Ciornei 653c91d114SIoana CiorneiSwitch tagging protocols 663c91d114SIoana Ciornei------------------------ 673c91d114SIoana Ciornei 687714ee15SVladimir OlteanDSA supports many vendor-specific tagging protocols, one software-defined 697714ee15SVladimir Olteantagging protocol, and a tag-less mode as well (``DSA_TAG_PROTO_NONE``). 703c91d114SIoana Ciornei 713c91d114SIoana CiorneiThe exact format of the tag protocol is vendor specific, but in general, they 723c91d114SIoana Ciorneiall contain something which: 733c91d114SIoana Ciornei 743c91d114SIoana Ciornei- identifies which port the Ethernet frame came from/should be sent to 753c91d114SIoana Ciornei- provides a reason why this frame was forwarded to the management interface 763c91d114SIoana Ciornei 777714ee15SVladimir OlteanAll tagging protocols are in ``net/dsa/tag_*.c`` files and implement the 787714ee15SVladimir Olteanmethods of the ``struct dsa_device_ops`` structure, which are detailed below. 797714ee15SVladimir Oltean 807714ee15SVladimir OlteanTagging protocols generally fall in one of three categories: 817714ee15SVladimir Oltean 827714ee15SVladimir Oltean1. The switch-specific frame header is located before the Ethernet header, 837714ee15SVladimir Oltean shifting to the right (from the perspective of the DSA master's frame 847714ee15SVladimir Oltean parser) the MAC DA, MAC SA, EtherType and the entire L2 payload. 857714ee15SVladimir Oltean2. The switch-specific frame header is located before the EtherType, keeping 867714ee15SVladimir Oltean the MAC DA and MAC SA in place from the DSA master's perspective, but 877714ee15SVladimir Oltean shifting the 'real' EtherType and L2 payload to the right. 887714ee15SVladimir Oltean3. The switch-specific frame header is located at the tail of the packet, 897714ee15SVladimir Oltean keeping all frame headers in place and not altering the view of the packet 907714ee15SVladimir Oltean that the DSA master's frame parser has. 917714ee15SVladimir Oltean 927714ee15SVladimir OlteanA tagging protocol may tag all packets with switch tags of the same length, or 937714ee15SVladimir Olteanthe tag length might vary (for example packets with PTP timestamps might 947714ee15SVladimir Olteanrequire an extended switch tag, or there might be one tag length on TX and a 957714ee15SVladimir Olteandifferent one on RX). Either way, the tagging protocol driver must populate the 964e500251SVladimir Oltean``struct dsa_device_ops::needed_headroom`` and/or ``struct dsa_device_ops::needed_tailroom`` 974e500251SVladimir Olteanwith the length in octets of the longest switch frame header/trailer. The DSA 984e500251SVladimir Olteanframework will automatically adjust the MTU of the master interface to 994e500251SVladimir Olteanaccommodate for this extra size in order for DSA user ports to support the 1004e500251SVladimir Olteanstandard MTU (L2 payload length) of 1500 octets. The ``needed_headroom`` and 1014e500251SVladimir Oltean``needed_tailroom`` properties are also used to request from the network stack, 1024e500251SVladimir Olteanon a best-effort basis, the allocation of packets with enough extra space such 1034e500251SVladimir Olteanthat the act of pushing the switch tag on transmission of a packet does not 1044e500251SVladimir Olteancause it to reallocate due to lack of memory. 1057714ee15SVladimir Oltean 1067714ee15SVladimir OlteanEven though applications are not expected to parse DSA-specific frame headers, 1077714ee15SVladimir Olteanthe format on the wire of the tagging protocol represents an Application Binary 1087714ee15SVladimir OlteanInterface exposed by the kernel towards user space, for decoders such as 1097714ee15SVladimir Oltean``libpcap``. The tagging protocol driver must populate the ``proto`` member of 1107714ee15SVladimir Oltean``struct dsa_device_ops`` with a value that uniquely describes the 1117714ee15SVladimir Olteancharacteristics of the interaction required between the switch hardware and the 1127714ee15SVladimir Olteandata path driver: the offset of each bit field within the frame header and any 1137714ee15SVladimir Olteanstateful processing required to deal with the frames (as may be required for 1147714ee15SVladimir OlteanPTP timestamping). 1157714ee15SVladimir Oltean 1167714ee15SVladimir OlteanFrom the perspective of the network stack, all switches within the same DSA 1177714ee15SVladimir Olteanswitch tree use the same tagging protocol. In case of a packet transiting a 1187714ee15SVladimir Olteanfabric with more than one switch, the switch-specific frame header is inserted 1197714ee15SVladimir Olteanby the first switch in the fabric that the packet was received on. This header 1207714ee15SVladimir Olteantypically contains information regarding its type (whether it is a control 1217714ee15SVladimir Olteanframe that must be trapped to the CPU, or a data frame to be forwarded). 1227714ee15SVladimir OlteanControl frames should be decapsulated only by the software data path, whereas 1237714ee15SVladimir Olteandata frames might also be autonomously forwarded towards other user ports of 1247714ee15SVladimir Olteanother switches from the same fabric, and in this case, the outermost switch 1257714ee15SVladimir Olteanports must decapsulate the packet. 1267714ee15SVladimir Oltean 1277714ee15SVladimir OlteanNote that in certain cases, it might be the case that the tagging format used 1285a48b743SBjorn Helgaasby a leaf switch (not connected directly to the CPU) is not the same as what 1297714ee15SVladimir Olteanthe network stack sees. This can be seen with Marvell switch trees, where the 1307714ee15SVladimir OlteanCPU port can be configured to use either the DSA or the Ethertype DSA (EDSA) 1317714ee15SVladimir Olteanformat, but the DSA links are configured to use the shorter (without Ethertype) 1327714ee15SVladimir OlteanDSA frame header, in order to reduce the autonomous packet forwarding overhead. 1337714ee15SVladimir OlteanIt still remains the case that, if the DSA switch tree is configured for the 1347714ee15SVladimir OlteanEDSA tagging protocol, the operating system sees EDSA-tagged packets from the 1357714ee15SVladimir Olteanleaf switches that tagged them with the shorter DSA header. This can be done 1367714ee15SVladimir Olteanbecause the Marvell switch connected directly to the CPU is configured to 1377714ee15SVladimir Olteanperform tag translation between DSA and EDSA (which is simply the operation of 1387714ee15SVladimir Olteanadding or removing the ``ETH_P_EDSA`` EtherType and some padding octets). 1397714ee15SVladimir Oltean 1407714ee15SVladimir OlteanIt is possible to construct cascaded setups of DSA switches even if their 1417714ee15SVladimir Olteantagging protocols are not compatible with one another. In this case, there are 1427714ee15SVladimir Olteanno DSA links in this fabric, and each switch constitutes a disjoint DSA switch 1437714ee15SVladimir Olteantree. The DSA links are viewed as simply a pair of a DSA master (the out-facing 1447714ee15SVladimir Olteanport of the upstream DSA switch) and a CPU port (the in-facing port of the 1457714ee15SVladimir Olteandownstream DSA switch). 1467714ee15SVladimir Oltean 1477714ee15SVladimir OlteanThe tagging protocol of the attached DSA switch tree can be viewed through the 1487714ee15SVladimir Oltean``dsa/tagging`` sysfs attribute of the DSA master:: 1497714ee15SVladimir Oltean 1507714ee15SVladimir Oltean cat /sys/class/net/eth0/dsa/tagging 1517714ee15SVladimir Oltean 1527714ee15SVladimir OlteanIf the hardware and driver are capable, the tagging protocol of the DSA switch 1537714ee15SVladimir Olteantree can be changed at runtime. This is done by writing the new tagging 1547714ee15SVladimir Olteanprotocol name to the same sysfs device attribute as above (the DSA master and 1557714ee15SVladimir Olteanall attached switch ports must be down while doing this). 1567714ee15SVladimir Oltean 1577714ee15SVladimir OlteanIt is desirable that all tagging protocols are testable with the ``dsa_loop`` 1587714ee15SVladimir Olteanmockup driver, which can be attached to any network interface. The goal is that 1597714ee15SVladimir Olteanany network interface should be capable of transmitting the same packet in the 1607714ee15SVladimir Olteansame way, and the tagger should decode the same received packet in the same way 1617714ee15SVladimir Olteanregardless of the driver used for the switch control path, and the driver used 1627714ee15SVladimir Olteanfor the DSA master. 1637714ee15SVladimir Oltean 1647714ee15SVladimir OlteanThe transmission of a packet goes through the tagger's ``xmit`` function. 1657714ee15SVladimir OlteanThe passed ``struct sk_buff *skb`` has ``skb->data`` pointing at 1667714ee15SVladimir Oltean``skb_mac_header(skb)``, i.e. at the destination MAC address, and the passed 1677714ee15SVladimir Oltean``struct net_device *dev`` represents the virtual DSA user network interface 1687714ee15SVladimir Olteanwhose hardware counterpart the packet must be steered to (i.e. ``swp0``). 1697714ee15SVladimir OlteanThe job of this method is to prepare the skb in a way that the switch will 1707714ee15SVladimir Olteanunderstand what egress port the packet is for (and not deliver it towards other 1717714ee15SVladimir Olteanports). Typically this is fulfilled by pushing a frame header. Checking for 1727714ee15SVladimir Olteaninsufficient size in the skb headroom or tailroom is unnecessary provided that 1734e500251SVladimir Olteanthe ``needed_headroom`` and ``needed_tailroom`` properties were filled out 1744e500251SVladimir Olteanproperly, because DSA ensures there is enough space before calling this method. 1757714ee15SVladimir Oltean 1767714ee15SVladimir OlteanThe reception of a packet goes through the tagger's ``rcv`` function. The 1777714ee15SVladimir Olteanpassed ``struct sk_buff *skb`` has ``skb->data`` pointing at 1787714ee15SVladimir Oltean``skb_mac_header(skb) + ETH_ALEN`` octets, i.e. to where the first octet after 1797714ee15SVladimir Olteanthe EtherType would have been, were this frame not tagged. The role of this 1807714ee15SVladimir Olteanmethod is to consume the frame header, adjust ``skb->data`` to really point at 1817714ee15SVladimir Olteanthe first octet after the EtherType, and to change ``skb->dev`` to point to the 1827714ee15SVladimir Olteanvirtual DSA user network interface corresponding to the physical front-facing 1837714ee15SVladimir Olteanswitch port that the packet was received on. 1847714ee15SVladimir Oltean 1857714ee15SVladimir OlteanSince tagging protocols in category 1 and 2 break software (and most often also 1867714ee15SVladimir Olteanhardware) packet dissection on the DSA master, features such as RPS (Receive 1877714ee15SVladimir OlteanPacket Steering) on the DSA master would be broken. The DSA framework deals 1887714ee15SVladimir Olteanwith this by hooking into the flow dissector and shifting the offset at which 1897714ee15SVladimir Olteanthe IP header is to be found in the tagged frame as seen by the DSA master. 1907714ee15SVladimir OlteanThis behavior is automatic based on the ``overhead`` value of the tagging 1917714ee15SVladimir Olteanprotocol. If not all packets are of equal size, the tagger can implement the 1927714ee15SVladimir Oltean``flow_dissect`` method of the ``struct dsa_device_ops`` and override this 1937714ee15SVladimir Olteandefault behavior by specifying the correct offset incurred by each individual 1947714ee15SVladimir OlteanRX packet. Tail taggers do not cause issues to the flow dissector. 1957714ee15SVladimir Oltean 196a997157eSLuiz Angelo Daros de LucaChecksum offload should work with category 1 and 2 taggers when the DSA master 197a997157eSLuiz Angelo Daros de Lucadriver declares NETIF_F_HW_CSUM in vlan_features and looks at csum_start and 198a997157eSLuiz Angelo Daros de Lucacsum_offset. For those cases, DSA will shift the checksum start and offset by 199a997157eSLuiz Angelo Daros de Lucathe tag size. If the DSA master driver still uses the legacy NETIF_F_IP_CSUM 200a997157eSLuiz Angelo Daros de Lucaor NETIF_F_IPV6_CSUM in vlan_features, the offload might only work if the 201a997157eSLuiz Angelo Daros de Lucaoffload hardware already expects that specific tag (perhaps due to matching 202a997157eSLuiz Angelo Daros de Lucavendors). DSA slaves inherit those flags from the master port, and it is up to 203a997157eSLuiz Angelo Daros de Lucathe driver to correctly fall back to software checksum when the IP header is not 204a997157eSLuiz Angelo Daros de Lucawhere the hardware expects. If that check is ineffective, the packets might go 205a997157eSLuiz Angelo Daros de Lucato the network without a proper checksum (the checksum field will have the 206a997157eSLuiz Angelo Daros de Lucapseudo IP header sum). For category 3, when the offload hardware does not 207a997157eSLuiz Angelo Daros de Lucaalready expect the switch tag in use, the checksum must be calculated before any 208a997157eSLuiz Angelo Daros de Lucatag is inserted (i.e. inside the tagger). Otherwise, the DSA master would 209a997157eSLuiz Angelo Daros de Lucainclude the tail tag in the (software or hardware) checksum calculation. Then, 210a997157eSLuiz Angelo Daros de Lucawhen the tag gets stripped by the switch during transmission, it will leave an 211a997157eSLuiz Angelo Daros de Lucaincorrect IP checksum in place. 212a997157eSLuiz Angelo Daros de Luca 2137714ee15SVladimir OlteanDue to various reasons (most common being category 1 taggers being associated 2147714ee15SVladimir Olteanwith DSA-unaware masters, mangling what the master perceives as MAC DA), the 2157714ee15SVladimir Olteantagging protocol may require the DSA master to operate in promiscuous mode, to 2167714ee15SVladimir Olteanreceive all frames regardless of the value of the MAC DA. This can be done by 2177714ee15SVladimir Olteansetting the ``promisc_on_master`` property of the ``struct dsa_device_ops``. 2187714ee15SVladimir OlteanNote that this assumes a DSA-unaware master driver, which is the norm. 2197714ee15SVladimir Oltean 2203c91d114SIoana CiorneiMaster network devices 2213c91d114SIoana Ciornei---------------------- 2223c91d114SIoana Ciornei 2233c91d114SIoana CiorneiMaster network devices are regular, unmodified Linux network device drivers for 2243c91d114SIoana Ciorneithe CPU/management Ethernet interface. Such a driver might occasionally need to 2253c91d114SIoana Ciorneiknow whether DSA is enabled (e.g.: to enable/disable specific offload features), 2263c91d114SIoana Ciorneibut the DSA subsystem has been proven to work with industry standard drivers: 2273c91d114SIoana Ciornei``e1000e,`` ``mv643xx_eth`` etc. without having to introduce modifications to these 2283c91d114SIoana Ciorneidrivers. Such network devices are also often referred to as conduit network 2293c91d114SIoana Ciorneidevices since they act as a pipe between the host processor and the hardware 2303c91d114SIoana CiorneiEthernet switch. 2313c91d114SIoana Ciornei 2323c91d114SIoana CiorneiNetworking stack hooks 2333c91d114SIoana Ciornei---------------------- 2343c91d114SIoana Ciornei 2354f6a009cSRandy DunlapWhen a master netdev is used with DSA, a small hook is placed in the 2363c91d114SIoana Ciorneinetworking stack is in order to have the DSA subsystem process the Ethernet 2373c91d114SIoana Ciorneiswitch specific tagging protocol. DSA accomplishes this by registering a 2383c91d114SIoana Ciorneispecific (and fake) Ethernet type (later becoming ``skb->protocol``) with the 2393c91d114SIoana Ciorneinetworking stack, this is also known as a ``ptype`` or ``packet_type``. A typical 2403c91d114SIoana CiorneiEthernet Frame receive sequence looks like this: 2413c91d114SIoana Ciornei 2423c91d114SIoana CiorneiMaster network device (e.g.: e1000e): 2433c91d114SIoana Ciornei 2443c91d114SIoana Ciornei1. Receive interrupt fires: 2453c91d114SIoana Ciornei 2463c91d114SIoana Ciornei - receive function is invoked 2473c91d114SIoana Ciornei - basic packet processing is done: getting length, status etc. 2483c91d114SIoana Ciornei - packet is prepared to be processed by the Ethernet layer by calling 2493c91d114SIoana Ciornei ``eth_type_trans`` 2503c91d114SIoana Ciornei 2513c91d114SIoana Ciornei2. net/ethernet/eth.c:: 2523c91d114SIoana Ciornei 2533c91d114SIoana Ciornei eth_type_trans(skb, dev) 2543c91d114SIoana Ciornei if (dev->dsa_ptr != NULL) 2553c91d114SIoana Ciornei -> skb->protocol = ETH_P_XDSA 2563c91d114SIoana Ciornei 2573c91d114SIoana Ciornei3. drivers/net/ethernet/\*:: 2583c91d114SIoana Ciornei 2593c91d114SIoana Ciornei netif_receive_skb(skb) 2603c91d114SIoana Ciornei -> iterate over registered packet_type 2613c91d114SIoana Ciornei -> invoke handler for ETH_P_XDSA, calls dsa_switch_rcv() 2623c91d114SIoana Ciornei 2633c91d114SIoana Ciornei4. net/dsa/dsa.c:: 2643c91d114SIoana Ciornei 2653c91d114SIoana Ciornei -> dsa_switch_rcv() 2663c91d114SIoana Ciornei -> invoke switch tag specific protocol handler in 'net/dsa/tag_*.c' 2673c91d114SIoana Ciornei 2683c91d114SIoana Ciornei5. net/dsa/tag_*.c: 2693c91d114SIoana Ciornei 2703c91d114SIoana Ciornei - inspect and strip switch tag protocol to determine originating port 2713c91d114SIoana Ciornei - locate per-port network device 2723c91d114SIoana Ciornei - invoke ``eth_type_trans()`` with the DSA slave network device 2733c91d114SIoana Ciornei - invoked ``netif_receive_skb()`` 2743c91d114SIoana Ciornei 2753c91d114SIoana CiorneiPast this point, the DSA slave network devices get delivered regular Ethernet 2763c91d114SIoana Ciorneiframes that can be processed by the networking stack. 2773c91d114SIoana Ciornei 2783c91d114SIoana CiorneiSlave network devices 2793c91d114SIoana Ciornei--------------------- 2803c91d114SIoana Ciornei 2813c91d114SIoana CiorneiSlave network devices created by DSA are stacked on top of their master network 2823c91d114SIoana Ciorneidevice, each of these network interfaces will be responsible for being a 2833c91d114SIoana Ciorneicontrolling and data-flowing end-point for each front-panel port of the switch. 2843c91d114SIoana CiorneiThese interfaces are specialized in order to: 2853c91d114SIoana Ciornei 2863c91d114SIoana Ciornei- insert/remove the switch tag protocol (if it exists) when sending traffic 2873c91d114SIoana Ciornei to/from specific switch ports 2883c91d114SIoana Ciornei- query the switch for ethtool operations: statistics, link state, 2893c91d114SIoana Ciornei Wake-on-LAN, register dumps... 2905a48b743SBjorn Helgaas- manage external/internal PHY: link, auto-negotiation, etc. 2913c91d114SIoana Ciornei 2923c91d114SIoana CiorneiThese slave network devices have custom net_device_ops and ethtool_ops function 2933c91d114SIoana Ciorneipointers which allow DSA to introduce a level of layering between the networking 2945a48b743SBjorn Helgaasstack/ethtool and the switch driver implementation. 2953c91d114SIoana Ciornei 2963c91d114SIoana CiorneiUpon frame transmission from these slave network devices, DSA will look up which 2975a48b743SBjorn Helgaasswitch tagging protocol is currently registered with these network devices and 2983c91d114SIoana Ciorneiinvoke a specific transmit routine which takes care of adding the relevant 2993c91d114SIoana Ciorneiswitch tag in the Ethernet frames. 3003c91d114SIoana Ciornei 3013c91d114SIoana CiorneiThese frames are then queued for transmission using the master network device 3025a48b743SBjorn Helgaas``ndo_start_xmit()`` function. Since they contain the appropriate switch tag, the 3033c91d114SIoana CiorneiEthernet switch will be able to process these incoming frames from the 3045a48b743SBjorn Helgaasmanagement interface and deliver them to the physical switch port. 3053c91d114SIoana Ciornei 3063c91d114SIoana CiorneiGraphical representation 3073c91d114SIoana Ciornei------------------------ 3083c91d114SIoana Ciornei 3093c91d114SIoana CiorneiSummarized, this is basically how DSA looks like from a network device 3103c91d114SIoana Ciorneiperspective:: 3113c91d114SIoana Ciornei 3120f455371SVladimir Oltean Unaware application 3130f455371SVladimir Oltean opens and binds socket 3140f455371SVladimir Oltean | ^ 3153c91d114SIoana Ciornei | | 3160f455371SVladimir Oltean +-----------v--|--------------------+ 3170f455371SVladimir Oltean |+------+ +------+ +------+ +------+| 3180f455371SVladimir Oltean || swp0 | | swp1 | | swp2 | | swp3 || 3190f455371SVladimir Oltean |+------+-+------+-+------+-+------+| 3200f455371SVladimir Oltean | DSA switch driver | 3210f455371SVladimir Oltean +-----------------------------------+ 3220f455371SVladimir Oltean | ^ 3230f455371SVladimir Oltean Tag added by | | Tag consumed by 3240f455371SVladimir Oltean switch driver | | switch driver 3250f455371SVladimir Oltean v | 3260f455371SVladimir Oltean +-----------------------------------+ 3270f455371SVladimir Oltean | Unmodified host interface driver | Software 3280f455371SVladimir Oltean --------+-----------------------------------+------------ 3290f455371SVladimir Oltean | Host interface (eth0) | Hardware 3300f455371SVladimir Oltean +-----------------------------------+ 3310f455371SVladimir Oltean | ^ 3320f455371SVladimir Oltean Tag consumed by | | Tag added by 3330f455371SVladimir Oltean switch hardware | | switch hardware 3340f455371SVladimir Oltean v | 3350f455371SVladimir Oltean +-----------------------------------+ 3360f455371SVladimir Oltean | Switch | 3370f455371SVladimir Oltean |+------+ +------+ +------+ +------+| 3380f455371SVladimir Oltean || swp0 | | swp1 | | swp2 | | swp3 || 3390f455371SVladimir Oltean ++------+-+------+-+------+-+------++ 3403c91d114SIoana Ciornei 3413c91d114SIoana CiorneiSlave MDIO bus 3423c91d114SIoana Ciornei-------------- 3433c91d114SIoana Ciornei 3443c91d114SIoana CiorneiIn order to be able to read to/from a switch PHY built into it, DSA creates a 3453c91d114SIoana Ciorneislave MDIO bus which allows a specific switch driver to divert and intercept 3463c91d114SIoana CiorneiMDIO reads/writes towards specific PHY addresses. In most MDIO-connected 3473c91d114SIoana Ciorneiswitches, these functions would utilize direct or indirect PHY addressing mode 3483c91d114SIoana Ciorneito return standard MII registers from the switch builtin PHYs, allowing the PHY 3493c91d114SIoana Ciorneilibrary and/or to return link status, link partner pages, auto-negotiation 3505a48b743SBjorn Helgaasresults, etc. 3513c91d114SIoana Ciornei 3525a48b743SBjorn HelgaasFor Ethernet switches which have both external and internal MDIO buses, the 3533c91d114SIoana Ciorneislave MII bus can be utilized to mux/demux MDIO reads and writes towards either 3543c91d114SIoana Ciorneiinternal or external MDIO devices this switch might be connected to: internal 3553c91d114SIoana CiorneiPHYs, external PHYs, or even external switches. 3563c91d114SIoana Ciornei 3573c91d114SIoana CiorneiData structures 3583c91d114SIoana Ciornei--------------- 3593c91d114SIoana Ciornei 3603c91d114SIoana CiorneiDSA data structures are defined in ``include/net/dsa.h`` as well as 3613c91d114SIoana Ciornei``net/dsa/dsa_priv.h``: 3623c91d114SIoana Ciornei 3633c91d114SIoana Ciornei- ``dsa_chip_data``: platform data configuration for a given switch device, 3643c91d114SIoana Ciornei this structure describes a switch device's parent device, its address, as 3653c91d114SIoana Ciornei well as various properties of its ports: names/labels, and finally a routing 3663c91d114SIoana Ciornei table indication (when cascading switches) 3673c91d114SIoana Ciornei 3683c91d114SIoana Ciornei- ``dsa_platform_data``: platform device configuration data which can reference 3695a48b743SBjorn Helgaas a collection of dsa_chip_data structures if multiple switches are cascaded, 3703c91d114SIoana Ciornei the master network device this switch tree is attached to needs to be 3713c91d114SIoana Ciornei referenced 3723c91d114SIoana Ciornei 3733c91d114SIoana Ciornei- ``dsa_switch_tree``: structure assigned to the master network device under 3743c91d114SIoana Ciornei ``dsa_ptr``, this structure references a dsa_platform_data structure as well as 3753c91d114SIoana Ciornei the tagging protocol supported by the switch tree, and which receive/transmit 3763c91d114SIoana Ciornei function hooks should be invoked, information about the directly attached 3773c91d114SIoana Ciornei switch is also provided: CPU port. Finally, a collection of dsa_switch are 3783c91d114SIoana Ciornei referenced to address individual switches in the tree. 3793c91d114SIoana Ciornei 3803c91d114SIoana Ciornei- ``dsa_switch``: structure describing a switch device in the tree, referencing 3813c91d114SIoana Ciornei a ``dsa_switch_tree`` as a backpointer, slave network devices, master network 3823c91d114SIoana Ciornei device, and a reference to the backing``dsa_switch_ops`` 3833c91d114SIoana Ciornei 3843c91d114SIoana Ciornei- ``dsa_switch_ops``: structure referencing function pointers, see below for a 3853c91d114SIoana Ciornei full description. 3863c91d114SIoana Ciornei 3873c91d114SIoana CiorneiDesign limitations 3883c91d114SIoana Ciornei================== 3893c91d114SIoana Ciornei 3903c91d114SIoana CiorneiLack of CPU/DSA network devices 3913c91d114SIoana Ciornei------------------------------- 3923c91d114SIoana Ciornei 3933c91d114SIoana CiorneiDSA does not currently create slave network devices for the CPU or DSA ports, as 3943c91d114SIoana Ciorneidescribed before. This might be an issue in the following cases: 3953c91d114SIoana Ciornei 3963c91d114SIoana Ciornei- inability to fetch switch CPU port statistics counters using ethtool, which 3973c91d114SIoana Ciornei can make it harder to debug MDIO switch connected using xMII interfaces 3983c91d114SIoana Ciornei 3993c91d114SIoana Ciornei- inability to configure the CPU port link parameters based on the Ethernet 4003c91d114SIoana Ciornei controller capabilities attached to it: http://patchwork.ozlabs.org/patch/509806/ 4013c91d114SIoana Ciornei 4023c91d114SIoana Ciornei- inability to configure specific VLAN IDs / trunking VLANs between switches 4033c91d114SIoana Ciornei when using a cascaded setup 4043c91d114SIoana Ciornei 4053c91d114SIoana CiorneiCommon pitfalls using DSA setups 4063c91d114SIoana Ciornei-------------------------------- 4073c91d114SIoana Ciornei 4083c91d114SIoana CiorneiOnce a master network device is configured to use DSA (dev->dsa_ptr becomes 4093c91d114SIoana Ciorneinon-NULL), and the switch behind it expects a tagging protocol, this network 4103c91d114SIoana Ciorneiinterface can only exclusively be used as a conduit interface. Sending packets 4113c91d114SIoana Ciorneidirectly through this interface (e.g.: opening a socket using this interface) 4123c91d114SIoana Ciorneiwill not make us go through the switch tagging protocol transmit function, so 4133c91d114SIoana Ciorneithe Ethernet switch on the other end, expecting a tag will typically drop this 4143c91d114SIoana Ciorneiframe. 4153c91d114SIoana Ciornei 4163c91d114SIoana CiorneiInteractions with other subsystems 4173c91d114SIoana Ciornei================================== 4183c91d114SIoana Ciornei 4193c91d114SIoana CiorneiDSA currently leverages the following subsystems: 4203c91d114SIoana Ciornei 4213c91d114SIoana Ciornei- MDIO/PHY library: ``drivers/net/phy/phy.c``, ``mdio_bus.c`` 4223c91d114SIoana Ciornei- Switchdev:``net/switchdev/*`` 4233c91d114SIoana Ciornei- Device Tree for various of_* functions 4248411abbcSVladimir Oltean- Devlink: ``net/core/devlink.c`` 4253c91d114SIoana Ciornei 4263c91d114SIoana CiorneiMDIO/PHY library 4273c91d114SIoana Ciornei---------------- 4283c91d114SIoana Ciornei 4293c91d114SIoana CiorneiSlave network devices exposed by DSA may or may not be interfacing with PHY 4303c91d114SIoana Ciorneidevices (``struct phy_device`` as defined in ``include/linux/phy.h)``, but the DSA 4313c91d114SIoana Ciorneisubsystem deals with all possible combinations: 4323c91d114SIoana Ciornei 4333c91d114SIoana Ciornei- internal PHY devices, built into the Ethernet switch hardware 4343c91d114SIoana Ciornei- external PHY devices, connected via an internal or external MDIO bus 4353c91d114SIoana Ciornei- internal PHY devices, connected via an internal MDIO bus 4363c91d114SIoana Ciornei- special, non-autonegotiated or non MDIO-managed PHY devices: SFPs, MoCA; a.k.a 4373c91d114SIoana Ciornei fixed PHYs 4383c91d114SIoana Ciornei 4393c91d114SIoana CiorneiThe PHY configuration is done by the ``dsa_slave_phy_setup()`` function and the 4403c91d114SIoana Ciorneilogic basically looks like this: 4413c91d114SIoana Ciornei 4423c91d114SIoana Ciornei- if Device Tree is used, the PHY device is looked up using the standard 4433c91d114SIoana Ciornei "phy-handle" property, if found, this PHY device is created and registered 4443c91d114SIoana Ciornei using ``of_phy_connect()`` 4453c91d114SIoana Ciornei 4465a48b743SBjorn Helgaas- if Device Tree is used and the PHY device is "fixed", that is, conforms to 4473c91d114SIoana Ciornei the definition of a non-MDIO managed PHY as defined in 4483c91d114SIoana Ciornei ``Documentation/devicetree/bindings/net/fixed-link.txt``, the PHY is registered 4493c91d114SIoana Ciornei and connected transparently using the special fixed MDIO bus driver 4503c91d114SIoana Ciornei 4513c91d114SIoana Ciornei- finally, if the PHY is built into the switch, as is very common with 4523c91d114SIoana Ciornei standalone switch packages, the PHY is probed using the slave MII bus created 4533c91d114SIoana Ciornei by DSA 4543c91d114SIoana Ciornei 4553c91d114SIoana Ciornei 4563c91d114SIoana CiorneiSWITCHDEV 4573c91d114SIoana Ciornei--------- 4583c91d114SIoana Ciornei 4593c91d114SIoana CiorneiDSA directly utilizes SWITCHDEV when interfacing with the bridge layer, and 4603c91d114SIoana Ciorneimore specifically with its VLAN filtering portion when configuring VLANs on top 461f8843991SVladimir Olteanof per-port slave network devices. As of today, the only SWITCHDEV objects 462f8843991SVladimir Olteansupported by DSA are the FDB and VLAN objects. 4633c91d114SIoana Ciornei 4648411abbcSVladimir OlteanDevlink 4658411abbcSVladimir Oltean------- 4668411abbcSVladimir Oltean 4678411abbcSVladimir OlteanDSA registers one devlink device per physical switch in the fabric. 4688411abbcSVladimir OlteanFor each devlink device, every physical port (i.e. user ports, CPU ports, DSA 4698411abbcSVladimir Olteanlinks or unused ports) is exposed as a devlink port. 4708411abbcSVladimir Oltean 4718411abbcSVladimir OlteanDSA drivers can make use of the following devlink features: 4728794be45SVladimir Oltean 4738411abbcSVladimir Oltean- Regions: debugging feature which allows user space to dump driver-defined 4748411abbcSVladimir Oltean areas of hardware information in a low-level, binary format. Both global 4758411abbcSVladimir Oltean regions as well as per-port regions are supported. It is possible to export 4768411abbcSVladimir Oltean devlink regions even for pieces of data that are already exposed in some way 4778411abbcSVladimir Oltean to the standard iproute2 user space programs (ip-link, bridge), like address 4788411abbcSVladimir Oltean tables and VLAN tables. For example, this might be useful if the tables 4798411abbcSVladimir Oltean contain additional hardware-specific details which are not visible through 4808411abbcSVladimir Oltean the iproute2 abstraction, or it might be useful to inspect these tables on 4818411abbcSVladimir Oltean the non-user ports too, which are invisible to iproute2 because no network 4828411abbcSVladimir Oltean interface is registered for them. 4838411abbcSVladimir Oltean- Params: a feature which enables user to configure certain low-level tunable 4848411abbcSVladimir Oltean knobs pertaining to the device. Drivers may implement applicable generic 4858411abbcSVladimir Oltean devlink params, or may add new device-specific devlink params. 4868411abbcSVladimir Oltean- Resources: a monitoring feature which enables users to see the degree of 4878411abbcSVladimir Oltean utilization of certain hardware tables in the device, such as FDB, VLAN, etc. 4888411abbcSVladimir Oltean- Shared buffers: a QoS feature for adjusting and partitioning memory and frame 4898411abbcSVladimir Oltean reservations per port and per traffic class, in the ingress and egress 4908411abbcSVladimir Oltean directions, such that low-priority bulk traffic does not impede the 4918411abbcSVladimir Oltean processing of high-priority critical traffic. 4928411abbcSVladimir Oltean 4938411abbcSVladimir OlteanFor more details, consult ``Documentation/networking/devlink/``. 4948411abbcSVladimir Oltean 4953c91d114SIoana CiorneiDevice Tree 4963c91d114SIoana Ciornei----------- 4973c91d114SIoana Ciornei 4983c91d114SIoana CiorneiDSA features a standardized binding which is documented in 4993c91d114SIoana Ciornei``Documentation/devicetree/bindings/net/dsa/dsa.txt``. PHY/MDIO library helper 5003c91d114SIoana Ciorneifunctions such as ``of_get_phy_mode()``, ``of_phy_connect()`` are also used to query 5015a48b743SBjorn Helgaasper-port PHY specific details: interface connection, MDIO bus location, etc. 5023c91d114SIoana Ciornei 5033c91d114SIoana CiorneiDriver development 5043c91d114SIoana Ciornei================== 5053c91d114SIoana Ciornei 50619b3b13cSVladimir OlteanDSA switch drivers need to implement a ``dsa_switch_ops`` structure which will 5073c91d114SIoana Ciorneicontain the various members described below. 5083c91d114SIoana Ciornei 50919b3b13cSVladimir OlteanProbing, registration and device lifetime 51019b3b13cSVladimir Oltean----------------------------------------- 5113c91d114SIoana Ciornei 51219b3b13cSVladimir OlteanDSA switches are regular ``device`` structures on buses (be they platform, SPI, 51319b3b13cSVladimir OlteanI2C, MDIO or otherwise). The DSA framework is not involved in their probing 51419b3b13cSVladimir Olteanwith the device core. 51519b3b13cSVladimir Oltean 51619b3b13cSVladimir OlteanSwitch registration from the perspective of a driver means passing a valid 51719b3b13cSVladimir Oltean``struct dsa_switch`` pointer to ``dsa_register_switch()``, usually from the 51819b3b13cSVladimir Olteanswitch driver's probing function. The following members must be valid in the 51919b3b13cSVladimir Olteanprovided structure: 52019b3b13cSVladimir Oltean 52119b3b13cSVladimir Oltean- ``ds->dev``: will be used to parse the switch's OF node or platform data. 52219b3b13cSVladimir Oltean 52319b3b13cSVladimir Oltean- ``ds->num_ports``: will be used to create the port list for this switch, and 52419b3b13cSVladimir Oltean to validate the port indices provided in the OF node. 52519b3b13cSVladimir Oltean 52619b3b13cSVladimir Oltean- ``ds->ops``: a pointer to the ``dsa_switch_ops`` structure holding the DSA 52719b3b13cSVladimir Oltean method implementations. 52819b3b13cSVladimir Oltean 52919b3b13cSVladimir Oltean- ``ds->priv``: backpointer to a driver-private data structure which can be 53019b3b13cSVladimir Oltean retrieved in all further DSA method callbacks. 53119b3b13cSVladimir Oltean 53219b3b13cSVladimir OlteanIn addition, the following flags in the ``dsa_switch`` structure may optionally 53319b3b13cSVladimir Olteanbe configured to obtain driver-specific behavior from the DSA core. Their 53419b3b13cSVladimir Olteanbehavior when set is documented through comments in ``include/net/dsa.h``. 53519b3b13cSVladimir Oltean 53619b3b13cSVladimir Oltean- ``ds->vlan_filtering_is_global`` 53719b3b13cSVladimir Oltean 53819b3b13cSVladimir Oltean- ``ds->needs_standalone_vlan_filtering`` 53919b3b13cSVladimir Oltean 54019b3b13cSVladimir Oltean- ``ds->configure_vlan_while_not_filtering`` 54119b3b13cSVladimir Oltean 54219b3b13cSVladimir Oltean- ``ds->untag_bridge_pvid`` 54319b3b13cSVladimir Oltean 54419b3b13cSVladimir Oltean- ``ds->assisted_learning_on_cpu_port`` 54519b3b13cSVladimir Oltean 54619b3b13cSVladimir Oltean- ``ds->mtu_enforcement_ingress`` 54719b3b13cSVladimir Oltean 54819b3b13cSVladimir Oltean- ``ds->fdb_isolation`` 54919b3b13cSVladimir Oltean 55019b3b13cSVladimir OlteanInternally, DSA keeps an array of switch trees (group of switches) global to 55119b3b13cSVladimir Olteanthe kernel, and attaches a ``dsa_switch`` structure to a tree on registration. 55219b3b13cSVladimir OlteanThe tree ID to which the switch is attached is determined by the first u32 55319b3b13cSVladimir Olteannumber of the ``dsa,member`` property of the switch's OF node (0 if missing). 55419b3b13cSVladimir OlteanThe switch ID within the tree is determined by the second u32 number of the 55519b3b13cSVladimir Olteansame OF property (0 if missing). Registering multiple switches with the same 55619b3b13cSVladimir Olteanswitch ID and tree ID is illegal and will cause an error. Using platform data, 55719b3b13cSVladimir Olteana single switch and a single switch tree is permitted. 55819b3b13cSVladimir Oltean 55919b3b13cSVladimir OlteanIn case of a tree with multiple switches, probing takes place asymmetrically. 56019b3b13cSVladimir OlteanThe first N-1 callers of ``dsa_register_switch()`` only add their ports to the 56119b3b13cSVladimir Olteanport list of the tree (``dst->ports``), each port having a backpointer to its 56219b3b13cSVladimir Olteanassociated switch (``dp->ds``). Then, these switches exit their 56319b3b13cSVladimir Oltean``dsa_register_switch()`` call early, because ``dsa_tree_setup_routing_table()`` 56419b3b13cSVladimir Olteanhas determined that the tree is not yet complete (not all ports referenced by 56519b3b13cSVladimir OlteanDSA links are present in the tree's port list). The tree becomes complete when 56619b3b13cSVladimir Olteanthe last switch calls ``dsa_register_switch()``, and this triggers the effective 56719b3b13cSVladimir Olteancontinuation of initialization (including the call to ``ds->ops->setup()``) for 56819b3b13cSVladimir Olteanall switches within that tree, all as part of the calling context of the last 56919b3b13cSVladimir Olteanswitch's probe function. 57019b3b13cSVladimir Oltean 57119b3b13cSVladimir OlteanThe opposite of registration takes place when calling ``dsa_unregister_switch()``, 57219b3b13cSVladimir Olteanwhich removes a switch's ports from the port list of the tree. The entire tree 57319b3b13cSVladimir Olteanis torn down when the first switch unregisters. 5743c91d114SIoana Ciornei 57554367831SVladimir OlteanIt is mandatory for DSA switch drivers to implement the ``shutdown()`` callback 57654367831SVladimir Olteanof their respective bus, and call ``dsa_switch_shutdown()`` from it (a minimal 57754367831SVladimir Olteanversion of the full teardown performed by ``dsa_unregister_switch()``). 57854367831SVladimir OlteanThe reason is that DSA keeps a reference on the master net device, and if the 57954367831SVladimir Olteandriver for the master device decides to unbind on shutdown, DSA's reference 58054367831SVladimir Olteanwill block that operation from finalizing. 58154367831SVladimir Oltean 58254367831SVladimir OlteanEither ``dsa_switch_shutdown()`` or ``dsa_unregister_switch()`` must be called, 58354367831SVladimir Olteanbut not both, and the device driver model permits the bus' ``remove()`` method 58454367831SVladimir Olteanto be called even if ``shutdown()`` was already called. Therefore, drivers are 58554367831SVladimir Olteanexpected to implement a mutual exclusion method between ``remove()`` and 58654367831SVladimir Oltean``shutdown()`` by setting their drvdata to NULL after any of these has run, and 58754367831SVladimir Olteanchecking whether the drvdata is NULL before proceeding to take any action. 58854367831SVladimir Oltean 58954367831SVladimir OlteanAfter ``dsa_switch_shutdown()`` or ``dsa_unregister_switch()`` was called, no 59054367831SVladimir Olteanfurther callbacks via the provided ``dsa_switch_ops`` may take place, and the 59154367831SVladimir Olteandriver may free the data structures associated with the ``dsa_switch``. 59254367831SVladimir Oltean 5933c91d114SIoana CiorneiSwitch configuration 5943c91d114SIoana Ciornei-------------------- 5953c91d114SIoana Ciornei 596c3f0e84dSVladimir Oltean- ``get_tag_protocol``: this is to indicate what kind of tagging protocol is 597c56313a4SVladimir Oltean supported, should be a valid value from the ``dsa_tag_protocol`` enum. 598c56313a4SVladimir Oltean The returned information does not have to be static; the driver is passed the 599c56313a4SVladimir Oltean CPU port number, as well as the tagging protocol of a possibly stacked 600c56313a4SVladimir Oltean upstream switch, in case there are hardware limitations in terms of supported 601c56313a4SVladimir Oltean tag formats. 6023c91d114SIoana Ciornei 603d6a0336aSVladimir Oltean- ``change_tag_protocol``: when the default tagging protocol has compatibility 604d6a0336aSVladimir Oltean problems with the master or other issues, the driver may support changing it 605d6a0336aSVladimir Oltean at runtime, either through a device tree property or through sysfs. In that 606d6a0336aSVladimir Oltean case, further calls to ``get_tag_protocol`` should report the protocol in 607d6a0336aSVladimir Oltean current use. 608d6a0336aSVladimir Oltean 6093c91d114SIoana Ciornei- ``setup``: setup function for the switch, this function is responsible for setting 6103c91d114SIoana Ciornei up the ``dsa_switch_ops`` private structure with all it needs: register maps, 6115a48b743SBjorn Helgaas interrupts, mutexes, locks, etc. This function is also expected to properly 6123c91d114SIoana Ciornei configure the switch to separate all network interfaces from each other, that 6133c91d114SIoana Ciornei is, they should be isolated by the switch hardware itself, typically by creating 6143c91d114SIoana Ciornei a Port-based VLAN ID for each port and allowing only the CPU port and the 6153c91d114SIoana Ciornei specific port to be in the forwarding vector. Ports that are unused by the 6163c91d114SIoana Ciornei platform should be disabled. Past this function, the switch is expected to be 6173c91d114SIoana Ciornei fully configured and ready to serve any kind of request. It is recommended 6183c91d114SIoana Ciornei to issue a software reset of the switch during this setup function in order to 6193c91d114SIoana Ciornei avoid relying on what a previous software agent such as a bootloader/firmware 620b763f50dSVladimir Oltean may have previously configured. The method responsible for undoing any 621b763f50dSVladimir Oltean applicable allocations or operations done here is ``teardown``. 6223c91d114SIoana Ciornei 6233c87237eSVladimir Oltean- ``port_setup`` and ``port_teardown``: methods for initialization and 6243c87237eSVladimir Oltean destruction of per-port data structures. It is mandatory for some operations 6253c87237eSVladimir Oltean such as registering and unregistering devlink port regions to be done from 6263c87237eSVladimir Oltean these methods, otherwise they are optional. A port will be torn down only if 6273c87237eSVladimir Oltean it has been previously set up. It is possible for a port to be set up during 6283c87237eSVladimir Oltean probing only to be torn down immediately afterwards, for example in case its 6293c87237eSVladimir Oltean PHY cannot be found. In this case, probing of the DSA switch continues 6303c87237eSVladimir Oltean without that particular port. 6313c87237eSVladimir Oltean 6323c91d114SIoana CiorneiPHY devices and link management 6333c91d114SIoana Ciornei------------------------------- 6343c91d114SIoana Ciornei 6353c91d114SIoana Ciornei- ``get_phy_flags``: Some switches are interfaced to various kinds of Ethernet PHYs, 6363c91d114SIoana Ciornei if the PHY library PHY driver needs to know about information it cannot obtain 6373c91d114SIoana Ciornei on its own (e.g.: coming from switch memory mapped registers), this function 6385a48b743SBjorn Helgaas should return a 32-bit bitmask of "flags" that is private between the switch 6393c91d114SIoana Ciornei driver and the Ethernet PHY driver in ``drivers/net/phy/\*``. 6403c91d114SIoana Ciornei 6413c91d114SIoana Ciornei- ``phy_read``: Function invoked by the DSA slave MDIO bus when attempting to read 6423c91d114SIoana Ciornei the switch port MDIO registers. If unavailable, return 0xffff for each read. 6433c91d114SIoana Ciornei For builtin switch Ethernet PHYs, this function should allow reading the link 6445a48b743SBjorn Helgaas status, auto-negotiation results, link partner pages, etc. 6453c91d114SIoana Ciornei 6463c91d114SIoana Ciornei- ``phy_write``: Function invoked by the DSA slave MDIO bus when attempting to write 6473c91d114SIoana Ciornei to the switch port MDIO registers. If unavailable return a negative error 6483c91d114SIoana Ciornei code. 6493c91d114SIoana Ciornei 6503c91d114SIoana Ciornei- ``adjust_link``: Function invoked by the PHY library when a slave network device 6513c91d114SIoana Ciornei is attached to a PHY device. This function is responsible for appropriately 6523c91d114SIoana Ciornei configuring the switch port link parameters: speed, duplex, pause based on 6533c91d114SIoana Ciornei what the ``phy_device`` is providing. 6543c91d114SIoana Ciornei 6553c91d114SIoana Ciornei- ``fixed_link_update``: Function invoked by the PHY library, and specifically by 6563c91d114SIoana Ciornei the fixed PHY driver asking the switch driver for link parameters that could 6573c91d114SIoana Ciornei not be auto-negotiated, or obtained by reading the PHY registers through MDIO. 6583c91d114SIoana Ciornei This is particularly useful for specific kinds of hardware such as QSGMII, 6593c91d114SIoana Ciornei MoCA or other kinds of non-MDIO managed PHYs where out of band link 6603c91d114SIoana Ciornei information is obtained 6613c91d114SIoana Ciornei 6623c91d114SIoana CiorneiEthtool operations 6633c91d114SIoana Ciornei------------------ 6643c91d114SIoana Ciornei 6653c91d114SIoana Ciornei- ``get_strings``: ethtool function used to query the driver's strings, will 6665a48b743SBjorn Helgaas typically return statistics strings, private flags strings, etc. 6673c91d114SIoana Ciornei 6683c91d114SIoana Ciornei- ``get_ethtool_stats``: ethtool function used to query per-port statistics and 6693c91d114SIoana Ciornei return their values. DSA overlays slave network devices general statistics: 6703c91d114SIoana Ciornei RX/TX counters from the network device, with switch driver specific statistics 6713c91d114SIoana Ciornei per port 6723c91d114SIoana Ciornei 6733c91d114SIoana Ciornei- ``get_sset_count``: ethtool function used to query the number of statistics items 6743c91d114SIoana Ciornei 6753c91d114SIoana Ciornei- ``get_wol``: ethtool function used to obtain Wake-on-LAN settings per-port, this 6765a48b743SBjorn Helgaas function may for certain implementations also query the master network device 6773c91d114SIoana Ciornei Wake-on-LAN settings if this interface needs to participate in Wake-on-LAN 6783c91d114SIoana Ciornei 6793c91d114SIoana Ciornei- ``set_wol``: ethtool function used to configure Wake-on-LAN settings per-port, 6803c91d114SIoana Ciornei direct counterpart to set_wol with similar restrictions 6813c91d114SIoana Ciornei 6823c91d114SIoana Ciornei- ``set_eee``: ethtool function which is used to configure a switch port EEE (Green 6833c91d114SIoana Ciornei Ethernet) settings, can optionally invoke the PHY library to enable EEE at the 6843c91d114SIoana Ciornei PHY level if relevant. This function should enable EEE at the switch port MAC 6853c91d114SIoana Ciornei controller and data-processing logic 6863c91d114SIoana Ciornei 6873c91d114SIoana Ciornei- ``get_eee``: ethtool function which is used to query a switch port EEE settings, 6883c91d114SIoana Ciornei this function should return the EEE state of the switch port MAC controller 6893c91d114SIoana Ciornei and data-processing logic as well as query the PHY for its currently configured 6903c91d114SIoana Ciornei EEE settings 6913c91d114SIoana Ciornei 6923c91d114SIoana Ciornei- ``get_eeprom_len``: ethtool function returning for a given switch the EEPROM 6933c91d114SIoana Ciornei length/size in bytes 6943c91d114SIoana Ciornei 6953c91d114SIoana Ciornei- ``get_eeprom``: ethtool function returning for a given switch the EEPROM contents 6963c91d114SIoana Ciornei 6973c91d114SIoana Ciornei- ``set_eeprom``: ethtool function writing specified data to a given switch EEPROM 6983c91d114SIoana Ciornei 6993c91d114SIoana Ciornei- ``get_regs_len``: ethtool function returning the register length for a given 7003c91d114SIoana Ciornei switch 7013c91d114SIoana Ciornei 7023c91d114SIoana Ciornei- ``get_regs``: ethtool function returning the Ethernet switch internal register 7033c91d114SIoana Ciornei contents. This function might require user-land code in ethtool to 7043c91d114SIoana Ciornei pretty-print register values and registers 7053c91d114SIoana Ciornei 7063c91d114SIoana CiorneiPower management 7073c91d114SIoana Ciornei---------------- 7083c91d114SIoana Ciornei 7093c91d114SIoana Ciornei- ``suspend``: function invoked by the DSA platform device when the system goes to 7103c91d114SIoana Ciornei suspend, should quiesce all Ethernet switch activities, but keep ports 7113c91d114SIoana Ciornei participating in Wake-on-LAN active as well as additional wake-up logic if 7123c91d114SIoana Ciornei supported 7133c91d114SIoana Ciornei 7143c91d114SIoana Ciornei- ``resume``: function invoked by the DSA platform device when the system resumes, 7153c91d114SIoana Ciornei should resume all Ethernet switch activities and re-configure the switch to be 7163c91d114SIoana Ciornei in a fully active state 7173c91d114SIoana Ciornei 7183c91d114SIoana Ciornei- ``port_enable``: function invoked by the DSA slave network device ndo_open 7195a48b743SBjorn Helgaas function when a port is administratively brought up, this function should 7205a48b743SBjorn Helgaas fully enable a given switch port. DSA takes care of marking the port with 7213c91d114SIoana Ciornei ``BR_STATE_BLOCKING`` if the port is a bridge member, or ``BR_STATE_FORWARDING`` if it 7223c91d114SIoana Ciornei was not, and propagating these changes down to the hardware 7233c91d114SIoana Ciornei 7243c91d114SIoana Ciornei- ``port_disable``: function invoked by the DSA slave network device ndo_close 7255a48b743SBjorn Helgaas function when a port is administratively brought down, this function should 7265a48b743SBjorn Helgaas fully disable a given switch port. DSA takes care of marking the port with 7273c91d114SIoana Ciornei ``BR_STATE_DISABLED`` and propagating changes to the hardware if this port is 7283c91d114SIoana Ciornei disabled while being a bridge member 7293c91d114SIoana Ciornei 7304e9d9bb6SVladimir OlteanAddress databases 7314e9d9bb6SVladimir Oltean----------------- 7324e9d9bb6SVladimir Oltean 7334e9d9bb6SVladimir OlteanSwitching hardware is expected to have a table for FDB entries, however not all 7344e9d9bb6SVladimir Olteanof them are active at the same time. An address database is the subset (partition) 7354e9d9bb6SVladimir Olteanof FDB entries that is active (can be matched by address learning on RX, or FDB 7364e9d9bb6SVladimir Olteanlookup on TX) depending on the state of the port. An address database may 7374e9d9bb6SVladimir Olteanoccasionally be called "FID" (Filtering ID) in this document, although the 7384e9d9bb6SVladimir Olteanunderlying implementation may choose whatever is available to the hardware. 7394e9d9bb6SVladimir Oltean 7404e9d9bb6SVladimir OlteanFor example, all ports that belong to a VLAN-unaware bridge (which is 7414e9d9bb6SVladimir Oltean*currently* VLAN-unaware) are expected to learn source addresses in the 7424e9d9bb6SVladimir Olteandatabase associated by the driver with that bridge (and not with other 7434e9d9bb6SVladimir OlteanVLAN-unaware bridges). During forwarding and FDB lookup, a packet received on a 7444e9d9bb6SVladimir OlteanVLAN-unaware bridge port should be able to find a VLAN-unaware FDB entry having 7454e9d9bb6SVladimir Olteanthe same MAC DA as the packet, which is present on another port member of the 7464e9d9bb6SVladimir Olteansame bridge. At the same time, the FDB lookup process must be able to not find 7474e9d9bb6SVladimir Olteanan FDB entry having the same MAC DA as the packet, if that entry points towards 7484e9d9bb6SVladimir Olteana port which is a member of a different VLAN-unaware bridge (and is therefore 7494e9d9bb6SVladimir Olteanassociated with a different address database). 7504e9d9bb6SVladimir Oltean 7514e9d9bb6SVladimir OlteanSimilarly, each VLAN of each offloaded VLAN-aware bridge should have an 7524e9d9bb6SVladimir Olteanassociated address database, which is shared by all ports which are members of 7534e9d9bb6SVladimir Olteanthat VLAN, but not shared by ports belonging to different bridges that are 7544e9d9bb6SVladimir Olteanmembers of the same VID. 7554e9d9bb6SVladimir Oltean 7564e9d9bb6SVladimir OlteanIn this context, a VLAN-unaware database means that all packets are expected to 7574e9d9bb6SVladimir Olteanmatch on it irrespective of VLAN ID (only MAC address lookup), whereas a 7584e9d9bb6SVladimir OlteanVLAN-aware database means that packets are supposed to match based on the VLAN 7594e9d9bb6SVladimir OlteanID from the classified 802.1Q header (or the pvid if untagged). 7604e9d9bb6SVladimir Oltean 7614e9d9bb6SVladimir OlteanAt the bridge layer, VLAN-unaware FDB entries have the special VID value of 0, 7624e9d9bb6SVladimir Olteanwhereas VLAN-aware FDB entries have non-zero VID values. Note that a 7634e9d9bb6SVladimir OlteanVLAN-unaware bridge may have VLAN-aware (non-zero VID) FDB entries, and a 7644e9d9bb6SVladimir OlteanVLAN-aware bridge may have VLAN-unaware FDB entries. As in hardware, the 7654e9d9bb6SVladimir Olteansoftware bridge keeps separate address databases, and offloads to hardware the 7664e9d9bb6SVladimir OlteanFDB entries belonging to these databases, through switchdev, asynchronously 7674e9d9bb6SVladimir Olteanrelative to the moment when the databases become active or inactive. 7684e9d9bb6SVladimir Oltean 7694e9d9bb6SVladimir OlteanWhen a user port operates in standalone mode, its driver should configure it to 7704e9d9bb6SVladimir Olteanuse a separate database called a port private database. This is different from 7714e9d9bb6SVladimir Olteanthe databases described above, and should impede operation as standalone port 7724e9d9bb6SVladimir Oltean(packet in, packet out to the CPU port) as little as possible. For example, 7734e9d9bb6SVladimir Olteanon ingress, it should not attempt to learn the MAC SA of ingress traffic, since 7744e9d9bb6SVladimir Olteanlearning is a bridging layer service and this is a standalone port, therefore 7754e9d9bb6SVladimir Olteanit would consume useless space. With no address learning, the port private 7764e9d9bb6SVladimir Olteandatabase should be empty in a naive implementation, and in this case, all 7774e9d9bb6SVladimir Olteanreceived packets should be trivially flooded to the CPU port. 7784e9d9bb6SVladimir Oltean 7794e9d9bb6SVladimir OlteanDSA (cascade) and CPU ports are also called "shared" ports because they service 7804e9d9bb6SVladimir Olteanmultiple address databases, and the database that a packet should be associated 7814e9d9bb6SVladimir Olteanto is usually embedded in the DSA tag. This means that the CPU port may 7824e9d9bb6SVladimir Olteansimultaneously transport packets coming from a standalone port (which were 7834e9d9bb6SVladimir Olteanclassified by hardware in one address database), and from a bridge port (which 7844e9d9bb6SVladimir Olteanwere classified to a different address database). 7854e9d9bb6SVladimir Oltean 7864e9d9bb6SVladimir OlteanSwitch drivers which satisfy certain criteria are able to optimize the naive 7874e9d9bb6SVladimir Olteanconfiguration by removing the CPU port from the flooding domain of the switch, 7884e9d9bb6SVladimir Olteanand just program the hardware with FDB entries pointing towards the CPU port 7894e9d9bb6SVladimir Olteanfor which it is known that software is interested in those MAC addresses. 7904e9d9bb6SVladimir OlteanPackets which do not match a known FDB entry will not be delivered to the CPU, 7914e9d9bb6SVladimir Olteanwhich will save CPU cycles required for creating an skb just to drop it. 7924e9d9bb6SVladimir Oltean 7934e9d9bb6SVladimir OlteanDSA is able to perform host address filtering for the following kinds of 7944e9d9bb6SVladimir Olteanaddresses: 7954e9d9bb6SVladimir Oltean 7964e9d9bb6SVladimir Oltean- Primary unicast MAC addresses of ports (``dev->dev_addr``). These are 7974e9d9bb6SVladimir Oltean associated with the port private database of the respective user port, 7984e9d9bb6SVladimir Oltean and the driver is notified to install them through ``port_fdb_add`` towards 7994e9d9bb6SVladimir Oltean the CPU port. 8004e9d9bb6SVladimir Oltean 8014e9d9bb6SVladimir Oltean- Secondary unicast and multicast MAC addresses of ports (addresses added 8024e9d9bb6SVladimir Oltean through ``dev_uc_add()`` and ``dev_mc_add()``). These are also associated 8034e9d9bb6SVladimir Oltean with the port private database of the respective user port. 8044e9d9bb6SVladimir Oltean 8054e9d9bb6SVladimir Oltean- Local/permanent bridge FDB entries (``BR_FDB_LOCAL``). These are the MAC 8064e9d9bb6SVladimir Oltean addresses of the bridge ports, for which packets must be terminated locally 8074e9d9bb6SVladimir Oltean and not forwarded. They are associated with the address database for that 8084e9d9bb6SVladimir Oltean bridge. 8094e9d9bb6SVladimir Oltean 8104e9d9bb6SVladimir Oltean- Static bridge FDB entries installed towards foreign (non-DSA) interfaces 8114e9d9bb6SVladimir Oltean present in the same bridge as some DSA switch ports. These are also 8124e9d9bb6SVladimir Oltean associated with the address database for that bridge. 8134e9d9bb6SVladimir Oltean 8144e9d9bb6SVladimir Oltean- Dynamically learned FDB entries on foreign interfaces present in the same 8154e9d9bb6SVladimir Oltean bridge as some DSA switch ports, only if ``ds->assisted_learning_on_cpu_port`` 8164e9d9bb6SVladimir Oltean is set to true by the driver. These are associated with the address database 8174e9d9bb6SVladimir Oltean for that bridge. 8184e9d9bb6SVladimir Oltean 8194e9d9bb6SVladimir OlteanFor various operations detailed below, DSA provides a ``dsa_db`` structure 8204e9d9bb6SVladimir Olteanwhich can be of the following types: 8214e9d9bb6SVladimir Oltean 8224e9d9bb6SVladimir Oltean- ``DSA_DB_PORT``: the FDB (or MDB) entry to be installed or deleted belongs to 8234e9d9bb6SVladimir Oltean the port private database of user port ``db->dp``. 8244e9d9bb6SVladimir Oltean- ``DSA_DB_BRIDGE``: the entry belongs to one of the address databases of bridge 8254e9d9bb6SVladimir Oltean ``db->bridge``. Separation between the VLAN-unaware database and the per-VID 8264e9d9bb6SVladimir Oltean databases of this bridge is expected to be done by the driver. 8274e9d9bb6SVladimir Oltean- ``DSA_DB_LAG``: the entry belongs to the address database of LAG ``db->lag``. 8284e9d9bb6SVladimir Oltean Note: ``DSA_DB_LAG`` is currently unused and may be removed in the future. 8294e9d9bb6SVladimir Oltean 8304e9d9bb6SVladimir OlteanThe drivers which act upon the ``dsa_db`` argument in ``port_fdb_add``, 8314e9d9bb6SVladimir Oltean``port_mdb_add`` etc should declare ``ds->fdb_isolation`` as true. 8324e9d9bb6SVladimir Oltean 8334e9d9bb6SVladimir OlteanDSA associates each offloaded bridge and each offloaded LAG with a one-based ID 8344e9d9bb6SVladimir Oltean(``struct dsa_bridge :: num``, ``struct dsa_lag :: id``) for the purposes of 8354e9d9bb6SVladimir Olteanrefcounting addresses on shared ports. Drivers may piggyback on DSA's numbering 8364e9d9bb6SVladimir Olteanscheme (the ID is readable through ``db->bridge.num`` and ``db->lag.id`` or may 8374e9d9bb6SVladimir Olteanimplement their own. 8384e9d9bb6SVladimir Oltean 8394e9d9bb6SVladimir OlteanOnly the drivers which declare support for FDB isolation are notified of FDB 8404e9d9bb6SVladimir Olteanentries on the CPU port belonging to ``DSA_DB_PORT`` databases. 8414e9d9bb6SVladimir OlteanFor compatibility/legacy reasons, ``DSA_DB_BRIDGE`` addresses are notified to 8424e9d9bb6SVladimir Olteandrivers even if they do not support FDB isolation. However, ``db->bridge.num`` 8434e9d9bb6SVladimir Olteanand ``db->lag.id`` are always set to 0 in that case (to denote the lack of 8444e9d9bb6SVladimir Olteanisolation, for refcounting purposes). 8454e9d9bb6SVladimir Oltean 8464e9d9bb6SVladimir OlteanNote that it is not mandatory for a switch driver to implement physically 8474e9d9bb6SVladimir Olteanseparate address databases for each standalone user port. Since FDB entries in 8484e9d9bb6SVladimir Olteanthe port private databases will always point to the CPU port, there is no risk 8494e9d9bb6SVladimir Olteanfor incorrect forwarding decisions. In this case, all standalone ports may 8504e9d9bb6SVladimir Olteanshare the same database, but the reference counting of host-filtered addresses 8514e9d9bb6SVladimir Oltean(not deleting the FDB entry for a port's MAC address if it's still in use by 8524e9d9bb6SVladimir Olteananother port) becomes the responsibility of the driver, because DSA is unaware 8534e9d9bb6SVladimir Olteanthat the port databases are in fact shared. This can be achieved by calling 8544e9d9bb6SVladimir Oltean``dsa_fdb_present_in_other_db()`` and ``dsa_mdb_present_in_other_db()``. 8554e9d9bb6SVladimir OlteanThe down side is that the RX filtering lists of each user port are in fact 8564e9d9bb6SVladimir Olteanshared, which means that user port A may accept a packet with a MAC DA it 8574e9d9bb6SVladimir Olteanshouldn't have, only because that MAC address was in the RX filtering list of 8584e9d9bb6SVladimir Olteanuser port B. These packets will still be dropped in software, however. 8594e9d9bb6SVladimir Oltean 8603c91d114SIoana CiorneiBridge layer 8613c91d114SIoana Ciornei------------ 8623c91d114SIoana Ciornei 86330836239SVladimir OlteanOffloading the bridge forwarding plane is optional and handled by the methods 86430836239SVladimir Olteanbelow. They may be absent, return -EOPNOTSUPP, or ``ds->max_num_bridges`` may 86530836239SVladimir Olteanbe non-zero and exceeded, and in this case, joining a bridge port is still 86630836239SVladimir Olteanpossible, but the packet forwarding will take place in software, and the ports 86730836239SVladimir Olteanunder a software bridge must remain configured in the same way as for 86830836239SVladimir Olteanstandalone operation, i.e. have all bridging service functions (address 86930836239SVladimir Olteanlearning etc) disabled, and send all received packets to the CPU port only. 87030836239SVladimir Oltean 87130836239SVladimir OlteanConcretely, a port starts offloading the forwarding plane of a bridge once it 87230836239SVladimir Olteanreturns success to the ``port_bridge_join`` method, and stops doing so after 87330836239SVladimir Oltean``port_bridge_leave`` has been called. Offloading the bridge means autonomously 87430836239SVladimir Olteanlearning FDB entries in accordance with the software bridge port's state, and 87530836239SVladimir Olteanautonomously forwarding (or flooding) received packets without CPU intervention. 87630836239SVladimir OlteanThis is optional even when offloading a bridge port. Tagging protocol drivers 87730836239SVladimir Olteanare expected to call ``dsa_default_offload_fwd_mark(skb)`` for packets which 87830836239SVladimir Olteanhave already been autonomously forwarded in the forwarding domain of the 87930836239SVladimir Olteaningress switch port. DSA, through ``dsa_port_devlink_setup()``, considers all 88030836239SVladimir Olteanswitch ports part of the same tree ID to be part of the same bridge forwarding 88130836239SVladimir Olteandomain (capable of autonomous forwarding to each other). 88230836239SVladimir Oltean 88330836239SVladimir OlteanOffloading the TX forwarding process of a bridge is a distinct concept from 88430836239SVladimir Olteansimply offloading its forwarding plane, and refers to the ability of certain 88530836239SVladimir Olteandriver and tag protocol combinations to transmit a single skb coming from the 88630836239SVladimir Olteanbridge device's transmit function to potentially multiple egress ports (and 88730836239SVladimir Olteanthereby avoid its cloning in software). 88830836239SVladimir Oltean 88930836239SVladimir OlteanPackets for which the bridge requests this behavior are called data plane 89030836239SVladimir Olteanpackets and have ``skb->offload_fwd_mark`` set to true in the tag protocol 89130836239SVladimir Olteandriver's ``xmit`` function. Data plane packets are subject to FDB lookup, 89230836239SVladimir Olteanhardware learning on the CPU port, and do not override the port STP state. 89330836239SVladimir OlteanAdditionally, replication of data plane packets (multicast, flooding) is 89430836239SVladimir Olteanhandled in hardware and the bridge driver will transmit a single skb for each 89530836239SVladimir Olteanpacket that may or may not need replication. 89630836239SVladimir Oltean 89730836239SVladimir OlteanWhen the TX forwarding offload is enabled, the tag protocol driver is 89830836239SVladimir Olteanresponsible to inject packets into the data plane of the hardware towards the 89930836239SVladimir Olteancorrect bridging domain (FID) that the port is a part of. The port may be 90030836239SVladimir OlteanVLAN-unaware, and in this case the FID must be equal to the FID used by the 90130836239SVladimir Olteandriver for its VLAN-unaware address database associated with that bridge. 90230836239SVladimir OlteanAlternatively, the bridge may be VLAN-aware, and in that case, it is guaranteed 90330836239SVladimir Olteanthat the packet is also VLAN-tagged with the VLAN ID that the bridge processed 90430836239SVladimir Olteanthis packet in. It is the responsibility of the hardware to untag the VID on 90530836239SVladimir Olteanthe egress-untagged ports, or keep the tag on the egress-tagged ones. 90630836239SVladimir Oltean 9073c91d114SIoana Ciornei- ``port_bridge_join``: bridge layer function invoked when a given switch port is 9085a48b743SBjorn Helgaas added to a bridge, this function should do what's necessary at the switch 9095a48b743SBjorn Helgaas level to permit the joining port to be added to the relevant logical 9103c91d114SIoana Ciornei domain for it to ingress/egress traffic with other members of the bridge. 91130836239SVladimir Oltean By setting the ``tx_fwd_offload`` argument to true, the TX forwarding process 91230836239SVladimir Oltean of this bridge is also offloaded. 9133c91d114SIoana Ciornei 9143c91d114SIoana Ciornei- ``port_bridge_leave``: bridge layer function invoked when a given switch port is 9155a48b743SBjorn Helgaas removed from a bridge, this function should do what's necessary at the 9163c91d114SIoana Ciornei switch level to deny the leaving port from ingress/egress traffic from the 9170cb8682eSVladimir Oltean remaining bridge members. 9183c91d114SIoana Ciornei 9193c91d114SIoana Ciornei- ``port_stp_state_set``: bridge layer function invoked when a given switch port STP 9203c91d114SIoana Ciornei state is computed by the bridge layer and should be propagated to switch 9210cb8682eSVladimir Oltean hardware to forward/block/learn traffic. 9223c91d114SIoana Ciornei 9235a275f4cSVladimir Oltean- ``port_bridge_flags``: bridge layer function invoked when a port must 9245a275f4cSVladimir Oltean configure its settings for e.g. flooding of unknown traffic or source address 9255a275f4cSVladimir Oltean learning. The switch driver is responsible for initial setup of the 9265a275f4cSVladimir Oltean standalone ports with address learning disabled and egress flooding of all 9275a275f4cSVladimir Oltean types of traffic, then the DSA core notifies of any change to the bridge port 9285a275f4cSVladimir Oltean flags when the port joins and leaves a bridge. DSA does not currently manage 9295a275f4cSVladimir Oltean the bridge port flags for the CPU port. The assumption is that address 9305a275f4cSVladimir Oltean learning should be statically enabled (if supported by the hardware) on the 9315a275f4cSVladimir Oltean CPU port, and flooding towards the CPU port should also be enabled, due to a 9325a275f4cSVladimir Oltean lack of an explicit address filtering mechanism in the DSA core. 9335a275f4cSVladimir Oltean 9340cb8682eSVladimir Oltean- ``port_fast_age``: bridge layer function invoked when flushing the 9350cb8682eSVladimir Oltean dynamically learned FDB entries on the port is necessary. This is called when 9360cb8682eSVladimir Oltean transitioning from an STP state where learning should take place to an STP 9370cb8682eSVladimir Oltean state where it shouldn't, or when leaving a bridge, or when address learning 9380cb8682eSVladimir Oltean is turned off via ``port_bridge_flags``. 9390cb8682eSVladimir Oltean 9403c91d114SIoana CiorneiBridge VLAN filtering 9413c91d114SIoana Ciornei--------------------- 9423c91d114SIoana Ciornei 9433c91d114SIoana Ciornei- ``port_vlan_filtering``: bridge layer function invoked when the bridge gets 9443c91d114SIoana Ciornei configured for turning on or off VLAN filtering. If nothing specific needs to 9453c91d114SIoana Ciornei be done at the hardware level, this callback does not need to be implemented. 9463c91d114SIoana Ciornei When VLAN filtering is turned on, the hardware must be programmed with 9473c91d114SIoana Ciornei rejecting 802.1Q frames which have VLAN IDs outside of the programmed allowed 9483c91d114SIoana Ciornei VLAN ID map/rules. If there is no PVID programmed into the switch port, 9493c91d114SIoana Ciornei untagged frames must be rejected as well. When turned off the switch must 9503c91d114SIoana Ciornei accept any 802.1Q frames irrespective of their VLAN ID, and untagged frames are 9513c91d114SIoana Ciornei allowed. 9523c91d114SIoana Ciornei 9533c91d114SIoana Ciornei- ``port_vlan_add``: bridge layer function invoked when a VLAN is configured 954*6ba1a4aaSVladimir Oltean (tagged or untagged) for the given switch port.. 9553c91d114SIoana Ciornei 9563c91d114SIoana Ciornei- ``port_vlan_del``: bridge layer function invoked when a VLAN is removed from the 9573c91d114SIoana Ciornei given switch port 9583c91d114SIoana Ciornei 9593c91d114SIoana Ciornei- ``port_fdb_add``: bridge layer function invoked when the bridge wants to install a 9603c91d114SIoana Ciornei Forwarding Database entry, the switch hardware should be programmed with the 9613c91d114SIoana Ciornei specified address in the specified VLAN Id in the forwarding database 962*6ba1a4aaSVladimir Oltean associated with this VLAN ID. 9633c91d114SIoana Ciornei 9643c91d114SIoana Ciornei- ``port_fdb_del``: bridge layer function invoked when the bridge wants to remove a 9653c91d114SIoana Ciornei Forwarding Database entry, the switch hardware should be programmed to delete 9663c91d114SIoana Ciornei the specified MAC address from the specified VLAN ID if it was mapped into 9673c91d114SIoana Ciornei this port forwarding database 9683c91d114SIoana Ciornei 969ea7006a7SVladimir Oltean- ``port_fdb_dump``: bridge bypass function invoked by ``ndo_fdb_dump`` on the 970ea7006a7SVladimir Oltean physical DSA port interfaces. Since DSA does not attempt to keep in sync its 971ea7006a7SVladimir Oltean hardware FDB entries with the software bridge, this method is implemented as 972ea7006a7SVladimir Oltean a means to view the entries visible on user ports in the hardware database. 973ea7006a7SVladimir Oltean The entries reported by this function have the ``self`` flag in the output of 974ea7006a7SVladimir Oltean the ``bridge fdb show`` command. 9753c91d114SIoana Ciornei 9763c91d114SIoana Ciornei- ``port_mdb_add``: bridge layer function invoked when the bridge wants to install 977*6ba1a4aaSVladimir Oltean a multicast database entry. The switch hardware should be programmed with the 9783c91d114SIoana Ciornei specified address in the specified VLAN ID in the forwarding database 9793c91d114SIoana Ciornei associated with this VLAN ID. 9803c91d114SIoana Ciornei 9813c91d114SIoana Ciornei- ``port_mdb_del``: bridge layer function invoked when the bridge wants to remove a 9823c91d114SIoana Ciornei multicast database entry, the switch hardware should be programmed to delete 9833c91d114SIoana Ciornei the specified MAC address from the specified VLAN ID if it was mapped into 9843c91d114SIoana Ciornei this port forwarding database. 9853c91d114SIoana Ciornei 986a9985444SVladimir OlteanLink aggregation 987a9985444SVladimir Oltean---------------- 988a9985444SVladimir Oltean 989a9985444SVladimir OlteanLink aggregation is implemented in the Linux networking stack by the bonding 990a9985444SVladimir Olteanand team drivers, which are modeled as virtual, stackable network interfaces. 991a9985444SVladimir OlteanDSA is capable of offloading a link aggregation group (LAG) to hardware that 992a9985444SVladimir Olteansupports the feature, and supports bridging between physical ports and LAGs, 993a9985444SVladimir Olteanas well as between LAGs. A bonding/team interface which holds multiple physical 994a9985444SVladimir Olteanports constitutes a logical port, although DSA has no explicit concept of a 995a9985444SVladimir Olteanlogical port at the moment. Due to this, events where a LAG joins/leaves a 996a9985444SVladimir Olteanbridge are treated as if all individual physical ports that are members of that 997a9985444SVladimir OlteanLAG join/leave the bridge. Switchdev port attributes (VLAN filtering, STP 998a9985444SVladimir Olteanstate, etc) and objects (VLANs, MDB entries) offloaded to a LAG as bridge port 999a9985444SVladimir Olteanare treated similarly: DSA offloads the same switchdev object / port attribute 1000a9985444SVladimir Olteanon all members of the LAG. Static bridge FDB entries on a LAG are not yet 1001a9985444SVladimir Olteansupported, since the DSA driver API does not have the concept of a logical port 1002a9985444SVladimir OlteanID. 1003a9985444SVladimir Oltean 1004a9985444SVladimir Oltean- ``port_lag_join``: function invoked when a given switch port is added to a 1005a9985444SVladimir Oltean LAG. The driver may return ``-EOPNOTSUPP``, and in this case, DSA will fall 1006a9985444SVladimir Oltean back to a software implementation where all traffic from this port is sent to 1007a9985444SVladimir Oltean the CPU. 1008a9985444SVladimir Oltean- ``port_lag_leave``: function invoked when a given switch port leaves a LAG 1009a9985444SVladimir Oltean and returns to operation as a standalone port. 1010a9985444SVladimir Oltean- ``port_lag_change``: function invoked when the link state of any member of 1011a9985444SVladimir Oltean the LAG changes, and the hashing function needs rebalancing to only make use 1012a9985444SVladimir Oltean of the subset of physical LAG member ports that are up. 1013a9985444SVladimir Oltean 1014a9985444SVladimir OlteanDrivers that benefit from having an ID associated with each offloaded LAG 1015a9985444SVladimir Olteancan optionally populate ``ds->num_lag_ids`` from the ``dsa_switch_ops::setup`` 1016a9985444SVladimir Olteanmethod. The LAG ID associated with a bonding/team interface can then be 1017a9985444SVladimir Olteanretrieved by a DSA switch driver using the ``dsa_lag_id`` function. 1018a9985444SVladimir Oltean 1019f8f3c20aSVladimir OlteanIEC 62439-2 (MRP) 1020f8f3c20aSVladimir Oltean----------------- 1021f8f3c20aSVladimir Oltean 1022f8f3c20aSVladimir OlteanThe Media Redundancy Protocol is a topology management protocol optimized for 1023f8f3c20aSVladimir Olteanfast fault recovery time for ring networks, which has some components 1024f8f3c20aSVladimir Olteanimplemented as a function of the bridge driver. MRP uses management PDUs 1025f8f3c20aSVladimir Oltean(Test, Topology, LinkDown/Up, Option) sent at a multicast destination MAC 1026f8f3c20aSVladimir Olteanaddress range of 01:15:4e:00:00:0x and with an EtherType of 0x88e3. 1027f8f3c20aSVladimir OlteanDepending on the node's role in the ring (MRM: Media Redundancy Manager, 1028f8f3c20aSVladimir OlteanMRC: Media Redundancy Client, MRA: Media Redundancy Automanager), certain MRP 1029f8f3c20aSVladimir OlteanPDUs might need to be terminated locally and others might need to be forwarded. 1030f8f3c20aSVladimir OlteanAn MRM might also benefit from offloading to hardware the creation and 1031f8f3c20aSVladimir Olteantransmission of certain MRP PDUs (Test). 1032f8f3c20aSVladimir Oltean 1033f8f3c20aSVladimir OlteanNormally an MRP instance can be created on top of any network interface, 1034f8f3c20aSVladimir Olteanhowever in the case of a device with an offloaded data path such as DSA, it is 1035f8f3c20aSVladimir Olteannecessary for the hardware, even if it is not MRP-aware, to be able to extract 1036f8f3c20aSVladimir Olteanthe MRP PDUs from the fabric before the driver can proceed with the software 1037f8f3c20aSVladimir Olteanimplementation. DSA today has no driver which is MRP-aware, therefore it only 1038f8f3c20aSVladimir Olteanlistens for the bare minimum switchdev objects required for the software assist 1039f8f3c20aSVladimir Olteanto work properly. The operations are detailed below. 1040f8f3c20aSVladimir Oltean 1041f8f3c20aSVladimir Oltean- ``port_mrp_add`` and ``port_mrp_del``: notifies driver when an MRP instance 1042f8f3c20aSVladimir Oltean with a certain ring ID, priority, primary port and secondary port is 1043f8f3c20aSVladimir Oltean created/deleted. 1044f8f3c20aSVladimir Oltean- ``port_mrp_add_ring_role`` and ``port_mrp_del_ring_role``: function invoked 1045f8f3c20aSVladimir Oltean when an MRP instance changes ring roles between MRM or MRC. This affects 1046f8f3c20aSVladimir Oltean which MRP PDUs should be trapped to software and which should be autonomously 1047f8f3c20aSVladimir Oltean forwarded. 1048f8f3c20aSVladimir Oltean 10496e9530f4SVladimir OlteanIEC 62439-3 (HSR/PRP) 10506e9530f4SVladimir Oltean--------------------- 10516e9530f4SVladimir Oltean 10526e9530f4SVladimir OlteanThe Parallel Redundancy Protocol (PRP) is a network redundancy protocol which 10536e9530f4SVladimir Olteanworks by duplicating and sequence numbering packets through two independent L2 10546e9530f4SVladimir Olteannetworks (which are unaware of the PRP tail tags carried in the packets), and 10556e9530f4SVladimir Olteaneliminating the duplicates at the receiver. The High-availability Seamless 10566e9530f4SVladimir OlteanRedundancy (HSR) protocol is similar in concept, except all nodes that carry 10576e9530f4SVladimir Olteanthe redundant traffic are aware of the fact that it is HSR-tagged (because HSR 10586e9530f4SVladimir Olteanuses a header with an EtherType of 0x892f) and are physically connected in a 10596e9530f4SVladimir Olteanring topology. Both HSR and PRP use supervision frames for monitoring the 10606e9530f4SVladimir Olteanhealth of the network and for discovery of other nodes. 10616e9530f4SVladimir Oltean 10626e9530f4SVladimir OlteanIn Linux, both HSR and PRP are implemented in the hsr driver, which 10636e9530f4SVladimir Olteaninstantiates a virtual, stackable network interface with two member ports. 10646e9530f4SVladimir OlteanThe driver only implements the basic roles of DANH (Doubly Attached Node 10656e9530f4SVladimir Olteanimplementing HSR) and DANP (Doubly Attached Node implementing PRP); the roles 10666e9530f4SVladimir Olteanof RedBox and QuadBox are not implemented (therefore, bridging a hsr network 10676e9530f4SVladimir Olteaninterface with a physical switch port does not produce the expected result). 10686e9530f4SVladimir Oltean 10696e9530f4SVladimir OlteanA driver which is able of offloading certain functions of a DANP or DANH should 10706e9530f4SVladimir Olteandeclare the corresponding netdev features as indicated by the documentation at 10716e9530f4SVladimir Oltean``Documentation/networking/netdev-features.rst``. Additionally, the following 10726e9530f4SVladimir Olteanmethods must be implemented: 10736e9530f4SVladimir Oltean 10746e9530f4SVladimir Oltean- ``port_hsr_join``: function invoked when a given switch port is added to a 10756e9530f4SVladimir Oltean DANP/DANH. The driver may return ``-EOPNOTSUPP`` and in this case, DSA will 10766e9530f4SVladimir Oltean fall back to a software implementation where all traffic from this port is 10776e9530f4SVladimir Oltean sent to the CPU. 10786e9530f4SVladimir Oltean- ``port_hsr_leave``: function invoked when a given switch port leaves a 10796e9530f4SVladimir Oltean DANP/DANH and returns to normal operation as a standalone port. 10806e9530f4SVladimir Oltean 10813c91d114SIoana CiorneiTODO 10823c91d114SIoana Ciornei==== 10833c91d114SIoana Ciornei 10843c91d114SIoana CiorneiMaking SWITCHDEV and DSA converge towards an unified codebase 10853c91d114SIoana Ciornei------------------------------------------------------------- 10863c91d114SIoana Ciornei 10873c91d114SIoana CiorneiSWITCHDEV properly takes care of abstracting the networking stack with offload 10883c91d114SIoana Ciorneicapable hardware, but does not enforce a strict switch device driver model. On 10893c91d114SIoana Ciorneithe other DSA enforces a fairly strict device driver model, and deals with most 10903c91d114SIoana Ciorneiof the switch specific. At some point we should envision a merger between these 10913c91d114SIoana Ciorneitwo subsystems and get the best of both worlds. 10923c91d114SIoana Ciornei 10933c91d114SIoana CiorneiOther hanging fruits 10943c91d114SIoana Ciornei-------------------- 10953c91d114SIoana Ciornei 10963c91d114SIoana Ciornei- allowing more than one CPU/management interface: 10973c91d114SIoana Ciornei http://comments.gmane.org/gmane.linux.network/365657 1098