1132db935SJakub Kicinski.. SPDX-License-Identifier: GPL-2.0+ 2132db935SJakub Kicinski 3132db935SJakub Kicinski================================================================= 4132db935SJakub KicinskiLinux Base Driver for the Intel(R) Ethernet Controller 700 Series 5132db935SJakub Kicinski================================================================= 6132db935SJakub Kicinski 7132db935SJakub KicinskiIntel 40 Gigabit Linux driver. 8132db935SJakub KicinskiCopyright(c) 1999-2018 Intel Corporation. 9132db935SJakub Kicinski 10132db935SJakub KicinskiContents 11132db935SJakub Kicinski======== 12132db935SJakub Kicinski 13132db935SJakub Kicinski- Overview 14132db935SJakub Kicinski- Identifying Your Adapter 15132db935SJakub Kicinski- Intel(R) Ethernet Flow Director 16132db935SJakub Kicinski- Additional Configurations 17132db935SJakub Kicinski- Known Issues 18132db935SJakub Kicinski- Support 19132db935SJakub Kicinski 20132db935SJakub Kicinski 21132db935SJakub KicinskiDriver information can be obtained using ethtool, lspci, and ifconfig. 22132db935SJakub KicinskiInstructions on updating ethtool can be found in the section Additional 23132db935SJakub KicinskiConfigurations later in this document. 24132db935SJakub Kicinski 25132db935SJakub KicinskiFor questions related to hardware requirements, refer to the documentation 26132db935SJakub Kicinskisupplied with your Intel adapter. All hardware requirements listed apply to use 27132db935SJakub Kicinskiwith Linux. 28132db935SJakub Kicinski 29132db935SJakub Kicinski 30132db935SJakub KicinskiIdentifying Your Adapter 31132db935SJakub Kicinski======================== 32132db935SJakub KicinskiThe driver is compatible with devices based on the following: 33132db935SJakub Kicinski 34132db935SJakub Kicinski * Intel(R) Ethernet Controller X710 35132db935SJakub Kicinski * Intel(R) Ethernet Controller XL710 36132db935SJakub Kicinski * Intel(R) Ethernet Network Connection X722 37132db935SJakub Kicinski * Intel(R) Ethernet Controller XXV710 38132db935SJakub Kicinski 39132db935SJakub KicinskiFor the best performance, make sure the latest NVM/FW is installed on your 40132db935SJakub Kicinskidevice. 41132db935SJakub Kicinski 42132db935SJakub KicinskiFor information on how to identify your adapter, and for the latest NVM/FW 43132db935SJakub Kicinskiimages and Intel network drivers, refer to the Intel Support website: 44132db935SJakub Kicinskihttps://www.intel.com/support 45132db935SJakub Kicinski 46132db935SJakub KicinskiSFP+ and QSFP+ Devices 47132db935SJakub Kicinski---------------------- 48132db935SJakub KicinskiFor information about supported media, refer to this document: 49132db935SJakub Kicinskihttps://www.intel.com/content/dam/www/public/us/en/documents/release-notes/xl710-ethernet-controller-feature-matrix.pdf 50132db935SJakub Kicinski 51132db935SJakub KicinskiNOTE: Some adapters based on the Intel(R) Ethernet Controller 700 Series only 52132db935SJakub Kicinskisupport Intel Ethernet Optics modules. On these adapters, other modules are not 53132db935SJakub Kicinskisupported and will not function. In all cases Intel recommends using Intel 54132db935SJakub KicinskiEthernet Optics; other modules may function but are not validated by Intel. 55132db935SJakub KicinskiContact Intel for supported media types. 56132db935SJakub Kicinski 57132db935SJakub KicinskiNOTE: For connections based on Intel(R) Ethernet Controller 700 Series, support 58132db935SJakub Kicinskiis dependent on your system board. Please see your vendor for details. 59132db935SJakub Kicinski 60132db935SJakub KicinskiNOTE: In systems that do not have adequate airflow to cool the adapter and 61132db935SJakub Kicinskioptical modules, you must use high temperature optical modules. 62132db935SJakub Kicinski 63132db935SJakub KicinskiVirtual Functions (VFs) 64132db935SJakub Kicinski----------------------- 65132db935SJakub KicinskiUse sysfs to enable VFs. For example:: 66132db935SJakub Kicinski 67132db935SJakub Kicinski #echo $num_vf_enabled > /sys/class/net/$dev/device/sriov_numvfs #enable VFs 68132db935SJakub Kicinski #echo 0 > /sys/class/net/$dev/device/sriov_numvfs #disable VFs 69132db935SJakub Kicinski 70132db935SJakub KicinskiFor example, the following instructions will configure PF eth0 and the first VF 71132db935SJakub Kicinskion VLAN 10:: 72132db935SJakub Kicinski 73132db935SJakub Kicinski $ ip link set dev eth0 vf 0 vlan 10 74132db935SJakub Kicinski 75132db935SJakub KicinskiVLAN Tag Packet Steering 76132db935SJakub Kicinski------------------------ 77132db935SJakub KicinskiAllows you to send all packets with a specific VLAN tag to a particular SR-IOV 78132db935SJakub Kicinskivirtual function (VF). Further, this feature allows you to designate a 79132db935SJakub Kicinskiparticular VF as trusted, and allows that trusted VF to request selective 80132db935SJakub Kicinskipromiscuous mode on the Physical Function (PF). 81132db935SJakub Kicinski 82132db935SJakub KicinskiTo set a VF as trusted or untrusted, enter the following command in the 83132db935SJakub KicinskiHypervisor:: 84132db935SJakub Kicinski 85132db935SJakub Kicinski # ip link set dev eth0 vf 1 trust [on|off] 86132db935SJakub Kicinski 87132db935SJakub KicinskiOnce the VF is designated as trusted, use the following commands in the VM to 88132db935SJakub Kicinskiset the VF to promiscuous mode. 89132db935SJakub Kicinski 90132db935SJakub Kicinski:: 91132db935SJakub Kicinski 92132db935SJakub Kicinski For promiscuous all: 93132db935SJakub Kicinski #ip link set eth2 promisc on 94132db935SJakub Kicinski Where eth2 is a VF interface in the VM 95132db935SJakub Kicinski 96132db935SJakub Kicinski For promiscuous Multicast: 97132db935SJakub Kicinski #ip link set eth2 allmulticast on 98132db935SJakub Kicinski Where eth2 is a VF interface in the VM 99132db935SJakub Kicinski 100132db935SJakub KicinskiNOTE: By default, the ethtool priv-flag vf-true-promisc-support is set to 101132db935SJakub Kicinski"off",meaning that promiscuous mode for the VF will be limited. To set the 102132db935SJakub Kicinskipromiscuous mode for the VF to true promiscuous and allow the VF to see all 103132db935SJakub Kicinskiingress traffic, use the following command:: 104132db935SJakub Kicinski 105132db935SJakub Kicinski #ethtool -set-priv-flags p261p1 vf-true-promisc-support on 106132db935SJakub Kicinski 107132db935SJakub KicinskiThe vf-true-promisc-support priv-flag does not enable promiscuous mode; rather, 108132db935SJakub Kicinskiit designates which type of promiscuous mode (limited or true) you will get 109132db935SJakub Kicinskiwhen you enable promiscuous mode using the ip link commands above. Note that 110132db935SJakub Kicinskithis is a global setting that affects the entire device. However,the 111132db935SJakub Kicinskivf-true-promisc-support priv-flag is only exposed to the first PF of the 112132db935SJakub Kicinskidevice. The PF remains in limited promiscuous mode (unless it is in MFP mode) 113132db935SJakub Kicinskiregardless of the vf-true-promisc-support setting. 114132db935SJakub Kicinski 115132db935SJakub KicinskiNow add a VLAN interface on the VF interface:: 116132db935SJakub Kicinski 117132db935SJakub Kicinski #ip link add link eth2 name eth2.100 type vlan id 100 118132db935SJakub Kicinski 119132db935SJakub KicinskiNote that the order in which you set the VF to promiscuous mode and add the 120132db935SJakub KicinskiVLAN interface does not matter (you can do either first). The end result in 121132db935SJakub Kicinskithis example is that the VF will get all traffic that is tagged with VLAN 100. 122132db935SJakub Kicinski 123132db935SJakub KicinskiIntel(R) Ethernet Flow Director 124132db935SJakub Kicinski------------------------------- 125132db935SJakub KicinskiThe Intel Ethernet Flow Director performs the following tasks: 126132db935SJakub Kicinski 127132db935SJakub Kicinski- Directs receive packets according to their flows to different queues. 128132db935SJakub Kicinski- Enables tight control on routing a flow in the platform. 129132db935SJakub Kicinski- Matches flows and CPU cores for flow affinity. 130132db935SJakub Kicinski- Supports multiple parameters for flexible flow classification and load 131132db935SJakub Kicinski balancing (in SFP mode only). 132132db935SJakub Kicinski 133132db935SJakub KicinskiNOTE: The Linux i40e driver supports the following flow types: IPv4, TCPv4, and 134132db935SJakub KicinskiUDPv4. For a given flow type, it supports valid combinations of IP addresses 135132db935SJakub Kicinski(source or destination) and UDP/TCP ports (source and destination). For 136132db935SJakub Kicinskiexample, you can supply only a source IP address, a source IP address and a 137132db935SJakub Kicinskidestination port, or any combination of one or more of these four parameters. 138132db935SJakub Kicinski 139132db935SJakub KicinskiNOTE: The Linux i40e driver allows you to filter traffic based on a 140132db935SJakub Kicinskiuser-defined flexible two-byte pattern and offset by using the ethtool user-def 141132db935SJakub Kicinskiand mask fields. Only L3 and L4 flow types are supported for user-defined 142132db935SJakub Kicinskiflexible filters. For a given flow type, you must clear all Intel Ethernet Flow 143132db935SJakub KicinskiDirector filters before changing the input set (for that flow type). 144132db935SJakub Kicinski 145132db935SJakub KicinskiTo enable or disable the Intel Ethernet Flow Director:: 146132db935SJakub Kicinski 147132db935SJakub Kicinski # ethtool -K ethX ntuple <on|off> 148132db935SJakub Kicinski 149132db935SJakub KicinskiWhen disabling ntuple filters, all the user programmed filters are flushed from 150132db935SJakub Kicinskithe driver cache and hardware. All needed filters must be re-added when ntuple 151132db935SJakub Kicinskiis re-enabled. 152132db935SJakub Kicinski 153132db935SJakub KicinskiTo add a filter that directs packet to queue 2, use -U or -N switch:: 154132db935SJakub Kicinski 155132db935SJakub Kicinski # ethtool -N ethX flow-type tcp4 src-ip 192.168.10.1 dst-ip \ 156132db935SJakub Kicinski 192.168.10.2 src-port 2000 dst-port 2001 action 2 [loc 1] 157132db935SJakub Kicinski 158132db935SJakub KicinskiTo set a filter using only the source and destination IP address:: 159132db935SJakub Kicinski 160132db935SJakub Kicinski # ethtool -N ethX flow-type tcp4 src-ip 192.168.10.1 dst-ip \ 161132db935SJakub Kicinski 192.168.10.2 action 2 [loc 1] 162132db935SJakub Kicinski 163132db935SJakub KicinskiTo see the list of filters currently present:: 164132db935SJakub Kicinski 165132db935SJakub Kicinski # ethtool <-u|-n> ethX 166132db935SJakub Kicinski 167132db935SJakub KicinskiApplication Targeted Routing (ATR) Perfect Filters 168132db935SJakub Kicinski-------------------------------------------------- 169132db935SJakub KicinskiATR is enabled by default when the kernel is in multiple transmit queue mode. 170132db935SJakub KicinskiAn ATR Intel Ethernet Flow Director filter rule is added when a TCP-IP flow 171132db935SJakub Kicinskistarts and is deleted when the flow ends. When a TCP-IP Intel Ethernet Flow 172132db935SJakub KicinskiDirector rule is added from ethtool (Sideband filter), ATR is turned off by the 173132db935SJakub Kicinskidriver. To re-enable ATR, the sideband can be disabled with the ethtool -K 174132db935SJakub Kicinskioption. For example:: 175132db935SJakub Kicinski 176132db935SJakub Kicinski ethtool –K [adapter] ntuple [off|on] 177132db935SJakub Kicinski 178132db935SJakub KicinskiIf sideband is re-enabled after ATR is re-enabled, ATR remains enabled until a 179132db935SJakub KicinskiTCP-IP flow is added. When all TCP-IP sideband rules are deleted, ATR is 180132db935SJakub Kicinskiautomatically re-enabled. 181132db935SJakub Kicinski 182132db935SJakub KicinskiPackets that match the ATR rules are counted in fdir_atr_match stats in 183132db935SJakub Kicinskiethtool, which also can be used to verify whether ATR rules still exist. 184132db935SJakub Kicinski 185132db935SJakub KicinskiSideband Perfect Filters 186132db935SJakub Kicinski------------------------ 187132db935SJakub KicinskiSideband Perfect Filters are used to direct traffic that matches specified 188132db935SJakub Kicinskicharacteristics. They are enabled through ethtool's ntuple interface. To add a 189132db935SJakub Kicinskinew filter use the following command:: 190132db935SJakub Kicinski 191132db935SJakub Kicinski ethtool -U <device> flow-type <type> src-ip <ip> dst-ip <ip> src-port <port> \ 192132db935SJakub Kicinski dst-port <port> action <queue> 193132db935SJakub Kicinski 194132db935SJakub KicinskiWhere: 195132db935SJakub Kicinski <device> - the ethernet device to program 196132db935SJakub Kicinski <type> - can be ip4, tcp4, udp4, or sctp4 197132db935SJakub Kicinski <ip> - the ip address to match on 198132db935SJakub Kicinski <port> - the port number to match on 199132db935SJakub Kicinski <queue> - the queue to direct traffic towards (-1 discards matching traffic) 200132db935SJakub Kicinski 201132db935SJakub KicinskiUse the following command to display all of the active filters:: 202132db935SJakub Kicinski 203132db935SJakub Kicinski ethtool -u <device> 204132db935SJakub Kicinski 205132db935SJakub KicinskiUse the following command to delete a filter:: 206132db935SJakub Kicinski 207132db935SJakub Kicinski ethtool -U <device> delete <N> 208132db935SJakub Kicinski 209132db935SJakub KicinskiWhere <N> is the filter id displayed when printing all the active filters, and 210132db935SJakub Kicinskimay also have been specified using "loc <N>" when adding the filter. 211132db935SJakub Kicinski 212132db935SJakub KicinskiThe following example matches TCP traffic sent from 192.168.0.1, port 5300, 213132db935SJakub Kicinskidirected to 192.168.0.5, port 80, and sends it to queue 7:: 214132db935SJakub Kicinski 215132db935SJakub Kicinski ethtool -U enp130s0 flow-type tcp4 src-ip 192.168.0.1 dst-ip 192.168.0.5 \ 216132db935SJakub Kicinski src-port 5300 dst-port 80 action 7 217132db935SJakub Kicinski 218132db935SJakub KicinskiFor each flow-type, the programmed filters must all have the same matching 219132db935SJakub Kicinskiinput set. For example, issuing the following two commands is acceptable:: 220132db935SJakub Kicinski 221132db935SJakub Kicinski ethtool -U enp130s0 flow-type ip4 src-ip 192.168.0.1 src-port 5300 action 7 222132db935SJakub Kicinski ethtool -U enp130s0 flow-type ip4 src-ip 192.168.0.5 src-port 55 action 10 223132db935SJakub Kicinski 224132db935SJakub KicinskiIssuing the next two commands, however, is not acceptable, since the first 225132db935SJakub Kicinskispecifies src-ip and the second specifies dst-ip:: 226132db935SJakub Kicinski 227132db935SJakub Kicinski ethtool -U enp130s0 flow-type ip4 src-ip 192.168.0.1 src-port 5300 action 7 228132db935SJakub Kicinski ethtool -U enp130s0 flow-type ip4 dst-ip 192.168.0.5 src-port 55 action 10 229132db935SJakub Kicinski 230132db935SJakub KicinskiThe second command will fail with an error. You may program multiple filters 231132db935SJakub Kicinskiwith the same fields, using different values, but, on one device, you may not 232132db935SJakub Kicinskiprogram two tcp4 filters with different matching fields. 233132db935SJakub Kicinski 234132db935SJakub KicinskiMatching on a sub-portion of a field is not supported by the i40e driver, thus 235132db935SJakub Kicinskipartial mask fields are not supported. 236132db935SJakub Kicinski 237132db935SJakub KicinskiThe driver also supports matching user-defined data within the packet payload. 238132db935SJakub KicinskiThis flexible data is specified using the "user-def" field of the ethtool 239132db935SJakub Kicinskicommand in the following way: 240132db935SJakub Kicinski 241132db935SJakub Kicinski+----------------------------+--------------------------+ 242132db935SJakub Kicinski| 31 28 24 20 16 | 15 12 8 4 0 | 243132db935SJakub Kicinski+----------------------------+--------------------------+ 244132db935SJakub Kicinski| offset into packet payload | 2 bytes of flexible data | 245132db935SJakub Kicinski+----------------------------+--------------------------+ 246132db935SJakub Kicinski 247132db935SJakub KicinskiFor example, 248132db935SJakub Kicinski 249132db935SJakub Kicinski:: 250132db935SJakub Kicinski 251132db935SJakub Kicinski ... user-def 0x4FFFF ... 252132db935SJakub Kicinski 253132db935SJakub Kicinskitells the filter to look 4 bytes into the payload and match that value against 254132db935SJakub Kicinski0xFFFF. The offset is based on the beginning of the payload, and not the 255132db935SJakub Kicinskibeginning of the packet. Thus 256132db935SJakub Kicinski 257132db935SJakub Kicinski:: 258132db935SJakub Kicinski 259132db935SJakub Kicinski flow-type tcp4 ... user-def 0x8BEAF ... 260132db935SJakub Kicinski 261132db935SJakub Kicinskiwould match TCP/IPv4 packets which have the value 0xBEAF 8 bytes into the 262132db935SJakub KicinskiTCP/IPv4 payload. 263132db935SJakub Kicinski 264132db935SJakub KicinskiNote that ICMP headers are parsed as 4 bytes of header and 4 bytes of payload. 265132db935SJakub KicinskiThus to match the first byte of the payload, you must actually add 4 bytes to 266132db935SJakub Kicinskithe offset. Also note that ip4 filters match both ICMP frames as well as raw 267132db935SJakub Kicinski(unknown) ip4 frames, where the payload will be the L3 payload of the IP4 frame. 268132db935SJakub Kicinski 269132db935SJakub KicinskiThe maximum offset is 64. The hardware will only read up to 64 bytes of data 270132db935SJakub Kicinskifrom the payload. The offset must be even because the flexible data is 2 bytes 271132db935SJakub Kicinskilong and must be aligned to byte 0 of the packet payload. 272132db935SJakub Kicinski 273132db935SJakub KicinskiThe user-defined flexible offset is also considered part of the input set and 274132db935SJakub Kicinskicannot be programmed separately for multiple filters of the same type. However, 275132db935SJakub Kicinskithe flexible data is not part of the input set and multiple filters may use the 276132db935SJakub Kicinskisame offset but match against different data. 277132db935SJakub Kicinski 278132db935SJakub KicinskiTo create filters that direct traffic to a specific Virtual Function, use the 279132db935SJakub Kicinski"action" parameter. Specify the action as a 64 bit value, where the lower 32 280132db935SJakub Kicinskibits represents the queue number, while the next 8 bits represent which VF. 281132db935SJakub KicinskiNote that 0 is the PF, so the VF identifier is offset by 1. For example:: 282132db935SJakub Kicinski 283132db935SJakub Kicinski ... action 0x800000002 ... 284132db935SJakub Kicinski 285132db935SJakub Kicinskispecifies to direct traffic to Virtual Function 7 (8 minus 1) into queue 2 of 286132db935SJakub Kicinskithat VF. 287132db935SJakub Kicinski 288132db935SJakub KicinskiNote that these filters will not break internal routing rules, and will not 289132db935SJakub Kicinskiroute traffic that otherwise would not have been sent to the specified Virtual 290132db935SJakub KicinskiFunction. 291132db935SJakub Kicinski 292132db935SJakub KicinskiSetting the link-down-on-close Private Flag 293132db935SJakub Kicinski------------------------------------------- 294132db935SJakub KicinskiWhen the link-down-on-close private flag is set to "on", the port's link will 295132db935SJakub Kicinskigo down when the interface is brought down using the ifconfig ethX down command. 296132db935SJakub Kicinski 297132db935SJakub KicinskiUse ethtool to view and set link-down-on-close, as follows:: 298132db935SJakub Kicinski 299132db935SJakub Kicinski ethtool --show-priv-flags ethX 300132db935SJakub Kicinski ethtool --set-priv-flags ethX link-down-on-close [on|off] 301132db935SJakub Kicinski 302132db935SJakub KicinskiViewing Link Messages 303132db935SJakub Kicinski--------------------- 304132db935SJakub KicinskiLink messages will not be displayed to the console if the distribution is 305132db935SJakub Kicinskirestricting system messages. In order to see network driver link messages on 306132db935SJakub Kicinskiyour console, set dmesg to eight by entering the following:: 307132db935SJakub Kicinski 308132db935SJakub Kicinski dmesg -n 8 309132db935SJakub Kicinski 310132db935SJakub KicinskiNOTE: This setting is not saved across reboots. 311132db935SJakub Kicinski 312132db935SJakub KicinskiJumbo Frames 313132db935SJakub Kicinski------------ 314132db935SJakub KicinskiJumbo Frames support is enabled by changing the Maximum Transmission Unit (MTU) 315132db935SJakub Kicinskito a value larger than the default value of 1500. 316132db935SJakub Kicinski 317132db935SJakub KicinskiUse the ifconfig command to increase the MTU size. For example, enter the 318132db935SJakub Kicinskifollowing where <x> is the interface number:: 319132db935SJakub Kicinski 320132db935SJakub Kicinski ifconfig eth<x> mtu 9000 up 321132db935SJakub Kicinski 322132db935SJakub KicinskiAlternatively, you can use the ip command as follows:: 323132db935SJakub Kicinski 324132db935SJakub Kicinski ip link set mtu 9000 dev eth<x> 325132db935SJakub Kicinski ip link set up dev eth<x> 326132db935SJakub Kicinski 327132db935SJakub KicinskiThis setting is not saved across reboots. The setting change can be made 328132db935SJakub Kicinskipermanent by adding 'MTU=9000' to the file:: 329132db935SJakub Kicinski 330132db935SJakub Kicinski /etc/sysconfig/network-scripts/ifcfg-eth<x> // for RHEL 331132db935SJakub Kicinski /etc/sysconfig/network/<config_file> // for SLES 332132db935SJakub Kicinski 333132db935SJakub KicinskiNOTE: The maximum MTU setting for Jumbo Frames is 9702. This value coincides 334132db935SJakub Kicinskiwith the maximum Jumbo Frames size of 9728 bytes. 335132db935SJakub Kicinski 336132db935SJakub KicinskiNOTE: This driver will attempt to use multiple page sized buffers to receive 337132db935SJakub Kicinskieach jumbo packet. This should help to avoid buffer starvation issues when 338132db935SJakub Kicinskiallocating receive packets. 339132db935SJakub Kicinski 340132db935SJakub Kicinskiethtool 341132db935SJakub Kicinski------- 342132db935SJakub KicinskiThe driver utilizes the ethtool interface for driver configuration and 343132db935SJakub Kicinskidiagnostics, as well as displaying statistical information. The latest ethtool 344132db935SJakub Kicinskiversion is required for this functionality. Download it at: 345132db935SJakub Kicinskihttps://www.kernel.org/pub/software/network/ethtool/ 346132db935SJakub Kicinski 347132db935SJakub KicinskiSupported ethtool Commands and Options for Filtering 348132db935SJakub Kicinski---------------------------------------------------- 349132db935SJakub Kicinski-n --show-nfc 350132db935SJakub Kicinski Retrieves the receive network flow classification configurations. 351132db935SJakub Kicinski 352132db935SJakub Kicinskirx-flow-hash tcp4|udp4|ah4|esp4|sctp4|tcp6|udp6|ah6|esp6|sctp6 353132db935SJakub Kicinski Retrieves the hash options for the specified network traffic type. 354132db935SJakub Kicinski 355132db935SJakub Kicinski-N --config-nfc 356132db935SJakub Kicinski Configures the receive network flow classification. 357132db935SJakub Kicinski 358132db935SJakub Kicinskirx-flow-hash tcp4|udp4|ah4|esp4|sctp4|tcp6|udp6|ah6|esp6|sctp6 m|v|t|s|d|f|n|r... 359132db935SJakub Kicinski Configures the hash options for the specified network traffic type. 360132db935SJakub Kicinski 361132db935SJakub Kicinskiudp4 UDP over IPv4 362132db935SJakub Kicinskiudp6 UDP over IPv6 363132db935SJakub Kicinski 364132db935SJakub Kicinskif Hash on bytes 0 and 1 of the Layer 4 header of the Rx packet. 365132db935SJakub Kicinskin Hash on bytes 2 and 3 of the Layer 4 header of the Rx packet. 366132db935SJakub Kicinski 367132db935SJakub KicinskiSpeed and Duplex Configuration 368132db935SJakub Kicinski------------------------------ 369132db935SJakub KicinskiIn addressing speed and duplex configuration issues, you need to distinguish 370132db935SJakub Kicinskibetween copper-based adapters and fiber-based adapters. 371132db935SJakub Kicinski 372132db935SJakub KicinskiIn the default mode, an Intel(R) Ethernet Network Adapter using copper 373132db935SJakub Kicinskiconnections will attempt to auto-negotiate with its link partner to determine 374132db935SJakub Kicinskithe best setting. If the adapter cannot establish link with the link partner 375132db935SJakub Kicinskiusing auto-negotiation, you may need to manually configure the adapter and link 376132db935SJakub Kicinskipartner to identical settings to establish link and pass packets. This should 377132db935SJakub Kicinskionly be needed when attempting to link with an older switch that does not 378132db935SJakub Kicinskisupport auto-negotiation or one that has been forced to a specific speed or 379132db935SJakub Kicinskiduplex mode. Your link partner must match the setting you choose. 1 Gbps speeds 380132db935SJakub Kicinskiand higher cannot be forced. Use the autonegotiation advertising setting to 381132db935SJakub Kicinskimanually set devices for 1 Gbps and higher. 382132db935SJakub Kicinski 383132db935SJakub KicinskiNOTE: You cannot set the speed for devices based on the Intel(R) Ethernet 384132db935SJakub KicinskiNetwork Adapter XXV710 based devices. 385132db935SJakub Kicinski 386132db935SJakub KicinskiSpeed, duplex, and autonegotiation advertising are configured through the 387132db935SJakub Kicinskiethtool utility. 388132db935SJakub Kicinski 389132db935SJakub KicinskiCaution: Only experienced network administrators should force speed and duplex 390132db935SJakub Kicinskior change autonegotiation advertising manually. The settings at the switch must 391132db935SJakub Kicinskialways match the adapter settings. Adapter performance may suffer or your 392132db935SJakub Kicinskiadapter may not operate if you configure the adapter differently from your 393132db935SJakub Kicinskiswitch. 394132db935SJakub Kicinski 395132db935SJakub KicinskiAn Intel(R) Ethernet Network Adapter using fiber-based connections, however, 396132db935SJakub Kicinskiwill not attempt to auto-negotiate with its link partner since those adapters 397132db935SJakub Kicinskioperate only in full duplex and only at their native speed. 398132db935SJakub Kicinski 399132db935SJakub KicinskiNAPI 400132db935SJakub Kicinski---- 401132db935SJakub KicinskiNAPI (Rx polling mode) is supported in the i40e driver. 402132db935SJakub KicinskiFor more information on NAPI, see 403132db935SJakub Kicinskihttps://wiki.linuxfoundation.org/networking/napi 404132db935SJakub Kicinski 405132db935SJakub KicinskiFlow Control 406132db935SJakub Kicinski------------ 407132db935SJakub KicinskiEthernet Flow Control (IEEE 802.3x) can be configured with ethtool to enable 408132db935SJakub Kicinskireceiving and transmitting pause frames for i40e. When transmit is enabled, 409132db935SJakub Kicinskipause frames are generated when the receive packet buffer crosses a predefined 410132db935SJakub Kicinskithreshold. When receive is enabled, the transmit unit will halt for the time 411132db935SJakub Kicinskidelay specified when a pause frame is received. 412132db935SJakub Kicinski 413132db935SJakub KicinskiNOTE: You must have a flow control capable link partner. 414132db935SJakub Kicinski 415132db935SJakub KicinskiFlow Control is on by default. 416132db935SJakub Kicinski 417132db935SJakub KicinskiUse ethtool to change the flow control settings. 418132db935SJakub Kicinski 419132db935SJakub KicinskiTo enable or disable Rx or Tx Flow Control:: 420132db935SJakub Kicinski 421132db935SJakub Kicinski ethtool -A eth? rx <on|off> tx <on|off> 422132db935SJakub Kicinski 423132db935SJakub KicinskiNote: This command only enables or disables Flow Control if auto-negotiation is 424132db935SJakub Kicinskidisabled. If auto-negotiation is enabled, this command changes the parameters 425132db935SJakub Kicinskiused for auto-negotiation with the link partner. 426132db935SJakub Kicinski 427132db935SJakub KicinskiTo enable or disable auto-negotiation:: 428132db935SJakub Kicinski 429132db935SJakub Kicinski ethtool -s eth? autoneg <on|off> 430132db935SJakub Kicinski 431132db935SJakub KicinskiNote: Flow Control auto-negotiation is part of link auto-negotiation. Depending 432132db935SJakub Kicinskion your device, you may not be able to change the auto-negotiation setting. 433132db935SJakub Kicinski 434132db935SJakub KicinskiRSS Hash Flow 435132db935SJakub Kicinski------------- 436132db935SJakub KicinskiAllows you to set the hash bytes per flow type and any combination of one or 437132db935SJakub Kicinskimore options for Receive Side Scaling (RSS) hash byte configuration. 438132db935SJakub Kicinski 439132db935SJakub Kicinski:: 440132db935SJakub Kicinski 441132db935SJakub Kicinski # ethtool -N <dev> rx-flow-hash <type> <option> 442132db935SJakub Kicinski 443132db935SJakub KicinskiWhere <type> is: 444132db935SJakub Kicinski tcp4 signifying TCP over IPv4 445132db935SJakub Kicinski udp4 signifying UDP over IPv4 446132db935SJakub Kicinski tcp6 signifying TCP over IPv6 447132db935SJakub Kicinski udp6 signifying UDP over IPv6 448132db935SJakub KicinskiAnd <option> is one or more of: 449132db935SJakub Kicinski s Hash on the IP source address of the Rx packet. 450132db935SJakub Kicinski d Hash on the IP destination address of the Rx packet. 451132db935SJakub Kicinski f Hash on bytes 0 and 1 of the Layer 4 header of the Rx packet. 452132db935SJakub Kicinski n Hash on bytes 2 and 3 of the Layer 4 header of the Rx packet. 453132db935SJakub Kicinski 454132db935SJakub KicinskiMAC and VLAN anti-spoofing feature 455132db935SJakub Kicinski---------------------------------- 456132db935SJakub KicinskiWhen a malicious driver attempts to send a spoofed packet, it is dropped by the 457132db935SJakub Kicinskihardware and not transmitted. 458132db935SJakub KicinskiNOTE: This feature can be disabled for a specific Virtual Function (VF):: 459132db935SJakub Kicinski 460132db935SJakub Kicinski ip link set <pf dev> vf <vf id> spoofchk {off|on} 461132db935SJakub Kicinski 462132db935SJakub KicinskiIEEE 1588 Precision Time Protocol (PTP) Hardware Clock (PHC) 463132db935SJakub Kicinski------------------------------------------------------------ 464132db935SJakub KicinskiPrecision Time Protocol (PTP) is used to synchronize clocks in a computer 465132db935SJakub Kicinskinetwork. PTP support varies among Intel devices that support this driver. Use 466132db935SJakub Kicinski"ethtool -T <netdev name>" to get a definitive list of PTP capabilities 467132db935SJakub Kicinskisupported by the device. 468132db935SJakub Kicinski 469132db935SJakub KicinskiIEEE 802.1ad (QinQ) Support 470132db935SJakub Kicinski--------------------------- 471132db935SJakub KicinskiThe IEEE 802.1ad standard, informally known as QinQ, allows for multiple VLAN 472132db935SJakub KicinskiIDs within a single Ethernet frame. VLAN IDs are sometimes referred to as 473132db935SJakub Kicinski"tags," and multiple VLAN IDs are thus referred to as a "tag stack." Tag stacks 474132db935SJakub Kicinskiallow L2 tunneling and the ability to segregate traffic within a particular 475132db935SJakub KicinskiVLAN ID, among other uses. 476132db935SJakub Kicinski 477132db935SJakub KicinskiThe following are examples of how to configure 802.1ad (QinQ):: 478132db935SJakub Kicinski 479132db935SJakub Kicinski ip link add link eth0 eth0.24 type vlan proto 802.1ad id 24 480132db935SJakub Kicinski ip link add link eth0.24 eth0.24.371 type vlan proto 802.1Q id 371 481132db935SJakub Kicinski 482132db935SJakub KicinskiWhere "24" and "371" are example VLAN IDs. 483132db935SJakub Kicinski 484132db935SJakub KicinskiNOTES: 485132db935SJakub Kicinski Receive checksum offloads, cloud filters, and VLAN acceleration are not 486132db935SJakub Kicinski supported for 802.1ad (QinQ) packets. 487132db935SJakub Kicinski 488132db935SJakub KicinskiVXLAN and GENEVE Overlay HW Offloading 489132db935SJakub Kicinski-------------------------------------- 490132db935SJakub KicinskiVirtual Extensible LAN (VXLAN) allows you to extend an L2 network over an L3 491132db935SJakub Kicinskinetwork, which may be useful in a virtualized or cloud environment. Some 492132db935SJakub KicinskiIntel(R) Ethernet Network devices perform VXLAN processing, offloading it from 493132db935SJakub Kicinskithe operating system. This reduces CPU utilization. 494132db935SJakub Kicinski 495132db935SJakub KicinskiVXLAN offloading is controlled by the Tx and Rx checksum offload options 496132db935SJakub Kicinskiprovided by ethtool. That is, if Tx checksum offload is enabled, and the 497132db935SJakub Kicinskiadapter has the capability, VXLAN offloading is also enabled. 498132db935SJakub Kicinski 499132db935SJakub KicinskiSupport for VXLAN and GENEVE HW offloading is dependent on kernel support of 500132db935SJakub Kicinskithe HW offloading features. 501132db935SJakub Kicinski 502132db935SJakub KicinskiMultiple Functions per Port 503132db935SJakub Kicinski--------------------------- 504132db935SJakub KicinskiSome adapters based on the Intel Ethernet Controller X710/XL710 support 505132db935SJakub Kicinskimultiple functions on a single physical port. Configure these functions through 506132db935SJakub Kicinskithe System Setup/BIOS. 507132db935SJakub Kicinski 508132db935SJakub KicinskiMinimum TX Bandwidth is the guaranteed minimum data transmission bandwidth, as 509132db935SJakub Kicinskia percentage of the full physical port link speed, that the partition will 510132db935SJakub Kicinskireceive. The bandwidth the partition is awarded will never fall below the level 511132db935SJakub Kicinskiyou specify. 512132db935SJakub Kicinski 513132db935SJakub KicinskiThe range for the minimum bandwidth values is: 514132db935SJakub Kicinski1 to ((100 minus # of partitions on the physical port) plus 1) 515132db935SJakub KicinskiFor example, if a physical port has 4 partitions, the range would be: 516132db935SJakub Kicinski1 to ((100 - 4) + 1 = 97) 517132db935SJakub Kicinski 518132db935SJakub KicinskiThe Maximum Bandwidth percentage represents the maximum transmit bandwidth 519132db935SJakub Kicinskiallocated to the partition as a percentage of the full physical port link 520132db935SJakub Kicinskispeed. The accepted range of values is 1-100. The value is used as a limiter, 521132db935SJakub Kicinskishould you chose that any one particular function not be able to consume 100% 522132db935SJakub Kicinskiof a port's bandwidth (should it be available). The sum of all the values for 523132db935SJakub KicinskiMaximum Bandwidth is not restricted, because no more than 100% of a port's 524132db935SJakub Kicinskibandwidth can ever be used. 525132db935SJakub Kicinski 526132db935SJakub KicinskiNOTE: X710/XXV710 devices fail to enable Max VFs (64) when Multiple Functions 527132db935SJakub Kicinskiper Port (MFP) and SR-IOV are enabled. An error from i40e is logged that says 528132db935SJakub Kicinski"add vsi failed for VF N, aq_err 16". To workaround the issue, enable less than 529132db935SJakub Kicinski64 virtual functions (VFs). 530132db935SJakub Kicinski 531132db935SJakub KicinskiData Center Bridging (DCB) 532132db935SJakub Kicinski-------------------------- 533132db935SJakub KicinskiDCB is a configuration Quality of Service implementation in hardware. It uses 534132db935SJakub Kicinskithe VLAN priority tag (802.1p) to filter traffic. That means that there are 8 535132db935SJakub Kicinskidifferent priorities that traffic can be filtered into. It also enables 536132db935SJakub Kicinskipriority flow control (802.1Qbb) which can limit or eliminate the number of 537132db935SJakub Kicinskidropped packets during network stress. Bandwidth can be allocated to each of 538132db935SJakub Kicinskithese priorities, which is enforced at the hardware level (802.1Qaz). 539132db935SJakub Kicinski 540132db935SJakub KicinskiAdapter firmware implements LLDP and DCBX protocol agents as per 802.1AB and 541132db935SJakub Kicinski802.1Qaz respectively. The firmware based DCBX agent runs in willing mode only 542132db935SJakub Kicinskiand can accept settings from a DCBX capable peer. Software configuration of 543132db935SJakub KicinskiDCBX parameters via dcbtool/lldptool are not supported. 544132db935SJakub Kicinski 545132db935SJakub KicinskiNOTE: Firmware LLDP can be disabled by setting the private flag disable-fw-lldp. 546132db935SJakub Kicinski 547132db935SJakub KicinskiThe i40e driver implements the DCB netlink interface layer to allow user-space 548132db935SJakub Kicinskito communicate with the driver and query DCB configuration for the port. 549132db935SJakub Kicinski 550132db935SJakub KicinskiNOTE: 551132db935SJakub KicinskiThe kernel assumes that TC0 is available, and will disable Priority Flow 552132db935SJakub KicinskiControl (PFC) on the device if TC0 is not available. To fix this, ensure TC0 is 553132db935SJakub Kicinskienabled when setting up DCB on your switch. 554132db935SJakub Kicinski 555132db935SJakub KicinskiInterrupt Rate Limiting 556132db935SJakub Kicinski----------------------- 557132db935SJakub Kicinski:Valid Range: 0-235 (0=no limit) 558132db935SJakub Kicinski 559132db935SJakub KicinskiThe Intel(R) Ethernet Controller XL710 family supports an interrupt rate 560132db935SJakub Kicinskilimiting mechanism. The user can control, via ethtool, the number of 561132db935SJakub Kicinskimicroseconds between interrupts. 562132db935SJakub Kicinski 563132db935SJakub KicinskiSyntax:: 564132db935SJakub Kicinski 565132db935SJakub Kicinski # ethtool -C ethX rx-usecs-high N 566132db935SJakub Kicinski 567132db935SJakub KicinskiThe range of 0-235 microseconds provides an effective range of 4,310 to 250,000 568132db935SJakub Kicinskiinterrupts per second. The value of rx-usecs-high can be set independently of 569132db935SJakub Kicinskirx-usecs and tx-usecs in the same ethtool command, and is also independent of 570132db935SJakub Kicinskithe adaptive interrupt moderation algorithm. The underlying hardware supports 571132db935SJakub Kicinskigranularity in 4-microsecond intervals, so adjacent values may result in the 572132db935SJakub Kicinskisame interrupt rate. 573132db935SJakub Kicinski 574132db935SJakub KicinskiOne possible use case is the following:: 575132db935SJakub Kicinski 576132db935SJakub Kicinski # ethtool -C ethX adaptive-rx off adaptive-tx off rx-usecs-high 20 rx-usecs \ 577132db935SJakub Kicinski 5 tx-usecs 5 578132db935SJakub Kicinski 579132db935SJakub KicinskiThe above command would disable adaptive interrupt moderation, and allow a 580132db935SJakub Kicinskimaximum of 5 microseconds before indicating a receive or transmit was complete. 581132db935SJakub KicinskiHowever, instead of resulting in as many as 200,000 interrupts per second, it 582132db935SJakub Kicinskilimits total interrupts per second to 50,000 via the rx-usecs-high parameter. 583132db935SJakub Kicinski 584132db935SJakub KicinskiPerformance Optimization 585132db935SJakub Kicinski======================== 586132db935SJakub KicinskiDriver defaults are meant to fit a wide variety of workloads, but if further 587132db935SJakub Kicinskioptimization is required we recommend experimenting with the following settings. 588132db935SJakub Kicinski 589132db935SJakub KicinskiNOTE: For better performance when processing small (64B) frame sizes, try 590132db935SJakub Kicinskienabling Hyper threading in the BIOS in order to increase the number of logical 591132db935SJakub Kicinskicores in the system and subsequently increase the number of queues available to 592132db935SJakub Kicinskithe adapter. 593132db935SJakub Kicinski 594132db935SJakub KicinskiVirtualized Environments 595132db935SJakub Kicinski------------------------ 596132db935SJakub Kicinski1. Disable XPS on both ends by using the included virt_perf_default script 597132db935SJakub Kicinskior by running the following command as root:: 598132db935SJakub Kicinski 599132db935SJakub Kicinski for file in `ls /sys/class/net/<ethX>/queues/tx-*/xps_cpus`; 600132db935SJakub Kicinski do echo 0 > $file; done 601132db935SJakub Kicinski 602132db935SJakub Kicinski2. Using the appropriate mechanism (vcpupin) in the vm, pin the cpu's to 603132db935SJakub Kicinskiindividual lcpu's, making sure to use a set of cpu's included in the 604132db935SJakub Kicinskidevice's local_cpulist: /sys/class/net/<ethX>/device/local_cpulist. 605132db935SJakub Kicinski 606132db935SJakub Kicinski3. Configure as many Rx/Tx queues in the VM as available. Do not rely on 607132db935SJakub Kicinskithe default setting of 1. 608132db935SJakub Kicinski 609132db935SJakub Kicinski 610132db935SJakub KicinskiNon-virtualized Environments 611132db935SJakub Kicinski---------------------------- 612132db935SJakub KicinskiPin the adapter's IRQs to specific cores by disabling the irqbalance service 613132db935SJakub Kicinskiand using the included set_irq_affinity script. Please see the script's help 614132db935SJakub Kicinskitext for further options. 615132db935SJakub Kicinski 616132db935SJakub Kicinski- The following settings will distribute the IRQs across all the cores evenly:: 617132db935SJakub Kicinski 618132db935SJakub Kicinski # scripts/set_irq_affinity -x all <interface1> , [ <interface2>, ... ] 619132db935SJakub Kicinski 620132db935SJakub Kicinski- The following settings will distribute the IRQs across all the cores that are 621132db935SJakub Kicinski local to the adapter (same NUMA node):: 622132db935SJakub Kicinski 623132db935SJakub Kicinski # scripts/set_irq_affinity -x local <interface1> ,[ <interface2>, ... ] 624132db935SJakub Kicinski 625132db935SJakub KicinskiFor very CPU intensive workloads, we recommend pinning the IRQs to all cores. 626132db935SJakub Kicinski 627132db935SJakub KicinskiFor IP Forwarding: Disable Adaptive ITR and lower Rx and Tx interrupts per 628132db935SJakub Kicinskiqueue using ethtool. 629132db935SJakub Kicinski 630132db935SJakub Kicinski- Setting rx-usecs and tx-usecs to 125 will limit interrupts to about 8000 631132db935SJakub Kicinski interrupts per second per queue. 632132db935SJakub Kicinski 633132db935SJakub Kicinski:: 634132db935SJakub Kicinski 635132db935SJakub Kicinski # ethtool -C <interface> adaptive-rx off adaptive-tx off rx-usecs 125 \ 636132db935SJakub Kicinski tx-usecs 125 637132db935SJakub Kicinski 638132db935SJakub KicinskiFor lower CPU utilization: Disable Adaptive ITR and lower Rx and Tx interrupts 639132db935SJakub Kicinskiper queue using ethtool. 640132db935SJakub Kicinski 641132db935SJakub Kicinski- Setting rx-usecs and tx-usecs to 250 will limit interrupts to about 4000 642132db935SJakub Kicinski interrupts per second per queue. 643132db935SJakub Kicinski 644132db935SJakub Kicinski:: 645132db935SJakub Kicinski 646132db935SJakub Kicinski # ethtool -C <interface> adaptive-rx off adaptive-tx off rx-usecs 250 \ 647132db935SJakub Kicinski tx-usecs 250 648132db935SJakub Kicinski 649132db935SJakub KicinskiFor lower latency: Disable Adaptive ITR and ITR by setting Rx and Tx to 0 using 650132db935SJakub Kicinskiethtool. 651132db935SJakub Kicinski 652132db935SJakub Kicinski:: 653132db935SJakub Kicinski 654132db935SJakub Kicinski # ethtool -C <interface> adaptive-rx off adaptive-tx off rx-usecs 0 \ 655132db935SJakub Kicinski tx-usecs 0 656132db935SJakub Kicinski 657132db935SJakub KicinskiApplication Device Queues (ADq) 658132db935SJakub Kicinski------------------------------- 659132db935SJakub KicinskiApplication Device Queues (ADq) allows you to dedicate one or more queues to a 660132db935SJakub Kicinskispecific application. This can reduce latency for the specified application, 661132db935SJakub Kicinskiand allow Tx traffic to be rate limited per application. Follow the steps below 662132db935SJakub Kicinskito set ADq. 663132db935SJakub Kicinski 664132db935SJakub Kicinski1. Create traffic classes (TCs). Maximum of 8 TCs can be created per interface. 665132db935SJakub KicinskiThe shaper bw_rlimit parameter is optional. 666132db935SJakub Kicinski 667132db935SJakub KicinskiExample: Sets up two tcs, tc0 and tc1, with 16 queues each and max tx rate set 668132db935SJakub Kicinskito 1Gbit for tc0 and 3Gbit for tc1. 669132db935SJakub Kicinski 670132db935SJakub Kicinski:: 671132db935SJakub Kicinski 672132db935SJakub Kicinski # tc qdisc add dev <interface> root mqprio num_tc 2 map 0 0 0 0 1 1 1 1 673132db935SJakub Kicinski queues 16@0 16@16 hw 1 mode channel shaper bw_rlimit min_rate 1Gbit 2Gbit 674132db935SJakub Kicinski max_rate 1Gbit 3Gbit 675132db935SJakub Kicinski 676132db935SJakub Kicinskimap: priority mapping for up to 16 priorities to tcs (e.g. map 0 0 0 0 1 1 1 1 677132db935SJakub Kicinskisets priorities 0-3 to use tc0 and 4-7 to use tc1) 678132db935SJakub Kicinski 679132db935SJakub Kicinskiqueues: for each tc, <num queues>@<offset> (e.g. queues 16@0 16@16 assigns 680132db935SJakub Kicinski16 queues to tc0 at offset 0 and 16 queues to tc1 at offset 16. Max total 681132db935SJakub Kicinskinumber of queues for all tcs is 64 or number of cores, whichever is lower.) 682132db935SJakub Kicinski 683132db935SJakub Kicinskihw 1 mode channel: ‘channel’ with ‘hw’ set to 1 is a new new hardware 684132db935SJakub Kicinskioffload mode in mqprio that makes full use of the mqprio options, the 685132db935SJakub KicinskiTCs, the queue configurations, and the QoS parameters. 686132db935SJakub Kicinski 687132db935SJakub Kicinskishaper bw_rlimit: for each tc, sets minimum and maximum bandwidth rates. 688132db935SJakub KicinskiTotals must be equal or less than port speed. 689132db935SJakub Kicinski 690132db935SJakub KicinskiFor example: min_rate 1Gbit 3Gbit: Verify bandwidth limit using network 691132db935SJakub Kicinskimonitoring tools such as ifstat or sar –n DEV [interval] [number of samples] 692132db935SJakub Kicinski 693132db935SJakub Kicinski2. Enable HW TC offload on interface:: 694132db935SJakub Kicinski 695132db935SJakub Kicinski # ethtool -K <interface> hw-tc-offload on 696132db935SJakub Kicinski 697132db935SJakub Kicinski3. Apply TCs to ingress (RX) flow of interface:: 698132db935SJakub Kicinski 699132db935SJakub Kicinski # tc qdisc add dev <interface> ingress 700132db935SJakub Kicinski 701132db935SJakub KicinskiNOTES: 702132db935SJakub Kicinski - Run all tc commands from the iproute2 <pathtoiproute2>/tc/ directory. 703132db935SJakub Kicinski - ADq is not compatible with cloud filters. 704132db935SJakub Kicinski - Setting up channels via ethtool (ethtool -L) is not supported when the 705132db935SJakub Kicinski TCs are configured using mqprio. 706132db935SJakub Kicinski - You must have iproute2 latest version 707132db935SJakub Kicinski - NVM version 6.01 or later is required. 708132db935SJakub Kicinski - ADq cannot be enabled when any the following features are enabled: Data 709132db935SJakub Kicinski Center Bridging (DCB), Multiple Functions per Port (MFP), or Sideband 710132db935SJakub Kicinski Filters. 711132db935SJakub Kicinski - If another driver (for example, DPDK) has set cloud filters, you cannot 712132db935SJakub Kicinski enable ADq. 713132db935SJakub Kicinski - Tunnel filters are not supported in ADq. If encapsulated packets do 714132db935SJakub Kicinski arrive in non-tunnel mode, filtering will be done on the inner headers. 715132db935SJakub Kicinski For example, for VXLAN traffic in non-tunnel mode, PCTYPE is identified 716132db935SJakub Kicinski as a VXLAN encapsulated packet, outer headers are ignored. Therefore, 717132db935SJakub Kicinski inner headers are matched. 718132db935SJakub Kicinski - If a TC filter on a PF matches traffic over a VF (on the PF), that 719132db935SJakub Kicinski traffic will be routed to the appropriate queue of the PF, and will 720132db935SJakub Kicinski not be passed on the VF. Such traffic will end up getting dropped higher 721132db935SJakub Kicinski up in the TCP/IP stack as it does not match PF address data. 722132db935SJakub Kicinski - If traffic matches multiple TC filters that point to different TCs, 723132db935SJakub Kicinski that traffic will be duplicated and sent to all matching TC queues. 724132db935SJakub Kicinski The hardware switch mirrors the packet to a VSI list when multiple 725132db935SJakub Kicinski filters are matched. 726132db935SJakub Kicinski 727132db935SJakub Kicinski 728132db935SJakub KicinskiKnown Issues/Troubleshooting 729132db935SJakub Kicinski============================ 730132db935SJakub Kicinski 731132db935SJakub KicinskiNOTE: 1 Gb devices based on the Intel(R) Ethernet Network Connection X722 do 732132db935SJakub Kicinskinot support the following features: 733132db935SJakub Kicinski 734132db935SJakub Kicinski * Data Center Bridging (DCB) 735132db935SJakub Kicinski * QOS 736132db935SJakub Kicinski * VMQ 737132db935SJakub Kicinski * SR-IOV 738132db935SJakub Kicinski * Task Encapsulation offload (VXLAN, NVGRE) 739132db935SJakub Kicinski * Energy Efficient Ethernet (EEE) 740132db935SJakub Kicinski * Auto-media detect 741132db935SJakub Kicinski 742132db935SJakub KicinskiUnexpected Issues when the device driver and DPDK share a device 743132db935SJakub Kicinski---------------------------------------------------------------- 744132db935SJakub KicinskiUnexpected issues may result when an i40e device is in multi driver mode and 745132db935SJakub Kicinskithe kernel driver and DPDK driver are sharing the device. This is because 746132db935SJakub Kicinskiaccess to the global NIC resources is not synchronized between multiple 747132db935SJakub Kicinskidrivers. Any change to the global NIC configuration (writing to a global 748132db935SJakub Kicinskiregister, setting global configuration by AQ, or changing switch modes) will 749132db935SJakub Kicinskiaffect all ports and drivers on the device. Loading DPDK with the 750132db935SJakub Kicinski"multi-driver" module parameter may mitigate some of the issues. 751132db935SJakub Kicinski 752132db935SJakub KicinskiTC0 must be enabled when setting up DCB on a switch 753132db935SJakub Kicinski--------------------------------------------------- 754132db935SJakub KicinskiThe kernel assumes that TC0 is available, and will disable Priority Flow 755132db935SJakub KicinskiControl (PFC) on the device if TC0 is not available. To fix this, ensure TC0 is 756132db935SJakub Kicinskienabled when setting up DCB on your switch. 757132db935SJakub Kicinski 758132db935SJakub Kicinski 759132db935SJakub KicinskiSupport 760132db935SJakub Kicinski======= 761132db935SJakub KicinskiFor general information, go to the Intel support website at: 762132db935SJakub Kicinski 763132db935SJakub Kicinskihttps://www.intel.com/support/ 764132db935SJakub Kicinski 765132db935SJakub Kicinskior the Intel Wired Networking project hosted by Sourceforge at: 766132db935SJakub Kicinski 767132db935SJakub Kicinskihttps://sourceforge.net/projects/e1000 768132db935SJakub Kicinski 769132db935SJakub KicinskiIf an issue is identified with the released source code on a supported kernel 770132db935SJakub Kicinskiwith a supported adapter, email the specific information related to the issue 771132db935SJakub Kicinskito e1000-devel@lists.sf.net. 772