1*132db935SJakub Kicinski.. SPDX-License-Identifier: GPL-2.0 2*132db935SJakub Kicinski 3*132db935SJakub Kicinski========================================================= 4*132db935SJakub KicinskiNeterion's (Formerly S2io) Xframe I/II PCI-X 10GbE driver 5*132db935SJakub Kicinski========================================================= 6*132db935SJakub Kicinski 7*132db935SJakub KicinskiRelease notes for Neterion's (Formerly S2io) Xframe I/II PCI-X 10GbE driver. 8*132db935SJakub Kicinski 9*132db935SJakub Kicinski.. Contents 10*132db935SJakub Kicinski - 1. Introduction 11*132db935SJakub Kicinski - 2. Identifying the adapter/interface 12*132db935SJakub Kicinski - 3. Features supported 13*132db935SJakub Kicinski - 4. Command line parameters 14*132db935SJakub Kicinski - 5. Performance suggestions 15*132db935SJakub Kicinski - 6. Available Downloads 16*132db935SJakub Kicinski 17*132db935SJakub Kicinski 18*132db935SJakub Kicinski1. Introduction 19*132db935SJakub Kicinski=============== 20*132db935SJakub KicinskiThis Linux driver supports Neterion's Xframe I PCI-X 1.0 and 21*132db935SJakub KicinskiXframe II PCI-X 2.0 adapters. It supports several features 22*132db935SJakub Kicinskisuch as jumbo frames, MSI/MSI-X, checksum offloads, TSO, UFO and so on. 23*132db935SJakub KicinskiSee below for complete list of features. 24*132db935SJakub Kicinski 25*132db935SJakub KicinskiAll features are supported for both IPv4 and IPv6. 26*132db935SJakub Kicinski 27*132db935SJakub Kicinski2. Identifying the adapter/interface 28*132db935SJakub Kicinski==================================== 29*132db935SJakub Kicinski 30*132db935SJakub Kicinskia. Insert the adapter(s) in your system. 31*132db935SJakub Kicinskib. Build and load driver:: 32*132db935SJakub Kicinski 33*132db935SJakub Kicinski # insmod s2io.ko 34*132db935SJakub Kicinski 35*132db935SJakub Kicinskic. View log messages:: 36*132db935SJakub Kicinski 37*132db935SJakub Kicinski # dmesg | tail -40 38*132db935SJakub Kicinski 39*132db935SJakub KicinskiYou will see messages similar to:: 40*132db935SJakub Kicinski 41*132db935SJakub Kicinski eth3: Neterion Xframe I 10GbE adapter (rev 3), Version 2.0.9.1, Intr type INTA 42*132db935SJakub Kicinski eth4: Neterion Xframe II 10GbE adapter (rev 2), Version 2.0.9.1, Intr type INTA 43*132db935SJakub Kicinski eth4: Device is on 64 bit 133MHz PCIX(M1) bus 44*132db935SJakub Kicinski 45*132db935SJakub KicinskiThe above messages identify the adapter type(Xframe I/II), adapter revision, 46*132db935SJakub Kicinskidriver version, interface name(eth3, eth4), Interrupt type(INTA, MSI, MSI-X). 47*132db935SJakub KicinskiIn case of Xframe II, the PCI/PCI-X bus width and frequency are displayed 48*132db935SJakub Kicinskias well. 49*132db935SJakub Kicinski 50*132db935SJakub KicinskiTo associate an interface with a physical adapter use "ethtool -p <ethX>". 51*132db935SJakub KicinskiThe corresponding adapter's LED will blink multiple times. 52*132db935SJakub Kicinski 53*132db935SJakub Kicinski3. Features supported 54*132db935SJakub Kicinski===================== 55*132db935SJakub Kicinskia. Jumbo frames. Xframe I/II supports MTU up to 9600 bytes, 56*132db935SJakub Kicinski modifiable using ip command. 57*132db935SJakub Kicinski 58*132db935SJakub Kicinskib. Offloads. Supports checksum offload(TCP/UDP/IP) on transmit 59*132db935SJakub Kicinski and receive, TSO. 60*132db935SJakub Kicinski 61*132db935SJakub Kicinskic. Multi-buffer receive mode. Scattering of packet across multiple 62*132db935SJakub Kicinski buffers. Currently driver supports 2-buffer mode which yields 63*132db935SJakub Kicinski significant performance improvement on certain platforms(SGI Altix, 64*132db935SJakub Kicinski IBM xSeries). 65*132db935SJakub Kicinski 66*132db935SJakub Kicinskid. MSI/MSI-X. Can be enabled on platforms which support this feature 67*132db935SJakub Kicinski (IA64, Xeon) resulting in noticeable performance improvement(up to 7% 68*132db935SJakub Kicinski on certain platforms). 69*132db935SJakub Kicinski 70*132db935SJakub Kicinskie. Statistics. Comprehensive MAC-level and software statistics displayed 71*132db935SJakub Kicinski using "ethtool -S" option. 72*132db935SJakub Kicinski 73*132db935SJakub Kicinskif. Multi-FIFO/Ring. Supports up to 8 transmit queues and receive rings, 74*132db935SJakub Kicinski with multiple steering options. 75*132db935SJakub Kicinski 76*132db935SJakub Kicinski4. Command line parameters 77*132db935SJakub Kicinski========================== 78*132db935SJakub Kicinski 79*132db935SJakub Kicinskia. tx_fifo_num 80*132db935SJakub Kicinski Number of transmit queues 81*132db935SJakub Kicinski 82*132db935SJakub KicinskiValid range: 1-8 83*132db935SJakub Kicinski 84*132db935SJakub KicinskiDefault: 1 85*132db935SJakub Kicinski 86*132db935SJakub Kicinskib. rx_ring_num 87*132db935SJakub Kicinski Number of receive rings 88*132db935SJakub Kicinski 89*132db935SJakub KicinskiValid range: 1-8 90*132db935SJakub Kicinski 91*132db935SJakub KicinskiDefault: 1 92*132db935SJakub Kicinski 93*132db935SJakub Kicinskic. tx_fifo_len 94*132db935SJakub Kicinski Size of each transmit queue 95*132db935SJakub Kicinski 96*132db935SJakub KicinskiValid range: Total length of all queues should not exceed 8192 97*132db935SJakub Kicinski 98*132db935SJakub KicinskiDefault: 4096 99*132db935SJakub Kicinski 100*132db935SJakub Kicinskid. rx_ring_sz 101*132db935SJakub Kicinski Size of each receive ring(in 4K blocks) 102*132db935SJakub Kicinski 103*132db935SJakub KicinskiValid range: Limited by memory on system 104*132db935SJakub Kicinski 105*132db935SJakub KicinskiDefault: 30 106*132db935SJakub Kicinski 107*132db935SJakub Kicinskie. intr_type 108*132db935SJakub Kicinski Specifies interrupt type. Possible values 0(INTA), 2(MSI-X) 109*132db935SJakub Kicinski 110*132db935SJakub KicinskiValid values: 0, 2 111*132db935SJakub Kicinski 112*132db935SJakub KicinskiDefault: 2 113*132db935SJakub Kicinski 114*132db935SJakub Kicinski5. Performance suggestions 115*132db935SJakub Kicinski========================== 116*132db935SJakub Kicinski 117*132db935SJakub KicinskiGeneral: 118*132db935SJakub Kicinski 119*132db935SJakub Kicinskia. Set MTU to maximum(9000 for switch setup, 9600 in back-to-back configuration) 120*132db935SJakub Kicinskib. Set TCP windows size to optimal value. 121*132db935SJakub Kicinski 122*132db935SJakub KicinskiFor instance, for MTU=1500 a value of 210K has been observed to result in 123*132db935SJakub Kicinskigood performance:: 124*132db935SJakub Kicinski 125*132db935SJakub Kicinski # sysctl -w net.ipv4.tcp_rmem="210000 210000 210000" 126*132db935SJakub Kicinski # sysctl -w net.ipv4.tcp_wmem="210000 210000 210000" 127*132db935SJakub Kicinski 128*132db935SJakub KicinskiFor MTU=9000, TCP window size of 10 MB is recommended:: 129*132db935SJakub Kicinski 130*132db935SJakub Kicinski # sysctl -w net.ipv4.tcp_rmem="10000000 10000000 10000000" 131*132db935SJakub Kicinski # sysctl -w net.ipv4.tcp_wmem="10000000 10000000 10000000" 132*132db935SJakub Kicinski 133*132db935SJakub KicinskiTransmit performance: 134*132db935SJakub Kicinski 135*132db935SJakub Kicinskia. By default, the driver respects BIOS settings for PCI bus parameters. 136*132db935SJakub Kicinski However, you may want to experiment with PCI bus parameters 137*132db935SJakub Kicinski max-split-transactions(MOST) and MMRBC (use setpci command). 138*132db935SJakub Kicinski 139*132db935SJakub Kicinski A MOST value of 2 has been found optimal for Opterons and 3 for Itanium. 140*132db935SJakub Kicinski 141*132db935SJakub Kicinski It could be different for your hardware. 142*132db935SJakub Kicinski 143*132db935SJakub Kicinski Set MMRBC to 4K**. 144*132db935SJakub Kicinski 145*132db935SJakub Kicinski For example you can set 146*132db935SJakub Kicinski 147*132db935SJakub Kicinski For opteron:: 148*132db935SJakub Kicinski 149*132db935SJakub Kicinski #setpci -d 17d5:* 62=1d 150*132db935SJakub Kicinski 151*132db935SJakub Kicinski For Itanium:: 152*132db935SJakub Kicinski 153*132db935SJakub Kicinski #setpci -d 17d5:* 62=3d 154*132db935SJakub Kicinski 155*132db935SJakub Kicinski For detailed description of the PCI registers, please see Xframe User Guide. 156*132db935SJakub Kicinski 157*132db935SJakub Kicinskib. Ensure Transmit Checksum offload is enabled. Use ethtool to set/verify this 158*132db935SJakub Kicinski parameter. 159*132db935SJakub Kicinski 160*132db935SJakub Kicinskic. Turn on TSO(using "ethtool -K"):: 161*132db935SJakub Kicinski 162*132db935SJakub Kicinski # ethtool -K <ethX> tso on 163*132db935SJakub Kicinski 164*132db935SJakub KicinskiReceive performance: 165*132db935SJakub Kicinski 166*132db935SJakub Kicinskia. By default, the driver respects BIOS settings for PCI bus parameters. 167*132db935SJakub Kicinski However, you may want to set PCI latency timer to 248:: 168*132db935SJakub Kicinski 169*132db935SJakub Kicinski #setpci -d 17d5:* LATENCY_TIMER=f8 170*132db935SJakub Kicinski 171*132db935SJakub Kicinski For detailed description of the PCI registers, please see Xframe User Guide. 172*132db935SJakub Kicinski 173*132db935SJakub Kicinskib. Use 2-buffer mode. This results in large performance boost on 174*132db935SJakub Kicinski certain platforms(eg. SGI Altix, IBM xSeries). 175*132db935SJakub Kicinski 176*132db935SJakub Kicinskic. Ensure Receive Checksum offload is enabled. Use "ethtool -K ethX" command to 177*132db935SJakub Kicinski set/verify this option. 178*132db935SJakub Kicinski 179*132db935SJakub Kicinskid. Enable NAPI feature(in kernel configuration Device Drivers ---> Network 180*132db935SJakub Kicinski device support ---> Ethernet (10000 Mbit) ---> S2IO 10Gbe Xframe NIC) to 181*132db935SJakub Kicinski bring down CPU utilization. 182*132db935SJakub Kicinski 183*132db935SJakub Kicinski.. note:: 184*132db935SJakub Kicinski 185*132db935SJakub Kicinski For AMD opteron platforms with 8131 chipset, MMRBC=1 and MOST=1 are 186*132db935SJakub Kicinski recommended as safe parameters. 187*132db935SJakub Kicinski 188*132db935SJakub KicinskiFor more information, please review the AMD8131 errata at 189*132db935SJakub Kicinskihttp://vip.amd.com/us-en/assets/content_type/white_papers_and_tech_docs/ 190*132db935SJakub Kicinski26310_AMD-8131_HyperTransport_PCI-X_Tunnel_Revision_Guide_rev_3_18.pdf 191*132db935SJakub Kicinski 192*132db935SJakub Kicinski6. Support 193*132db935SJakub Kicinski========== 194*132db935SJakub Kicinski 195*132db935SJakub KicinskiFor further support please contact either your 10GbE Xframe NIC vendor (IBM, 196*132db935SJakub KicinskiHP, SGI etc.) 197