1d2ba7ca3SMauro Carvalho Chehab.. SPDX-License-Identifier: GPL-2.0 2d2ba7ca3SMauro Carvalho Chehab 3d2ba7ca3SMauro Carvalho Chehab================================= 4d2ba7ca3SMauro Carvalho ChehabChelsio S3 iSCSI Driver for Linux 5d2ba7ca3SMauro Carvalho Chehab================================= 6d2ba7ca3SMauro Carvalho Chehab 7d2ba7ca3SMauro Carvalho ChehabIntroduction 8d2ba7ca3SMauro Carvalho Chehab============ 9d2ba7ca3SMauro Carvalho Chehab 10d2ba7ca3SMauro Carvalho ChehabThe Chelsio T3 ASIC based Adapters (S310, S320, S302, S304, Mezz cards, etc. 11d2ba7ca3SMauro Carvalho Chehabseries of products) support iSCSI acceleration and iSCSI Direct Data Placement 12d2ba7ca3SMauro Carvalho Chehab(DDP) where the hardware handles the expensive byte touching operations, such 13d2ba7ca3SMauro Carvalho Chehabas CRC computation and verification, and direct DMA to the final host memory 14d2ba7ca3SMauro Carvalho Chehabdestination: 15d2ba7ca3SMauro Carvalho Chehab 16d2ba7ca3SMauro Carvalho Chehab - iSCSI PDU digest generation and verification 17d2ba7ca3SMauro Carvalho Chehab 18d2ba7ca3SMauro Carvalho Chehab On transmitting, Chelsio S3 h/w computes and inserts the Header and 19d2ba7ca3SMauro Carvalho Chehab Data digest into the PDUs. 20d2ba7ca3SMauro Carvalho Chehab On receiving, Chelsio S3 h/w computes and verifies the Header and 21d2ba7ca3SMauro Carvalho Chehab Data digest of the PDUs. 22d2ba7ca3SMauro Carvalho Chehab 23d2ba7ca3SMauro Carvalho Chehab - Direct Data Placement (DDP) 24d2ba7ca3SMauro Carvalho Chehab 25d2ba7ca3SMauro Carvalho Chehab S3 h/w can directly place the iSCSI Data-In or Data-Out PDU's 26d2ba7ca3SMauro Carvalho Chehab payload into pre-posted final destination host-memory buffers based 27d2ba7ca3SMauro Carvalho Chehab on the Initiator Task Tag (ITT) in Data-In or Target Task Tag (TTT) 28d2ba7ca3SMauro Carvalho Chehab in Data-Out PDUs. 29d2ba7ca3SMauro Carvalho Chehab 30d2ba7ca3SMauro Carvalho Chehab - PDU Transmit and Recovery 31d2ba7ca3SMauro Carvalho Chehab 32d2ba7ca3SMauro Carvalho Chehab On transmitting, S3 h/w accepts the complete PDU (header + data) 33d2ba7ca3SMauro Carvalho Chehab from the host driver, computes and inserts the digests, decomposes 34d2ba7ca3SMauro Carvalho Chehab the PDU into multiple TCP segments if necessary, and transmit all 35d2ba7ca3SMauro Carvalho Chehab the TCP segments onto the wire. It handles TCP retransmission if 36d2ba7ca3SMauro Carvalho Chehab needed. 37d2ba7ca3SMauro Carvalho Chehab 38d2ba7ca3SMauro Carvalho Chehab On receiving, S3 h/w recovers the iSCSI PDU by reassembling TCP 39d2ba7ca3SMauro Carvalho Chehab segments, separating the header and data, calculating and verifying 40d2ba7ca3SMauro Carvalho Chehab the digests, then forwarding the header to the host. The payload data, 41d2ba7ca3SMauro Carvalho Chehab if possible, will be directly placed into the pre-posted host DDP 42d2ba7ca3SMauro Carvalho Chehab buffer. Otherwise, the payload data will be sent to the host too. 43d2ba7ca3SMauro Carvalho Chehab 44d2ba7ca3SMauro Carvalho ChehabThe cxgb3i driver interfaces with open-iscsi initiator and provides the iSCSI 45d2ba7ca3SMauro Carvalho Chehabacceleration through Chelsio hardware wherever applicable. 46d2ba7ca3SMauro Carvalho Chehab 47d2ba7ca3SMauro Carvalho ChehabUsing the cxgb3i Driver 48d2ba7ca3SMauro Carvalho Chehab======================= 49d2ba7ca3SMauro Carvalho Chehab 50d2ba7ca3SMauro Carvalho ChehabThe following steps need to be taken to accelerates the open-iscsi initiator: 51d2ba7ca3SMauro Carvalho Chehab 52d2ba7ca3SMauro Carvalho Chehab1. Load the cxgb3i driver: "modprobe cxgb3i" 53d2ba7ca3SMauro Carvalho Chehab 54d2ba7ca3SMauro Carvalho Chehab The cxgb3i module registers a new transport class "cxgb3i" with open-iscsi. 55d2ba7ca3SMauro Carvalho Chehab 56d2ba7ca3SMauro Carvalho Chehab * in the case of recompiling the kernel, the cxgb3i selection is located at:: 57d2ba7ca3SMauro Carvalho Chehab 58d2ba7ca3SMauro Carvalho Chehab Device Drivers 59d2ba7ca3SMauro Carvalho Chehab SCSI device support ---> 60d2ba7ca3SMauro Carvalho Chehab [*] SCSI low-level drivers ---> 61d2ba7ca3SMauro Carvalho Chehab <M> Chelsio S3xx iSCSI support 62d2ba7ca3SMauro Carvalho Chehab 63d2ba7ca3SMauro Carvalho Chehab2. Create an interface file located under /etc/iscsi/ifaces/ for the new 64d2ba7ca3SMauro Carvalho Chehab transport class "cxgb3i". 65d2ba7ca3SMauro Carvalho Chehab 66d2ba7ca3SMauro Carvalho Chehab The content of the file should be in the following format:: 67d2ba7ca3SMauro Carvalho Chehab 68d2ba7ca3SMauro Carvalho Chehab iface.transport_name = cxgb3i 69d2ba7ca3SMauro Carvalho Chehab iface.net_ifacename = <ethX> 70d2ba7ca3SMauro Carvalho Chehab iface.ipaddress = <iscsi ip address> 71d2ba7ca3SMauro Carvalho Chehab 72d2ba7ca3SMauro Carvalho Chehab * if iface.ipaddress is specified, <iscsi ip address> needs to be either the 73d2ba7ca3SMauro Carvalho Chehab same as the ethX's ip address or an address on the same subnet. Make 74d2ba7ca3SMauro Carvalho Chehab sure the ip address is unique in the network. 75d2ba7ca3SMauro Carvalho Chehab 76d2ba7ca3SMauro Carvalho Chehab3. edit /etc/iscsi/iscsid.conf 77d2ba7ca3SMauro Carvalho Chehab The default setting for MaxRecvDataSegmentLength (131072) is too big; 78d2ba7ca3SMauro Carvalho Chehab replace with a value no bigger than 15360 (for example 8192):: 79d2ba7ca3SMauro Carvalho Chehab 80d2ba7ca3SMauro Carvalho Chehab node.conn[0].iscsi.MaxRecvDataSegmentLength = 8192 81d2ba7ca3SMauro Carvalho Chehab 82d2ba7ca3SMauro Carvalho Chehab * The login would fail for a normal session if MaxRecvDataSegmentLength is 83d2ba7ca3SMauro Carvalho Chehab too big. A error message in the format of 84d2ba7ca3SMauro Carvalho Chehab "cxgb3i: ERR! MaxRecvSegmentLength <X> too big. Need to be <= <Y>." 85d2ba7ca3SMauro Carvalho Chehab would be logged to dmesg. 86d2ba7ca3SMauro Carvalho Chehab 87d2ba7ca3SMauro Carvalho Chehab4. To direct open-iscsi traffic to go through cxgb3i's accelerated path, 88d2ba7ca3SMauro Carvalho Chehab "-I <iface file name>" option needs to be specified with most of the 89d2ba7ca3SMauro Carvalho Chehab iscsiadm command. <iface file name> is the transport interface file created 90d2ba7ca3SMauro Carvalho Chehab in step 2. 91