1.. SPDX-License-Identifier: GPL-2.0
2
3============================================================
4Linux kernel driver for Elastic Network Adapter (ENA) family
5============================================================
6
7Overview
8========
9
10ENA is a networking interface designed to make good use of modern CPU
11features and system architectures.
12
13The ENA device exposes a lightweight management interface with a
14minimal set of memory mapped registers and extendable command set
15through an Admin Queue.
16
17The driver supports a range of ENA devices, is link-speed independent
18(i.e., the same driver is used for 10GbE, 25GbE, 40GbE, etc.), and has
19a negotiated and extendable feature set.
20
21Some ENA devices support SR-IOV. This driver is used for both the
22SR-IOV Physical Function (PF) and Virtual Function (VF) devices.
23
24ENA devices enable high speed and low overhead network traffic
25processing by providing multiple Tx/Rx queue pairs (the maximum number
26is advertised by the device via the Admin Queue), a dedicated MSI-X
27interrupt vector per Tx/Rx queue pair, adaptive interrupt moderation,
28and CPU cacheline optimized data placement.
29
30The ENA driver supports industry standard TCP/IP offload features such
31as checksum offload and TCP transmit segmentation offload (TSO).
32Receive-side scaling (RSS) is supported for multi-core scaling.
33
34The ENA driver and its corresponding devices implement health
35monitoring mechanisms such as watchdog, enabling the device and driver
36to recover in a manner transparent to the application, as well as
37debug logs.
38
39Some of the ENA devices support a working mode called Low-latency
40Queue (LLQ), which saves several more microseconds.
41
42ENA Source Code Directory Structure
43===================================
44
45=================   ======================================================
46ena_com.[ch]        Management communication layer. This layer is
47		    responsible for the handling all the management
48		    (admin) communication between the device and the
49		    driver.
50ena_eth_com.[ch]    Tx/Rx data path.
51ena_admin_defs.h    Definition of ENA management interface.
52ena_eth_io_defs.h   Definition of ENA data path interface.
53ena_common_defs.h   Common definitions for ena_com layer.
54ena_regs_defs.h     Definition of ENA PCI memory-mapped (MMIO) registers.
55ena_netdev.[ch]     Main Linux kernel driver.
56ena_syfsfs.[ch]     Sysfs files.
57ena_ethtool.c       ethtool callbacks.
58ena_pci_id_tbl.h    Supported device IDs.
59=================   ======================================================
60
61Management Interface:
62=====================
63
64ENA management interface is exposed by means of:
65
66- PCIe Configuration Space
67- Device Registers
68- Admin Queue (AQ) and Admin Completion Queue (ACQ)
69- Asynchronous Event Notification Queue (AENQ)
70
71ENA device MMIO Registers are accessed only during driver
72initialization and are not involved in further normal device
73operation.
74
75AQ is used for submitting management commands, and the
76results/responses are reported asynchronously through ACQ.
77
78ENA introduces a small set of management commands with room for
79vendor-specific extensions. Most of the management operations are
80framed in a generic Get/Set feature command.
81
82The following admin queue commands are supported:
83
84- Create I/O submission queue
85- Create I/O completion queue
86- Destroy I/O submission queue
87- Destroy I/O completion queue
88- Get feature
89- Set feature
90- Configure AENQ
91- Get statistics
92
93Refer to ena_admin_defs.h for the list of supported Get/Set Feature
94properties.
95
96The Asynchronous Event Notification Queue (AENQ) is a uni-directional
97queue used by the ENA device to send to the driver events that cannot
98be reported using ACQ. AENQ events are subdivided into groups. Each
99group may have multiple syndromes, as shown below
100
101The events are:
102
103	====================	===============
104	Group			Syndrome
105	====================	===============
106	Link state change	**X**
107	Fatal error		**X**
108	Notification		Suspend traffic
109	Notification		Resume traffic
110	Keep-Alive		**X**
111	====================	===============
112
113ACQ and AENQ share the same MSI-X vector.
114
115Keep-Alive is a special mechanism that allows monitoring of the
116device's health. The driver maintains a watchdog (WD) handler which,
117if fired, logs the current state and statistics then resets and
118restarts the ENA device and driver. A Keep-Alive event is delivered by
119the device every second. The driver re-arms the WD upon reception of a
120Keep-Alive event. A missed Keep-Alive event causes the WD handler to
121fire.
122
123Data Path Interface
124===================
125I/O operations are based on Tx and Rx Submission Queues (Tx SQ and Rx
126SQ correspondingly). Each SQ has a completion queue (CQ) associated
127with it.
128
129The SQs and CQs are implemented as descriptor rings in contiguous
130physical memory.
131
132The ENA driver supports two Queue Operation modes for Tx SQs:
133
134- Regular mode
135
136  * In this mode the Tx SQs reside in the host's memory. The ENA
137    device fetches the ENA Tx descriptors and packet data from host
138    memory.
139
140- Low Latency Queue (LLQ) mode or "push-mode".
141
142  * In this mode the driver pushes the transmit descriptors and the
143    first 128 bytes of the packet directly to the ENA device memory
144    space. The rest of the packet payload is fetched by the
145    device. For this operation mode, the driver uses a dedicated PCI
146    device memory BAR, which is mapped with write-combine capability.
147
148The Rx SQs support only the regular mode.
149
150Note: Not all ENA devices support LLQ, and this feature is negotiated
151      with the device upon initialization. If the ENA device does not
152      support LLQ mode, the driver falls back to the regular mode.
153
154The driver supports multi-queue for both Tx and Rx. This has various
155benefits:
156
157- Reduced CPU/thread/process contention on a given Ethernet interface.
158- Cache miss rate on completion is reduced, particularly for data
159  cache lines that hold the sk_buff structures.
160- Increased process-level parallelism when handling received packets.
161- Increased data cache hit rate, by steering kernel processing of
162  packets to the CPU, where the application thread consuming the
163  packet is running.
164- In hardware interrupt re-direction.
165
166Interrupt Modes
167===============
168The driver assigns a single MSI-X vector per queue pair (for both Tx
169and Rx directions). The driver assigns an additional dedicated MSI-X vector
170for management (for ACQ and AENQ).
171
172Management interrupt registration is performed when the Linux kernel
173probes the adapter, and it is de-registered when the adapter is
174removed. I/O queue interrupt registration is performed when the Linux
175interface of the adapter is opened, and it is de-registered when the
176interface is closed.
177
178The management interrupt is named::
179
180   ena-mgmnt@pci:<PCI domain:bus:slot.function>
181
182and for each queue pair, an interrupt is named::
183
184   <interface name>-Tx-Rx-<queue index>
185
186The ENA device operates in auto-mask and auto-clear interrupt
187modes. That is, once MSI-X is delivered to the host, its Cause bit is
188automatically cleared and the interrupt is masked. The interrupt is
189unmasked by the driver after NAPI processing is complete.
190
191Interrupt Moderation
192====================
193ENA driver and device can operate in conventional or adaptive interrupt
194moderation mode.
195
196In conventional mode the driver instructs device to postpone interrupt
197posting according to static interrupt delay value. The interrupt delay
198value can be configured through ethtool(8). The following ethtool
199parameters are supported by the driver: tx-usecs, rx-usecs
200
201In adaptive interrupt moderation mode the interrupt delay value is
202updated by the driver dynamically and adjusted every NAPI cycle
203according to the traffic nature.
204
205Adaptive coalescing can be switched on/off through ethtool(8)
206adaptive_rx on|off parameter.
207
208More information about Adaptive Interrupt Moderation (DIM) can be found in
209Documentation/networking/net_dim.rst
210
211RX copybreak
212============
213The rx_copybreak is initialized by default to ENA_DEFAULT_RX_COPYBREAK
214and can be configured by the ETHTOOL_STUNABLE command of the
215SIOCETHTOOL ioctl.
216
217SKB
218===
219The driver-allocated SKB for frames received from Rx handling using
220NAPI context. The allocation method depends on the size of the packet.
221If the frame length is larger than rx_copybreak, napi_get_frags()
222is used, otherwise netdev_alloc_skb_ip_align() is used, the buffer
223content is copied (by CPU) to the SKB, and the buffer is recycled.
224
225Statistics
226==========
227The user can obtain ENA device and driver statistics using ethtool.
228The driver can collect regular or extended statistics (including
229per-queue stats) from the device.
230
231In addition the driver logs the stats to syslog upon device reset.
232
233MTU
234===
235The driver supports an arbitrarily large MTU with a maximum that is
236negotiated with the device. The driver configures MTU using the
237SetFeature command (ENA_ADMIN_MTU property). The user can change MTU
238via ip(8) and similar legacy tools.
239
240Stateless Offloads
241==================
242The ENA driver supports:
243
244- TSO over IPv4/IPv6
245- TSO with ECN
246- IPv4 header checksum offload
247- TCP/UDP over IPv4/IPv6 checksum offloads
248
249RSS
250===
251- The ENA device supports RSS that allows flexible Rx traffic
252  steering.
253- Toeplitz and CRC32 hash functions are supported.
254- Different combinations of L2/L3/L4 fields can be configured as
255  inputs for hash functions.
256- The driver configures RSS settings using the AQ SetFeature command
257  (ENA_ADMIN_RSS_HASH_FUNCTION, ENA_ADMIN_RSS_HASH_INPUT and
258  ENA_ADMIN_RSS_INDIRECTION_TABLE_CONFIG properties).
259- If the NETIF_F_RXHASH flag is set, the 32-bit result of the hash
260  function delivered in the Rx CQ descriptor is set in the received
261  SKB.
262- The user can provide a hash key, hash function, and configure the
263  indirection table through ethtool(8).
264
265DATA PATH
266=========
267Tx
268--
269
270ena_start_xmit() is called by the stack. This function does the following:
271
272- Maps data buffers (skb->data and frags).
273- Populates ena_buf for the push buffer (if the driver and device are
274  in push mode.)
275- Prepares ENA bufs for the remaining frags.
276- Allocates a new request ID from the empty req_id ring. The request
277  ID is the index of the packet in the Tx info. This is used for
278  out-of-order TX completions.
279- Adds the packet to the proper place in the Tx ring.
280- Calls ena_com_prepare_tx(), an ENA communication layer that converts
281  the ena_bufs to ENA descriptors (and adds meta ENA descriptors as
282  needed.)
283
284  * This function also copies the ENA descriptors and the push buffer
285    to the Device memory space (if in push mode.)
286
287- Writes doorbell to the ENA device.
288- When the ENA device finishes sending the packet, a completion
289  interrupt is raised.
290- The interrupt handler schedules NAPI.
291- The ena_clean_tx_irq() function is called. This function handles the
292  completion descriptors generated by the ENA, with a single
293  completion descriptor per completed packet.
294
295  * req_id is retrieved from the completion descriptor. The tx_info of
296    the packet is retrieved via the req_id. The data buffers are
297    unmapped and req_id is returned to the empty req_id ring.
298  * The function stops when the completion descriptors are completed or
299    the budget is reached.
300
301Rx
302--
303
304- When a packet is received from the ENA device.
305- The interrupt handler schedules NAPI.
306- The ena_clean_rx_irq() function is called. This function calls
307  ena_rx_pkt(), an ENA communication layer function, which returns the
308  number of descriptors used for a new unhandled packet, and zero if
309  no new packet is found.
310- Then it calls the ena_clean_rx_irq() function.
311- ena_eth_rx_skb() checks packet length:
312
313  * If the packet is small (len < rx_copybreak), the driver allocates
314    a SKB for the new packet, and copies the packet payload into the
315    SKB data buffer.
316
317    - In this way the original data buffer is not passed to the stack
318      and is reused for future Rx packets.
319
320  * Otherwise the function unmaps the Rx buffer, then allocates the
321    new SKB structure and hooks the Rx buffer to the SKB frags.
322
323- The new SKB is updated with the necessary information (protocol,
324  checksum hw verify result, etc.), and then passed to the network
325  stack, using the NAPI interface function napi_gro_receive().
326