1.. SPDX-License-Identifier: GPL-2.0
2
3============================================================
4Linux kernel driver for Elastic Network Adapter (ENA) family
5============================================================
6
7Overview
8========
9
10ENA is a networking interface designed to make good use of modern CPU
11features and system architectures.
12
13The ENA device exposes a lightweight management interface with a
14minimal set of memory mapped registers and extendible command set
15through an Admin Queue.
16
17The driver supports a range of ENA devices, is link-speed independent
18(i.e., the same driver is used for 10GbE, 25GbE, 40GbE, etc), and has
19a negotiated and extendible feature set.
20
21Some ENA devices support SR-IOV. This driver is used for both the
22SR-IOV Physical Function (PF) and Virtual Function (VF) devices.
23
24ENA devices enable high speed and low overhead network traffic
25processing by providing multiple Tx/Rx queue pairs (the maximum number
26is advertised by the device via the Admin Queue), a dedicated MSI-X
27interrupt vector per Tx/Rx queue pair, adaptive interrupt moderation,
28and CPU cacheline optimized data placement.
29
30The ENA driver supports industry standard TCP/IP offload features such as
31checksum offload. Receive-side scaling (RSS) is supported for multi-core
32scaling.
33
34The ENA driver and its corresponding devices implement health
35monitoring mechanisms such as watchdog, enabling the device and driver
36to recover in a manner transparent to the application, as well as
37debug logs.
38
39Some of the ENA devices support a working mode called Low-latency
40Queue (LLQ), which saves several more microseconds.
41ENA Source Code Directory Structure
42===================================
43
44=================   ======================================================
45ena_com.[ch]        Management communication layer. This layer is
46                    responsible for the handling all the management
47                    (admin) communication between the device and the
48                    driver.
49ena_eth_com.[ch]    Tx/Rx data path.
50ena_admin_defs.h    Definition of ENA management interface.
51ena_eth_io_defs.h   Definition of ENA data path interface.
52ena_common_defs.h   Common definitions for ena_com layer.
53ena_regs_defs.h     Definition of ENA PCI memory-mapped (MMIO) registers.
54ena_netdev.[ch]     Main Linux kernel driver.
55ena_ethtool.c       ethtool callbacks.
56ena_pci_id_tbl.h    Supported device IDs.
57=================   ======================================================
58
59Management Interface:
60=====================
61
62ENA management interface is exposed by means of:
63
64- PCIe Configuration Space
65- Device Registers
66- Admin Queue (AQ) and Admin Completion Queue (ACQ)
67- Asynchronous Event Notification Queue (AENQ)
68
69ENA device MMIO Registers are accessed only during driver
70initialization and are not used during further normal device
71operation.
72
73AQ is used for submitting management commands, and the
74results/responses are reported asynchronously through ACQ.
75
76ENA introduces a small set of management commands with room for
77vendor-specific extensions. Most of the management operations are
78framed in a generic Get/Set feature command.
79
80The following admin queue commands are supported:
81
82- Create I/O submission queue
83- Create I/O completion queue
84- Destroy I/O submission queue
85- Destroy I/O completion queue
86- Get feature
87- Set feature
88- Configure AENQ
89- Get statistics
90
91Refer to ena_admin_defs.h for the list of supported Get/Set Feature
92properties.
93
94The Asynchronous Event Notification Queue (AENQ) is a uni-directional
95queue used by the ENA device to send to the driver events that cannot
96be reported using ACQ. AENQ events are subdivided into groups. Each
97group may have multiple syndromes, as shown below
98
99The events are:
100
101====================    ===============
102Group                   Syndrome
103====================    ===============
104Link state change       **X**
105Fatal error             **X**
106Notification            Suspend traffic
107Notification            Resume traffic
108Keep-Alive              **X**
109====================    ===============
110
111ACQ and AENQ share the same MSI-X vector.
112
113Keep-Alive is a special mechanism that allows monitoring the device's health.
114A Keep-Alive event is delivered by the device every second.
115The driver maintains a watchdog (WD) handler which logs the current state and
116statistics. If the keep-alive events aren't delivered as expected the WD resets
117the device and the driver.
118
119Data Path Interface
120===================
121
122I/O operations are based on Tx and Rx Submission Queues (Tx SQ and Rx
123SQ correspondingly). Each SQ has a completion queue (CQ) associated
124with it.
125
126The SQs and CQs are implemented as descriptor rings in contiguous
127physical memory.
128
129The ENA driver supports two Queue Operation modes for Tx SQs:
130
131- **Regular mode:**
132  In this mode the Tx SQs reside in the host's memory. The ENA
133  device fetches the ENA Tx descriptors and packet data from host
134  memory.
135
136- **Low Latency Queue (LLQ) mode or "push-mode":**
137  In this mode the driver pushes the transmit descriptors and the
138  first 128 bytes of the packet directly to the ENA device memory
139  space. The rest of the packet payload is fetched by the
140  device. For this operation mode, the driver uses a dedicated PCI
141  device memory BAR, which is mapped with write-combine capability.
142
143  **Note that** not all ENA devices support LLQ, and this feature is negotiated
144  with the device upon initialization. If the ENA device does not
145  support LLQ mode, the driver falls back to the regular mode.
146
147The Rx SQs support only the regular mode.
148
149The driver supports multi-queue for both Tx and Rx. This has various
150benefits:
151
152- Reduced CPU/thread/process contention on a given Ethernet interface.
153- Cache miss rate on completion is reduced, particularly for data
154  cache lines that hold the sk_buff structures.
155- Increased process-level parallelism when handling received packets.
156- Increased data cache hit rate, by steering kernel processing of
157  packets to the CPU, where the application thread consuming the
158  packet is running.
159- In hardware interrupt re-direction.
160
161Interrupt Modes
162===============
163
164The driver assigns a single MSI-X vector per queue pair (for both Tx
165and Rx directions). The driver assigns an additional dedicated MSI-X vector
166for management (for ACQ and AENQ).
167
168Management interrupt registration is performed when the Linux kernel
169probes the adapter, and it is de-registered when the adapter is
170removed. I/O queue interrupt registration is performed when the Linux
171interface of the adapter is opened, and it is de-registered when the
172interface is closed.
173
174The management interrupt is named::
175
176   ena-mgmnt@pci:<PCI domain:bus:slot.function>
177
178and for each queue pair, an interrupt is named::
179
180   <interface name>-Tx-Rx-<queue index>
181
182The ENA device operates in auto-mask and auto-clear interrupt
183modes. That is, once MSI-X is delivered to the host, its Cause bit is
184automatically cleared and the interrupt is masked. The interrupt is
185unmasked by the driver after NAPI processing is complete.
186
187Interrupt Moderation
188====================
189
190ENA driver and device can operate in conventional or adaptive interrupt
191moderation mode.
192
193**In conventional mode** the driver instructs device to postpone interrupt
194posting according to static interrupt delay value. The interrupt delay
195value can be configured through `ethtool(8)`. The following `ethtool`
196parameters are supported by the driver: ``tx-usecs``, ``rx-usecs``
197
198**In adaptive interrupt** moderation mode the interrupt delay value is
199updated by the driver dynamically and adjusted every NAPI cycle
200according to the traffic nature.
201
202Adaptive coalescing can be switched on/off through `ethtool(8)`'s
203:code:`adaptive_rx on|off` parameter.
204
205More information about Adaptive Interrupt Moderation (DIM) can be found in
206Documentation/networking/net_dim.rst
207
208RX copybreak
209============
210The rx_copybreak is initialized by default to ENA_DEFAULT_RX_COPYBREAK
211and can be configured by the ETHTOOL_STUNABLE command of the
212SIOCETHTOOL ioctl.
213
214Statistics
215==========
216
217The user can obtain ENA device and driver statistics using `ethtool`.
218The driver can collect regular or extended statistics (including
219per-queue stats) from the device.
220
221In addition the driver logs the stats to syslog upon device reset.
222
223MTU
224===
225
226The driver supports an arbitrarily large MTU with a maximum that is
227negotiated with the device. The driver configures MTU using the
228SetFeature command (ENA_ADMIN_MTU property). The user can change MTU
229via `ip(8)` and similar legacy tools.
230
231Stateless Offloads
232==================
233
234The ENA driver supports:
235
236- IPv4 header checksum offload
237- TCP/UDP over IPv4/IPv6 checksum offloads
238
239RSS
240===
241
242- The ENA device supports RSS that allows flexible Rx traffic
243  steering.
244- Toeplitz and CRC32 hash functions are supported.
245- Different combinations of L2/L3/L4 fields can be configured as
246  inputs for hash functions.
247- The driver configures RSS settings using the AQ SetFeature command
248  (ENA_ADMIN_RSS_HASH_FUNCTION, ENA_ADMIN_RSS_HASH_INPUT and
249  ENA_ADMIN_RSS_INDIRECTION_TABLE_CONFIG properties).
250- If the NETIF_F_RXHASH flag is set, the 32-bit result of the hash
251  function delivered in the Rx CQ descriptor is set in the received
252  SKB.
253- The user can provide a hash key, hash function, and configure the
254  indirection table through `ethtool(8)`.
255
256DATA PATH
257=========
258
259Tx
260--
261
262:code:`ena_start_xmit()` is called by the stack. This function does the following:
263
264- Maps data buffers (``skb->data`` and frags).
265- Populates ``ena_buf`` for the push buffer (if the driver and device are
266  in push mode).
267- Prepares ENA bufs for the remaining frags.
268- Allocates a new request ID from the empty ``req_id`` ring. The request
269  ID is the index of the packet in the Tx info. This is used for
270  out-of-order Tx completions.
271- Adds the packet to the proper place in the Tx ring.
272- Calls :code:`ena_com_prepare_tx()`, an ENA communication layer that converts
273  the ``ena_bufs`` to ENA descriptors (and adds meta ENA descriptors as
274  needed).
275
276  * This function also copies the ENA descriptors and the push buffer
277    to the Device memory space (if in push mode).
278
279- Writes a doorbell to the ENA device.
280- When the ENA device finishes sending the packet, a completion
281  interrupt is raised.
282- The interrupt handler schedules NAPI.
283- The :code:`ena_clean_tx_irq()` function is called. This function handles the
284  completion descriptors generated by the ENA, with a single
285  completion descriptor per completed packet.
286
287  * ``req_id`` is retrieved from the completion descriptor. The ``tx_info`` of
288    the packet is retrieved via the ``req_id``. The data buffers are
289    unmapped and ``req_id`` is returned to the empty ``req_id`` ring.
290  * The function stops when the completion descriptors are completed or
291    the budget is reached.
292
293Rx
294--
295
296- When a packet is received from the ENA device.
297- The interrupt handler schedules NAPI.
298- The :code:`ena_clean_rx_irq()` function is called. This function calls
299  :code:`ena_com_rx_pkt()`, an ENA communication layer function, which returns the
300  number of descriptors used for a new packet, and zero if
301  no new packet is found.
302- :code:`ena_rx_skb()` checks packet length:
303
304  * If the packet is small (len < rx_copybreak), the driver allocates
305    a SKB for the new packet, and copies the packet payload into the
306    SKB data buffer.
307
308    - In this way the original data buffer is not passed to the stack
309      and is reused for future Rx packets.
310
311  * Otherwise the function unmaps the Rx buffer, sets the first
312    descriptor as `skb`'s linear part and the other descriptors as the
313    `skb`'s frags.
314
315- The new SKB is updated with the necessary information (protocol,
316  checksum hw verify result, etc), and then passed to the network
317  stack, using the NAPI interface function :code:`napi_gro_receive()`.
318