1.. SPDX-License-Identifier: GPL-2.0
2
3====================
4Interface statistics
5====================
6
7Overview
8========
9
10This document is a guide to Linux network interface statistics.
11
12There are three main sources of interface statistics in Linux:
13
14 - standard interface statistics based on
15   :c:type:`struct rtnl_link_stats64 <rtnl_link_stats64>`;
16 - protocol-specific statistics; and
17 - driver-defined statistics available via ethtool.
18
19Standard interface statistics
20-----------------------------
21
22There are multiple interfaces to reach the standard statistics.
23Most commonly used is the `ip` command from `iproute2`::
24
25  $ ip -s -s link show dev ens4u1u1
26  6: ens4u1u1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP mode DEFAULT group default qlen 1000
27    link/ether 48:2a:e3:4c:b1:d1 brd ff:ff:ff:ff:ff:ff
28    RX: bytes  packets  errors  dropped overrun mcast
29    74327665117 69016965 0       0       0       0
30    RX errors: length   crc     frame   fifo    missed
31               0        0       0       0       0
32    TX: bytes  packets  errors  dropped carrier collsns
33    21405556176 44608960 0       0       0       0
34    TX errors: aborted  fifo   window heartbeat transns
35               0        0       0       0       128
36    altname enp58s0u1u1
37
38Note that `-s` has been specified twice to see all members of
39:c:type:`struct rtnl_link_stats64 <rtnl_link_stats64>`.
40If `-s` is specified once the detailed errors won't be shown.
41
42`ip` supports JSON formatting via the `-j` option.
43
44Protocol-specific statistics
45----------------------------
46
47Protocol-specific statistics are exposed via relevant interfaces,
48the same interfaces as are used to configure them.
49
50ethtool
51~~~~~~~
52
53Ethtool exposes common low-level statistics.
54All the standard statistics are expected to be maintained
55by the device, not the driver (as opposed to driver-defined stats
56described in the next section which mix software and hardware stats).
57For devices which contain unmanaged
58switches (e.g. legacy SR-IOV or multi-host NICs) the events counted
59may not pertain exclusively to the packets destined to
60the local host interface. In other words the events may
61be counted at the network port (MAC/PHY blocks) without separation
62for different host side (PCIe) devices. Such ambiguity must not
63be present when internal switch is managed by Linux (so called
64switchdev mode for NICs).
65
66Standard ethtool statistics can be accessed via the interfaces used
67for configuration. For example ethtool interface used
68to configure pause frames can report corresponding hardware counters::
69
70  $ ethtool --include-statistics -a eth0
71  Pause parameters for eth0:
72  Autonegotiate:	on
73  RX:			on
74  TX:			on
75  Statistics:
76    tx_pause_frames: 1
77    rx_pause_frames: 1
78
79General Ethernet statistics not associated with any particular
80functionality are exposed via ``ethtool -S $ifc`` by specifying
81the ``--groups`` parameter::
82
83  $ ethtool -S eth0 --groups eth-phy eth-mac eth-ctrl rmon
84  Stats for eth0:
85  eth-phy-SymbolErrorDuringCarrier: 0
86  eth-mac-FramesTransmittedOK: 1
87  eth-mac-FrameTooLongErrors: 1
88  eth-ctrl-MACControlFramesTransmitted: 1
89  eth-ctrl-MACControlFramesReceived: 0
90  eth-ctrl-UnsupportedOpcodesReceived: 1
91  rmon-etherStatsUndersizePkts: 1
92  rmon-etherStatsJabbers: 0
93  rmon-rx-etherStatsPkts64Octets: 1
94  rmon-rx-etherStatsPkts65to127Octets: 0
95  rmon-rx-etherStatsPkts128to255Octets: 0
96  rmon-tx-etherStatsPkts64Octets: 2
97  rmon-tx-etherStatsPkts65to127Octets: 3
98  rmon-tx-etherStatsPkts128to255Octets: 0
99
100Driver-defined statistics
101-------------------------
102
103Driver-defined ethtool statistics can be dumped using `ethtool -S $ifc`, e.g.::
104
105  $ ethtool -S ens4u1u1
106  NIC statistics:
107     tx_single_collisions: 0
108     tx_multi_collisions: 0
109
110uAPIs
111=====
112
113procfs
114------
115
116The historical `/proc/net/dev` text interface gives access to the list
117of interfaces as well as their statistics.
118
119Note that even though this interface is using
120:c:type:`struct rtnl_link_stats64 <rtnl_link_stats64>`
121internally it combines some of the fields.
122
123sysfs
124-----
125
126Each device directory in sysfs contains a `statistics` directory (e.g.
127`/sys/class/net/lo/statistics/`) with files corresponding to
128members of :c:type:`struct rtnl_link_stats64 <rtnl_link_stats64>`.
129
130This simple interface is convenient especially in constrained/embedded
131environments without access to tools. However, it's inefficient when
132reading multiple stats as it internally performs a full dump of
133:c:type:`struct rtnl_link_stats64 <rtnl_link_stats64>`
134and reports only the stat corresponding to the accessed file.
135
136Sysfs files are documented in
137`Documentation/ABI/testing/sysfs-class-net-statistics`.
138
139
140netlink
141-------
142
143`rtnetlink` (`NETLINK_ROUTE`) is the preferred method of accessing
144:c:type:`struct rtnl_link_stats64 <rtnl_link_stats64>` stats.
145
146Statistics are reported both in the responses to link information
147requests (`RTM_GETLINK`) and statistic requests (`RTM_GETSTATS`,
148when `IFLA_STATS_LINK_64` bit is set in the `.filter_mask` of the request).
149
150ethtool
151-------
152
153Ethtool IOCTL interface allows drivers to report implementation
154specific statistics. Historically it has also been used to report
155statistics for which other APIs did not exist, like per-device-queue
156statistics, or standard-based statistics (e.g. RFC 2863).
157
158Statistics and their string identifiers are retrieved separately.
159Identifiers via `ETHTOOL_GSTRINGS` with `string_set` set to `ETH_SS_STATS`,
160and values via `ETHTOOL_GSTATS`. User space should use `ETHTOOL_GDRVINFO`
161to retrieve the number of statistics (`.n_stats`).
162
163ethtool-netlink
164---------------
165
166Ethtool netlink is a replacement for the older IOCTL interface.
167
168Protocol-related statistics can be requested in get commands by setting
169the `ETHTOOL_FLAG_STATS` flag in `ETHTOOL_A_HEADER_FLAGS`. Currently
170statistics are supported in the following commands:
171
172  - `ETHTOOL_MSG_PAUSE_GET`
173  - `ETHTOOL_MSG_FEC_GET`
174
175debugfs
176-------
177
178Some drivers expose extra statistics via `debugfs`.
179
180struct rtnl_link_stats64
181========================
182
183.. kernel-doc:: include/uapi/linux/if_link.h
184    :identifiers: rtnl_link_stats64
185
186Notes for driver authors
187========================
188
189Drivers should report all statistics which have a matching member in
190:c:type:`struct rtnl_link_stats64 <rtnl_link_stats64>` exclusively
191via `.ndo_get_stats64`. Reporting such standard stats via ethtool
192or debugfs will not be accepted.
193
194Drivers must ensure best possible compliance with
195:c:type:`struct rtnl_link_stats64 <rtnl_link_stats64>`.
196Please note for example that detailed error statistics must be
197added into the general `rx_error` / `tx_error` counters.
198
199The `.ndo_get_stats64` callback can not sleep because of accesses
200via `/proc/net/dev`. If driver may sleep when retrieving the statistics
201from the device it should do so periodically asynchronously and only return
202a recent copy from `.ndo_get_stats64`. Ethtool interrupt coalescing interface
203allows setting the frequency of refreshing statistics, if needed.
204
205Retrieving ethtool statistics is a multi-syscall process, drivers are advised
206to keep the number of statistics constant to avoid race conditions with
207user space trying to read them.
208
209Statistics must persist across routine operations like bringing the interface
210down and up.
211
212Kernel-internal data structures
213-------------------------------
214
215The following structures are internal to the kernel, their members are
216translated to netlink attributes when dumped. Drivers must not overwrite
217the statistics they don't report with 0.
218
219- ethtool_pause_stats()
220- ethtool_fec_stats()
221