1Using TopDown metrics in user space
2-----------------------------------
3
4Intel CPUs (since Sandy Bridge and Silvermont) support a TopDown
5methodology to break down CPU pipeline execution into 4 bottlenecks:
6frontend bound, backend bound, bad speculation, retiring.
7
8For more details on Topdown see [1][5]
9
10Traditionally this was implemented by events in generic counters
11and specific formulas to compute the bottlenecks.
12
13perf stat --topdown implements this.
14
15Full Top Down includes more levels that can break down the
16bottlenecks further. This is not directly implemented in perf,
17but available in other tools that can run on top of perf,
18such as toplev[2] or vtune[3]
19
20New Topdown features in Ice Lake
21===============================
22
23With Ice Lake CPUs the TopDown metrics are directly available as
24fixed counters and do not require generic counters. This allows
25to collect TopDown always in addition to other events.
26
27% perf stat -a --topdown -I1000
28#           time             retiring      bad speculation       frontend bound        backend bound
29     1.001281330                23.0%                15.3%                29.6%                32.1%
30     2.003009005                 5.0%                 6.8%                46.6%                41.6%
31     3.004646182                 6.7%                 6.7%                46.0%                40.6%
32     4.006326375                 5.0%                 6.4%                47.6%                41.0%
33     5.007991804                 5.1%                 6.3%                46.3%                42.3%
34     6.009626773                 6.2%                 7.1%                47.3%                39.3%
35     7.011296356                 4.7%                 6.7%                46.2%                42.4%
36     8.012951831                 4.7%                 6.7%                47.5%                41.1%
37...
38
39This also enables measuring TopDown per thread/process instead
40of only per core.
41
42Using TopDown through RDPMC in applications on Ice Lake
43======================================================
44
45For more fine grained measurements it can be useful to
46access the new  directly from user space. This is more complicated,
47but drastically lowers overhead.
48
49On Ice Lake, there is a new fixed counter 3: SLOTS, which reports
50"pipeline SLOTS" (cycles multiplied by core issue width) and a
51metric register that reports slots ratios for the different bottleneck
52categories.
53
54The metrics counter is CPU model specific and is not available on older
55CPUs.
56
57Example code
58============
59
60Library functions to do the functionality described below
61is also available in libjevents [4]
62
63The application opens a group with fixed counter 3 (SLOTS) and any
64metric event, and allow user programs to read the performance counters.
65
66Fixed counter 3 is mapped to a pseudo event event=0x00, umask=04,
67so the perf_event_attr structure should be initialized with
68{ .config = 0x0400, .type = PERF_TYPE_RAW }
69The metric events are mapped to the pseudo event event=0x00, umask=0x8X.
70For example, the perf_event_attr structure can be initialized with
71{ .config = 0x8000, .type = PERF_TYPE_RAW } for Retiring metric event
72The Fixed counter 3 must be the leader of the group.
73
74#include <linux/perf_event.h>
75#include <sys/mman.h>
76#include <sys/syscall.h>
77#include <unistd.h>
78
79/* Provide own perf_event_open stub because glibc doesn't */
80__attribute__((weak))
81int perf_event_open(struct perf_event_attr *attr, pid_t pid,
82		    int cpu, int group_fd, unsigned long flags)
83{
84	return syscall(__NR_perf_event_open, attr, pid, cpu, group_fd, flags);
85}
86
87/* Open slots counter file descriptor for current task. */
88struct perf_event_attr slots = {
89	.type = PERF_TYPE_RAW,
90	.size = sizeof(struct perf_event_attr),
91	.config = 0x400,
92	.exclude_kernel = 1,
93};
94
95int slots_fd = perf_event_open(&slots, 0, -1, -1, 0);
96if (slots_fd < 0)
97	... error ...
98
99/* Memory mapping the fd permits _rdpmc calls from userspace */
100void *slots_p = mmap(0, getpagesize(), PROT_READ, MAP_SHARED, slots_fd, 0);
101if (!slot_p)
102	.... error ...
103
104/*
105 * Open metrics event file descriptor for current task.
106 * Set slots event as the leader of the group.
107 */
108struct perf_event_attr metrics = {
109	.type = PERF_TYPE_RAW,
110	.size = sizeof(struct perf_event_attr),
111	.config = 0x8000,
112	.exclude_kernel = 1,
113};
114
115int metrics_fd = perf_event_open(&metrics, 0, -1, slots_fd, 0);
116if (metrics_fd < 0)
117	... error ...
118
119/* Memory mapping the fd permits _rdpmc calls from userspace */
120void *metrics_p = mmap(0, getpagesize(), PROT_READ, MAP_SHARED, metrics_fd, 0);
121if (!metrics_p)
122	... error ...
123
124Note: the file descriptors returned by the perf_event_open calls must be memory
125mapped to permit calls to the _rdpmd instruction. Permission may also be granted
126by writing the /sys/devices/cpu/rdpmc sysfs node.
127
128The RDPMC instruction (or _rdpmc compiler intrinsic) can now be used
129to read slots and the topdown metrics at different points of the program:
130
131#include <stdint.h>
132#include <x86intrin.h>
133
134#define RDPMC_FIXED	(1 << 30)	/* return fixed counters */
135#define RDPMC_METRIC	(1 << 29)	/* return metric counters */
136
137#define FIXED_COUNTER_SLOTS		3
138#define METRIC_COUNTER_TOPDOWN_L1_L2	0
139
140static inline uint64_t read_slots(void)
141{
142	return _rdpmc(RDPMC_FIXED | FIXED_COUNTER_SLOTS);
143}
144
145static inline uint64_t read_metrics(void)
146{
147	return _rdpmc(RDPMC_METRIC | METRIC_COUNTER_TOPDOWN_L1_L2);
148}
149
150Then the program can be instrumented to read these metrics at different
151points.
152
153It's not a good idea to do this with too short code regions,
154as the parallelism and overlap in the CPU program execution will
155cause too much measurement inaccuracy. For example instrumenting
156individual basic blocks is definitely too fine grained.
157
158_rdpmc calls should not be mixed with reading the metrics and slots counters
159through system calls, as the kernel will reset these counters after each system
160call.
161
162Decoding metrics values
163=======================
164
165The value reported by read_metrics() contains four 8 bit fields
166that represent a scaled ratio that represent the Level 1 bottleneck.
167All four fields add up to 0xff (= 100%)
168
169The binary ratios in the metric value can be converted to float ratios:
170
171#define GET_METRIC(m, i) (((m) >> (i*8)) & 0xff)
172
173/* L1 Topdown metric events */
174#define TOPDOWN_RETIRING(val)	((float)GET_METRIC(val, 0) / 0xff)
175#define TOPDOWN_BAD_SPEC(val)	((float)GET_METRIC(val, 1) / 0xff)
176#define TOPDOWN_FE_BOUND(val)	((float)GET_METRIC(val, 2) / 0xff)
177#define TOPDOWN_BE_BOUND(val)	((float)GET_METRIC(val, 3) / 0xff)
178
179/*
180 * L2 Topdown metric events.
181 * Available on Sapphire Rapids and later platforms.
182 */
183#define TOPDOWN_HEAVY_OPS(val)		((float)GET_METRIC(val, 4) / 0xff)
184#define TOPDOWN_BR_MISPREDICT(val)	((float)GET_METRIC(val, 5) / 0xff)
185#define TOPDOWN_FETCH_LAT(val)		((float)GET_METRIC(val, 6) / 0xff)
186#define TOPDOWN_MEM_BOUND(val)		((float)GET_METRIC(val, 7) / 0xff)
187
188and then converted to percent for printing.
189
190The ratios in the metric accumulate for the time when the counter
191is enabled. For measuring programs it is often useful to measure
192specific sections. For this it is needed to deltas on metrics.
193
194This can be done by scaling the metrics with the slots counter
195read at the same time.
196
197Then it's possible to take deltas of these slots counts
198measured at different points, and determine the metrics
199for that time period.
200
201	slots_a = read_slots();
202	metric_a = read_metrics();
203
204	... larger code region ...
205
206	slots_b = read_slots()
207	metric_b = read_metrics()
208
209	# compute scaled metrics for measurement a
210	retiring_slots_a = GET_METRIC(metric_a, 0) * slots_a
211	bad_spec_slots_a = GET_METRIC(metric_a, 1) * slots_a
212	fe_bound_slots_a = GET_METRIC(metric_a, 2) * slots_a
213	be_bound_slots_a = GET_METRIC(metric_a, 3) * slots_a
214
215	# compute delta scaled metrics between b and a
216	retiring_slots = GET_METRIC(metric_b, 0) * slots_b - retiring_slots_a
217	bad_spec_slots = GET_METRIC(metric_b, 1) * slots_b - bad_spec_slots_a
218	fe_bound_slots = GET_METRIC(metric_b, 2) * slots_b - fe_bound_slots_a
219	be_bound_slots = GET_METRIC(metric_b, 3) * slots_b - be_bound_slots_a
220
221Later the individual ratios of L1 metric events for the measurement period can
222be recreated from these counts.
223
224	slots_delta = slots_b - slots_a
225	retiring_ratio = (float)retiring_slots / slots_delta
226	bad_spec_ratio = (float)bad_spec_slots / slots_delta
227	fe_bound_ratio = (float)fe_bound_slots / slots_delta
228	be_bound_ratio = (float)be_bound_slots / slota_delta
229
230	printf("Retiring %.2f%% Bad Speculation %.2f%% FE Bound %.2f%% BE Bound %.2f%%\n",
231		retiring_ratio * 100.,
232		bad_spec_ratio * 100.,
233		fe_bound_ratio * 100.,
234		be_bound_ratio * 100.);
235
236The individual ratios of L2 metric events for the measurement period can be
237recreated from L1 and L2 metric counters. (Available on Sapphire Rapids and
238later platforms)
239
240	# compute scaled metrics for measurement a
241	heavy_ops_slots_a = GET_METRIC(metric_a, 4) * slots_a
242	br_mispredict_slots_a = GET_METRIC(metric_a, 5) * slots_a
243	fetch_lat_slots_a = GET_METRIC(metric_a, 6) * slots_a
244	mem_bound_slots_a = GET_METRIC(metric_a, 7) * slots_a
245
246	# compute delta scaled metrics between b and a
247	heavy_ops_slots = GET_METRIC(metric_b, 4) * slots_b - heavy_ops_slots_a
248	br_mispredict_slots = GET_METRIC(metric_b, 5) * slots_b - br_mispredict_slots_a
249	fetch_lat_slots = GET_METRIC(metric_b, 6) * slots_b - fetch_lat_slots_a
250	mem_bound_slots = GET_METRIC(metric_b, 7) * slots_b - mem_bound_slots_a
251
252	slots_delta = slots_b - slots_a
253	heavy_ops_ratio = (float)heavy_ops_slots / slots_delta
254	light_ops_ratio = retiring_ratio - heavy_ops_ratio;
255
256	br_mispredict_ratio = (float)br_mispredict_slots / slots_delta
257	machine_clears_ratio = bad_spec_ratio - br_mispredict_ratio;
258
259	fetch_lat_ratio = (float)fetch_lat_slots / slots_delta
260	fetch_bw_ratio = fe_bound_ratio - fetch_lat_ratio;
261
262	mem_bound_ratio = (float)mem_bound_slots / slota_delta
263	core_bound_ratio = be_bound_ratio - mem_bound_ratio;
264
265	printf("Heavy Operations %.2f%% Light Operations %.2f%% "
266	       "Branch Mispredict %.2f%% Machine Clears %.2f%% "
267	       "Fetch Latency %.2f%% Fetch Bandwidth %.2f%% "
268	       "Mem Bound %.2f%% Core Bound %.2f%%\n",
269		heavy_ops_ratio * 100.,
270		light_ops_ratio * 100.,
271		br_mispredict_ratio * 100.,
272		machine_clears_ratio * 100.,
273		fetch_lat_ratio * 100.,
274		fetch_bw_ratio * 100.,
275		mem_bound_ratio * 100.,
276		core_bound_ratio * 100.);
277
278Resetting metrics counters
279==========================
280
281Since the individual metrics are only 8bit they lose precision for
282short regions over time because the number of cycles covered by each
283fraction bit shrinks. So the counters need to be reset regularly.
284
285When using the kernel perf API the kernel resets on every read.
286So as long as the reading is at reasonable intervals (every few
287seconds) the precision is good.
288
289When using perf stat it is recommended to always use the -I option,
290with no longer interval than a few seconds
291
292	perf stat -I 1000 --topdown ...
293
294For user programs using RDPMC directly the counter can
295be reset explicitly using ioctl:
296
297	ioctl(perf_fd, PERF_EVENT_IOC_RESET, 0);
298
299This "opens" a new measurement period.
300
301A program using RDPMC for TopDown should schedule such a reset
302regularly, as in every few seconds.
303
304Limits on Ice Lake
305==================
306
307Four pseudo TopDown metric events are exposed for the end-users,
308topdown-retiring, topdown-bad-spec, topdown-fe-bound and topdown-be-bound.
309They can be used to collect the TopDown value under the following
310rules:
311- All the TopDown metric events must be in a group with the SLOTS event.
312- The SLOTS event must be the leader of the group.
313- The PERF_FORMAT_GROUP flag must be applied for each TopDown metric
314  events
315
316The SLOTS event and the TopDown metric events can be counting members of
317a sampling read group. Since the SLOTS event must be the leader of a TopDown
318group, the second event of the group is the sampling event.
319For example, perf record -e '{slots, $sampling_event, topdown-retiring}:S'
320
321Extension on Sapphire Rapids Server
322===================================
323The metrics counter is extended to support TMA method level 2 metrics.
324The lower half of the register is the TMA level 1 metrics (legacy).
325The upper half is also divided into four 8-bit fields for the new level 2
326metrics. Four more TopDown metric events are exposed for the end-users,
327topdown-heavy-ops, topdown-br-mispredict, topdown-fetch-lat and
328topdown-mem-bound.
329
330Each of the new level 2 metrics in the upper half is a subset of the
331corresponding level 1 metric in the lower half. Software can deduce the
332other four level 2 metrics by subtracting corresponding metrics as below.
333
334    Light_Operations = Retiring - Heavy_Operations
335    Machine_Clears = Bad_Speculation - Branch_Mispredicts
336    Fetch_Bandwidth = Frontend_Bound - Fetch_Latency
337    Core_Bound = Backend_Bound - Memory_Bound
338
339
340[1] https://software.intel.com/en-us/top-down-microarchitecture-analysis-method-win
341[2] https://github.com/andikleen/pmu-tools/wiki/toplev-manual
342[3] https://software.intel.com/en-us/intel-vtune-amplifier-xe
343[4] https://github.com/andikleen/pmu-tools/tree/master/jevents
344[5] https://sites.google.com/site/analysismethods/yasin-pubs
345