1.. SPDX-License-Identifier: GPL-2.0
2
3.. include:: <isonum.txt>
4
5===============================
6Bus lock detection and handling
7===============================
8
9:Copyright: |copy| 2021 Intel Corporation
10:Authors: - Fenghua Yu <fenghua.yu@intel.com>
11          - Tony Luck <tony.luck@intel.com>
12
13Problem
14=======
15
16A split lock is any atomic operation whose operand crosses two cache lines.
17Since the operand spans two cache lines and the operation must be atomic,
18the system locks the bus while the CPU accesses the two cache lines.
19
20A bus lock is acquired through either split locked access to writeback (WB)
21memory or any locked access to non-WB memory. This is typically thousands of
22cycles slower than an atomic operation within a cache line. It also disrupts
23performance on other cores and brings the whole system to its knees.
24
25Detection
26=========
27
28Intel processors may support either or both of the following hardware
29mechanisms to detect split locks and bus locks.
30
31#AC exception for split lock detection
32--------------------------------------
33
34Beginning with the Tremont Atom CPU split lock operations may raise an
35Alignment Check (#AC) exception when a split lock operation is attemped.
36
37#DB exception for bus lock detection
38------------------------------------
39
40Some CPUs have the ability to notify the kernel by an #DB trap after a user
41instruction acquires a bus lock and is executed. This allows the kernel to
42terminate the application or to enforce throttling.
43
44Software handling
45=================
46
47The kernel #AC and #DB handlers handle bus lock based on the kernel
48parameter "split_lock_detect". Here is a summary of different options:
49
50+------------------+----------------------------+-----------------------+
51|split_lock_detect=|#AC for split lock		|#DB for bus lock	|
52+------------------+----------------------------+-----------------------+
53|off	  	   |Do nothing			|Do nothing		|
54+------------------+----------------------------+-----------------------+
55|warn		   |Kernel OOPs			|Warn once per task and |
56|(default)	   |Warn once per task, add a	|and continues to run.  |
57|		   |delay, add synchronization	|			|
58|		   |to prevent more than one	|			|
59|		   |core from executing a	|			|
60|		   |split lock in parallel.	|			|
61|		   |sysctl split_lock_mitigate	|			|
62|		   |can be used to avoid the	|			|
63|		   |delay and synchronization	|			|
64|		   |When both features are	|			|
65|		   |supported, warn in #AC	|			|
66+------------------+----------------------------+-----------------------+
67|fatal		   |Kernel OOPs			|Send SIGBUS to user.	|
68|		   |Send SIGBUS to user		|			|
69|		   |When both features are	|			|
70|		   |supported, fatal in #AC	|			|
71+------------------+----------------------------+-----------------------+
72|ratelimit:N	   |Do nothing			|Limit bus lock rate to	|
73|(0 < N <= 1000)   |				|N bus locks per second	|
74|		   |				|system wide and warn on|
75|		   |				|bus locks.		|
76+------------------+----------------------------+-----------------------+
77
78Usages
79======
80
81Detecting and handling bus lock may find usages in various areas:
82
83It is critical for real time system designers who build consolidated real
84time systems. These systems run hard real time code on some cores and run
85"untrusted" user processes on other cores. The hard real time cannot afford
86to have any bus lock from the untrusted processes to hurt real time
87performance. To date the designers have been unable to deploy these
88solutions as they have no way to prevent the "untrusted" user code from
89generating split lock and bus lock to block the hard real time code to
90access memory during bus locking.
91
92It's also useful for general computing to prevent guests or user
93applications from slowing down the overall system by executing instructions
94with bus lock.
95
96
97Guidance
98========
99off
100---
101
102Disable checking for split lock and bus lock. This option can be useful if
103there are legacy applications that trigger these events at a low rate so
104that mitigation is not needed.
105
106warn
107----
108
109A warning is emitted when a bus lock is detected which allows to identify
110the offending application. This is the default behavior.
111
112fatal
113-----
114
115In this case, the bus lock is not tolerated and the process is killed.
116
117ratelimit
118---------
119
120A system wide bus lock rate limit N is specified where 0 < N <= 1000. This
121allows a bus lock rate up to N bus locks per second. When the bus lock rate
122is exceeded then any task which is caught via the buslock #DB exception is
123throttled by enforced sleeps until the rate goes under the limit again.
124
125This is an effective mitigation in cases where a minimal impact can be
126tolerated, but an eventual Denial of Service attack has to be prevented. It
127allows to identify the offending processes and analyze whether they are
128malicious or just badly written.
129
130Selecting a rate limit of 1000 allows the bus to be locked for up to about
131seven million cycles each second (assuming 7000 cycles for each bus
132lock). On a 2 GHz processor that would be about 0.35% system slowdown.
133