1======================================
2NO_HZ: Reducing Scheduling-Clock Ticks
3======================================
4
5
6This document describes Kconfig options and boot parameters that can
7reduce the number of scheduling-clock interrupts, thereby improving energy
8efficiency and reducing OS jitter.  Reducing OS jitter is important for
9some types of computationally intensive high-performance computing (HPC)
10applications and for real-time applications.
11
12There are three main ways of managing scheduling-clock interrupts
13(also known as "scheduling-clock ticks" or simply "ticks"):
14
151.	Never omit scheduling-clock ticks (CONFIG_HZ_PERIODIC=y or
16	CONFIG_NO_HZ=n for older kernels).  You normally will -not-
17	want to choose this option.
18
192.	Omit scheduling-clock ticks on idle CPUs (CONFIG_NO_HZ_IDLE=y or
20	CONFIG_NO_HZ=y for older kernels).  This is the most common
21	approach, and should be the default.
22
233.	Omit scheduling-clock ticks on CPUs that are either idle or that
24	have only one runnable task (CONFIG_NO_HZ_FULL=y).  Unless you
25	are running realtime applications or certain types of HPC
26	workloads, you will normally -not- want this option.
27
28These three cases are described in the following three sections, followed
29by a third section on RCU-specific considerations, a fourth section
30discussing testing, and a fifth and final section listing known issues.
31
32
33Never Omit Scheduling-Clock Ticks
34=================================
35
36Very old versions of Linux from the 1990s and the very early 2000s
37are incapable of omitting scheduling-clock ticks.  It turns out that
38there are some situations where this old-school approach is still the
39right approach, for example, in heavy workloads with lots of tasks
40that use short bursts of CPU, where there are very frequent idle
41periods, but where these idle periods are also quite short (tens or
42hundreds of microseconds).  For these types of workloads, scheduling
43clock interrupts will normally be delivered any way because there
44will frequently be multiple runnable tasks per CPU.  In these cases,
45attempting to turn off the scheduling clock interrupt will have no effect
46other than increasing the overhead of switching to and from idle and
47transitioning between user and kernel execution.
48
49This mode of operation can be selected using CONFIG_HZ_PERIODIC=y (or
50CONFIG_NO_HZ=n for older kernels).
51
52However, if you are instead running a light workload with long idle
53periods, failing to omit scheduling-clock interrupts will result in
54excessive power consumption.  This is especially bad on battery-powered
55devices, where it results in extremely short battery lifetimes.  If you
56are running light workloads, you should therefore read the following
57section.
58
59In addition, if you are running either a real-time workload or an HPC
60workload with short iterations, the scheduling-clock interrupts can
61degrade your applications performance.  If this describes your workload,
62you should read the following two sections.
63
64
65Omit Scheduling-Clock Ticks For Idle CPUs
66=========================================
67
68If a CPU is idle, there is little point in sending it a scheduling-clock
69interrupt.  After all, the primary purpose of a scheduling-clock interrupt
70is to force a busy CPU to shift its attention among multiple duties,
71and an idle CPU has no duties to shift its attention among.
72
73The CONFIG_NO_HZ_IDLE=y Kconfig option causes the kernel to avoid sending
74scheduling-clock interrupts to idle CPUs, which is critically important
75both to battery-powered devices and to highly virtualized mainframes.
76A battery-powered device running a CONFIG_HZ_PERIODIC=y kernel would
77drain its battery very quickly, easily 2-3 times as fast as would the
78same device running a CONFIG_NO_HZ_IDLE=y kernel.  A mainframe running
791,500 OS instances might find that half of its CPU time was consumed by
80unnecessary scheduling-clock interrupts.  In these situations, there
81is strong motivation to avoid sending scheduling-clock interrupts to
82idle CPUs.  That said, dyntick-idle mode is not free:
83
841.	It increases the number of instructions executed on the path
85	to and from the idle loop.
86
872.	On many architectures, dyntick-idle mode also increases the
88	number of expensive clock-reprogramming operations.
89
90Therefore, systems with aggressive real-time response constraints often
91run CONFIG_HZ_PERIODIC=y kernels (or CONFIG_NO_HZ=n for older kernels)
92in order to avoid degrading from-idle transition latencies.
93
94An idle CPU that is not receiving scheduling-clock interrupts is said to
95be "dyntick-idle", "in dyntick-idle mode", "in nohz mode", or "running
96tickless".  The remainder of this document will use "dyntick-idle mode".
97
98There is also a boot parameter "nohz=" that can be used to disable
99dyntick-idle mode in CONFIG_NO_HZ_IDLE=y kernels by specifying "nohz=off".
100By default, CONFIG_NO_HZ_IDLE=y kernels boot with "nohz=on", enabling
101dyntick-idle mode.
102
103
104Omit Scheduling-Clock Ticks For CPUs With Only One Runnable Task
105================================================================
106
107If a CPU has only one runnable task, there is little point in sending it
108a scheduling-clock interrupt because there is no other task to switch to.
109Note that omitting scheduling-clock ticks for CPUs with only one runnable
110task implies also omitting them for idle CPUs.
111
112The CONFIG_NO_HZ_FULL=y Kconfig option causes the kernel to avoid
113sending scheduling-clock interrupts to CPUs with a single runnable task,
114and such CPUs are said to be "adaptive-ticks CPUs".  This is important
115for applications with aggressive real-time response constraints because
116it allows them to improve their worst-case response times by the maximum
117duration of a scheduling-clock interrupt.  It is also important for
118computationally intensive short-iteration workloads:  If any CPU is
119delayed during a given iteration, all the other CPUs will be forced to
120wait idle while the delayed CPU finishes.  Thus, the delay is multiplied
121by one less than the number of CPUs.  In these situations, there is
122again strong motivation to avoid sending scheduling-clock interrupts.
123
124By default, no CPU will be an adaptive-ticks CPU.  The "nohz_full="
125boot parameter specifies the adaptive-ticks CPUs.  For example,
126"nohz_full=1,6-8" says that CPUs 1, 6, 7, and 8 are to be adaptive-ticks
127CPUs.  Note that you are prohibited from marking all of the CPUs as
128adaptive-tick CPUs:  At least one non-adaptive-tick CPU must remain
129online to handle timekeeping tasks in order to ensure that system
130calls like gettimeofday() returns accurate values on adaptive-tick CPUs.
131(This is not an issue for CONFIG_NO_HZ_IDLE=y because there are no running
132user processes to observe slight drifts in clock rate.)  Therefore, the
133boot CPU is prohibited from entering adaptive-ticks mode.  Specifying a
134"nohz_full=" mask that includes the boot CPU will result in a boot-time
135error message, and the boot CPU will be removed from the mask.  Note that
136this means that your system must have at least two CPUs in order for
137CONFIG_NO_HZ_FULL=y to do anything for you.
138
139Finally, adaptive-ticks CPUs must have their RCU callbacks offloaded.
140This is covered in the "RCU IMPLICATIONS" section below.
141
142Normally, a CPU remains in adaptive-ticks mode as long as possible.
143In particular, transitioning to kernel mode does not automatically change
144the mode.  Instead, the CPU will exit adaptive-ticks mode only if needed,
145for example, if that CPU enqueues an RCU callback.
146
147Just as with dyntick-idle mode, the benefits of adaptive-tick mode do
148not come for free:
149
1501.	CONFIG_NO_HZ_FULL selects CONFIG_NO_HZ_COMMON, so you cannot run
151	adaptive ticks without also running dyntick idle.  This dependency
152	extends down into the implementation, so that all of the costs
153	of CONFIG_NO_HZ_IDLE are also incurred by CONFIG_NO_HZ_FULL.
154
1552.	The user/kernel transitions are slightly more expensive due
156	to the need to inform kernel subsystems (such as RCU) about
157	the change in mode.
158
1593.	POSIX CPU timers prevent CPUs from entering adaptive-tick mode.
160	Real-time applications needing to take actions based on CPU time
161	consumption need to use other means of doing so.
162
1634.	If there are more perf events pending than the hardware can
164	accommodate, they are normally round-robined so as to collect
165	all of them over time.  Adaptive-tick mode may prevent this
166	round-robining from happening.  This will likely be fixed by
167	preventing CPUs with large numbers of perf events pending from
168	entering adaptive-tick mode.
169
1705.	Scheduler statistics for adaptive-tick CPUs may be computed
171	slightly differently than those for non-adaptive-tick CPUs.
172	This might in turn perturb load-balancing of real-time tasks.
173
1746.	The LB_BIAS scheduler feature is disabled by adaptive ticks.
175
176Although improvements are expected over time, adaptive ticks is quite
177useful for many types of real-time and compute-intensive applications.
178However, the drawbacks listed above mean that adaptive ticks should not
179(yet) be enabled by default.
180
181
182RCU Implications
183================
184
185There are situations in which idle CPUs cannot be permitted to
186enter either dyntick-idle mode or adaptive-tick mode, the most
187common being when that CPU has RCU callbacks pending.
188
189The CONFIG_RCU_FAST_NO_HZ=y Kconfig option may be used to cause such CPUs
190to enter dyntick-idle mode or adaptive-tick mode anyway.  In this case,
191a timer will awaken these CPUs every four jiffies in order to ensure
192that the RCU callbacks are processed in a timely fashion.
193
194Another approach is to offload RCU callback processing to "rcuo" kthreads
195using the CONFIG_RCU_NOCB_CPU=y Kconfig option.  The specific CPUs to
196offload may be selected using The "rcu_nocbs=" kernel boot parameter,
197which takes a comma-separated list of CPUs and CPU ranges, for example,
198"1,3-5" selects CPUs 1, 3, 4, and 5.
199
200The offloaded CPUs will never queue RCU callbacks, and therefore RCU
201never prevents offloaded CPUs from entering either dyntick-idle mode
202or adaptive-tick mode.  That said, note that it is up to userspace to
203pin the "rcuo" kthreads to specific CPUs if desired.  Otherwise, the
204scheduler will decide where to run them, which might or might not be
205where you want them to run.
206
207
208Testing
209=======
210
211So you enable all the OS-jitter features described in this document,
212but do not see any change in your workload's behavior.  Is this because
213your workload isn't affected that much by OS jitter, or is it because
214something else is in the way?  This section helps answer this question
215by providing a simple OS-jitter test suite, which is available on branch
216master of the following git archive:
217
218git://git.kernel.org/pub/scm/linux/kernel/git/frederic/dynticks-testing.git
219
220Clone this archive and follow the instructions in the README file.
221This test procedure will produce a trace that will allow you to evaluate
222whether or not you have succeeded in removing OS jitter from your system.
223If this trace shows that you have removed OS jitter as much as is
224possible, then you can conclude that your workload is not all that
225sensitive to OS jitter.
226
227Note: this test requires that your system have at least two CPUs.
228We do not currently have a good way to remove OS jitter from single-CPU
229systems.
230
231
232Known Issues
233============
234
235*	Dyntick-idle slows transitions to and from idle slightly.
236	In practice, this has not been a problem except for the most
237	aggressive real-time workloads, which have the option of disabling
238	dyntick-idle mode, an option that most of them take.  However,
239	some workloads will no doubt want to use adaptive ticks to
240	eliminate scheduling-clock interrupt latencies.  Here are some
241	options for these workloads:
242
243	a.	Use PMQOS from userspace to inform the kernel of your
244		latency requirements (preferred).
245
246	b.	On x86 systems, use the "idle=mwait" boot parameter.
247
248	c.	On x86 systems, use the "intel_idle.max_cstate=" to limit
249	`	the maximum C-state depth.
250
251	d.	On x86 systems, use the "idle=poll" boot parameter.
252		However, please note that use of this parameter can cause
253		your CPU to overheat, which may cause thermal throttling
254		to degrade your latencies -- and that this degradation can
255		be even worse than that of dyntick-idle.  Furthermore,
256		this parameter effectively disables Turbo Mode on Intel
257		CPUs, which can significantly reduce maximum performance.
258
259*	Adaptive-ticks slows user/kernel transitions slightly.
260	This is not expected to be a problem for computationally intensive
261	workloads, which have few such transitions.  Careful benchmarking
262	will be required to determine whether or not other workloads
263	are significantly affected by this effect.
264
265*	Adaptive-ticks does not do anything unless there is only one
266	runnable task for a given CPU, even though there are a number
267	of other situations where the scheduling-clock tick is not
268	needed.  To give but one example, consider a CPU that has one
269	runnable high-priority SCHED_FIFO task and an arbitrary number
270	of low-priority SCHED_OTHER tasks.  In this case, the CPU is
271	required to run the SCHED_FIFO task until it either blocks or
272	some other higher-priority task awakens on (or is assigned to)
273	this CPU, so there is no point in sending a scheduling-clock
274	interrupt to this CPU.	However, the current implementation
275	nevertheless sends scheduling-clock interrupts to CPUs having a
276	single runnable SCHED_FIFO task and multiple runnable SCHED_OTHER
277	tasks, even though these interrupts are unnecessary.
278
279	And even when there are multiple runnable tasks on a given CPU,
280	there is little point in interrupting that CPU until the current
281	running task's timeslice expires, which is almost always way
282	longer than the time of the next scheduling-clock interrupt.
283
284	Better handling of these sorts of situations is future work.
285
286*	A reboot is required to reconfigure both adaptive idle and RCU
287	callback offloading.  Runtime reconfiguration could be provided
288	if needed, however, due to the complexity of reconfiguring RCU at
289	runtime, there would need to be an earthshakingly good reason.
290	Especially given that you have the straightforward option of
291	simply offloading RCU callbacks from all CPUs and pinning them
292	where you want them whenever you want them pinned.
293
294*	Additional configuration is required to deal with other sources
295	of OS jitter, including interrupts and system-utility tasks
296	and processes.  This configuration normally involves binding
297	interrupts and tasks to particular CPUs.
298
299*	Some sources of OS jitter can currently be eliminated only by
300	constraining the workload.  For example, the only way to eliminate
301	OS jitter due to global TLB shootdowns is to avoid the unmapping
302	operations (such as kernel module unload operations) that
303	result in these shootdowns.  For another example, page faults
304	and TLB misses can be reduced (and in some cases eliminated) by
305	using huge pages and by constraining the amount of memory used
306	by the application.  Pre-faulting the working set can also be
307	helpful, especially when combined with the mlock() and mlockall()
308	system calls.
309
310*	Unless all CPUs are idle, at least one CPU must keep the
311	scheduling-clock interrupt going in order to support accurate
312	timekeeping.
313
314*	If there might potentially be some adaptive-ticks CPUs, there
315	will be at least one CPU keeping the scheduling-clock interrupt
316	going, even if all CPUs are otherwise idle.
317
318	Better handling of this situation is ongoing work.
319
320*	Some process-handling operations still require the occasional
321	scheduling-clock tick.	These operations include calculating CPU
322	load, maintaining sched average, computing CFS entity vruntime,
323	computing avenrun, and carrying out load balancing.  They are
324	currently accommodated by scheduling-clock tick every second
325	or so.	On-going work will eliminate the need even for these
326	infrequent scheduling-clock ticks.
327