Lines Matching +full:4 +full:- +full:cpu
2 CPU hotplug in the Kernel
19 insertion and removal require support for CPU hotplug.
22 provisioning reasons, or for RAS purposes to keep an offending CPU off
23 system execution path. Hence the need for CPU hotplug support in the
26 A more novel use of CPU-hotplug support is its use today in suspend resume
27 support for SMP. Dual-core and HT support makes even a laptop run SMP kernels
59 CPU maps
72 after a CPU is available for kernel scheduling and ready to receive
73 interrupts from devices. Its cleared when a CPU is brought down using
75 migrated to another target CPU.
81 from the map depending on the event is hot-add/hot-remove. There are currently
85 You really don't need to manipulate any of the system CPU maps. They should
86 be read-only for most use. When setting up per-cpu resources almost always use
88 ``for_each_cpu()`` can be used to iterate over a custom CPU mask.
93 Using CPU hotplug
100 $ ls -lh /sys/devices/system/cpu
102 drwxr-xr-x 9 root root 0 Dec 21 16:33 cpu0
103 drwxr-xr-x 9 root root 0 Dec 21 16:33 cpu1
104 drwxr-xr-x 9 root root 0 Dec 21 16:33 cpu2
105 drwxr-xr-x 9 root root 0 Dec 21 16:33 cpu3
106 drwxr-xr-x 9 root root 0 Dec 21 16:33 cpu4
107 drwxr-xr-x 9 root root 0 Dec 21 16:33 cpu5
108 drwxr-xr-x 9 root root 0 Dec 21 16:33 cpu6
109 drwxr-xr-x 9 root root 0 Dec 21 16:33 cpu7
110 drwxr-xr-x 2 root root 0 Dec 21 16:33 hotplug
111 -r--r--r-- 1 root root 4.0K Dec 21 16:33 offline
112 -r--r--r-- 1 root root 4.0K Dec 21 16:33 online
113 -r--r--r-- 1 root root 4.0K Dec 21 16:33 possible
114 -r--r--r-- 1 root root 4.0K Dec 21 16:33 present
116 The files *offline*, *online*, *possible*, *present* represent the CPU masks.
117 Each CPU folder contains an *online* file which controls the logical on (1) and
120 $ echo 0 > /sys/devices/system/cpu/cpu4/online
121 smpboot: CPU 4 is now offline
123 Once the CPU is shutdown, it will be removed from */proc/interrupts*,
127 $ echo 1 > /sys/devices/system/cpu/cpu4/online
128 smpboot: Booting Node 0 Processor 4 APIC 0x1
130 The CPU is usable again. This should work on all CPUs, but CPU0 is often special
131 and excluded from CPU hotplug.
133 The CPU hotplug coordination
137 ----------------
139 Once a CPU has been logically shutdown the teardown callbacks of registered
145 * All processes are migrated away from this outgoing CPU to new CPUs.
146 The new CPU is chosen from each process' current cpuset, which may be
148 * All interrupts targeted to this CPU are migrated to a new CPU
149 * timers are also migrated to a new CPU
154 The CPU hotplug API
157 CPU hotplug state machine
158 -------------------------
160 CPU hotplug uses a trivial state machine with a linear state space from
164 When a CPU is onlined, the startup callbacks are invoked sequentially until
166 callbacks of a state are set up or an instance is added to a multi-instance
169 When a CPU is offlined the teardown callbacks are invoked in the reverse
172 removed from a multi-instance state.
175 operations (CPU online or CPU offline) then the other not-required callback
185 The startup callbacks in this section are invoked before the CPU is
186 started during a CPU online operation. The teardown callbacks are invoked
187 after the CPU has become dysfunctional during a CPU offline operation.
189 The callbacks are invoked on a control CPU as they can't obviously run on
190 the hotplugged CPU which is either not yet started or has become
194 bring a CPU successfully online. The teardown callbacks are used to free
195 resources or to move pending work to an online CPU after the hotplugged
196 CPU became dysfunctional.
198 The startup callbacks are allowed to fail. If a callback fails, the CPU
199 online operation is aborted and the CPU is brought down to the previous
209 The startup callbacks in this section are invoked on the hotplugged CPU
210 with interrupts disabled during a CPU online operation in the early CPU
212 on the hotplugged CPU during a CPU offline operation shortly before the
213 CPU is completely shut down.
225 The startup callbacks in this section are invoked on the hotplugged CPU
226 during a CPU online operation. The teardown callbacks are invoked on the
227 hotplugged CPU during a CPU offline operation.
229 The callbacks are invoked in the context of the per CPU hotplug thread,
230 which is pinned on the hotplugged CPU. The callbacks are invoked with
234 operation is aborted and the CPU is brought back to the previous state.
236 CPU online/offline operations
237 -----------------------------
242 [CPUHP_OFFLINE + 1]->startup() -> success
243 [CPUHP_OFFLINE + 2]->startup() -> success
244 [CPUHP_OFFLINE + 3] -> skipped because startup == NULL
246 [CPUHP_BRINGUP_CPU]->startup() -> success
248 [CPUHP_BRINGUP_CPU + 1]->startup() -> success
250 [CPUHP_AP_ONLINE]->startup() -> success
252 [CPUHP_AP_ONLINE + 1]->startup() -> success
254 [CPUHP_ONLINE - 1]->startup() -> success
260 [CPUHP_ONLINE - 1]->teardown() -> success
262 [CPUHP_AP_ONLINE + 1]->teardown() -> success
264 [CPUHP_AP_ONLINE]->teardown() -> success
266 [CPUHP_BRINGUP_ONLINE - 1]->teardown()
269 [CPUHP_BRINGUP_CPU]->teardown()
270 [CPUHP_OFFLINE + 3]->teardown()
271 [CPUHP_OFFLINE + 2] -> skipped because teardown == NULL
272 [CPUHP_OFFLINE + 1]->teardown()
278 [CPUHP_OFFLINE + 1]->startup() -> success
279 [CPUHP_OFFLINE + 2]->startup() -> success
280 [CPUHP_OFFLINE + 3] -> skipped because startup == NULL
282 [CPUHP_BRINGUP_CPU]->startup() -> success
284 [CPUHP_BRINGUP_CPU + 1]->startup() -> success
286 [CPUHP_AP_ONLINE]->startup() -> success
288 [CPUHP_AP_ONLINE + 1]->startup() -> success
289 ---
290 [CPUHP_AP_ONLINE + N]->startup() -> fail
291 [CPUHP_AP_ONLINE + (N - 1)]->teardown()
293 [CPUHP_AP_ONLINE + 1]->teardown()
295 [CPUHP_AP_ONLINE]->teardown()
297 [CPUHP_BRINGUP_ONLINE - 1]->teardown()
300 [CPUHP_BRINGUP_CPU]->teardown()
301 [CPUHP_OFFLINE + 3]->teardown()
302 [CPUHP_OFFLINE + 2] -> skipped because teardown == NULL
303 [CPUHP_OFFLINE + 1]->teardown()
309 [CPUHP_ONLINE - 1]->teardown() -> success
311 [CPUHP_ONLINE - N]->teardown() -> fail
312 [CPUHP_ONLINE - (N - 1)]->startup()
314 [CPUHP_ONLINE - 1]->startup()
321 [CPUHP_ONLINE - 1]->teardown() -> success
323 [CPUHP_ONLINE - N]->teardown() -> fail
324 [CPUHP_ONLINE - (N - 1)]->startup() -> success
325 [CPUHP_ONLINE - (N - 2)]->startup() -> fail
327 The CPU hotplug state machine stops right here and does not try to go back
330 [CPUHP_ONLINE - (N - 1)]->teardown() -> success
331 [CPUHP_ONLINE - N]->teardown() -> fail
332 [CPUHP_ONLINE - (N - 1)]->startup() -> success
333 [CPUHP_ONLINE - (N - 2)]->startup() -> fail
334 [CPUHP_ONLINE - (N - 1)]->teardown() -> success
335 [CPUHP_ONLINE - N]->teardown() -> fail
337 Lather, rinse and repeat. In this case the CPU left in state::
339 [CPUHP_ONLINE - (N - 1)]
345 ------------------
347 There are two ways to allocate a CPU hotplug state:
352 ordering requirements versus other CPU hotplug states. E.g. the PERF core
354 callbacks during a CPU online operation. During a CPU offline operation
364 up at runtime and are part of the initializer of the CPU hotplug state
365 array in kernel/cpu.c.
377 Setup of a CPU hotplug state
378 ----------------------------
388 CPU hotplug state callbacks need to be invoked for each instance, the CPU
389 hotplug core provides multi-instance support. The advantage over driver
391 serialized against CPU hotplug operations and provide the automatic
393 multi-instance state the following function is available:
398 constants for dynamically allocated states - CPUHP_BP_PREPARE_DYN,
399 CPUHP_AP_ONLINE_DYN - depending on the state section (PREPARE, ONLINE) for
431 invoked during a CPU online operation. If the usage site does not require a
435 be invoked during a CPU offline operation. If the usage site does not
447 the current CPU (PREPARE section) or on each online CPU (ONLINE
448 section) in the context of the CPU's hotplug thread.
450 If a callback fails for CPU N then the teardown callback for CPU
451 0 .. N-1 is invoked to rollback the operation. The state setup fails,
455 The state setup and the callback invocations are serialized against CPU
456 hotplug operations. If the setup function has to be called from a CPU
458 used. These functions cannot be used from within CPU hotplug callbacks.
470 multi-instance states the dynamically allocated state number is
477 Removal of a CPU hotplug state
478 ------------------------------
500 callback is either invoked on the current CPU (PREPARE section) or on
501 each online CPU (ONLINE section) in the context of the CPU's hotplug
506 The state removal and the callback invocations are serialized against CPU
507 hotplug operations. If the remove function has to be called from a CPU
509 used. These functions cannot be used from within CPU hotplug callbacks.
511 If a multi-instance state is removed then the caller has to remove all
514 Multi-Instance state instance management
515 ----------------------------------------
517 Once the multi-instance state is set up, instances can be added to the
527 instance's data structure. The pointer is handed to the multi-instance
534 multi-instance state's node list.
540 the callback is either invoked on the current CPU (PREPARE section) or
541 on each online CPU (ONLINE section) in the context of the CPU's hotplug
544 If a callback fails for CPU N then the teardown callback for CPU
545 0 .. N-1 is invoked to rollback the operation, the function fails and
546 the instance is not added to the node list of the multi-instance state.
566 section the callback is either invoked on the current CPU (PREPARE
567 section) or on each online CPU (ONLINE section) in the context of the
568 CPU's hotplug thread.
573 serialized against CPU hotplug operations. These functions cannot be used
574 from within CPU hotplug callbacks and CPU hotplug read locked regions.
577 --------
606 Setup, use and teardown a dynamically allocated multi-instance state in the
613 ret = cpuhp_state_add_instance(state, &inst1->node);
617 ret = cpuhp_state_add_instance(state, &inst2->node);
621 cpuhp_remove_instance(state, &inst1->node);
623 cpuhp_remove_instance(state, &inst2->node);
632 shutdown a CPU and then put it online again. It is also possible to put the CPU
637 All registered states are enumerated in ``/sys/devices/system/cpu/hotplug/states`` ::
639 $ tail /sys/devices/system/cpu/hotplug/states
643 141: acpi/cpu-drv:online
653 $ cat /sys/devices/system/cpu/cpu4/hotplug/state
655 $ echo 140 > /sys/devices/system/cpu/cpu4/hotplug/target
656 $ cat /sys/devices/system/cpu/cpu4/hotplug/state
662 $ echo 169 > /sys/devices/system/cpu/cpu4/hotplug/target
663 $ cat /sys/devices/system/cpu/cpu4/hotplug/state
668 # TASK-PID CPU# TIMESTAMP FUNCTION
670 bash-394 [001] 22.976: cpuhp_enter: cpu: 0004 target: 140 step: 169 (cpuhp_kick_ap_work)
671 cpuhp/4-31 [004] 22.977: cpuhp_enter: cpu: 0004 target: 140 step: 168 (sched_cpu_deactivate)
672 cpuhp/4-31 [004] 22.990: cpuhp_exit: cpu: 0004 state: 168 step: 168 ret: 0
673 cpuhp/4-31 [004] 22.991: cpuhp_enter: cpu: 0004 target: 140 step: 144 (mce_cpu_pre_down)
674 cpuhp/4-31 [004] 22.992: cpuhp_exit: cpu: 0004 state: 144 step: 144 ret: 0
675 …cpuhp/4-31 [004] 22.993: cpuhp_multi_enter: cpu: 0004 target: 140 step: 143 (virtnet_cpu_down_p…
676 cpuhp/4-31 [004] 22.994: cpuhp_exit: cpu: 0004 state: 143 step: 143 ret: 0
677 cpuhp/4-31 [004] 22.995: cpuhp_enter: cpu: 0004 target: 140 step: 142 (cacheinfo_cpu_pre_down)
678 cpuhp/4-31 [004] 22.996: cpuhp_exit: cpu: 0004 state: 142 step: 142 ret: 0
679 bash-394 [001] 22.997: cpuhp_exit: cpu: 0004 state: 140 step: 169 ret: 0
680 bash-394 [005] 95.540: cpuhp_enter: cpu: 0004 target: 169 step: 140 (cpuhp_kick_ap_work)
681 cpuhp/4-31 [004] 95.541: cpuhp_enter: cpu: 0004 target: 169 step: 141 (acpi_soft_cpu_online)
682 cpuhp/4-31 [004] 95.542: cpuhp_exit: cpu: 0004 state: 141 step: 141 ret: 0
683 cpuhp/4-31 [004] 95.543: cpuhp_enter: cpu: 0004 target: 169 step: 142 (cacheinfo_cpu_online)
684 cpuhp/4-31 [004] 95.544: cpuhp_exit: cpu: 0004 state: 142 step: 142 ret: 0
685 …cpuhp/4-31 [004] 95.545: cpuhp_multi_enter: cpu: 0004 target: 169 step: 143 (virtnet_cpu_online)
686 cpuhp/4-31 [004] 95.546: cpuhp_exit: cpu: 0004 state: 143 step: 143 ret: 0
687 cpuhp/4-31 [004] 95.547: cpuhp_enter: cpu: 0004 target: 169 step: 144 (mce_cpu_online)
688 cpuhp/4-31 [004] 95.548: cpuhp_exit: cpu: 0004 state: 144 step: 144 ret: 0
689 cpuhp/4-31 [004] 95.549: cpuhp_enter: cpu: 0004 target: 169 step: 145 (console_cpu_notify)
690 cpuhp/4-31 [004] 95.550: cpuhp_exit: cpu: 0004 state: 145 step: 145 ret: 0
691 cpuhp/4-31 [004] 95.551: cpuhp_enter: cpu: 0004 target: 169 step: 168 (sched_cpu_activate)
692 cpuhp/4-31 [004] 95.552: cpuhp_exit: cpu: 0004 state: 168 step: 168 ret: 0
693 bash-394 [005] 95.553: cpuhp_exit: cpu: 0004 state: 169 step: 140 ret: 0
708 Arch interface to bring up a CPU
711 Arch interface to shutdown a CPU, no more interrupts can be handled by the
715 This actually supposed to ensure death of the CPU. Actually look at some
716 example code in other arch that implement CPU hotplug. The processor is taken
724 After CPU successfully onlined or offline udev events are sent. A udev rule like::
726 …SUBSYSTEM=="cpu", DRIVERS=="processor", DEVPATH=="/devices/system/cpu/*", RUN+="the_hotplug_receiv…
734 echo "CPU ${DEVPATH##*/} offline"
738 echo "CPU ${DEVPATH##*/} online"
745 /sys/devices/system/cpu/crash_hotplug contains '1' if the kernel
752 To skip userspace processing of CPU hot un/plug events for kdump
753 (i.e. the unload-then-reload to obtain a current list of CPUs), this sysfs
756 SUBSYSTEM=="cpu", ATTRS{crash_hotplug}=="1", GOTO="kdump_reload_end"
758 For a CPU hot un/plug event, if the architecture supports kernel updates
760 the unload-then-reload of the kdump capture kernel.
765 .. kernel-doc:: include/linux/cpuhotplug.h