Lines Matching +full:memory +full:- +full:controller
1 .. _cgroup-v2:
11 conventions of cgroup v2. It describes all userland-visible aspects
12 of cgroup including core and specific controller behaviors. All
14 v1 is available under :ref:`Documentation/admin-guide/cgroup-v1/index.rst <cgroup-v1>`.
19 1-1. Terminology
20 1-2. What is cgroup?
22 2-1. Mounting
23 2-2. Organizing Processes and Threads
24 2-2-1. Processes
25 2-2-2. Threads
26 2-3. [Un]populated Notification
27 2-4. Controlling Controllers
28 2-4-1. Enabling and Disabling
29 2-4-2. Top-down Constraint
30 2-4-3. No Internal Process Constraint
31 2-5. Delegation
32 2-5-1. Model of Delegation
33 2-5-2. Delegation Containment
34 2-6. Guidelines
35 2-6-1. Organize Once and Control
36 2-6-2. Avoid Name Collisions
38 3-1. Weights
39 3-2. Limits
40 3-3. Protections
41 3-4. Allocations
43 4-1. Format
44 4-2. Conventions
45 4-3. Core Interface Files
47 5-1. CPU
48 5-1-1. CPU Interface Files
49 5-2. Memory
50 5-2-1. Memory Interface Files
51 5-2-2. Usage Guidelines
52 5-2-3. Memory Ownership
53 5-3. IO
54 5-3-1. IO Interface Files
55 5-3-2. Writeback
56 5-3-3. IO Latency
57 5-3-3-1. How IO Latency Throttling Works
58 5-3-3-2. IO Latency Interface Files
59 5-3-4. IO Priority
60 5-4. PID
61 5-4-1. PID Interface Files
62 5-5. Cpuset
63 5.5-1. Cpuset Interface Files
64 5-6. Device
65 5-7. RDMA
66 5-7-1. RDMA Interface Files
67 5-8. HugeTLB
68 5.8-1. HugeTLB Interface Files
69 5-9. Misc
70 5.9-1 Miscellaneous cgroup Interface Files
71 5.9-2 Migration and Ownership
72 5-10. Others
73 5-10-1. perf_event
74 5-N. Non-normative information
75 5-N-1. CPU controller root cgroup process behaviour
76 5-N-2. IO controller root cgroup process behaviour
78 6-1. Basics
79 6-2. The Root and Views
80 6-3. Migration and setns(2)
81 6-4. Interaction with Other Namespaces
83 P-1. Filesystem Support for Writeback
86 R-1. Multiple Hierarchies
87 R-2. Thread Granularity
88 R-3. Competition Between Inner Nodes and Threads
89 R-4. Other Interface Issues
90 R-5. Controller Issues and Remedies
91 R-5-1. Memory
98 -----------
107 ---------------
113 cgroup is largely composed of two parts - the core and controllers.
115 processes. A cgroup controller is usually responsible for
128 disabled selectively on a cgroup. All controller behaviors are
129 hierarchical - if a controller is enabled on a cgroup, it affects all
131 sub-hierarchy of the cgroup. When a controller is enabled on a nested
141 --------
146 # mount -t cgroup2 none $MOUNT_POINT
155 A controller can be moved across hierarchies only after the controller
156 is no longer referenced in its current hierarchy. Because per-cgroup
157 controller states are destroyed asynchronously and controllers may
158 have lingering references, a controller may not show up immediately on
160 Similarly, a controller should be fully disabled to be moved out of
162 controller to become available for other hierarchies; furthermore, due
163 to inter-controller dependencies, other controllers may need to be
169 the hierarchies and controller associations before starting using the
184 ignored on non-init namespace mounts. Please refer to the
189 task migrations and controller on/offs at the cost of making
196 Only populate memory.events with data for the current cgroup,
201 option is ignored on non-init namespace mounts.
204 Recursively apply memory.min and memory.low protection to
209 behavior but is a mount-option to avoid regressing setups
215 --------------------------------
221 A child cgroup can be created by creating a sub-directory::
226 structure. Each cgroup has a read-writable interface file
228 belong to the cgroup one-per-line. The PIDs are not ordered and the
259 0::/test-cgroup/test-cgroup-nested
266 0::/test-cgroup/test-cgroup-nested (deleted)
292 constraint - threaded controllers can be enabled on non-leaf cgroups
316 - As the cgroup will join the parent's resource domain. The parent
319 - When the parent is an unthreaded domain, it must not have any domain
323 Topology-wise, a cgroup can be in an invalid state. Please consider
326 A (threaded domain) - B (threaded) - C (domain, just created)
341 threads in the cgroup. Except that the operations are per-thread
342 instead of per-process, "cgroup.threads" has the same format and
357 a threaded controller is enabled inside a threaded subtree, it only
363 constraint, a threaded controller must be able to handle competition
364 between threads in a non-leaf cgroup and its child cgroups. Each
365 threaded controller defines how such competitions are handled.
369 --------------------------
371 Each non-root cgroup has a "cgroup.events" file which contains
372 "populated" field indicating whether the cgroup's sub-hierarchy has
376 example, to start a clean-up operation after all processes of a given
377 sub-hierarchy have exited. The populated state updates and
378 notifications are recursive. Consider the following sub-hierarchy
382 A(4) - B(0) - C(1)
392 -----------------------
401 cpu io memory
403 No controller is enabled by default. Controllers can be enabled and
406 # echo "+cpu +memory -io" > cgroup.subtree_control
410 all succeed or fail. If multiple operations on the same controller
413 Enabling a controller in a cgroup indicates that the distribution of
415 Consider the following sub-hierarchy. The enabled controllers are
418 A(cpu,memory) - B(memory) - C()
421 As A has "cpu" and "memory" enabled, A will control the distribution
422 of CPU cycles and memory to its children, in this case, B. As B has
423 "memory" enabled but not "CPU", C and D will compete freely on CPU
424 cycles but their division of memory available to B will be controlled.
426 As a controller regulates the distribution of the target resource to
427 the cgroup's children, enabling it creates the controller's interface
429 would create the "cpu." prefixed controller interface files in C and
430 D. Likewise, disabling "memory" from B would remove the "memory."
431 prefixed controller interface files from C and D. This means that the
432 controller interface files - anything which doesn't start with
436 Top-down Constraint
439 Resources are distributed top-down and a cgroup can further distribute
441 parent. This means that all non-root "cgroup.subtree_control" files
443 "cgroup.subtree_control" file. A controller can be enabled only if
444 the parent has the controller enabled and a controller can't be
451 Non-root cgroups can distribute domain resources to their children
456 This guarantees that, when a domain controller is looking at the part
465 is up to each controller (for more information on this topic please
466 refer to the Non-normative information section in the Controllers
470 enabled controller in the cgroup's "cgroup.subtree_control". This is
479 ----------
499 delegated, the user can build sub-hierarchy under the directory,
503 happens in the delegated sub-hierarchy, nothing can escape the
507 cgroups in or nesting depth of a delegated sub-hierarchy; however,
514 A delegated sub-hierarchy is contained in the sense that processes
515 can't be moved into or out of the sub-hierarchy by the delegatee.
518 requiring the following conditions for a process with a non-root euid
522 - The writer must have write access to the "cgroup.procs" file.
524 - The writer must have write access to the "cgroup.procs" file of the
528 processes around freely in the delegated sub-hierarchy it can't pull
529 in from or push out to outside the sub-hierarchy.
535 ~~~~~~~~~~~~~ - C0 - C00
538 ~~~~~~~~~~~~~ - C1 - C10
545 will be denied with -EACCES.
550 is not reachable, the migration is rejected with -ENOENT.
554 ----------
560 and stateful resources such as memory are not moved together with the
562 inherent trade-offs between migration and various hot paths in terms
568 resource structure once on start-up. Dynamic adjustments to resource
569 distribution can be made by changing controller configuration through
581 controller's interface files are prefixed with the controller name and
582 a dot. A controller's name is composed of lower case alphabets and
601 -------
607 work-conserving. Due to the dynamic nature, this model is usually
622 .. _cgroupv2-limits-distributor:
625 ------
628 Limits can be over-committed - the sum of the limits of children can
633 As limits can be over-committed, all configuration combinations are
640 .. _cgroupv2-protections-distributor:
643 -----------
648 soft boundaries. Protections can also be over-committed in which case
655 As protections can be over-committed, all configuration combinations
659 "memory.low" implements best-effort memory protection and is an
664 -----------
667 resource. Allocations can't be over-committed - the sum of the
674 As allocations can't be over-committed, some configuration
679 "cpu.rt.max" hard-allocates realtime slices and is an example of this
687 ------
692 New-line separated values
700 (when read-only or multiple values can be written at once)
726 -----------
728 - Settings for a single feature should be contained in a single file.
730 - The root cgroup should be exempt from resource control and thus
733 - The default time unit is microseconds. If a different unit is ever
736 - A parts-per quantity should use a percentage decimal with at least
737 two digit fractional part - e.g. 13.40.
739 - If a controller implements weight based resource distribution, its
745 - If a controller implements an absolute resource guarantee and/or
747 respectively. If a controller implements best effort resource
754 - If a setting has a configurable default value and keyed specific
768 # cat cgroup-example-interface-file
774 # echo 125 > cgroup-example-interface-file
778 # echo "default 125" > cgroup-example-interface-file
782 # echo "8:16 170" > cgroup-example-interface-file
786 # echo "8:0 default" > cgroup-example-interface-file
787 # cat cgroup-example-interface-file
791 - For events which are not very high frequency, an interface file
798 --------------------
803 A read-write single value file which exists on non-root
809 - "domain" : A normal valid domain cgroup.
811 - "domain threaded" : A threaded domain cgroup which is
814 - "domain invalid" : A cgroup which is in an invalid state.
818 - "threaded" : A threaded cgroup which is a member of a
825 A read-write new-line separated values file which exists on
829 the cgroup one-per-line. The PIDs are not ordered and the
838 - It must have write access to the "cgroup.procs" file.
840 - It must have write access to the "cgroup.procs" file of the
843 When delegating a sub-hierarchy, write access to this file
851 A read-write new-line separated values file which exists on
855 the cgroup one-per-line. The TIDs are not ordered and the
864 - It must have write access to the "cgroup.threads" file.
866 - The cgroup that the thread is currently in must be in the
869 - It must have write access to the "cgroup.procs" file of the
872 When delegating a sub-hierarchy, write access to this file
876 A read-only space separated values file which exists on all
883 A read-write space separated values file which exists on all
890 Space separated list of controllers prefixed with '+' or '-'
891 can be written to enable or disable controllers. A controller
892 name prefixed with '+' enables the controller and '-'
893 disables. If a controller appears more than once on the list,
898 A read-only flat-keyed file which exists on non-root cgroups.
910 A read-write single value files. The default is "max".
917 A read-write single value files. The default is "max".
924 A read-only flat-keyed file with the following entries:
942 A read-write single value file which exists on non-root cgroups.
965 create new sub-cgroups.
968 A write-only single value file which exists in non-root cgroups.
980 the whole thread-group.
983 A read-write single value file that allowed values are "0" and "1".
987 Writing "1" to the file will re-enable the cgroup PSI accounting.
995 This may cause non-negligible overhead for some workloads when under
997 be used to disable PSI accounting in the non-leaf cgroups.
1000 A read-write nested-keyed file.
1008 .. _cgroup-v2-cpu:
1011 ---
1014 controller implements weight and absolute bandwidth limit models for
1026 the cpu controller can only be enabled when all RT processes are in
1030 before the cpu controller can be enabled.
1039 A read-only flat-keyed file.
1040 This file exists whether the controller is enabled or not.
1044 - usage_usec
1045 - user_usec
1046 - system_usec
1048 and the following five when the controller is enabled:
1050 - nr_periods
1051 - nr_throttled
1052 - throttled_usec
1053 - nr_bursts
1054 - burst_usec
1057 A read-write single value file which exists on non-root
1063 A read-write single value file which exists on non-root
1066 The nice value is in the range [-20, 19].
1075 A read-write two value file which exists on non-root cgroups.
1087 A read-write single value file which exists on non-root
1093 A read-write nested-keyed file.
1099 A read-write single value file which exists on non-root cgroups.
1114 A read-write single value file which exists on non-root cgroups.
1126 Memory section in Controllers
1127 ------
1129 The "memory" controller regulates distribution of memory. Memory is
1131 intertwining between memory usage and reclaim pressure and the
1132 stateful nature of memory, the distribution model is relatively
1135 While not completely water-tight, all major memory usages by a given
1136 cgroup are tracked so that the total memory consumption can be
1138 following types of memory usages are tracked.
1140 - Userland memory - page cache and anonymous memory.
1142 - Kernel data structures such as dentries and inodes.
1144 - TCP socket buffers.
1149 Memory Interface Files argument
1152 All memory amounts are in bytes. If a value which is not aligned to
1156 memory.current
1157 A read-only single value file which exists on non-root
1160 The total amount of memory currently being used by the cgroup
1163 memory.min
1164 A read-write single value file which exists on non-root
1167 Hard memory protection. If the memory usage of a cgroup
1168 is within its effective min boundary, the cgroup's memory
1170 unprotected reclaimable memory available, OOM killer
1176 Effective min boundary is limited by memory.min values of
1177 all ancestor cgroups. If there is memory.min overcommitment
1178 (child cgroup or cgroups are requiring more protected memory
1181 actual memory usage below memory.min.
1183 Putting more memory than generally available under this
1186 If a memory cgroup is not populated with processes,
1187 its memory.min is ignored.
1189 memory.low
1190 A read-write single value file which exists on non-root
1193 Best-effort memory protection. If the memory usage of a
1195 memory won't be reclaimed unless there is no reclaimable
1196 memory available in unprotected cgroups.
1202 Effective low boundary is limited by memory.low values of
1203 all ancestor cgroups. If there is memory.low overcommitment
1204 (child cgroup or cgroups are requiring more protected memory
1207 actual memory usage below memory.low.
1209 Putting more memory than generally available under this
1212 memory.high
1213 A read-write single value file which exists on non-root
1216 Memory usage throttle limit. If a cgroup's usage goes
1226 memory.max
1227 A read-write single value file which exists on non-root
1230 Memory usage hard limit. This is the main mechanism to limit
1231 memory usage of a cgroup. If a cgroup's memory usage reaches
1236 In default configuration regular 0-order allocations always
1241 as -ENOMEM or silently ignore in cases like disk readahead.
1243 memory.reclaim
1244 A write-only nested-keyed file which exists for all cgroups.
1246 This is a simple interface to trigger memory reclaim in the
1254 echo "1G" > memory.reclaim
1258 type of memory to reclaim from (anon, file, ..).
1262 specified amount, -EAGAIN is returned.
1265 interface) is not meant to indicate memory pressure on the
1266 memory cgroup. Therefore socket memory balancing triggered by
1267 the memory reclaim normally is not exercised in this case.
1269 reclaim induced by memory.reclaim.
1271 memory.peak
1272 A read-only single value file which exists on non-root
1275 The max memory usage recorded for the cgroup and its
1278 memory.oom.group
1279 A read-write single value file which exists on non-root
1285 (if the memory cgroup is not a leaf cgroup) are killed
1289 Tasks with the OOM protection (oom_score_adj set to -1000)
1294 memory.oom.group values of ancestor cgroups.
1296 memory.events
1297 A read-only flat-keyed file which exists on non-root cgroups.
1305 memory.events.local.
1309 high memory pressure even though its usage is under
1311 boundary is over-committed.
1315 throttled and routed to perform direct memory reclaim
1316 because the high memory boundary was exceeded. For a
1317 cgroup whose memory usage is capped by the high limit
1318 rather than global memory pressure, this event's
1322 The number of times the cgroup's memory usage was
1327 The number of time the cgroup's memory usage was
1331 considered as an option, e.g. for failed high-order
1341 memory.events.local
1342 Similar to memory.events but the fields in the file are local
1346 memory.stat
1347 A read-only flat-keyed file which exists on non-root cgroups.
1349 This breaks down the cgroup's memory footprint into different
1350 types of memory, type-specific details, and other information
1351 on the state and past events of the memory management system.
1353 All memory amounts are in bytes.
1359 If the entry has no per-node counter (or not show in the
1360 memory.numa_stat). We use 'npn' (non-per-node) as the tag
1361 to indicate that it will not show in the memory.numa_stat.
1364 Amount of memory used in anonymous mappings such as
1368 Amount of memory used to cache filesystem data,
1369 including tmpfs and shared memory.
1372 Amount of total kernel memory, including
1374 addition to other kernel memory use cases.
1377 Amount of memory allocated to kernel stacks.
1380 Amount of memory allocated for page tables.
1383 Amount of memory allocated for secondary page tables,
1388 Amount of memory used for storing per-cpu kernel
1392 Amount of memory used in network transmission buffers
1395 Amount of memory used for vmap backed memory.
1398 Amount of cached filesystem data that is swap-backed,
1402 Amount of memory consumed by the zswap compression backend.
1405 Amount of application memory swapped out to zswap.
1419 Amount of swap cached in memory. The swapcache is accounted
1420 against both memory and swap usage.
1423 Amount of memory used in anonymous mappings backed by
1435 Amount of memory, swap-backed and filesystem-backed,
1436 on the internal memory management lists used by the
1440 memory management lists), inactive_foo + active_foo may not be equal to
1441 the value for the foo counter, since the foo counter is type-based, not
1442 list-based.
1449 Part of "slab" that cannot be reclaimed on memory
1453 Amount of memory used for storing in-kernel data
1520 Amount of pages postponed to be freed under memory pressure
1535 memory.numa_stat
1536 A read-only nested-keyed file which exists on non-root cgroups.
1538 This breaks down the cgroup's memory footprint into different
1539 types of memory, type-specific details, and other information
1540 per node on the state of the memory management system.
1548 All memory amounts are in bytes.
1550 The output format of memory.numa_stat is::
1558 The entries can refer to the memory.stat.
1560 memory.swap.current
1561 A read-only single value file which exists on non-root
1567 memory.swap.high
1568 A read-write single value file which exists on non-root
1573 allow userspace to implement custom out-of-memory procedures.
1577 during regular operation. Compare to memory.swap.max, which
1579 continue unimpeded as long as other memory can be reclaimed.
1583 memory.swap.peak
1584 A read-only single value file which exists on non-root
1590 memory.swap.max
1591 A read-write single value file which exists on non-root
1595 limit, anonymous memory of the cgroup will not be swapped out.
1597 memory.swap.events
1598 A read-only flat-keyed file which exists on non-root cgroups.
1614 because of running out of swap system-wide or max
1620 reduces the impact on the workload and memory management.
1622 memory.zswap.current
1623 A read-only single value file which exists on non-root
1626 The total amount of memory consumed by the zswap compression
1629 memory.zswap.max
1630 A read-write single value file which exists on non-root
1637 memory.pressure
1638 A read-only nested-keyed file.
1640 Shows pressure stall information for memory. See
1647 "memory.high" is the main mechanism to control memory usage.
1648 Over-committing on high limit (sum of high limits > available memory)
1649 and letting global memory pressure to distribute memory according to
1655 more memory or terminating the workload.
1657 Determining whether a cgroup has enough memory is not trivial as
1658 memory usage doesn't indicate whether the workload can benefit from
1659 more memory. For example, a workload which writes data received from
1660 network to a file can use all available memory but can also operate as
1661 performant with a small amount of memory. A measure of memory
1662 pressure - how much the workload is being impacted due to lack of
1663 memory - is necessary to determine whether a workload needs more
1664 memory; unfortunately, memory pressure monitoring mechanism isn't
1668 Memory Ownership argument
1671 A memory area is charged to the cgroup which instantiated it and stays
1673 to a different cgroup doesn't move the memory usages that it
1676 A memory area may be used by processes belonging to different cgroups.
1677 To which cgroup the area will be charged is in-deterministic; however,
1678 over time, the memory area is likely to end up in a cgroup which has
1679 enough memory allowance to avoid high reclaim pressure.
1681 If a cgroup sweeps a considerable amount of memory which is expected
1683 POSIX_FADV_DONTNEED to relinquish the ownership of memory areas
1684 belonging to the affected files to ensure correct memory ownership.
1688 --
1690 The "io" controller regulates the distribution of IO resources. This
1691 controller implements both weight based and absolute bandwidth or IOPS
1693 only if cfq-iosched is in use and neither scheme is available for
1694 blk-mq devices.
1701 A read-only nested-keyed file.
1721 A read-write nested-keyed file which exists only on the root
1725 model based controller (CONFIG_BLK_CGROUP_IOCOST) which
1733 enable Weight-based control enable
1743 The controller is disabled by default and can be enabled by
1745 to zero and the controller uses internal device saturation
1753 shows that on sdb, the controller is enabled, will consider
1765 devices which show wide temporary behavior changes - e.g. a
1776 A read-write nested-keyed file which exists only on the root
1780 controller (CONFIG_BLK_CGROUP_IOCOST) which currently
1789 model The cost model in use - "linear"
1815 generate device-specific coefficients.
1818 A read-write flat-keyed file which exists on non-root cgroups.
1838 A read-write nested-keyed file which exists on non-root
1852 When writing, any number of nested key-value pairs can be
1877 A read-only nested-keyed file.
1888 mechanism. Writeback sits between the memory and IO domains and
1889 regulates the proportion of dirty memory by balancing dirtying and
1892 The io controller, in conjunction with the memory controller,
1893 implements control of page cache writeback IOs. The memory controller
1894 defines the memory domain that dirty memory ratio is calculated and
1895 maintained for and the io controller defines the io domain which
1896 writes out dirty pages for the memory domain. Both system-wide and
1897 per-cgroup dirty memory states are examined and the more restrictive
1905 There are inherent differences in memory and writeback management
1906 which affects how cgroup ownership is tracked. Memory is tracked per
1911 As cgroup ownership for memory is tracked per page, there can be pages
1923 As memory controller assigns page ownership on the first use and
1934 amount of available memory capped by limits imposed by the
1935 memory controller and system-wide clean memory.
1939 total available memory and applied the same way as
1946 This is a cgroup v2 controller for IO workload protection. You provide a group
1948 controller will throttle any peers that have a lower latency target than the
1968 your real setting, setting at 10-15% higher than the value in io.stat.
1974 target the controller doesn't do anything. Once a group starts missing its
1978 - Queue depth throttling. This is the number of outstanding IO's a group is
1982 - Artificial delay induction. There are certain types of IO that cannot be
2005 If the controller is enabled you will see extra stats in io.stat in
2029 no-change
2032 promote-to-rt
2033 For requests that have a non-RT I/O priority class, change it into RT.
2037 restrict-to-be
2047 none-to-rt
2048 Deprecated. Just an alias for promote-to-rt.
2052 +----------------+---+
2053 | no-change | 0 |
2054 +----------------+---+
2055 | rt-to-be | 2 |
2056 +----------------+---+
2057 | all-to-idle | 3 |
2058 +----------------+---+
2062 +-------------------------------+---+
2064 +-------------------------------+---+
2065 | IOPRIO_CLASS_RT (real-time) | 1 |
2066 +-------------------------------+---+
2068 +-------------------------------+---+
2070 +-------------------------------+---+
2074 - If I/O priority class policy is promote-to-rt, change the request I/O
2077 - If I/O priorityt class is not promote-to-rt, translate the I/O priority
2083 ---
2085 The process number controller is used to allow a cgroup to stop any
2090 controllers cannot prevent, thus warranting its own controller. For
2092 hitting memory restrictions.
2094 Note that PIDs used in this controller refer to TIDs, process IDs as
2102 A read-write single value file which exists on non-root
2108 A read-only single value file which exists on all cgroups.
2118 through fork() or clone(). These will return -EAGAIN if the creation
2123 ------
2125 The "cpuset" controller provides a mechanism for constraining
2126 the CPU and memory node placement of tasks to only the resources
2130 memory placement to reduce cross-node memory access and contention
2133 The "cpuset" controller is hierarchical. That means the controller
2134 cannot use CPUs or memory nodes not allowed in its parent.
2141 A read-write multiple values file which exists on non-root
2142 cpuset-enabled cgroups.
2149 The CPU numbers are comma-separated numbers or ranges.
2153 0-4,6,8-10
2156 setting as the nearest cgroup ancestor with a non-empty
2163 A read-only multiple values file which exists on all
2164 cpuset-enabled cgroups.
2180 A read-write multiple values file which exists on non-root
2181 cpuset-enabled cgroups.
2183 It lists the requested memory nodes to be used by tasks within
2184 this cgroup. The actual list of memory nodes granted, however,
2186 from the requested memory nodes.
2188 The memory node numbers are comma-separated numbers or ranges.
2192 0-1,3
2195 setting as the nearest cgroup ancestor with a non-empty
2196 "cpuset.mems" or all the available memory nodes if none
2200 and won't be affected by any memory nodes hotplug events.
2202 Setting a non-empty value to "cpuset.mems" causes memory of
2204 they are currently using memory outside of the designated nodes.
2206 There is a cost for this memory migration. The migration
2207 may not be complete and some memory pages may be left behind.
2214 A read-only multiple values file which exists on all
2215 cpuset-enabled cgroups.
2217 It lists the onlined memory nodes that are actually granted to
2218 this cgroup by its parent. These memory nodes are allowed to
2221 If "cpuset.mems" is empty, it shows all the memory nodes from the
2224 the memory nodes listed in "cpuset.mems" can be granted. In this
2227 Its value will be affected by memory nodes hotplug events.
2230 A read-write single value file which exists on non-root
2231 cpuset-enabled cgroups. This flag is owned by the parent cgroup
2237 "member" Non-root member of a partition
2243 cannot be changed. All other non-root cgroups start out as
2263 two possible states - valid or invalid. An invalid partition
2274 "member" Non-root member of a partition
2306 A valid non-root parent partition may distribute out all its CPUs
2325 Device controller
2326 -----------------
2328 Device controller manages access to device files. It includes both
2332 Cgroup v2 device controller has no interface files and is implemented
2337 on the return value the attempt will succeed or fail with -EPERM.
2342 If the program returns 0, the attempt fails with -EPERM, otherwise it
2350 ----
2352 The "rdma" controller regulates the distribution and accounting of
2359 A readwrite nested-keyed file that exists for all the cgroups
2380 A read-only file that describes current resource usage.
2389 -------
2391 The HugeTLB controller allows to limit the HugeTLB usage per control group and
2392 enforces the controller limit during page fault.
2406 A read-only flat-keyed file which exists on non-root cgroups.
2417 Similar to memory.numa_stat, it shows the numa information of the
2419 use hugetlb pages are included. The per-node values are in bytes.
2422 ----
2426 cgroup resources. Controller is enabled by the CONFIG_CGROUP_MISC config
2429 A resource can be added to the controller via enum misc_res_type{} in the
2435 uncharge APIs. All of the APIs to interact with misc controller are in
2441 Miscellaneous controller provides 3 interface files. If two misc resources (res_a and res_b) are re…
2444 A read-only flat-keyed file shown only in the root cgroup. It shows
2453 A read-only flat-keyed file shown in the all cgroups. It shows
2461 A read-write flat-keyed file shown in the non root cgroups. Allowed
2480 A read-only flat-keyed file which exists on non-root cgroups. The
2498 ------
2503 perf_event controller, if not mounted on a legacy hierarchy, is
2505 always be filtered by cgroup v2 path. The controller can still be
2509 Non-normative information
2510 -------------------------
2516 CPU controller root cgroup process behaviour
2526 appropriately so the neutral - nice 0 - value is 100 instead of 1024).
2529 IO controller root cgroup process behaviour
2542 ------
2561 The path '/batchjobs/container_id1' can be considered as system-data
2566 # ls -l /proc/self/ns/cgroup
2567 lrwxrwxrwx 1 root root 0 2014-07-15 10:37 /proc/self/ns/cgroup -> cgroup:[4026531835]
2573 # ls -l /proc/self/ns/cgroup
2574 lrwxrwxrwx 1 root root 0 2014-07-15 10:35 /proc/self/ns/cgroup -> cgroup:[4026532183]
2578 When some thread from a multi-threaded process unshares its cgroup
2590 ------------------
2601 # ~/unshare -c # unshare cgroupns in some cgroup
2609 Each process gets its namespace-specific view of "/proc/$PID/cgroup"
2640 ----------------------
2669 ---------------------------------
2672 running inside a non-init cgroup namespace::
2674 # mount -t cgroup2 none $MOUNT_POINT
2681 the view of cgroup hierarchy by namespace-private cgroupfs mount
2694 --------------------------------
2697 address_space_operations->writepage[s]() to annotate bio's using the
2714 super_block by setting SB_I_CGROUPWB in ->s_iflags. This allows for
2731 - Multiple hierarchies including named ones are not supported.
2733 - All v1 mount options are not supported.
2735 - The "tasks" file is removed and "cgroup.procs" is not sorted.
2737 - "cgroup.clone_children" is removed.
2739 - /proc/cgroups is meaningless for v2. Use "cgroup.controllers" file
2747 --------------------
2753 For example, as there is only one instance of each controller, utility
2760 the specific controller.
2764 each controller on its own hierarchy. Only closely related ones, such
2783 Also, as a controller couldn't have any expectation regarding the
2785 controller had to assume that all other controllers were attached to
2792 depending on the specific controller. In other words, hierarchy may
2795 how memory is distributed beyond a certain level while still wanting
2800 ------------------
2808 Generally, in-process knowledge is available only to the process
2809 itself; thus, unlike service-level organization of processes,
2816 sub-hierarchies and control resource distributions along them. This
2817 effectively raised cgroup to the status of a syscall-like API exposed
2827 that the process would actually be operating on its own sub-hierarchy.
2831 system-management pseudo filesystem. cgroup ended up with interface
2834 individual applications through the ill-defined delegation mechanism
2844 -------------------------------------------
2852 The cpu controller considered threads and cgroups as equivalents and
2855 cycles and the number of internal threads fluctuated - the ratios
2861 The io controller implicitly created a hidden leaf node for each
2869 The memory controller didn't have a way to control what happened
2871 clearly defined. There were attempts to add ad-hoc behaviors and
2885 ----------------------
2889 was how an empty cgroup was notified - a userland helper binary was
2892 to in-kernel event delivery filtering mechanism further complicating
2895 Controller interfaces were problematic too. An extreme example is
2907 formats and units even in the same controller.
2913 Controller Issues and Remedies
2914 ------------------------------
2916 Memory subsection
2921 global reclaim prefers is opt-in, rather than opt-out. The costs for
2931 becomes self-defeating.
2933 The memory.low boundary on the other hand is a top-down allocated
2942 available memory. The memory consumption of workloads varies during
2950 The memory.high boundary on the other hand can be set much more
2956 and make corrections until the minimal memory footprint that still
2963 system than killing the group. Otherwise, memory.max is there to
2967 Setting the original memory.limit_in_bytes below the current usage was
2969 limit setting to fail. memory.max on the other hand will first set the
2971 new limit is met - or the task writing to memory.max is killed.
2973 The combined memory+swap accounting and limiting is replaced by real
2976 The main argument for a combined memory+swap facility in the original
2978 able to swap all anonymous memory of a child group, regardless of the
2980 groups can sabotage swapping by other means - such as referencing its
2981 anonymous memory in a tight loop - and an admin can not assume full