1.. _admin_guide_ksm:
2
3=======================
4Kernel Samepage Merging
5=======================
6
7Overview
8========
9
10KSM is a memory-saving de-duplication feature, enabled by CONFIG_KSM=y,
11added to the Linux kernel in 2.6.32.  See ``mm/ksm.c`` for its implementation,
12and http://lwn.net/Articles/306704/ and https://lwn.net/Articles/330589/
13
14KSM was originally developed for use with KVM (where it was known as
15Kernel Shared Memory), to fit more virtual machines into physical memory,
16by sharing the data common between them.  But it can be useful to any
17application which generates many instances of the same data.
18
19The KSM daemon ksmd periodically scans those areas of user memory
20which have been registered with it, looking for pages of identical
21content which can be replaced by a single write-protected page (which
22is automatically copied if a process later wants to update its
23content). The amount of pages that KSM daemon scans in a single pass
24and the time between the passes are configured using :ref:`sysfs
25intraface <ksm_sysfs>`
26
27KSM only merges anonymous (private) pages, never pagecache (file) pages.
28KSM's merged pages were originally locked into kernel memory, but can now
29be swapped out just like other user pages (but sharing is broken when they
30are swapped back in: ksmd must rediscover their identity and merge again).
31
32Controlling KSM with madvise
33============================
34
35KSM only operates on those areas of address space which an application
36has advised to be likely candidates for merging, by using the madvise(2)
37system call::
38
39	int madvise(addr, length, MADV_MERGEABLE)
40
41The app may call
42
43::
44
45	int madvise(addr, length, MADV_UNMERGEABLE)
46
47to cancel that advice and restore unshared pages: whereupon KSM
48unmerges whatever it merged in that range.  Note: this unmerging call
49may suddenly require more memory than is available - possibly failing
50with EAGAIN, but more probably arousing the Out-Of-Memory killer.
51
52If KSM is not configured into the running kernel, madvise MADV_MERGEABLE
53and MADV_UNMERGEABLE simply fail with EINVAL.  If the running kernel was
54built with CONFIG_KSM=y, those calls will normally succeed: even if the
55KSM daemon is not currently running, MADV_MERGEABLE still registers
56the range for whenever the KSM daemon is started; even if the range
57cannot contain any pages which KSM could actually merge; even if
58MADV_UNMERGEABLE is applied to a range which was never MADV_MERGEABLE.
59
60If a region of memory must be split into at least one new MADV_MERGEABLE
61or MADV_UNMERGEABLE region, the madvise may return ENOMEM if the process
62will exceed ``vm.max_map_count`` (see Documentation/admin-guide/sysctl/vm.rst).
63
64Like other madvise calls, they are intended for use on mapped areas of
65the user address space: they will report ENOMEM if the specified range
66includes unmapped gaps (though working on the intervening mapped areas),
67and might fail with EAGAIN if not enough memory for internal structures.
68
69Applications should be considerate in their use of MADV_MERGEABLE,
70restricting its use to areas likely to benefit.  KSM's scans may use a lot
71of processing power: some installations will disable KSM for that reason.
72
73.. _ksm_sysfs:
74
75KSM daemon sysfs interface
76==========================
77
78The KSM daemon is controlled by sysfs files in ``/sys/kernel/mm/ksm/``,
79readable by all but writable only by root:
80
81pages_to_scan
82        how many pages to scan before ksmd goes to sleep
83        e.g. ``echo 100 > /sys/kernel/mm/ksm/pages_to_scan``.
84
85        Default: 100 (chosen for demonstration purposes)
86
87sleep_millisecs
88        how many milliseconds ksmd should sleep before next scan
89        e.g. ``echo 20 > /sys/kernel/mm/ksm/sleep_millisecs``
90
91        Default: 20 (chosen for demonstration purposes)
92
93merge_across_nodes
94        specifies if pages from different NUMA nodes can be merged.
95        When set to 0, ksm merges only pages which physically reside
96        in the memory area of same NUMA node. That brings lower
97        latency to access of shared pages. Systems with more nodes, at
98        significant NUMA distances, are likely to benefit from the
99        lower latency of setting 0. Smaller systems, which need to
100        minimize memory usage, are likely to benefit from the greater
101        sharing of setting 1 (default). You may wish to compare how
102        your system performs under each setting, before deciding on
103        which to use. ``merge_across_nodes`` setting can be changed only
104        when there are no ksm shared pages in the system: set run 2 to
105        unmerge pages first, then to 1 after changing
106        ``merge_across_nodes``, to remerge according to the new setting.
107
108        Default: 1 (merging across nodes as in earlier releases)
109
110run
111        * set to 0 to stop ksmd from running but keep merged pages,
112        * set to 1 to run ksmd e.g. ``echo 1 > /sys/kernel/mm/ksm/run``,
113        * set to 2 to stop ksmd and unmerge all pages currently merged, but
114	  leave mergeable areas registered for next run.
115
116        Default: 0 (must be changed to 1 to activate KSM, except if
117        CONFIG_SYSFS is disabled)
118
119use_zero_pages
120        specifies whether empty pages (i.e. allocated pages that only
121        contain zeroes) should be treated specially.  When set to 1,
122        empty pages are merged with the kernel zero page(s) instead of
123        with each other as it would happen normally. This can improve
124        the performance on architectures with coloured zero pages,
125        depending on the workload. Care should be taken when enabling
126        this setting, as it can potentially degrade the performance of
127        KSM for some workloads, for example if the checksums of pages
128        candidate for merging match the checksum of an empty
129        page. This setting can be changed at any time, it is only
130        effective for pages merged after the change.
131
132        Default: 0 (normal KSM behaviour as in earlier releases)
133
134max_page_sharing
135        Maximum sharing allowed for each KSM page. This enforces a
136        deduplication limit to avoid high latency for virtual memory
137        operations that involve traversal of the virtual mappings that
138        share the KSM page. The minimum value is 2 as a newly created
139        KSM page will have at least two sharers. The higher this value
140        the faster KSM will merge the memory and the higher the
141        deduplication factor will be, but the slower the worst case
142        virtual mappings traversal could be for any given KSM
143        page. Slowing down this traversal means there will be higher
144        latency for certain virtual memory operations happening during
145        swapping, compaction, NUMA balancing and page migration, in
146        turn decreasing responsiveness for the caller of those virtual
147        memory operations. The scheduler latency of other tasks not
148        involved with the VM operations doing the virtual mappings
149        traversal is not affected by this parameter as these
150        traversals are always schedule friendly themselves.
151
152stable_node_chains_prune_millisecs
153        specifies how frequently KSM checks the metadata of the pages
154        that hit the deduplication limit for stale information.
155        Smaller milllisecs values will free up the KSM metadata with
156        lower latency, but they will make ksmd use more CPU during the
157        scan. It's a noop if not a single KSM page hit the
158        ``max_page_sharing`` yet.
159
160The effectiveness of KSM and MADV_MERGEABLE is shown in ``/sys/kernel/mm/ksm/``:
161
162pages_shared
163        how many shared pages are being used
164pages_sharing
165        how many more sites are sharing them i.e. how much saved
166pages_unshared
167        how many pages unique but repeatedly checked for merging
168pages_volatile
169        how many pages changing too fast to be placed in a tree
170full_scans
171        how many times all mergeable areas have been scanned
172stable_node_chains
173        the number of KSM pages that hit the ``max_page_sharing`` limit
174stable_node_dups
175        number of duplicated KSM pages
176
177A high ratio of ``pages_sharing`` to ``pages_shared`` indicates good
178sharing, but a high ratio of ``pages_unshared`` to ``pages_sharing``
179indicates wasted effort.  ``pages_volatile`` embraces several
180different kinds of activity, but a high proportion there would also
181indicate poor use of madvise MADV_MERGEABLE.
182
183The maximum possible ``pages_sharing/pages_shared`` ratio is limited by the
184``max_page_sharing`` tunable. To increase the ratio ``max_page_sharing`` must
185be increased accordingly.
186
187--
188Izik Eidus,
189Hugh Dickins, 17 Nov 2009
190