1.. SPDX-License-Identifier: GPL-2.0
2
3===============
4Detailed Usages
5===============
6
7DAMON provides below interfaces for different users.
8
9- *DAMON user space tool.*
10  `This <https://github.com/awslabs/damo>`_ is for privileged people such as
11  system administrators who want a just-working human-friendly interface.
12  Using this, users can use the DAMON’s major features in a human-friendly way.
13  It may not be highly tuned for special cases, though.  For more detail,
14  please refer to its `usage document
15  <https://github.com/awslabs/damo/blob/next/USAGE.md>`_.
16- *sysfs interface.*
17  :ref:`This <sysfs_interface>` is for privileged user space programmers who
18  want more optimized use of DAMON.  Using this, users can use DAMON’s major
19  features by reading from and writing to special sysfs files.  Therefore,
20  you can write and use your personalized DAMON sysfs wrapper programs that
21  reads/writes the sysfs files instead of you.  The `DAMON user space tool
22  <https://github.com/awslabs/damo>`_ is one example of such programs.
23- *debugfs interface. (DEPRECATED!)*
24  :ref:`This <debugfs_interface>` is almost identical to :ref:`sysfs interface
25  <sysfs_interface>`.  This is deprecated, so users should move to the
26  :ref:`sysfs interface <sysfs_interface>`.  If you depend on this and cannot
27  move, please report your usecase to damon@lists.linux.dev and
28  linux-mm@kvack.org.
29- *Kernel Space Programming Interface.*
30  :doc:`This </mm/damon/api>` is for kernel space programmers.  Using this,
31  users can utilize every feature of DAMON most flexibly and efficiently by
32  writing kernel space DAMON application programs for you.  You can even extend
33  DAMON for various address spaces.  For detail, please refer to the interface
34  :doc:`document </mm/damon/api>`.
35
36.. _sysfs_interface:
37
38sysfs Interface
39===============
40
41DAMON sysfs interface is built when ``CONFIG_DAMON_SYSFS`` is defined.  It
42creates multiple directories and files under its sysfs directory,
43``<sysfs>/kernel/mm/damon/``.  You can control DAMON by writing to and reading
44from the files under the directory.
45
46For a short example, users can monitor the virtual address space of a given
47workload as below. ::
48
49    # cd /sys/kernel/mm/damon/admin/
50    # echo 1 > kdamonds/nr_kdamonds && echo 1 > kdamonds/0/contexts/nr_contexts
51    # echo vaddr > kdamonds/0/contexts/0/operations
52    # echo 1 > kdamonds/0/contexts/0/targets/nr_targets
53    # echo $(pidof <workload>) > kdamonds/0/contexts/0/targets/0/pid_target
54    # echo on > kdamonds/0/state
55
56Files Hierarchy
57---------------
58
59The files hierarchy of DAMON sysfs interface is shown below.  In the below
60figure, parents-children relations are represented with indentations, each
61directory is having ``/`` suffix, and files in each directory are separated by
62comma (","). ::
63
64    /sys/kernel/mm/damon/admin
65kdamonds/nr_kdamonds
66    │ │ 0/state,pid
67    │ │ │ contexts/nr_contexts
68    │ │ │ │ 0/avail_operations,operations
69    │ │ │ │ │ monitoring_attrs/
70    │ │ │ │ │ │ intervals/sample_us,aggr_us,update_us
71    │ │ │ │ │ │ nr_regions/min,max
72    │ │ │ │ │ targets/nr_targets
73    │ │ │ │ │ │ 0/pid_target
74    │ │ │ │ │ │ │ regions/nr_regions
75    │ │ │ │ │ │ │ │ 0/start,end
76    │ │ │ │ │ │ │ │ ...
77    │ │ │ │ │ │ ...
78    │ │ │ │ │ schemes/nr_schemes
79    │ │ │ │ │ │ 0/action
80    │ │ │ │ │ │ │ access_pattern/
81    │ │ │ │ │ │ │ │ sz/min,max
82    │ │ │ │ │ │ │ │ nr_accesses/min,max
83    │ │ │ │ │ │ │ │ age/min,max
84    │ │ │ │ │ │ │ quotas/ms,bytes,reset_interval_ms
85    │ │ │ │ │ │ │ │ weights/sz_permil,nr_accesses_permil,age_permil
86    │ │ │ │ │ │ │ watermarks/metric,interval_us,high,mid,low
87    │ │ │ │ │ │ │ filters/nr_filters
88    │ │ │ │ │ │ │ │ 0/type,matching,memcg_id
89    │ │ │ │ │ │ │ stats/nr_tried,sz_tried,nr_applied,sz_applied,qt_exceeds
90    │ │ │ │ │ │ │ tried_regions/total_bytes
91    │ │ │ │ │ │ │ │ 0/start,end,nr_accesses,age
92    │ │ │ │ │ │ │ │ ...
93    │ │ │ │ │ │ ...
94    │ │ │ │ ...
95    │ │ ...
96
97Root
98----
99
100The root of the DAMON sysfs interface is ``<sysfs>/kernel/mm/damon/``, and it
101has one directory named ``admin``.  The directory contains the files for
102privileged user space programs' control of DAMON.  User space tools or daemons
103having the root permission could use this directory.
104
105kdamonds/
106---------
107
108The monitoring-related information including request specifications and results
109are called DAMON context.  DAMON executes each context with a kernel thread
110called kdamond, and multiple kdamonds could run in parallel.
111
112Under the ``admin`` directory, one directory, ``kdamonds``, which has files for
113controlling the kdamonds exist.  In the beginning, this directory has only one
114file, ``nr_kdamonds``.  Writing a number (``N``) to the file creates the number
115of child directories named ``0`` to ``N-1``.  Each directory represents each
116kdamond.
117
118kdamonds/<N>/
119-------------
120
121In each kdamond directory, two files (``state`` and ``pid``) and one directory
122(``contexts``) exist.
123
124Reading ``state`` returns ``on`` if the kdamond is currently running, or
125``off`` if it is not running.  Writing ``on`` or ``off`` makes the kdamond be
126in the state.  Writing ``commit`` to the ``state`` file makes kdamond reads the
127user inputs in the sysfs files except ``state`` file again.  Writing
128``update_schemes_stats`` to ``state`` file updates the contents of stats files
129for each DAMON-based operation scheme of the kdamond.  For details of the
130stats, please refer to :ref:`stats section <sysfs_schemes_stats>`.
131
132Writing ``update_schemes_tried_regions`` to ``state`` file updates the
133DAMON-based operation scheme action tried regions directory for each
134DAMON-based operation scheme of the kdamond.  Writing
135``update_schemes_tried_bytes`` to ``state`` file updates only
136``.../tried_regions/total_bytes`` files.  Writing
137``clear_schemes_tried_regions`` to ``state`` file clears the DAMON-based
138operating scheme action tried regions directory for each DAMON-based operation
139scheme of the kdamond.  For details of the DAMON-based operation scheme action
140tried regions directory, please refer to :ref:`tried_regions section
141<sysfs_schemes_tried_regions>`.
142
143If the state is ``on``, reading ``pid`` shows the pid of the kdamond thread.
144
145``contexts`` directory contains files for controlling the monitoring contexts
146that this kdamond will execute.
147
148kdamonds/<N>/contexts/
149----------------------
150
151In the beginning, this directory has only one file, ``nr_contexts``.  Writing a
152number (``N``) to the file creates the number of child directories named as
153``0`` to ``N-1``.  Each directory represents each monitoring context.  At the
154moment, only one context per kdamond is supported, so only ``0`` or ``1`` can
155be written to the file.
156
157.. _sysfs_contexts:
158
159contexts/<N>/
160-------------
161
162In each context directory, two files (``avail_operations`` and ``operations``)
163and three directories (``monitoring_attrs``, ``targets``, and ``schemes``)
164exist.
165
166DAMON supports multiple types of monitoring operations, including those for
167virtual address space and the physical address space.  You can get the list of
168available monitoring operations set on the currently running kernel by reading
169``avail_operations`` file.  Based on the kernel configuration, the file will
170list some or all of below keywords.
171
172 - vaddr: Monitor virtual address spaces of specific processes
173 - fvaddr: Monitor fixed virtual address ranges
174 - paddr: Monitor the physical address space of the system
175
176Please refer to :ref:`regions sysfs directory <sysfs_regions>` for detailed
177differences between the operations sets in terms of the monitoring target
178regions.
179
180You can set and get what type of monitoring operations DAMON will use for the
181context by writing one of the keywords listed in ``avail_operations`` file and
182reading from the ``operations`` file.
183
184.. _sysfs_monitoring_attrs:
185
186contexts/<N>/monitoring_attrs/
187------------------------------
188
189Files for specifying attributes of the monitoring including required quality
190and efficiency of the monitoring are in ``monitoring_attrs`` directory.
191Specifically, two directories, ``intervals`` and ``nr_regions`` exist in this
192directory.
193
194Under ``intervals`` directory, three files for DAMON's sampling interval
195(``sample_us``), aggregation interval (``aggr_us``), and update interval
196(``update_us``) exist.  You can set and get the values in micro-seconds by
197writing to and reading from the files.
198
199Under ``nr_regions`` directory, two files for the lower-bound and upper-bound
200of DAMON's monitoring regions (``min`` and ``max``, respectively), which
201controls the monitoring overhead, exist.  You can set and get the values by
202writing to and rading from the files.
203
204For more details about the intervals and monitoring regions range, please refer
205to the Design document (:doc:`/mm/damon/design`).
206
207contexts/<N>/targets/
208---------------------
209
210In the beginning, this directory has only one file, ``nr_targets``.  Writing a
211number (``N``) to the file creates the number of child directories named ``0``
212to ``N-1``.  Each directory represents each monitoring target.
213
214targets/<N>/
215------------
216
217In each target directory, one file (``pid_target``) and one directory
218(``regions``) exist.
219
220If you wrote ``vaddr`` to the ``contexts/<N>/operations``, each target should
221be a process.  You can specify the process to DAMON by writing the pid of the
222process to the ``pid_target`` file.
223
224.. _sysfs_regions:
225
226targets/<N>/regions
227-------------------
228
229When ``vaddr`` monitoring operations set is being used (``vaddr`` is written to
230the ``contexts/<N>/operations`` file), DAMON automatically sets and updates the
231monitoring target regions so that entire memory mappings of target processes
232can be covered.  However, users could want to set the initial monitoring region
233to specific address ranges.
234
235In contrast, DAMON do not automatically sets and updates the monitoring target
236regions when ``fvaddr`` or ``paddr`` monitoring operations sets are being used
237(``fvaddr`` or ``paddr`` have written to the ``contexts/<N>/operations``).
238Therefore, users should set the monitoring target regions by themselves in the
239cases.
240
241For such cases, users can explicitly set the initial monitoring target regions
242as they want, by writing proper values to the files under this directory.
243
244In the beginning, this directory has only one file, ``nr_regions``.  Writing a
245number (``N``) to the file creates the number of child directories named ``0``
246to ``N-1``.  Each directory represents each initial monitoring target region.
247
248regions/<N>/
249------------
250
251In each region directory, you will find two files (``start`` and ``end``).  You
252can set and get the start and end addresses of the initial monitoring target
253region by writing to and reading from the files, respectively.
254
255Each region should not overlap with others.  ``end`` of directory ``N`` should
256be equal or smaller than ``start`` of directory ``N+1``.
257
258contexts/<N>/schemes/
259---------------------
260
261The directory for DAMON-based Operation Schemes (:ref:`DAMOS
262<damon_design_damos>`).  Users can get and set the schemes by reading from and
263writing to files under this directory.
264
265In the beginning, this directory has only one file, ``nr_schemes``.  Writing a
266number (``N``) to the file creates the number of child directories named ``0``
267to ``N-1``.  Each directory represents each DAMON-based operation scheme.
268
269schemes/<N>/
270------------
271
272In each scheme directory, five directories (``access_pattern``, ``quotas``,
273``watermarks``, ``filters``, ``stats``, and ``tried_regions``) and one file
274(``action``) exist.
275
276The ``action`` file is for setting and getting the scheme's :ref:`action
277<damon_design_damos_action>`.  The keywords that can be written to and read
278from the file and their meaning are as below.
279
280Note that support of each action depends on the running DAMON operations set
281:ref:`implementation <sysfs_contexts>`.
282
283 - ``willneed``: Call ``madvise()`` for the region with ``MADV_WILLNEED``.
284   Supported by ``vaddr`` and ``fvaddr`` operations set.
285 - ``cold``: Call ``madvise()`` for the region with ``MADV_COLD``.
286   Supported by ``vaddr`` and ``fvaddr`` operations set.
287 - ``pageout``: Call ``madvise()`` for the region with ``MADV_PAGEOUT``.
288   Supported by ``vaddr``, ``fvaddr`` and ``paddr`` operations set.
289 - ``hugepage``: Call ``madvise()`` for the region with ``MADV_HUGEPAGE``.
290   Supported by ``vaddr`` and ``fvaddr`` operations set.
291 - ``nohugepage``: Call ``madvise()`` for the region with ``MADV_NOHUGEPAGE``.
292   Supported by ``vaddr`` and ``fvaddr`` operations set.
293 - ``lru_prio``: Prioritize the region on its LRU lists.
294   Supported by ``paddr`` operations set.
295 - ``lru_deprio``: Deprioritize the region on its LRU lists.
296   Supported by ``paddr`` operations set.
297 - ``stat``: Do nothing but count the statistics.
298   Supported by all operations sets.
299
300schemes/<N>/access_pattern/
301---------------------------
302
303The directory for the target access :ref:`pattern
304<damon_design_damos_access_pattern>` of the given DAMON-based operation scheme.
305
306Under the ``access_pattern`` directory, three directories (``sz``,
307``nr_accesses``, and ``age``) each having two files (``min`` and ``max``)
308exist.  You can set and get the access pattern for the given scheme by writing
309to and reading from the ``min`` and ``max`` files under ``sz``,
310``nr_accesses``, and ``age`` directories, respectively.  Note that the ``min``
311and the ``max`` form a closed interval.
312
313schemes/<N>/quotas/
314-------------------
315
316The directory for the :ref:`quotas <damon_design_damos_quotas>` of the given
317DAMON-based operation scheme.
318
319Under ``quotas`` directory, three files (``ms``, ``bytes``,
320``reset_interval_ms``) and one directory (``weights``) having three files
321(``sz_permil``, ``nr_accesses_permil``, and ``age_permil``) in it exist.
322
323You can set the ``time quota`` in milliseconds, ``size quota`` in bytes, and
324``reset interval`` in milliseconds by writing the values to the three files,
325respectively.  Then, DAMON tries to use only up to ``time quota`` milliseconds
326for applying the ``action`` to memory regions of the ``access_pattern``, and to
327apply the action to only up to ``bytes`` bytes of memory regions within the
328``reset_interval_ms``.  Setting both ``ms`` and ``bytes`` zero disables the
329quota limits.
330
331You can also set the :ref:`prioritization weights
332<damon_design_damos_quotas_prioritization>` for size, access frequency, and age
333in per-thousand unit by writing the values to the three files under the
334``weights`` directory.
335
336schemes/<N>/watermarks/
337-----------------------
338
339The directory for the :ref:`watermarks <damon_design_damos_watermarks>` of the
340given DAMON-based operation scheme.
341
342Under the watermarks directory, five files (``metric``, ``interval_us``,
343``high``, ``mid``, and ``low``) for setting the metric, the time interval
344between check of the metric, and the three watermarks exist.  You can set and
345get the five values by writing to the files, respectively.
346
347Keywords and meanings of those that can be written to the ``metric`` file are
348as below.
349
350 - none: Ignore the watermarks
351 - free_mem_rate: System's free memory rate (per thousand)
352
353The ``interval`` should written in microseconds unit.
354
355schemes/<N>/filters/
356--------------------
357
358The directory for the :ref:`filters <damon_design_damos_filters>` of the given
359DAMON-based operation scheme.
360
361In the beginning, this directory has only one file, ``nr_filters``.  Writing a
362number (``N``) to the file creates the number of child directories named ``0``
363to ``N-1``.  Each directory represents each filter.  The filters are evaluated
364in the numeric order.
365
366Each filter directory contains six files, namely ``type``, ``matcing``,
367``memcg_path``, ``addr_start``, ``addr_end``, and ``target_idx``.  To ``type``
368file, you can write one of four special keywords: ``anon`` for anonymous pages,
369``memcg`` for specific memory cgroup, ``addr`` for specific address range (an
370open-ended interval), or ``target`` for specific DAMON monitoring target
371filtering.  In case of the memory cgroup filtering, you can specify the memory
372cgroup of the interest by writing the path of the memory cgroup from the
373cgroups mount point to ``memcg_path`` file.  In case of the address range
374filtering, you can specify the start and end address of the range to
375``addr_start`` and ``addr_end`` files, respectively.  For the DAMON monitoring
376target filtering, you can specify the index of the target between the list of
377the DAMON context's monitoring targets list to ``target_idx`` file.  You can
378write ``Y`` or ``N`` to ``matching`` file to filter out pages that does or does
379not match to the type, respectively.  Then, the scheme's action will not be
380applied to the pages that specified to be filtered out.
381
382For example, below restricts a DAMOS action to be applied to only non-anonymous
383pages of all memory cgroups except ``/having_care_already``.::
384
385    # echo 2 > nr_filters
386    # # filter out anonymous pages
387    echo anon > 0/type
388    echo Y > 0/matching
389    # # further filter out all cgroups except one at '/having_care_already'
390    echo memcg > 1/type
391    echo /having_care_already > 1/memcg_path
392    echo N > 1/matching
393
394Note that ``anon`` and ``memcg`` filters are currently supported only when
395``paddr`` `implementation <sysfs_contexts>` is being used.
396
397Also, memory regions that are filtered out by ``addr`` or ``target`` filters
398are not counted as the scheme has tried to those, while regions that filtered
399out by other type filters are counted as the scheme has tried to.  The
400difference is applied to :ref:`stats <damos_stats>` and
401:ref:`tried regions <sysfs_schemes_tried_regions>`.
402
403.. _sysfs_schemes_stats:
404
405schemes/<N>/stats/
406------------------
407
408DAMON counts the total number and bytes of regions that each scheme is tried to
409be applied, the two numbers for the regions that each scheme is successfully
410applied, and the total number of the quota limit exceeds.  This statistics can
411be used for online analysis or tuning of the schemes.
412
413The statistics can be retrieved by reading the files under ``stats`` directory
414(``nr_tried``, ``sz_tried``, ``nr_applied``, ``sz_applied``, and
415``qt_exceeds``), respectively.  The files are not updated in real time, so you
416should ask DAMON sysfs interface to update the content of the files for the
417stats by writing a special keyword, ``update_schemes_stats`` to the relevant
418``kdamonds/<N>/state`` file.
419
420.. _sysfs_schemes_tried_regions:
421
422schemes/<N>/tried_regions/
423--------------------------
424
425This directory initially has one file, ``total_bytes``.
426
427When a special keyword, ``update_schemes_tried_regions``, is written to the
428relevant ``kdamonds/<N>/state`` file, DAMON updates the ``total_bytes`` file so
429that reading it returns the total size of the scheme tried regions, and creates
430directories named integer starting from ``0`` under this directory.  Each
431directory contains files exposing detailed information about each of the memory
432region that the corresponding scheme's ``action`` has tried to be applied under
433this directory, during next :ref:`aggregation interval
434<sysfs_monitoring_attrs>`.  The information includes address range,
435``nr_accesses``, and ``age`` of the region.
436
437Writing ``update_schemes_tried_bytes`` to the relevant ``kdamonds/<N>/state``
438file will only update the ``total_bytes`` file, and will not create the
439subdirectories.
440
441The directories will be removed when another special keyword,
442``clear_schemes_tried_regions``, is written to the relevant
443``kdamonds/<N>/state`` file.
444
445The expected usage of this directory is investigations of schemes' behaviors,
446and query-like efficient data access monitoring results retrievals.  For the
447latter use case, in particular, users can set the ``action`` as ``stat`` and
448set the ``access pattern`` as their interested pattern that they want to query.
449
450tried_regions/<N>/
451------------------
452
453In each region directory, you will find four files (``start``, ``end``,
454``nr_accesses``, and ``age``).  Reading the files will show the start and end
455addresses, ``nr_accesses``, and ``age`` of the region that corresponding
456DAMON-based operation scheme ``action`` has tried to be applied.
457
458Example
459~~~~~~~
460
461Below commands applies a scheme saying "If a memory region of size in [4KiB,
4628KiB] is showing accesses per aggregate interval in [0, 5] for aggregate
463interval in [10, 20], page out the region.  For the paging out, use only up to
46410ms per second, and also don't page out more than 1GiB per second.  Under the
465limitation, page out memory regions having longer age first.  Also, check the
466free memory rate of the system every 5 seconds, start the monitoring and paging
467out when the free memory rate becomes lower than 50%, but stop it if the free
468memory rate becomes larger than 60%, or lower than 30%". ::
469
470    # cd <sysfs>/kernel/mm/damon/admin
471    # # populate directories
472    # echo 1 > kdamonds/nr_kdamonds; echo 1 > kdamonds/0/contexts/nr_contexts;
473    # echo 1 > kdamonds/0/contexts/0/schemes/nr_schemes
474    # cd kdamonds/0/contexts/0/schemes/0
475    # # set the basic access pattern and the action
476    # echo 4096 > access_pattern/sz/min
477    # echo 8192 > access_pattern/sz/max
478    # echo 0 > access_pattern/nr_accesses/min
479    # echo 5 > access_pattern/nr_accesses/max
480    # echo 10 > access_pattern/age/min
481    # echo 20 > access_pattern/age/max
482    # echo pageout > action
483    # # set quotas
484    # echo 10 > quotas/ms
485    # echo $((1024*1024*1024)) > quotas/bytes
486    # echo 1000 > quotas/reset_interval_ms
487    # # set watermark
488    # echo free_mem_rate > watermarks/metric
489    # echo 5000000 > watermarks/interval_us
490    # echo 600 > watermarks/high
491    # echo 500 > watermarks/mid
492    # echo 300 > watermarks/low
493
494Please note that it's highly recommended to use user space tools like `damo
495<https://github.com/awslabs/damo>`_ rather than manually reading and writing
496the files as above.  Above is only for an example.
497
498.. _debugfs_interface:
499
500debugfs Interface (DEPRECATED!)
501===============================
502
503.. note::
504
505  THIS IS DEPRECATED!
506
507  DAMON debugfs interface is deprecated, so users should move to the
508  :ref:`sysfs interface <sysfs_interface>`.  If you depend on this and cannot
509  move, please report your usecase to damon@lists.linux.dev and
510  linux-mm@kvack.org.
511
512DAMON exports eight files, ``attrs``, ``target_ids``, ``init_regions``,
513``schemes``, ``monitor_on``, ``kdamond_pid``, ``mk_contexts`` and
514``rm_contexts`` under its debugfs directory, ``<debugfs>/damon/``.
515
516
517Attributes
518----------
519
520Users can get and set the ``sampling interval``, ``aggregation interval``,
521``update interval``, and min/max number of monitoring target regions by
522reading from and writing to the ``attrs`` file.  To know about the monitoring
523attributes in detail, please refer to the :doc:`/mm/damon/design`.  For
524example, below commands set those values to 5 ms, 100 ms, 1,000 ms, 10 and
5251000, and then check it again::
526
527    # cd <debugfs>/damon
528    # echo 5000 100000 1000000 10 1000 > attrs
529    # cat attrs
530    5000 100000 1000000 10 1000
531
532
533Target IDs
534----------
535
536Some types of address spaces supports multiple monitoring target.  For example,
537the virtual memory address spaces monitoring can have multiple processes as the
538monitoring targets.  Users can set the targets by writing relevant id values of
539the targets to, and get the ids of the current targets by reading from the
540``target_ids`` file.  In case of the virtual address spaces monitoring, the
541values should be pids of the monitoring target processes.  For example, below
542commands set processes having pids 42 and 4242 as the monitoring targets and
543check it again::
544
545    # cd <debugfs>/damon
546    # echo 42 4242 > target_ids
547    # cat target_ids
548    42 4242
549
550Users can also monitor the physical memory address space of the system by
551writing a special keyword, "``paddr\n``" to the file.  Because physical address
552space monitoring doesn't support multiple targets, reading the file will show a
553fake value, ``42``, as below::
554
555    # cd <debugfs>/damon
556    # echo paddr > target_ids
557    # cat target_ids
558    42
559
560Note that setting the target ids doesn't start the monitoring.
561
562
563Initial Monitoring Target Regions
564---------------------------------
565
566In case of the virtual address space monitoring, DAMON automatically sets and
567updates the monitoring target regions so that entire memory mappings of target
568processes can be covered.  However, users can want to limit the monitoring
569region to specific address ranges, such as the heap, the stack, or specific
570file-mapped area.  Or, some users can know the initial access pattern of their
571workloads and therefore want to set optimal initial regions for the 'adaptive
572regions adjustment'.
573
574In contrast, DAMON do not automatically sets and updates the monitoring target
575regions in case of physical memory monitoring.  Therefore, users should set the
576monitoring target regions by themselves.
577
578In such cases, users can explicitly set the initial monitoring target regions
579as they want, by writing proper values to the ``init_regions`` file.  The input
580should be a sequence of three integers separated by white spaces that represent
581one region in below form.::
582
583    <target idx> <start address> <end address>
584
585The ``target idx`` should be the index of the target in ``target_ids`` file,
586starting from ``0``, and the regions should be passed in address order.  For
587example, below commands will set a couple of address ranges, ``1-100`` and
588``100-200`` as the initial monitoring target region of pid 42, which is the
589first one (index ``0``) in ``target_ids``, and another couple of address
590ranges, ``20-40`` and ``50-100`` as that of pid 4242, which is the second one
591(index ``1``) in ``target_ids``.::
592
593    # cd <debugfs>/damon
594    # cat target_ids
595    42 4242
596    # echo "0   1       100 \
597            0   100     200 \
598            1   20      40  \
599            1   50      100" > init_regions
600
601Note that this sets the initial monitoring target regions only.  In case of
602virtual memory monitoring, DAMON will automatically updates the boundary of the
603regions after one ``update interval``.  Therefore, users should set the
604``update interval`` large enough in this case, if they don't want the
605update.
606
607
608Schemes
609-------
610
611Users can get and set the DAMON-based operation :ref:`schemes
612<damon_design_damos>` by reading from and writing to ``schemes`` debugfs file.
613Reading the file also shows the statistics of each scheme.  To the file, each
614of the schemes should be represented in each line in below form::
615
616    <target access pattern> <action> <quota> <watermarks>
617
618You can disable schemes by simply writing an empty string to the file.
619
620Target Access Pattern
621~~~~~~~~~~~~~~~~~~~~~
622
623The target access :ref:`pattern <damon_design_damos_access_pattern>` of the
624scheme.  The ``<target access pattern>`` is constructed with three ranges in
625below form::
626
627    min-size max-size min-acc max-acc min-age max-age
628
629Specifically, bytes for the size of regions (``min-size`` and ``max-size``),
630number of monitored accesses per aggregate interval for access frequency
631(``min-acc`` and ``max-acc``), number of aggregate intervals for the age of
632regions (``min-age`` and ``max-age``) are specified.  Note that the ranges are
633closed interval.
634
635Action
636~~~~~~
637
638The ``<action>`` is a predefined integer for memory management :ref:`actions
639<damon_design_damos_action>`.  The supported numbers and their meanings are as
640below.
641
642 - 0: Call ``madvise()`` for the region with ``MADV_WILLNEED``.  Ignored if
643   ``target`` is ``paddr``.
644 - 1: Call ``madvise()`` for the region with ``MADV_COLD``.  Ignored if
645   ``target`` is ``paddr``.
646 - 2: Call ``madvise()`` for the region with ``MADV_PAGEOUT``.
647 - 3: Call ``madvise()`` for the region with ``MADV_HUGEPAGE``.  Ignored if
648   ``target`` is ``paddr``.
649 - 4: Call ``madvise()`` for the region with ``MADV_NOHUGEPAGE``.  Ignored if
650   ``target`` is ``paddr``.
651 - 5: Do nothing but count the statistics
652
653Quota
654~~~~~
655
656Users can set the :ref:`quotas <damon_design_damos_quotas>` of the given scheme
657via the ``<quota>`` in below form::
658
659    <ms> <sz> <reset interval> <priority weights>
660
661This makes DAMON to try to use only up to ``<ms>`` milliseconds for applying
662the action to memory regions of the ``target access pattern`` within the
663``<reset interval>`` milliseconds, and to apply the action to only up to
664``<sz>`` bytes of memory regions within the ``<reset interval>``.  Setting both
665``<ms>`` and ``<sz>`` zero disables the quota limits.
666
667For the :ref:`prioritization <damon_design_damos_quotas_prioritization>`, users
668can set the weights for the three properties in ``<priority weights>`` in below
669form::
670
671    <size weight> <access frequency weight> <age weight>
672
673Watermarks
674~~~~~~~~~~
675
676Users can specify :ref:`watermarks <damon_design_damos_watermarks>` of the
677given scheme via ``<watermarks>`` in below form::
678
679    <metric> <check interval> <high mark> <middle mark> <low mark>
680
681``<metric>`` is a predefined integer for the metric to be checked.  The
682supported numbers and their meanings are as below.
683
684 - 0: Ignore the watermarks
685 - 1: System's free memory rate (per thousand)
686
687The value of the metric is checked every ``<check interval>`` microseconds.
688
689If the value is higher than ``<high mark>`` or lower than ``<low mark>``, the
690scheme is deactivated.  If the value is lower than ``<mid mark>``, the scheme
691is activated.
692
693.. _damos_stats:
694
695Statistics
696~~~~~~~~~~
697
698It also counts the total number and bytes of regions that each scheme is tried
699to be applied, the two numbers for the regions that each scheme is successfully
700applied, and the total number of the quota limit exceeds.  This statistics can
701be used for online analysis or tuning of the schemes.
702
703The statistics can be shown by reading the ``schemes`` file.  Reading the file
704will show each scheme you entered in each line, and the five numbers for the
705statistics will be added at the end of each line.
706
707Example
708~~~~~~~
709
710Below commands applies a scheme saying "If a memory region of size in [4KiB,
7118KiB] is showing accesses per aggregate interval in [0, 5] for aggregate
712interval in [10, 20], page out the region.  For the paging out, use only up to
71310ms per second, and also don't page out more than 1GiB per second.  Under the
714limitation, page out memory regions having longer age first.  Also, check the
715free memory rate of the system every 5 seconds, start the monitoring and paging
716out when the free memory rate becomes lower than 50%, but stop it if the free
717memory rate becomes larger than 60%, or lower than 30%".::
718
719    # cd <debugfs>/damon
720    # scheme="4096 8192  0 5    10 20    2"  # target access pattern and action
721    # scheme+=" 10 $((1024*1024*1024)) 1000" # quotas
722    # scheme+=" 0 0 100"                     # prioritization weights
723    # scheme+=" 1 5000000 600 500 300"       # watermarks
724    # echo "$scheme" > schemes
725
726
727Turning On/Off
728--------------
729
730Setting the files as described above doesn't incur effect unless you explicitly
731start the monitoring.  You can start, stop, and check the current status of the
732monitoring by writing to and reading from the ``monitor_on`` file.  Writing
733``on`` to the file starts the monitoring of the targets with the attributes.
734Writing ``off`` to the file stops those.  DAMON also stops if every target
735process is terminated.  Below example commands turn on, off, and check the
736status of DAMON::
737
738    # cd <debugfs>/damon
739    # echo on > monitor_on
740    # echo off > monitor_on
741    # cat monitor_on
742    off
743
744Please note that you cannot write to the above-mentioned debugfs files while
745the monitoring is turned on.  If you write to the files while DAMON is running,
746an error code such as ``-EBUSY`` will be returned.
747
748
749Monitoring Thread PID
750---------------------
751
752DAMON does requested monitoring with a kernel thread called ``kdamond``.  You
753can get the pid of the thread by reading the ``kdamond_pid`` file.  When the
754monitoring is turned off, reading the file returns ``none``. ::
755
756    # cd <debugfs>/damon
757    # cat monitor_on
758    off
759    # cat kdamond_pid
760    none
761    # echo on > monitor_on
762    # cat kdamond_pid
763    18594
764
765
766Using Multiple Monitoring Threads
767---------------------------------
768
769One ``kdamond`` thread is created for each monitoring context.  You can create
770and remove monitoring contexts for multiple ``kdamond`` required use case using
771the ``mk_contexts`` and ``rm_contexts`` files.
772
773Writing the name of the new context to the ``mk_contexts`` file creates a
774directory of the name on the DAMON debugfs directory.  The directory will have
775DAMON debugfs files for the context. ::
776
777    # cd <debugfs>/damon
778    # ls foo
779    # ls: cannot access 'foo': No such file or directory
780    # echo foo > mk_contexts
781    # ls foo
782    # attrs  init_regions  kdamond_pid  schemes  target_ids
783
784If the context is not needed anymore, you can remove it and the corresponding
785directory by putting the name of the context to the ``rm_contexts`` file. ::
786
787    # echo foo > rm_contexts
788    # ls foo
789    # ls: cannot access 'foo': No such file or directory
790
791Note that ``mk_contexts``, ``rm_contexts``, and ``monitor_on`` files are in the
792root directory only.
793
794
795.. _tracepoint:
796
797Tracepoint for Monitoring Results
798=================================
799
800Users can get the monitoring results via the :ref:`tried_regions
801<sysfs_schemes_tried_regions>` or a tracepoint, ``damon:damon_aggregated``.
802While the tried regions directory is useful for getting a snapshot, the
803tracepoint is useful for getting a full record of the results.  While the
804monitoring is turned on, you could record the tracepoint events and show
805results using tracepoint supporting tools like ``perf``.  For example::
806
807    # echo on > monitor_on
808    # perf record -e damon:damon_aggregated &
809    # sleep 5
810    # kill 9 $(pidof perf)
811    # echo off > monitor_on
812    # perf script
813