1.. SPDX-License-Identifier: GPL-2.0 2 3=============== 4Detailed Usages 5=============== 6 7DAMON provides below interfaces for different users. 8 9- *DAMON user space tool.* 10 `This <https://github.com/awslabs/damo>`_ is for privileged people such as 11 system administrators who want a just-working human-friendly interface. 12 Using this, users can use the DAMON’s major features in a human-friendly way. 13 It may not be highly tuned for special cases, though. For more detail, 14 please refer to its `usage document 15 <https://github.com/awslabs/damo/blob/next/USAGE.md>`_. 16- *sysfs interface.* 17 :ref:`This <sysfs_interface>` is for privileged user space programmers who 18 want more optimized use of DAMON. Using this, users can use DAMON’s major 19 features by reading from and writing to special sysfs files. Therefore, 20 you can write and use your personalized DAMON sysfs wrapper programs that 21 reads/writes the sysfs files instead of you. The `DAMON user space tool 22 <https://github.com/awslabs/damo>`_ is one example of such programs. Note 23 that this interface provides only simple :ref:`statistics <damos_stats>` for 24 the monitoring results. For detailed monitoring results, DAMON provides a 25 :ref:`tracepoint <tracepoint>`. 26- *debugfs interface. (DEPRECATED!)* 27 :ref:`This <debugfs_interface>` is almost identical to :ref:`sysfs interface 28 <sysfs_interface>`. This is deprecated, so users should move to the 29 :ref:`sysfs interface <sysfs_interface>`. If you depend on this and cannot 30 move, please report your usecase to damon@lists.linux.dev and 31 linux-mm@kvack.org. 32- *Kernel Space Programming Interface.* 33 :doc:`This </mm/damon/api>` is for kernel space programmers. Using this, 34 users can utilize every feature of DAMON most flexibly and efficiently by 35 writing kernel space DAMON application programs for you. You can even extend 36 DAMON for various address spaces. For detail, please refer to the interface 37 :doc:`document </mm/damon/api>`. 38 39.. _sysfs_interface: 40 41sysfs Interface 42=============== 43 44DAMON sysfs interface is built when ``CONFIG_DAMON_SYSFS`` is defined. It 45creates multiple directories and files under its sysfs directory, 46``<sysfs>/kernel/mm/damon/``. You can control DAMON by writing to and reading 47from the files under the directory. 48 49For a short example, users can monitor the virtual address space of a given 50workload as below. :: 51 52 # cd /sys/kernel/mm/damon/admin/ 53 # echo 1 > kdamonds/nr_kdamonds && echo 1 > kdamonds/0/contexts/nr_contexts 54 # echo vaddr > kdamonds/0/contexts/0/operations 55 # echo 1 > kdamonds/0/contexts/0/targets/nr_targets 56 # echo $(pidof <workload>) > kdamonds/0/contexts/0/targets/0/pid_target 57 # echo on > kdamonds/0/state 58 59Files Hierarchy 60--------------- 61 62The files hierarchy of DAMON sysfs interface is shown below. In the below 63figure, parents-children relations are represented with indentations, each 64directory is having ``/`` suffix, and files in each directory are separated by 65comma (","). :: 66 67 /sys/kernel/mm/damon/admin 68 │ kdamonds/nr_kdamonds 69 │ │ 0/state,pid 70 │ │ │ contexts/nr_contexts 71 │ │ │ │ 0/avail_operations,operations 72 │ │ │ │ │ monitoring_attrs/ 73 │ │ │ │ │ │ intervals/sample_us,aggr_us,update_us 74 │ │ │ │ │ │ nr_regions/min,max 75 │ │ │ │ │ targets/nr_targets 76 │ │ │ │ │ │ 0/pid_target 77 │ │ │ │ │ │ │ regions/nr_regions 78 │ │ │ │ │ │ │ │ 0/start,end 79 │ │ │ │ │ │ │ │ ... 80 │ │ │ │ │ │ ... 81 │ │ │ │ │ schemes/nr_schemes 82 │ │ │ │ │ │ 0/action 83 │ │ │ │ │ │ │ access_pattern/ 84 │ │ │ │ │ │ │ │ sz/min,max 85 │ │ │ │ │ │ │ │ nr_accesses/min,max 86 │ │ │ │ │ │ │ │ age/min,max 87 │ │ │ │ │ │ │ quotas/ms,bytes,reset_interval_ms 88 │ │ │ │ │ │ │ │ weights/sz_permil,nr_accesses_permil,age_permil 89 │ │ │ │ │ │ │ watermarks/metric,interval_us,high,mid,low 90 │ │ │ │ │ │ │ filters/nr_filters 91 │ │ │ │ │ │ │ │ 0/type,matching,memcg_id 92 │ │ │ │ │ │ │ stats/nr_tried,sz_tried,nr_applied,sz_applied,qt_exceeds 93 │ │ │ │ │ │ │ tried_regions/ 94 │ │ │ │ │ │ │ │ 0/start,end,nr_accesses,age 95 │ │ │ │ │ │ │ │ ... 96 │ │ │ │ │ │ ... 97 │ │ │ │ ... 98 │ │ ... 99 100Root 101---- 102 103The root of the DAMON sysfs interface is ``<sysfs>/kernel/mm/damon/``, and it 104has one directory named ``admin``. The directory contains the files for 105privileged user space programs' control of DAMON. User space tools or deamons 106having the root permission could use this directory. 107 108kdamonds/ 109--------- 110 111The monitoring-related information including request specifications and results 112are called DAMON context. DAMON executes each context with a kernel thread 113called kdamond, and multiple kdamonds could run in parallel. 114 115Under the ``admin`` directory, one directory, ``kdamonds``, which has files for 116controlling the kdamonds exist. In the beginning, this directory has only one 117file, ``nr_kdamonds``. Writing a number (``N``) to the file creates the number 118of child directories named ``0`` to ``N-1``. Each directory represents each 119kdamond. 120 121kdamonds/<N>/ 122------------- 123 124In each kdamond directory, two files (``state`` and ``pid``) and one directory 125(``contexts``) exist. 126 127Reading ``state`` returns ``on`` if the kdamond is currently running, or 128``off`` if it is not running. Writing ``on`` or ``off`` makes the kdamond be 129in the state. Writing ``commit`` to the ``state`` file makes kdamond reads the 130user inputs in the sysfs files except ``state`` file again. Writing 131``update_schemes_stats`` to ``state`` file updates the contents of stats files 132for each DAMON-based operation scheme of the kdamond. For details of the 133stats, please refer to :ref:`stats section <sysfs_schemes_stats>`. Writing 134``update_schemes_tried_regions`` to ``state`` file updates the DAMON-based 135operation scheme action tried regions directory for each DAMON-based operation 136scheme of the kdamond. Writing ``clear_schemes_tried_regions`` to ``state`` 137file clears the DAMON-based operating scheme action tried regions directory for 138each DAMON-based operation scheme of the kdamond. For details of the 139DAMON-based operation scheme action tried regions directory, please refer to 140:ref:`tried_regions section <sysfs_schemes_tried_regions>`. 141 142If the state is ``on``, reading ``pid`` shows the pid of the kdamond thread. 143 144``contexts`` directory contains files for controlling the monitoring contexts 145that this kdamond will execute. 146 147kdamonds/<N>/contexts/ 148---------------------- 149 150In the beginning, this directory has only one file, ``nr_contexts``. Writing a 151number (``N``) to the file creates the number of child directories named as 152``0`` to ``N-1``. Each directory represents each monitoring context. At the 153moment, only one context per kdamond is supported, so only ``0`` or ``1`` can 154be written to the file. 155 156.. _sysfs_contexts: 157 158contexts/<N>/ 159------------- 160 161In each context directory, two files (``avail_operations`` and ``operations``) 162and three directories (``monitoring_attrs``, ``targets``, and ``schemes``) 163exist. 164 165DAMON supports multiple types of monitoring operations, including those for 166virtual address space and the physical address space. You can get the list of 167available monitoring operations set on the currently running kernel by reading 168``avail_operations`` file. Based on the kernel configuration, the file will 169list some or all of below keywords. 170 171 - vaddr: Monitor virtual address spaces of specific processes 172 - fvaddr: Monitor fixed virtual address ranges 173 - paddr: Monitor the physical address space of the system 174 175Please refer to :ref:`regions sysfs directory <sysfs_regions>` for detailed 176differences between the operations sets in terms of the monitoring target 177regions. 178 179You can set and get what type of monitoring operations DAMON will use for the 180context by writing one of the keywords listed in ``avail_operations`` file and 181reading from the ``operations`` file. 182 183.. _sysfs_monitoring_attrs: 184 185contexts/<N>/monitoring_attrs/ 186------------------------------ 187 188Files for specifying attributes of the monitoring including required quality 189and efficiency of the monitoring are in ``monitoring_attrs`` directory. 190Specifically, two directories, ``intervals`` and ``nr_regions`` exist in this 191directory. 192 193Under ``intervals`` directory, three files for DAMON's sampling interval 194(``sample_us``), aggregation interval (``aggr_us``), and update interval 195(``update_us``) exist. You can set and get the values in micro-seconds by 196writing to and reading from the files. 197 198Under ``nr_regions`` directory, two files for the lower-bound and upper-bound 199of DAMON's monitoring regions (``min`` and ``max``, respectively), which 200controls the monitoring overhead, exist. You can set and get the values by 201writing to and rading from the files. 202 203For more details about the intervals and monitoring regions range, please refer 204to the Design document (:doc:`/mm/damon/design`). 205 206contexts/<N>/targets/ 207--------------------- 208 209In the beginning, this directory has only one file, ``nr_targets``. Writing a 210number (``N``) to the file creates the number of child directories named ``0`` 211to ``N-1``. Each directory represents each monitoring target. 212 213targets/<N>/ 214------------ 215 216In each target directory, one file (``pid_target``) and one directory 217(``regions``) exist. 218 219If you wrote ``vaddr`` to the ``contexts/<N>/operations``, each target should 220be a process. You can specify the process to DAMON by writing the pid of the 221process to the ``pid_target`` file. 222 223.. _sysfs_regions: 224 225targets/<N>/regions 226------------------- 227 228When ``vaddr`` monitoring operations set is being used (``vaddr`` is written to 229the ``contexts/<N>/operations`` file), DAMON automatically sets and updates the 230monitoring target regions so that entire memory mappings of target processes 231can be covered. However, users could want to set the initial monitoring region 232to specific address ranges. 233 234In contrast, DAMON do not automatically sets and updates the monitoring target 235regions when ``fvaddr`` or ``paddr`` monitoring operations sets are being used 236(``fvaddr`` or ``paddr`` have written to the ``contexts/<N>/operations``). 237Therefore, users should set the monitoring target regions by themselves in the 238cases. 239 240For such cases, users can explicitly set the initial monitoring target regions 241as they want, by writing proper values to the files under this directory. 242 243In the beginning, this directory has only one file, ``nr_regions``. Writing a 244number (``N``) to the file creates the number of child directories named ``0`` 245to ``N-1``. Each directory represents each initial monitoring target region. 246 247regions/<N>/ 248------------ 249 250In each region directory, you will find two files (``start`` and ``end``). You 251can set and get the start and end addresses of the initial monitoring target 252region by writing to and reading from the files, respectively. 253 254Each region should not overlap with others. ``end`` of directory ``N`` should 255be equal or smaller than ``start`` of directory ``N+1``. 256 257contexts/<N>/schemes/ 258--------------------- 259 260The directory for DAMON-based Operation Schemes (:ref:`DAMOS 261<damon_design_damos>`). Users can get and set the schemes by reading from and 262writing to files under this directory. 263 264In the beginning, this directory has only one file, ``nr_schemes``. Writing a 265number (``N``) to the file creates the number of child directories named ``0`` 266to ``N-1``. Each directory represents each DAMON-based operation scheme. 267 268schemes/<N>/ 269------------ 270 271In each scheme directory, five directories (``access_pattern``, ``quotas``, 272``watermarks``, ``filters``, ``stats``, and ``tried_regions``) and one file 273(``action``) exist. 274 275The ``action`` file is for setting and getting the scheme's :ref:`action 276<damon_design_damos_action>`. The keywords that can be written to and read 277from the file and their meaning are as below. 278 279Note that support of each action depends on the running DAMON operations set 280:ref:`implementation <sysfs_contexts>`. 281 282 - ``willneed``: Call ``madvise()`` for the region with ``MADV_WILLNEED``. 283 Supported by ``vaddr`` and ``fvaddr`` operations set. 284 - ``cold``: Call ``madvise()`` for the region with ``MADV_COLD``. 285 Supported by ``vaddr`` and ``fvaddr`` operations set. 286 - ``pageout``: Call ``madvise()`` for the region with ``MADV_PAGEOUT``. 287 Supported by ``vaddr``, ``fvaddr`` and ``paddr`` operations set. 288 - ``hugepage``: Call ``madvise()`` for the region with ``MADV_HUGEPAGE``. 289 Supported by ``vaddr`` and ``fvaddr`` operations set. 290 - ``nohugepage``: Call ``madvise()`` for the region with ``MADV_NOHUGEPAGE``. 291 Supported by ``vaddr`` and ``fvaddr`` operations set. 292 - ``lru_prio``: Prioritize the region on its LRU lists. 293 Supported by ``paddr`` operations set. 294 - ``lru_deprio``: Deprioritize the region on its LRU lists. 295 Supported by ``paddr`` operations set. 296 - ``stat``: Do nothing but count the statistics. 297 Supported by all operations sets. 298 299schemes/<N>/access_pattern/ 300--------------------------- 301 302The directory for the target access :ref:`pattern 303<damon_design_damos_access_pattern>` of the given DAMON-based operation scheme. 304 305Under the ``access_pattern`` directory, three directories (``sz``, 306``nr_accesses``, and ``age``) each having two files (``min`` and ``max``) 307exist. You can set and get the access pattern for the given scheme by writing 308to and reading from the ``min`` and ``max`` files under ``sz``, 309``nr_accesses``, and ``age`` directories, respectively. Note that the ``min`` 310and the ``max`` form a closed interval. 311 312schemes/<N>/quotas/ 313------------------- 314 315The directory for the :ref:`quotas <damon_design_damos_quotas>` of the given 316DAMON-based operation scheme. 317 318Under ``quotas`` directory, three files (``ms``, ``bytes``, 319``reset_interval_ms``) and one directory (``weights``) having three files 320(``sz_permil``, ``nr_accesses_permil``, and ``age_permil``) in it exist. 321 322You can set the ``time quota`` in milliseconds, ``size quota`` in bytes, and 323``reset interval`` in milliseconds by writing the values to the three files, 324respectively. Then, DAMON tries to use only up to ``time quota`` milliseconds 325for applying the ``action`` to memory regions of the ``access_pattern``, and to 326apply the action to only up to ``bytes`` bytes of memory regions within the 327``reset_interval_ms``. Setting both ``ms`` and ``bytes`` zero disables the 328quota limits. 329 330You can also set the :ref:`prioritization weights 331<damon_design_damos_quotas_prioritization>` for size, access frequency, and age 332in per-thousand unit by writing the values to the three files under the 333``weights`` directory. 334 335schemes/<N>/watermarks/ 336----------------------- 337 338The directory for the :ref:`watermarks <damon_design_damos_watermarks>` of the 339given DAMON-based operation scheme. 340 341Under the watermarks directory, five files (``metric``, ``interval_us``, 342``high``, ``mid``, and ``low``) for setting the metric, the time interval 343between check of the metric, and the three watermarks exist. You can set and 344get the five values by writing to the files, respectively. 345 346Keywords and meanings of those that can be written to the ``metric`` file are 347as below. 348 349 - none: Ignore the watermarks 350 - free_mem_rate: System's free memory rate (per thousand) 351 352The ``interval`` should written in microseconds unit. 353 354schemes/<N>/filters/ 355-------------------- 356 357The directory for the :ref:`filters <damon_design_damos_filters>` of the given 358DAMON-based operation scheme. 359 360In the beginning, this directory has only one file, ``nr_filters``. Writing a 361number (``N``) to the file creates the number of child directories named ``0`` 362to ``N-1``. Each directory represents each filter. The filters are evaluated 363in the numeric order. 364 365Each filter directory contains three files, namely ``type``, ``matcing``, and 366``memcg_path``. You can write one of two special keywords, ``anon`` for 367anonymous pages, or ``memcg`` for specific memory cgroup filtering. In case of 368the memory cgroup filtering, you can specify the memory cgroup of the interest 369by writing the path of the memory cgroup from the cgroups mount point to 370``memcg_path`` file. You can write ``Y`` or ``N`` to ``matching`` file to 371filter out pages that does or does not match to the type, respectively. Then, 372the scheme's action will not be applied to the pages that specified to be 373filtered out. 374 375For example, below restricts a DAMOS action to be applied to only non-anonymous 376pages of all memory cgroups except ``/having_care_already``.:: 377 378 # echo 2 > nr_filters 379 # # filter out anonymous pages 380 echo anon > 0/type 381 echo Y > 0/matching 382 # # further filter out all cgroups except one at '/having_care_already' 383 echo memcg > 1/type 384 echo /having_care_already > 1/memcg_path 385 echo N > 1/matching 386 387Note that filters are currently supported only when ``paddr`` 388`implementation <sysfs_contexts>` is being used. 389 390.. _sysfs_schemes_stats: 391 392schemes/<N>/stats/ 393------------------ 394 395DAMON counts the total number and bytes of regions that each scheme is tried to 396be applied, the two numbers for the regions that each scheme is successfully 397applied, and the total number of the quota limit exceeds. This statistics can 398be used for online analysis or tuning of the schemes. 399 400The statistics can be retrieved by reading the files under ``stats`` directory 401(``nr_tried``, ``sz_tried``, ``nr_applied``, ``sz_applied``, and 402``qt_exceeds``), respectively. The files are not updated in real time, so you 403should ask DAMON sysfs interface to updte the content of the files for the 404stats by writing a special keyword, ``update_schemes_stats`` to the relevant 405``kdamonds/<N>/state`` file. 406 407.. _sysfs_schemes_tried_regions: 408 409schemes/<N>/tried_regions/ 410-------------------------- 411 412When a special keyword, ``update_schemes_tried_regions``, is written to the 413relevant ``kdamonds/<N>/state`` file, DAMON creates directories named integer 414starting from ``0`` under this directory. Each directory contains files 415exposing detailed information about each of the memory region that the 416corresponding scheme's ``action`` has tried to be applied under this directory, 417during next :ref:`aggregation interval <sysfs_monitoring_attrs>`. The 418information includes address range, ``nr_accesses``, and ``age`` of the region. 419 420The directories will be removed when another special keyword, 421``clear_schemes_tried_regions``, is written to the relevant 422``kdamonds/<N>/state`` file. 423 424tried_regions/<N>/ 425------------------ 426 427In each region directory, you will find four files (``start``, ``end``, 428``nr_accesses``, and ``age``). Reading the files will show the start and end 429addresses, ``nr_accesses``, and ``age`` of the region that corresponding 430DAMON-based operation scheme ``action`` has tried to be applied. 431 432Example 433~~~~~~~ 434 435Below commands applies a scheme saying "If a memory region of size in [4KiB, 4368KiB] is showing accesses per aggregate interval in [0, 5] for aggregate 437interval in [10, 20], page out the region. For the paging out, use only up to 43810ms per second, and also don't page out more than 1GiB per second. Under the 439limitation, page out memory regions having longer age first. Also, check the 440free memory rate of the system every 5 seconds, start the monitoring and paging 441out when the free memory rate becomes lower than 50%, but stop it if the free 442memory rate becomes larger than 60%, or lower than 30%". :: 443 444 # cd <sysfs>/kernel/mm/damon/admin 445 # # populate directories 446 # echo 1 > kdamonds/nr_kdamonds; echo 1 > kdamonds/0/contexts/nr_contexts; 447 # echo 1 > kdamonds/0/contexts/0/schemes/nr_schemes 448 # cd kdamonds/0/contexts/0/schemes/0 449 # # set the basic access pattern and the action 450 # echo 4096 > access_pattern/sz/min 451 # echo 8192 > access_pattern/sz/max 452 # echo 0 > access_pattern/nr_accesses/min 453 # echo 5 > access_pattern/nr_accesses/max 454 # echo 10 > access_pattern/age/min 455 # echo 20 > access_pattern/age/max 456 # echo pageout > action 457 # # set quotas 458 # echo 10 > quotas/ms 459 # echo $((1024*1024*1024)) > quotas/bytes 460 # echo 1000 > quotas/reset_interval_ms 461 # # set watermark 462 # echo free_mem_rate > watermarks/metric 463 # echo 5000000 > watermarks/interval_us 464 # echo 600 > watermarks/high 465 # echo 500 > watermarks/mid 466 # echo 300 > watermarks/low 467 468Please note that it's highly recommended to use user space tools like `damo 469<https://github.com/awslabs/damo>`_ rather than manually reading and writing 470the files as above. Above is only for an example. 471 472.. _debugfs_interface: 473 474debugfs Interface (DEPRECATED!) 475=============================== 476 477.. note:: 478 479 THIS IS DEPRECATED! 480 481 DAMON debugfs interface is deprecated, so users should move to the 482 :ref:`sysfs interface <sysfs_interface>`. If you depend on this and cannot 483 move, please report your usecase to damon@lists.linux.dev and 484 linux-mm@kvack.org. 485 486DAMON exports eight files, ``attrs``, ``target_ids``, ``init_regions``, 487``schemes``, ``monitor_on``, ``kdamond_pid``, ``mk_contexts`` and 488``rm_contexts`` under its debugfs directory, ``<debugfs>/damon/``. 489 490 491Attributes 492---------- 493 494Users can get and set the ``sampling interval``, ``aggregation interval``, 495``update interval``, and min/max number of monitoring target regions by 496reading from and writing to the ``attrs`` file. To know about the monitoring 497attributes in detail, please refer to the :doc:`/mm/damon/design`. For 498example, below commands set those values to 5 ms, 100 ms, 1,000 ms, 10 and 4991000, and then check it again:: 500 501 # cd <debugfs>/damon 502 # echo 5000 100000 1000000 10 1000 > attrs 503 # cat attrs 504 5000 100000 1000000 10 1000 505 506 507Target IDs 508---------- 509 510Some types of address spaces supports multiple monitoring target. For example, 511the virtual memory address spaces monitoring can have multiple processes as the 512monitoring targets. Users can set the targets by writing relevant id values of 513the targets to, and get the ids of the current targets by reading from the 514``target_ids`` file. In case of the virtual address spaces monitoring, the 515values should be pids of the monitoring target processes. For example, below 516commands set processes having pids 42 and 4242 as the monitoring targets and 517check it again:: 518 519 # cd <debugfs>/damon 520 # echo 42 4242 > target_ids 521 # cat target_ids 522 42 4242 523 524Users can also monitor the physical memory address space of the system by 525writing a special keyword, "``paddr\n``" to the file. Because physical address 526space monitoring doesn't support multiple targets, reading the file will show a 527fake value, ``42``, as below:: 528 529 # cd <debugfs>/damon 530 # echo paddr > target_ids 531 # cat target_ids 532 42 533 534Note that setting the target ids doesn't start the monitoring. 535 536 537Initial Monitoring Target Regions 538--------------------------------- 539 540In case of the virtual address space monitoring, DAMON automatically sets and 541updates the monitoring target regions so that entire memory mappings of target 542processes can be covered. However, users can want to limit the monitoring 543region to specific address ranges, such as the heap, the stack, or specific 544file-mapped area. Or, some users can know the initial access pattern of their 545workloads and therefore want to set optimal initial regions for the 'adaptive 546regions adjustment'. 547 548In contrast, DAMON do not automatically sets and updates the monitoring target 549regions in case of physical memory monitoring. Therefore, users should set the 550monitoring target regions by themselves. 551 552In such cases, users can explicitly set the initial monitoring target regions 553as they want, by writing proper values to the ``init_regions`` file. The input 554should be a sequence of three integers separated by white spaces that represent 555one region in below form.:: 556 557 <target idx> <start address> <end address> 558 559The ``target idx`` should be the index of the target in ``target_ids`` file, 560starting from ``0``, and the regions should be passed in address order. For 561example, below commands will set a couple of address ranges, ``1-100`` and 562``100-200`` as the initial monitoring target region of pid 42, which is the 563first one (index ``0``) in ``target_ids``, and another couple of address 564ranges, ``20-40`` and ``50-100`` as that of pid 4242, which is the second one 565(index ``1``) in ``target_ids``.:: 566 567 # cd <debugfs>/damon 568 # cat target_ids 569 42 4242 570 # echo "0 1 100 \ 571 0 100 200 \ 572 1 20 40 \ 573 1 50 100" > init_regions 574 575Note that this sets the initial monitoring target regions only. In case of 576virtual memory monitoring, DAMON will automatically updates the boundary of the 577regions after one ``update interval``. Therefore, users should set the 578``update interval`` large enough in this case, if they don't want the 579update. 580 581 582Schemes 583------- 584 585Users can get and set the DAMON-based operation :ref:`schemes 586<damon_design_damos>` by reading from and writing to ``schemes`` debugfs file. 587Reading the file also shows the statistics of each scheme. To the file, each 588of the schemes should be represented in each line in below form:: 589 590 <target access pattern> <action> <quota> <watermarks> 591 592You can disable schemes by simply writing an empty string to the file. 593 594Target Access Pattern 595~~~~~~~~~~~~~~~~~~~~~ 596 597The target access :ref:`pattern <damon_design_damos_access_pattern>` of the 598scheme. The ``<target access pattern>`` is constructed with three ranges in 599below form:: 600 601 min-size max-size min-acc max-acc min-age max-age 602 603Specifically, bytes for the size of regions (``min-size`` and ``max-size``), 604number of monitored accesses per aggregate interval for access frequency 605(``min-acc`` and ``max-acc``), number of aggregate intervals for the age of 606regions (``min-age`` and ``max-age``) are specified. Note that the ranges are 607closed interval. 608 609Action 610~~~~~~ 611 612The ``<action>`` is a predefined integer for memory management :ref:`actions 613<damon_design_damos_action>`. The supported numbers and their meanings are as 614below. 615 616 - 0: Call ``madvise()`` for the region with ``MADV_WILLNEED``. Ignored if 617 ``target`` is ``paddr``. 618 - 1: Call ``madvise()`` for the region with ``MADV_COLD``. Ignored if 619 ``target`` is ``paddr``. 620 - 2: Call ``madvise()`` for the region with ``MADV_PAGEOUT``. 621 - 3: Call ``madvise()`` for the region with ``MADV_HUGEPAGE``. Ignored if 622 ``target`` is ``paddr``. 623 - 4: Call ``madvise()`` for the region with ``MADV_NOHUGEPAGE``. Ignored if 624 ``target`` is ``paddr``. 625 - 5: Do nothing but count the statistics 626 627Quota 628~~~~~ 629 630Users can set the :ref:`quotas <damon_design_damos_quotas>` of the given scheme 631via the ``<quota>`` in below form:: 632 633 <ms> <sz> <reset interval> <priority weights> 634 635This makes DAMON to try to use only up to ``<ms>`` milliseconds for applying 636the action to memory regions of the ``target access pattern`` within the 637``<reset interval>`` milliseconds, and to apply the action to only up to 638``<sz>`` bytes of memory regions within the ``<reset interval>``. Setting both 639``<ms>`` and ``<sz>`` zero disables the quota limits. 640 641For the :ref:`prioritization <damon_design_damos_quotas_prioritization>`, users 642can set the weights for the three properties in ``<priority weights>`` in below 643form:: 644 645 <size weight> <access frequency weight> <age weight> 646 647Watermarks 648~~~~~~~~~~ 649 650Users can specify :ref:`watermarks <damon_design_damos_watermarks>` of the 651given scheme via ``<watermarks>`` in below form:: 652 653 <metric> <check interval> <high mark> <middle mark> <low mark> 654 655``<metric>`` is a predefined integer for the metric to be checked. The 656supported numbers and their meanings are as below. 657 658 - 0: Ignore the watermarks 659 - 1: System's free memory rate (per thousand) 660 661The value of the metric is checked every ``<check interval>`` microseconds. 662 663If the value is higher than ``<high mark>`` or lower than ``<low mark>``, the 664scheme is deactivated. If the value is lower than ``<mid mark>``, the scheme 665is activated. 666 667.. _damos_stats: 668 669Statistics 670~~~~~~~~~~ 671 672It also counts the total number and bytes of regions that each scheme is tried 673to be applied, the two numbers for the regions that each scheme is successfully 674applied, and the total number of the quota limit exceeds. This statistics can 675be used for online analysis or tuning of the schemes. 676 677The statistics can be shown by reading the ``schemes`` file. Reading the file 678will show each scheme you entered in each line, and the five numbers for the 679statistics will be added at the end of each line. 680 681Example 682~~~~~~~ 683 684Below commands applies a scheme saying "If a memory region of size in [4KiB, 6858KiB] is showing accesses per aggregate interval in [0, 5] for aggregate 686interval in [10, 20], page out the region. For the paging out, use only up to 68710ms per second, and also don't page out more than 1GiB per second. Under the 688limitation, page out memory regions having longer age first. Also, check the 689free memory rate of the system every 5 seconds, start the monitoring and paging 690out when the free memory rate becomes lower than 50%, but stop it if the free 691memory rate becomes larger than 60%, or lower than 30%".:: 692 693 # cd <debugfs>/damon 694 # scheme="4096 8192 0 5 10 20 2" # target access pattern and action 695 # scheme+=" 10 $((1024*1024*1024)) 1000" # quotas 696 # scheme+=" 0 0 100" # prioritization weights 697 # scheme+=" 1 5000000 600 500 300" # watermarks 698 # echo "$scheme" > schemes 699 700 701Turning On/Off 702-------------- 703 704Setting the files as described above doesn't incur effect unless you explicitly 705start the monitoring. You can start, stop, and check the current status of the 706monitoring by writing to and reading from the ``monitor_on`` file. Writing 707``on`` to the file starts the monitoring of the targets with the attributes. 708Writing ``off`` to the file stops those. DAMON also stops if every target 709process is terminated. Below example commands turn on, off, and check the 710status of DAMON:: 711 712 # cd <debugfs>/damon 713 # echo on > monitor_on 714 # echo off > monitor_on 715 # cat monitor_on 716 off 717 718Please note that you cannot write to the above-mentioned debugfs files while 719the monitoring is turned on. If you write to the files while DAMON is running, 720an error code such as ``-EBUSY`` will be returned. 721 722 723Monitoring Thread PID 724--------------------- 725 726DAMON does requested monitoring with a kernel thread called ``kdamond``. You 727can get the pid of the thread by reading the ``kdamond_pid`` file. When the 728monitoring is turned off, reading the file returns ``none``. :: 729 730 # cd <debugfs>/damon 731 # cat monitor_on 732 off 733 # cat kdamond_pid 734 none 735 # echo on > monitor_on 736 # cat kdamond_pid 737 18594 738 739 740Using Multiple Monitoring Threads 741--------------------------------- 742 743One ``kdamond`` thread is created for each monitoring context. You can create 744and remove monitoring contexts for multiple ``kdamond`` required use case using 745the ``mk_contexts`` and ``rm_contexts`` files. 746 747Writing the name of the new context to the ``mk_contexts`` file creates a 748directory of the name on the DAMON debugfs directory. The directory will have 749DAMON debugfs files for the context. :: 750 751 # cd <debugfs>/damon 752 # ls foo 753 # ls: cannot access 'foo': No such file or directory 754 # echo foo > mk_contexts 755 # ls foo 756 # attrs init_regions kdamond_pid schemes target_ids 757 758If the context is not needed anymore, you can remove it and the corresponding 759directory by putting the name of the context to the ``rm_contexts`` file. :: 760 761 # echo foo > rm_contexts 762 # ls foo 763 # ls: cannot access 'foo': No such file or directory 764 765Note that ``mk_contexts``, ``rm_contexts``, and ``monitor_on`` files are in the 766root directory only. 767 768 769.. _tracepoint: 770 771Tracepoint for Monitoring Results 772================================= 773 774DAMON provides the monitoring results via a tracepoint, 775``damon:damon_aggregated``. While the monitoring is turned on, you could 776record the tracepoint events and show results using tracepoint supporting tools 777like ``perf``. For example:: 778 779 # echo on > monitor_on 780 # perf record -e damon:damon_aggregated & 781 # sleep 5 782 # kill 9 $(pidof perf) 783 # echo off > monitor_on 784 # perf script 785