1.. SPDX-License-Identifier: GPL-2.0 2 3=============== 4Detailed Usages 5=============== 6 7DAMON provides below interfaces for different users. 8 9- *DAMON user space tool.* 10 `This <https://github.com/awslabs/damo>`_ is for privileged people such as 11 system administrators who want a just-working human-friendly interface. 12 Using this, users can use the DAMON’s major features in a human-friendly way. 13 It may not be highly tuned for special cases, though. For more detail, 14 please refer to its `usage document 15 <https://github.com/awslabs/damo/blob/next/USAGE.md>`_. 16- *sysfs interface.* 17 :ref:`This <sysfs_interface>` is for privileged user space programmers who 18 want more optimized use of DAMON. Using this, users can use DAMON’s major 19 features by reading from and writing to special sysfs files. Therefore, 20 you can write and use your personalized DAMON sysfs wrapper programs that 21 reads/writes the sysfs files instead of you. The `DAMON user space tool 22 <https://github.com/awslabs/damo>`_ is one example of such programs. 23- *debugfs interface. (DEPRECATED!)* 24 :ref:`This <debugfs_interface>` is almost identical to :ref:`sysfs interface 25 <sysfs_interface>`. This is deprecated, so users should move to the 26 :ref:`sysfs interface <sysfs_interface>`. If you depend on this and cannot 27 move, please report your usecase to damon@lists.linux.dev and 28 linux-mm@kvack.org. 29- *Kernel Space Programming Interface.* 30 :doc:`This </mm/damon/api>` is for kernel space programmers. Using this, 31 users can utilize every feature of DAMON most flexibly and efficiently by 32 writing kernel space DAMON application programs for you. You can even extend 33 DAMON for various address spaces. For detail, please refer to the interface 34 :doc:`document </mm/damon/api>`. 35 36.. _sysfs_interface: 37 38sysfs Interface 39=============== 40 41DAMON sysfs interface is built when ``CONFIG_DAMON_SYSFS`` is defined. It 42creates multiple directories and files under its sysfs directory, 43``<sysfs>/kernel/mm/damon/``. You can control DAMON by writing to and reading 44from the files under the directory. 45 46For a short example, users can monitor the virtual address space of a given 47workload as below. :: 48 49 # cd /sys/kernel/mm/damon/admin/ 50 # echo 1 > kdamonds/nr_kdamonds && echo 1 > kdamonds/0/contexts/nr_contexts 51 # echo vaddr > kdamonds/0/contexts/0/operations 52 # echo 1 > kdamonds/0/contexts/0/targets/nr_targets 53 # echo $(pidof <workload>) > kdamonds/0/contexts/0/targets/0/pid_target 54 # echo on > kdamonds/0/state 55 56Files Hierarchy 57--------------- 58 59The files hierarchy of DAMON sysfs interface is shown below. In the below 60figure, parents-children relations are represented with indentations, each 61directory is having ``/`` suffix, and files in each directory are separated by 62comma (","). :: 63 64 /sys/kernel/mm/damon/admin 65 │ kdamonds/nr_kdamonds 66 │ │ 0/state,pid 67 │ │ │ contexts/nr_contexts 68 │ │ │ │ 0/avail_operations,operations 69 │ │ │ │ │ monitoring_attrs/ 70 │ │ │ │ │ │ intervals/sample_us,aggr_us,update_us 71 │ │ │ │ │ │ nr_regions/min,max 72 │ │ │ │ │ targets/nr_targets 73 │ │ │ │ │ │ 0/pid_target 74 │ │ │ │ │ │ │ regions/nr_regions 75 │ │ │ │ │ │ │ │ 0/start,end 76 │ │ │ │ │ │ │ │ ... 77 │ │ │ │ │ │ ... 78 │ │ │ │ │ schemes/nr_schemes 79 │ │ │ │ │ │ 0/action 80 │ │ │ │ │ │ │ access_pattern/ 81 │ │ │ │ │ │ │ │ sz/min,max 82 │ │ │ │ │ │ │ │ nr_accesses/min,max 83 │ │ │ │ │ │ │ │ age/min,max 84 │ │ │ │ │ │ │ quotas/ms,bytes,reset_interval_ms 85 │ │ │ │ │ │ │ │ weights/sz_permil,nr_accesses_permil,age_permil 86 │ │ │ │ │ │ │ watermarks/metric,interval_us,high,mid,low 87 │ │ │ │ │ │ │ filters/nr_filters 88 │ │ │ │ │ │ │ │ 0/type,matching,memcg_id 89 │ │ │ │ │ │ │ stats/nr_tried,sz_tried,nr_applied,sz_applied,qt_exceeds 90 │ │ │ │ │ │ │ tried_regions/total_bytes 91 │ │ │ │ │ │ │ │ 0/start,end,nr_accesses,age 92 │ │ │ │ │ │ │ │ ... 93 │ │ │ │ │ │ ... 94 │ │ │ │ ... 95 │ │ ... 96 97Root 98---- 99 100The root of the DAMON sysfs interface is ``<sysfs>/kernel/mm/damon/``, and it 101has one directory named ``admin``. The directory contains the files for 102privileged user space programs' control of DAMON. User space tools or deamons 103having the root permission could use this directory. 104 105kdamonds/ 106--------- 107 108The monitoring-related information including request specifications and results 109are called DAMON context. DAMON executes each context with a kernel thread 110called kdamond, and multiple kdamonds could run in parallel. 111 112Under the ``admin`` directory, one directory, ``kdamonds``, which has files for 113controlling the kdamonds exist. In the beginning, this directory has only one 114file, ``nr_kdamonds``. Writing a number (``N``) to the file creates the number 115of child directories named ``0`` to ``N-1``. Each directory represents each 116kdamond. 117 118kdamonds/<N>/ 119------------- 120 121In each kdamond directory, two files (``state`` and ``pid``) and one directory 122(``contexts``) exist. 123 124Reading ``state`` returns ``on`` if the kdamond is currently running, or 125``off`` if it is not running. Writing ``on`` or ``off`` makes the kdamond be 126in the state. Writing ``commit`` to the ``state`` file makes kdamond reads the 127user inputs in the sysfs files except ``state`` file again. Writing 128``update_schemes_stats`` to ``state`` file updates the contents of stats files 129for each DAMON-based operation scheme of the kdamond. For details of the 130stats, please refer to :ref:`stats section <sysfs_schemes_stats>`. 131 132Writing ``update_schemes_tried_regions`` to ``state`` file updates the 133DAMON-based operation scheme action tried regions directory for each 134DAMON-based operation scheme of the kdamond. Writing 135``update_schemes_tried_bytes`` to ``state`` file updates only 136``.../tried_regions/total_bytes`` files. Writing 137``clear_schemes_tried_regions`` to ``state`` file clears the DAMON-based 138operating scheme action tried regions directory for each DAMON-based operation 139scheme of the kdamond. For details of the DAMON-based operation scheme action 140tried regions directory, please refer to :ref:`tried_regions section 141<sysfs_schemes_tried_regions>`. 142 143If the state is ``on``, reading ``pid`` shows the pid of the kdamond thread. 144 145``contexts`` directory contains files for controlling the monitoring contexts 146that this kdamond will execute. 147 148kdamonds/<N>/contexts/ 149---------------------- 150 151In the beginning, this directory has only one file, ``nr_contexts``. Writing a 152number (``N``) to the file creates the number of child directories named as 153``0`` to ``N-1``. Each directory represents each monitoring context. At the 154moment, only one context per kdamond is supported, so only ``0`` or ``1`` can 155be written to the file. 156 157.. _sysfs_contexts: 158 159contexts/<N>/ 160------------- 161 162In each context directory, two files (``avail_operations`` and ``operations``) 163and three directories (``monitoring_attrs``, ``targets``, and ``schemes``) 164exist. 165 166DAMON supports multiple types of monitoring operations, including those for 167virtual address space and the physical address space. You can get the list of 168available monitoring operations set on the currently running kernel by reading 169``avail_operations`` file. Based on the kernel configuration, the file will 170list some or all of below keywords. 171 172 - vaddr: Monitor virtual address spaces of specific processes 173 - fvaddr: Monitor fixed virtual address ranges 174 - paddr: Monitor the physical address space of the system 175 176Please refer to :ref:`regions sysfs directory <sysfs_regions>` for detailed 177differences between the operations sets in terms of the monitoring target 178regions. 179 180You can set and get what type of monitoring operations DAMON will use for the 181context by writing one of the keywords listed in ``avail_operations`` file and 182reading from the ``operations`` file. 183 184.. _sysfs_monitoring_attrs: 185 186contexts/<N>/monitoring_attrs/ 187------------------------------ 188 189Files for specifying attributes of the monitoring including required quality 190and efficiency of the monitoring are in ``monitoring_attrs`` directory. 191Specifically, two directories, ``intervals`` and ``nr_regions`` exist in this 192directory. 193 194Under ``intervals`` directory, three files for DAMON's sampling interval 195(``sample_us``), aggregation interval (``aggr_us``), and update interval 196(``update_us``) exist. You can set and get the values in micro-seconds by 197writing to and reading from the files. 198 199Under ``nr_regions`` directory, two files for the lower-bound and upper-bound 200of DAMON's monitoring regions (``min`` and ``max``, respectively), which 201controls the monitoring overhead, exist. You can set and get the values by 202writing to and rading from the files. 203 204For more details about the intervals and monitoring regions range, please refer 205to the Design document (:doc:`/mm/damon/design`). 206 207contexts/<N>/targets/ 208--------------------- 209 210In the beginning, this directory has only one file, ``nr_targets``. Writing a 211number (``N``) to the file creates the number of child directories named ``0`` 212to ``N-1``. Each directory represents each monitoring target. 213 214targets/<N>/ 215------------ 216 217In each target directory, one file (``pid_target``) and one directory 218(``regions``) exist. 219 220If you wrote ``vaddr`` to the ``contexts/<N>/operations``, each target should 221be a process. You can specify the process to DAMON by writing the pid of the 222process to the ``pid_target`` file. 223 224.. _sysfs_regions: 225 226targets/<N>/regions 227------------------- 228 229When ``vaddr`` monitoring operations set is being used (``vaddr`` is written to 230the ``contexts/<N>/operations`` file), DAMON automatically sets and updates the 231monitoring target regions so that entire memory mappings of target processes 232can be covered. However, users could want to set the initial monitoring region 233to specific address ranges. 234 235In contrast, DAMON do not automatically sets and updates the monitoring target 236regions when ``fvaddr`` or ``paddr`` monitoring operations sets are being used 237(``fvaddr`` or ``paddr`` have written to the ``contexts/<N>/operations``). 238Therefore, users should set the monitoring target regions by themselves in the 239cases. 240 241For such cases, users can explicitly set the initial monitoring target regions 242as they want, by writing proper values to the files under this directory. 243 244In the beginning, this directory has only one file, ``nr_regions``. Writing a 245number (``N``) to the file creates the number of child directories named ``0`` 246to ``N-1``. Each directory represents each initial monitoring target region. 247 248regions/<N>/ 249------------ 250 251In each region directory, you will find two files (``start`` and ``end``). You 252can set and get the start and end addresses of the initial monitoring target 253region by writing to and reading from the files, respectively. 254 255Each region should not overlap with others. ``end`` of directory ``N`` should 256be equal or smaller than ``start`` of directory ``N+1``. 257 258contexts/<N>/schemes/ 259--------------------- 260 261The directory for DAMON-based Operation Schemes (:ref:`DAMOS 262<damon_design_damos>`). Users can get and set the schemes by reading from and 263writing to files under this directory. 264 265In the beginning, this directory has only one file, ``nr_schemes``. Writing a 266number (``N``) to the file creates the number of child directories named ``0`` 267to ``N-1``. Each directory represents each DAMON-based operation scheme. 268 269schemes/<N>/ 270------------ 271 272In each scheme directory, five directories (``access_pattern``, ``quotas``, 273``watermarks``, ``filters``, ``stats``, and ``tried_regions``) and one file 274(``action``) exist. 275 276The ``action`` file is for setting and getting the scheme's :ref:`action 277<damon_design_damos_action>`. The keywords that can be written to and read 278from the file and their meaning are as below. 279 280Note that support of each action depends on the running DAMON operations set 281:ref:`implementation <sysfs_contexts>`. 282 283 - ``willneed``: Call ``madvise()`` for the region with ``MADV_WILLNEED``. 284 Supported by ``vaddr`` and ``fvaddr`` operations set. 285 - ``cold``: Call ``madvise()`` for the region with ``MADV_COLD``. 286 Supported by ``vaddr`` and ``fvaddr`` operations set. 287 - ``pageout``: Call ``madvise()`` for the region with ``MADV_PAGEOUT``. 288 Supported by ``vaddr``, ``fvaddr`` and ``paddr`` operations set. 289 - ``hugepage``: Call ``madvise()`` for the region with ``MADV_HUGEPAGE``. 290 Supported by ``vaddr`` and ``fvaddr`` operations set. 291 - ``nohugepage``: Call ``madvise()`` for the region with ``MADV_NOHUGEPAGE``. 292 Supported by ``vaddr`` and ``fvaddr`` operations set. 293 - ``lru_prio``: Prioritize the region on its LRU lists. 294 Supported by ``paddr`` operations set. 295 - ``lru_deprio``: Deprioritize the region on its LRU lists. 296 Supported by ``paddr`` operations set. 297 - ``stat``: Do nothing but count the statistics. 298 Supported by all operations sets. 299 300schemes/<N>/access_pattern/ 301--------------------------- 302 303The directory for the target access :ref:`pattern 304<damon_design_damos_access_pattern>` of the given DAMON-based operation scheme. 305 306Under the ``access_pattern`` directory, three directories (``sz``, 307``nr_accesses``, and ``age``) each having two files (``min`` and ``max``) 308exist. You can set and get the access pattern for the given scheme by writing 309to and reading from the ``min`` and ``max`` files under ``sz``, 310``nr_accesses``, and ``age`` directories, respectively. Note that the ``min`` 311and the ``max`` form a closed interval. 312 313schemes/<N>/quotas/ 314------------------- 315 316The directory for the :ref:`quotas <damon_design_damos_quotas>` of the given 317DAMON-based operation scheme. 318 319Under ``quotas`` directory, three files (``ms``, ``bytes``, 320``reset_interval_ms``) and one directory (``weights``) having three files 321(``sz_permil``, ``nr_accesses_permil``, and ``age_permil``) in it exist. 322 323You can set the ``time quota`` in milliseconds, ``size quota`` in bytes, and 324``reset interval`` in milliseconds by writing the values to the three files, 325respectively. Then, DAMON tries to use only up to ``time quota`` milliseconds 326for applying the ``action`` to memory regions of the ``access_pattern``, and to 327apply the action to only up to ``bytes`` bytes of memory regions within the 328``reset_interval_ms``. Setting both ``ms`` and ``bytes`` zero disables the 329quota limits. 330 331You can also set the :ref:`prioritization weights 332<damon_design_damos_quotas_prioritization>` for size, access frequency, and age 333in per-thousand unit by writing the values to the three files under the 334``weights`` directory. 335 336schemes/<N>/watermarks/ 337----------------------- 338 339The directory for the :ref:`watermarks <damon_design_damos_watermarks>` of the 340given DAMON-based operation scheme. 341 342Under the watermarks directory, five files (``metric``, ``interval_us``, 343``high``, ``mid``, and ``low``) for setting the metric, the time interval 344between check of the metric, and the three watermarks exist. You can set and 345get the five values by writing to the files, respectively. 346 347Keywords and meanings of those that can be written to the ``metric`` file are 348as below. 349 350 - none: Ignore the watermarks 351 - free_mem_rate: System's free memory rate (per thousand) 352 353The ``interval`` should written in microseconds unit. 354 355schemes/<N>/filters/ 356-------------------- 357 358The directory for the :ref:`filters <damon_design_damos_filters>` of the given 359DAMON-based operation scheme. 360 361In the beginning, this directory has only one file, ``nr_filters``. Writing a 362number (``N``) to the file creates the number of child directories named ``0`` 363to ``N-1``. Each directory represents each filter. The filters are evaluated 364in the numeric order. 365 366Each filter directory contains five files, namely ``type``, ``matcing``, 367``memcg_path``, ``addr_start``, and ``addr_end``. To ``type`` file, you can 368write one of three special keywords: ``anon`` for anonymous pages, ``memcg`` 369for specific memory cgroup, or ``addr`` for specific address range (an 370open-ended interval) filtering. In case of the memory cgroup filtering, you 371can specify the memory cgroup of the interest by writing the path of the memory 372cgroup from the cgroups mount point to ``memcg_path`` file. In case of the 373address range filtering, you can specify the start and end address of the range 374to ``addr_start`` and ``addr_end`` files, respectively. You can write ``Y`` or 375``N`` to ``matching`` file to filter out pages that does or does not match to 376the type, respectively. Then, the scheme's action will not be applied to the 377pages that specified to be filtered out. 378 379For example, below restricts a DAMOS action to be applied to only non-anonymous 380pages of all memory cgroups except ``/having_care_already``.:: 381 382 # echo 2 > nr_filters 383 # # filter out anonymous pages 384 echo anon > 0/type 385 echo Y > 0/matching 386 # # further filter out all cgroups except one at '/having_care_already' 387 echo memcg > 1/type 388 echo /having_care_already > 1/memcg_path 389 echo N > 1/matching 390 391Note that ``anon`` and ``memcg`` filters are currently supported only when 392``paddr`` `implementation <sysfs_contexts>` is being used. 393 394Also, memory regions that are filtered out by ``addr`` filters are not counted 395as the scheme has tried to those, while regions that filtered out by other type 396filters are counted as the scheme has tried to. The difference is applied to 397:ref:`stats <damos_stats>` and :ref:`tried regions 398<sysfs_schemes_tried_regions>`. 399 400.. _sysfs_schemes_stats: 401 402schemes/<N>/stats/ 403------------------ 404 405DAMON counts the total number and bytes of regions that each scheme is tried to 406be applied, the two numbers for the regions that each scheme is successfully 407applied, and the total number of the quota limit exceeds. This statistics can 408be used for online analysis or tuning of the schemes. 409 410The statistics can be retrieved by reading the files under ``stats`` directory 411(``nr_tried``, ``sz_tried``, ``nr_applied``, ``sz_applied``, and 412``qt_exceeds``), respectively. The files are not updated in real time, so you 413should ask DAMON sysfs interface to updte the content of the files for the 414stats by writing a special keyword, ``update_schemes_stats`` to the relevant 415``kdamonds/<N>/state`` file. 416 417.. _sysfs_schemes_tried_regions: 418 419schemes/<N>/tried_regions/ 420-------------------------- 421 422This directory initially has one file, ``total_bytes``. 423 424When a special keyword, ``update_schemes_tried_regions``, is written to the 425relevant ``kdamonds/<N>/state`` file, DAMON updates the ``total_bytes`` file so 426that reading it returns the total size of the scheme tried regions, and creates 427directories named integer starting from ``0`` under this directory. Each 428directory contains files exposing detailed information about each of the memory 429region that the corresponding scheme's ``action`` has tried to be applied under 430this directory, during next :ref:`aggregation interval 431<sysfs_monitoring_attrs>`. The information includes address range, 432``nr_accesses``, and ``age`` of the region. 433 434Writing ``update_schemes_tried_bytes`` to the relevant ``kdamonds/<N>/state`` 435file will only update the ``total_bytes`` file, and will not create the 436subdirectories. 437 438The directories will be removed when another special keyword, 439``clear_schemes_tried_regions``, is written to the relevant 440``kdamonds/<N>/state`` file. 441 442The expected usage of this directory is investigations of schemes' behaviors, 443and query-like efficient data access monitoring results retrievals. For the 444latter use case, in particular, users can set the ``action`` as ``stat`` and 445set the ``access pattern`` as their interested pattern that they want to query. 446 447tried_regions/<N>/ 448------------------ 449 450In each region directory, you will find four files (``start``, ``end``, 451``nr_accesses``, and ``age``). Reading the files will show the start and end 452addresses, ``nr_accesses``, and ``age`` of the region that corresponding 453DAMON-based operation scheme ``action`` has tried to be applied. 454 455Example 456~~~~~~~ 457 458Below commands applies a scheme saying "If a memory region of size in [4KiB, 4598KiB] is showing accesses per aggregate interval in [0, 5] for aggregate 460interval in [10, 20], page out the region. For the paging out, use only up to 46110ms per second, and also don't page out more than 1GiB per second. Under the 462limitation, page out memory regions having longer age first. Also, check the 463free memory rate of the system every 5 seconds, start the monitoring and paging 464out when the free memory rate becomes lower than 50%, but stop it if the free 465memory rate becomes larger than 60%, or lower than 30%". :: 466 467 # cd <sysfs>/kernel/mm/damon/admin 468 # # populate directories 469 # echo 1 > kdamonds/nr_kdamonds; echo 1 > kdamonds/0/contexts/nr_contexts; 470 # echo 1 > kdamonds/0/contexts/0/schemes/nr_schemes 471 # cd kdamonds/0/contexts/0/schemes/0 472 # # set the basic access pattern and the action 473 # echo 4096 > access_pattern/sz/min 474 # echo 8192 > access_pattern/sz/max 475 # echo 0 > access_pattern/nr_accesses/min 476 # echo 5 > access_pattern/nr_accesses/max 477 # echo 10 > access_pattern/age/min 478 # echo 20 > access_pattern/age/max 479 # echo pageout > action 480 # # set quotas 481 # echo 10 > quotas/ms 482 # echo $((1024*1024*1024)) > quotas/bytes 483 # echo 1000 > quotas/reset_interval_ms 484 # # set watermark 485 # echo free_mem_rate > watermarks/metric 486 # echo 5000000 > watermarks/interval_us 487 # echo 600 > watermarks/high 488 # echo 500 > watermarks/mid 489 # echo 300 > watermarks/low 490 491Please note that it's highly recommended to use user space tools like `damo 492<https://github.com/awslabs/damo>`_ rather than manually reading and writing 493the files as above. Above is only for an example. 494 495.. _debugfs_interface: 496 497debugfs Interface (DEPRECATED!) 498=============================== 499 500.. note:: 501 502 THIS IS DEPRECATED! 503 504 DAMON debugfs interface is deprecated, so users should move to the 505 :ref:`sysfs interface <sysfs_interface>`. If you depend on this and cannot 506 move, please report your usecase to damon@lists.linux.dev and 507 linux-mm@kvack.org. 508 509DAMON exports eight files, ``attrs``, ``target_ids``, ``init_regions``, 510``schemes``, ``monitor_on``, ``kdamond_pid``, ``mk_contexts`` and 511``rm_contexts`` under its debugfs directory, ``<debugfs>/damon/``. 512 513 514Attributes 515---------- 516 517Users can get and set the ``sampling interval``, ``aggregation interval``, 518``update interval``, and min/max number of monitoring target regions by 519reading from and writing to the ``attrs`` file. To know about the monitoring 520attributes in detail, please refer to the :doc:`/mm/damon/design`. For 521example, below commands set those values to 5 ms, 100 ms, 1,000 ms, 10 and 5221000, and then check it again:: 523 524 # cd <debugfs>/damon 525 # echo 5000 100000 1000000 10 1000 > attrs 526 # cat attrs 527 5000 100000 1000000 10 1000 528 529 530Target IDs 531---------- 532 533Some types of address spaces supports multiple monitoring target. For example, 534the virtual memory address spaces monitoring can have multiple processes as the 535monitoring targets. Users can set the targets by writing relevant id values of 536the targets to, and get the ids of the current targets by reading from the 537``target_ids`` file. In case of the virtual address spaces monitoring, the 538values should be pids of the monitoring target processes. For example, below 539commands set processes having pids 42 and 4242 as the monitoring targets and 540check it again:: 541 542 # cd <debugfs>/damon 543 # echo 42 4242 > target_ids 544 # cat target_ids 545 42 4242 546 547Users can also monitor the physical memory address space of the system by 548writing a special keyword, "``paddr\n``" to the file. Because physical address 549space monitoring doesn't support multiple targets, reading the file will show a 550fake value, ``42``, as below:: 551 552 # cd <debugfs>/damon 553 # echo paddr > target_ids 554 # cat target_ids 555 42 556 557Note that setting the target ids doesn't start the monitoring. 558 559 560Initial Monitoring Target Regions 561--------------------------------- 562 563In case of the virtual address space monitoring, DAMON automatically sets and 564updates the monitoring target regions so that entire memory mappings of target 565processes can be covered. However, users can want to limit the monitoring 566region to specific address ranges, such as the heap, the stack, or specific 567file-mapped area. Or, some users can know the initial access pattern of their 568workloads and therefore want to set optimal initial regions for the 'adaptive 569regions adjustment'. 570 571In contrast, DAMON do not automatically sets and updates the monitoring target 572regions in case of physical memory monitoring. Therefore, users should set the 573monitoring target regions by themselves. 574 575In such cases, users can explicitly set the initial monitoring target regions 576as they want, by writing proper values to the ``init_regions`` file. The input 577should be a sequence of three integers separated by white spaces that represent 578one region in below form.:: 579 580 <target idx> <start address> <end address> 581 582The ``target idx`` should be the index of the target in ``target_ids`` file, 583starting from ``0``, and the regions should be passed in address order. For 584example, below commands will set a couple of address ranges, ``1-100`` and 585``100-200`` as the initial monitoring target region of pid 42, which is the 586first one (index ``0``) in ``target_ids``, and another couple of address 587ranges, ``20-40`` and ``50-100`` as that of pid 4242, which is the second one 588(index ``1``) in ``target_ids``.:: 589 590 # cd <debugfs>/damon 591 # cat target_ids 592 42 4242 593 # echo "0 1 100 \ 594 0 100 200 \ 595 1 20 40 \ 596 1 50 100" > init_regions 597 598Note that this sets the initial monitoring target regions only. In case of 599virtual memory monitoring, DAMON will automatically updates the boundary of the 600regions after one ``update interval``. Therefore, users should set the 601``update interval`` large enough in this case, if they don't want the 602update. 603 604 605Schemes 606------- 607 608Users can get and set the DAMON-based operation :ref:`schemes 609<damon_design_damos>` by reading from and writing to ``schemes`` debugfs file. 610Reading the file also shows the statistics of each scheme. To the file, each 611of the schemes should be represented in each line in below form:: 612 613 <target access pattern> <action> <quota> <watermarks> 614 615You can disable schemes by simply writing an empty string to the file. 616 617Target Access Pattern 618~~~~~~~~~~~~~~~~~~~~~ 619 620The target access :ref:`pattern <damon_design_damos_access_pattern>` of the 621scheme. The ``<target access pattern>`` is constructed with three ranges in 622below form:: 623 624 min-size max-size min-acc max-acc min-age max-age 625 626Specifically, bytes for the size of regions (``min-size`` and ``max-size``), 627number of monitored accesses per aggregate interval for access frequency 628(``min-acc`` and ``max-acc``), number of aggregate intervals for the age of 629regions (``min-age`` and ``max-age``) are specified. Note that the ranges are 630closed interval. 631 632Action 633~~~~~~ 634 635The ``<action>`` is a predefined integer for memory management :ref:`actions 636<damon_design_damos_action>`. The supported numbers and their meanings are as 637below. 638 639 - 0: Call ``madvise()`` for the region with ``MADV_WILLNEED``. Ignored if 640 ``target`` is ``paddr``. 641 - 1: Call ``madvise()`` for the region with ``MADV_COLD``. Ignored if 642 ``target`` is ``paddr``. 643 - 2: Call ``madvise()`` for the region with ``MADV_PAGEOUT``. 644 - 3: Call ``madvise()`` for the region with ``MADV_HUGEPAGE``. Ignored if 645 ``target`` is ``paddr``. 646 - 4: Call ``madvise()`` for the region with ``MADV_NOHUGEPAGE``. Ignored if 647 ``target`` is ``paddr``. 648 - 5: Do nothing but count the statistics 649 650Quota 651~~~~~ 652 653Users can set the :ref:`quotas <damon_design_damos_quotas>` of the given scheme 654via the ``<quota>`` in below form:: 655 656 <ms> <sz> <reset interval> <priority weights> 657 658This makes DAMON to try to use only up to ``<ms>`` milliseconds for applying 659the action to memory regions of the ``target access pattern`` within the 660``<reset interval>`` milliseconds, and to apply the action to only up to 661``<sz>`` bytes of memory regions within the ``<reset interval>``. Setting both 662``<ms>`` and ``<sz>`` zero disables the quota limits. 663 664For the :ref:`prioritization <damon_design_damos_quotas_prioritization>`, users 665can set the weights for the three properties in ``<priority weights>`` in below 666form:: 667 668 <size weight> <access frequency weight> <age weight> 669 670Watermarks 671~~~~~~~~~~ 672 673Users can specify :ref:`watermarks <damon_design_damos_watermarks>` of the 674given scheme via ``<watermarks>`` in below form:: 675 676 <metric> <check interval> <high mark> <middle mark> <low mark> 677 678``<metric>`` is a predefined integer for the metric to be checked. The 679supported numbers and their meanings are as below. 680 681 - 0: Ignore the watermarks 682 - 1: System's free memory rate (per thousand) 683 684The value of the metric is checked every ``<check interval>`` microseconds. 685 686If the value is higher than ``<high mark>`` or lower than ``<low mark>``, the 687scheme is deactivated. If the value is lower than ``<mid mark>``, the scheme 688is activated. 689 690.. _damos_stats: 691 692Statistics 693~~~~~~~~~~ 694 695It also counts the total number and bytes of regions that each scheme is tried 696to be applied, the two numbers for the regions that each scheme is successfully 697applied, and the total number of the quota limit exceeds. This statistics can 698be used for online analysis or tuning of the schemes. 699 700The statistics can be shown by reading the ``schemes`` file. Reading the file 701will show each scheme you entered in each line, and the five numbers for the 702statistics will be added at the end of each line. 703 704Example 705~~~~~~~ 706 707Below commands applies a scheme saying "If a memory region of size in [4KiB, 7088KiB] is showing accesses per aggregate interval in [0, 5] for aggregate 709interval in [10, 20], page out the region. For the paging out, use only up to 71010ms per second, and also don't page out more than 1GiB per second. Under the 711limitation, page out memory regions having longer age first. Also, check the 712free memory rate of the system every 5 seconds, start the monitoring and paging 713out when the free memory rate becomes lower than 50%, but stop it if the free 714memory rate becomes larger than 60%, or lower than 30%".:: 715 716 # cd <debugfs>/damon 717 # scheme="4096 8192 0 5 10 20 2" # target access pattern and action 718 # scheme+=" 10 $((1024*1024*1024)) 1000" # quotas 719 # scheme+=" 0 0 100" # prioritization weights 720 # scheme+=" 1 5000000 600 500 300" # watermarks 721 # echo "$scheme" > schemes 722 723 724Turning On/Off 725-------------- 726 727Setting the files as described above doesn't incur effect unless you explicitly 728start the monitoring. You can start, stop, and check the current status of the 729monitoring by writing to and reading from the ``monitor_on`` file. Writing 730``on`` to the file starts the monitoring of the targets with the attributes. 731Writing ``off`` to the file stops those. DAMON also stops if every target 732process is terminated. Below example commands turn on, off, and check the 733status of DAMON:: 734 735 # cd <debugfs>/damon 736 # echo on > monitor_on 737 # echo off > monitor_on 738 # cat monitor_on 739 off 740 741Please note that you cannot write to the above-mentioned debugfs files while 742the monitoring is turned on. If you write to the files while DAMON is running, 743an error code such as ``-EBUSY`` will be returned. 744 745 746Monitoring Thread PID 747--------------------- 748 749DAMON does requested monitoring with a kernel thread called ``kdamond``. You 750can get the pid of the thread by reading the ``kdamond_pid`` file. When the 751monitoring is turned off, reading the file returns ``none``. :: 752 753 # cd <debugfs>/damon 754 # cat monitor_on 755 off 756 # cat kdamond_pid 757 none 758 # echo on > monitor_on 759 # cat kdamond_pid 760 18594 761 762 763Using Multiple Monitoring Threads 764--------------------------------- 765 766One ``kdamond`` thread is created for each monitoring context. You can create 767and remove monitoring contexts for multiple ``kdamond`` required use case using 768the ``mk_contexts`` and ``rm_contexts`` files. 769 770Writing the name of the new context to the ``mk_contexts`` file creates a 771directory of the name on the DAMON debugfs directory. The directory will have 772DAMON debugfs files for the context. :: 773 774 # cd <debugfs>/damon 775 # ls foo 776 # ls: cannot access 'foo': No such file or directory 777 # echo foo > mk_contexts 778 # ls foo 779 # attrs init_regions kdamond_pid schemes target_ids 780 781If the context is not needed anymore, you can remove it and the corresponding 782directory by putting the name of the context to the ``rm_contexts`` file. :: 783 784 # echo foo > rm_contexts 785 # ls foo 786 # ls: cannot access 'foo': No such file or directory 787 788Note that ``mk_contexts``, ``rm_contexts``, and ``monitor_on`` files are in the 789root directory only. 790 791 792.. _tracepoint: 793 794Tracepoint for Monitoring Results 795================================= 796 797Users can get the monitoring results via the :ref:`tried_regions 798<sysfs_schemes_tried_regions>` or a tracepoint, ``damon:damon_aggregated``. 799While the tried regions directory is useful for getting a snapshot, the 800tracepoint is useful for getting a full record of the results. While the 801monitoring is turned on, you could record the tracepoint events and show 802results using tracepoint supporting tools like ``perf``. For example:: 803 804 # echo on > monitor_on 805 # perf record -e damon:damon_aggregated & 806 # sleep 5 807 # kill 9 $(pidof perf) 808 # echo off > monitor_on 809 # perf script 810