1.. SPDX-License-Identifier: GPL-2.0 2 3=============== 4Detailed Usages 5=============== 6 7DAMON provides below three interfaces for different users. 8 9- *DAMON user space tool.* 10 `This <https://github.com/awslabs/damo>`_ is for privileged people such as 11 system administrators who want a just-working human-friendly interface. 12 Using this, users can use the DAMON’s major features in a human-friendly way. 13 It may not be highly tuned for special cases, though. It supports both 14 virtual and physical address spaces monitoring. For more detail, please 15 refer to its `usage document 16 <https://github.com/awslabs/damo/blob/next/USAGE.md>`_. 17- *debugfs interface.* 18 :ref:`This <debugfs_interface>` is for privileged user space programmers who 19 want more optimized use of DAMON. Using this, users can use DAMON’s major 20 features by reading from and writing to special debugfs files. Therefore, 21 you can write and use your personalized DAMON debugfs wrapper programs that 22 reads/writes the debugfs files instead of you. The `DAMON user space tool 23 <https://github.com/awslabs/damo>`_ is one example of such programs. It 24 supports both virtual and physical address spaces monitoring. Note that this 25 interface provides only simple :ref:`statistics <damos_stats>` for the 26 monitoring results. For detailed monitoring results, DAMON provides a 27 :ref:`tracepoint <tracepoint>`. 28- *Kernel Space Programming Interface.* 29 :doc:`This </vm/damon/api>` is for kernel space programmers. Using this, 30 users can utilize every feature of DAMON most flexibly and efficiently by 31 writing kernel space DAMON application programs for you. You can even extend 32 DAMON for various address spaces. For detail, please refer to the interface 33 :doc:`document </vm/damon/api>`. 34 35 36.. _debugfs_interface: 37 38debugfs Interface 39================= 40 41DAMON exports eight files, ``attrs``, ``target_ids``, ``init_regions``, 42``schemes``, ``monitor_on``, ``kdamond_pid``, ``mk_contexts`` and 43``rm_contexts`` under its debugfs directory, ``<debugfs>/damon/``. 44 45 46Attributes 47---------- 48 49Users can get and set the ``sampling interval``, ``aggregation interval``, 50``regions update interval``, and min/max number of monitoring target regions by 51reading from and writing to the ``attrs`` file. To know about the monitoring 52attributes in detail, please refer to the :doc:`/vm/damon/design`. For 53example, below commands set those values to 5 ms, 100 ms, 1,000 ms, 10 and 541000, and then check it again:: 55 56 # cd <debugfs>/damon 57 # echo 5000 100000 1000000 10 1000 > attrs 58 # cat attrs 59 5000 100000 1000000 10 1000 60 61 62Target IDs 63---------- 64 65Some types of address spaces supports multiple monitoring target. For example, 66the virtual memory address spaces monitoring can have multiple processes as the 67monitoring targets. Users can set the targets by writing relevant id values of 68the targets to, and get the ids of the current targets by reading from the 69``target_ids`` file. In case of the virtual address spaces monitoring, the 70values should be pids of the monitoring target processes. For example, below 71commands set processes having pids 42 and 4242 as the monitoring targets and 72check it again:: 73 74 # cd <debugfs>/damon 75 # echo 42 4242 > target_ids 76 # cat target_ids 77 42 4242 78 79Users can also monitor the physical memory address space of the system by 80writing a special keyword, "``paddr\n``" to the file. Because physical address 81space monitoring doesn't support multiple targets, reading the file will show a 82fake value, ``42``, as below:: 83 84 # cd <debugfs>/damon 85 # echo paddr > target_ids 86 # cat target_ids 87 42 88 89Note that setting the target ids doesn't start the monitoring. 90 91 92Initial Monitoring Target Regions 93--------------------------------- 94 95In case of the virtual address space monitoring, DAMON automatically sets and 96updates the monitoring target regions so that entire memory mappings of target 97processes can be covered. However, users can want to limit the monitoring 98region to specific address ranges, such as the heap, the stack, or specific 99file-mapped area. Or, some users can know the initial access pattern of their 100workloads and therefore want to set optimal initial regions for the 'adaptive 101regions adjustment'. 102 103In contrast, DAMON do not automatically sets and updates the monitoring target 104regions in case of physical memory monitoring. Therefore, users should set the 105monitoring target regions by themselves. 106 107In such cases, users can explicitly set the initial monitoring target regions 108as they want, by writing proper values to the ``init_regions`` file. Each line 109of the input should represent one region in below form.:: 110 111 <target idx> <start address> <end address> 112 113The ``target idx`` should be the index of the target in ``target_ids`` file, 114starting from ``0``, and the regions should be passed in address order. For 115example, below commands will set a couple of address ranges, ``1-100`` and 116``100-200`` as the initial monitoring target region of pid 42, which is the 117first one (index ``0``) in ``target_ids``, and another couple of address 118ranges, ``20-40`` and ``50-100`` as that of pid 4242, which is the second one 119(index ``1``) in ``target_ids``.:: 120 121 # cd <debugfs>/damon 122 # cat target_ids 123 42 4242 124 # echo "0 1 100 125 0 100 200 126 1 20 40 127 1 50 100" > init_regions 128 129Note that this sets the initial monitoring target regions only. In case of 130virtual memory monitoring, DAMON will automatically updates the boundary of the 131regions after one ``regions update interval``. Therefore, users should set the 132``regions update interval`` large enough in this case, if they don't want the 133update. 134 135 136Schemes 137------- 138 139For usual DAMON-based data access aware memory management optimizations, users 140would simply want the system to apply a memory management action to a memory 141region of a specific access pattern. DAMON receives such formalized operation 142schemes from the user and applies those to the target processes. 143 144Users can get and set the schemes by reading from and writing to ``schemes`` 145debugfs file. Reading the file also shows the statistics of each scheme. To 146the file, each of the schemes should be represented in each line in below 147form:: 148 149 <target access pattern> <action> <quota> <watermarks> 150 151You can disable schemes by simply writing an empty string to the file. 152 153Target Access Pattern 154~~~~~~~~~~~~~~~~~~~~~ 155 156The ``<target access pattern>`` is constructed with three ranges in below 157form:: 158 159 min-size max-size min-acc max-acc min-age max-age 160 161Specifically, bytes for the size of regions (``min-size`` and ``max-size``), 162number of monitored accesses per aggregate interval for access frequency 163(``min-acc`` and ``max-acc``), number of aggregate intervals for the age of 164regions (``min-age`` and ``max-age``) are specified. Note that the ranges are 165closed interval. 166 167Action 168~~~~~~ 169 170The ``<action>`` is a predefined integer for memory management actions, which 171DAMON will apply to the regions having the target access pattern. The 172supported numbers and their meanings are as below. 173 174 - 0: Call ``madvise()`` for the region with ``MADV_WILLNEED`` 175 - 1: Call ``madvise()`` for the region with ``MADV_COLD`` 176 - 2: Call ``madvise()`` for the region with ``MADV_PAGEOUT`` 177 - 3: Call ``madvise()`` for the region with ``MADV_HUGEPAGE`` 178 - 4: Call ``madvise()`` for the region with ``MADV_NOHUGEPAGE`` 179 - 5: Do nothing but count the statistics 180 181Quota 182~~~~~ 183 184Optimal ``target access pattern`` for each ``action`` is workload dependent, so 185not easy to find. Worse yet, setting a scheme of some action too aggressive 186can cause severe overhead. To avoid such overhead, users can limit time and 187size quota for the scheme via the ``<quota>`` in below form:: 188 189 <ms> <sz> <reset interval> <priority weights> 190 191This makes DAMON to try to use only up to ``<ms>`` milliseconds for applying 192the action to memory regions of the ``target access pattern`` within the 193``<reset interval>`` milliseconds, and to apply the action to only up to 194``<sz>`` bytes of memory regions within the ``<reset interval>``. Setting both 195``<ms>`` and ``<sz>`` zero disables the quota limits. 196 197When the quota limit is expected to be exceeded, DAMON prioritizes found memory 198regions of the ``target access pattern`` based on their size, access frequency, 199and age. For personalized prioritization, users can set the weights for the 200three properties in ``<priority weights>`` in below form:: 201 202 <size weight> <access frequency weight> <age weight> 203 204Watermarks 205~~~~~~~~~~ 206 207Some schemes would need to run based on current value of the system's specific 208metrics like free memory ratio. For such cases, users can specify watermarks 209for the condition.:: 210 211 <metric> <check interval> <high mark> <middle mark> <low mark> 212 213``<metric>`` is a predefined integer for the metric to be checked. The 214supported numbers and their meanings are as below. 215 216 - 0: Ignore the watermarks 217 - 1: System's free memory rate (per thousand) 218 219The value of the metric is checked every ``<check interval>`` microseconds. 220 221If the value is higher than ``<high mark>`` or lower than ``<low mark>``, the 222scheme is deactivated. If the value is lower than ``<mid mark>``, the scheme 223is activated. 224 225.. _damos_stats: 226 227Statistics 228~~~~~~~~~~ 229 230It also counts the total number and bytes of regions that each scheme is tried 231to be applied, the two numbers for the regions that each scheme is successfully 232applied, and the total number of the quota limit exceeds. This statistics can 233be used for online analysis or tuning of the schemes. 234 235The statistics can be shown by reading the ``schemes`` file. Reading the file 236will show each scheme you entered in each line, and the five numbers for the 237statistics will be added at the end of each line. 238 239Example 240~~~~~~~ 241 242Below commands applies a scheme saying "If a memory region of size in [4KiB, 2438KiB] is showing accesses per aggregate interval in [0, 5] for aggregate 244interval in [10, 20], page out the region. For the paging out, use only up to 24510ms per second, and also don't page out more than 1GiB per second. Under the 246limitation, page out memory regions having longer age first. Also, check the 247free memory rate of the system every 5 seconds, start the monitoring and paging 248out when the free memory rate becomes lower than 50%, but stop it if the free 249memory rate becomes larger than 60%, or lower than 30%".:: 250 251 # cd <debugfs>/damon 252 # scheme="4096 8192 0 5 10 20 2" # target access pattern and action 253 # scheme+=" 10 $((1024*1024*1024)) 1000" # quotas 254 # scheme+=" 0 0 100" # prioritization weights 255 # scheme+=" 1 5000000 600 500 300" # watermarks 256 # echo "$scheme" > schemes 257 258 259Turning On/Off 260-------------- 261 262Setting the files as described above doesn't incur effect unless you explicitly 263start the monitoring. You can start, stop, and check the current status of the 264monitoring by writing to and reading from the ``monitor_on`` file. Writing 265``on`` to the file starts the monitoring of the targets with the attributes. 266Writing ``off`` to the file stops those. DAMON also stops if every target 267process is terminated. Below example commands turn on, off, and check the 268status of DAMON:: 269 270 # cd <debugfs>/damon 271 # echo on > monitor_on 272 # echo off > monitor_on 273 # cat monitor_on 274 off 275 276Please note that you cannot write to the above-mentioned debugfs files while 277the monitoring is turned on. If you write to the files while DAMON is running, 278an error code such as ``-EBUSY`` will be returned. 279 280 281Monitoring Thread PID 282--------------------- 283 284DAMON does requested monitoring with a kernel thread called ``kdamond``. You 285can get the pid of the thread by reading the ``kdamond_pid`` file. When the 286monitoring is turned off, reading the file returns ``none``. :: 287 288 # cd <debugfs>/damon 289 # cat monitor_on 290 off 291 # cat kdamond_pid 292 none 293 # echo on > monitor_on 294 # cat kdamond_pid 295 18594 296 297 298Using Multiple Monitoring Threads 299--------------------------------- 300 301One ``kdamond`` thread is created for each monitoring context. You can create 302and remove monitoring contexts for multiple ``kdamond`` required use case using 303the ``mk_contexts`` and ``rm_contexts`` files. 304 305Writing the name of the new context to the ``mk_contexts`` file creates a 306directory of the name on the DAMON debugfs directory. The directory will have 307DAMON debugfs files for the context. :: 308 309 # cd <debugfs>/damon 310 # ls foo 311 # ls: cannot access 'foo': No such file or directory 312 # echo foo > mk_contexts 313 # ls foo 314 # attrs init_regions kdamond_pid schemes target_ids 315 316If the context is not needed anymore, you can remove it and the corresponding 317directory by putting the name of the context to the ``rm_contexts`` file. :: 318 319 # echo foo > rm_contexts 320 # ls foo 321 # ls: cannot access 'foo': No such file or directory 322 323Note that ``mk_contexts``, ``rm_contexts``, and ``monitor_on`` files are in the 324root directory only. 325 326 327.. _tracepoint: 328 329Tracepoint for Monitoring Results 330================================= 331 332DAMON provides the monitoring results via a tracepoint, 333``damon:damon_aggregated``. While the monitoring is turned on, you could 334record the tracepoint events and show results using tracepoint supporting tools 335like ``perf``. For example:: 336 337 # echo on > monitor_on 338 # perf record -e damon:damon_aggregated & 339 # sleep 5 340 # kill 9 $(pidof perf) 341 # echo off > monitor_on 342 # perf script 343