1.. SPDX-License-Identifier: GPL-2.0 2 3=============== 4Detailed Usages 5=============== 6 7DAMON provides below three interfaces for different users. 8 9- *DAMON user space tool.* 10 `This <https://github.com/awslabs/damo>`_ is for privileged people such as 11 system administrators who want a just-working human-friendly interface. 12 Using this, users can use the DAMON’s major features in a human-friendly way. 13 It may not be highly tuned for special cases, though. It supports both 14 virtual and physical address spaces monitoring. For more detail, please 15 refer to its `usage document 16 <https://github.com/awslabs/damo/blob/next/USAGE.md>`_. 17- *debugfs interface.* 18 :ref:`This <debugfs_interface>` is for privileged user space programmers who 19 want more optimized use of DAMON. Using this, users can use DAMON’s major 20 features by reading from and writing to special debugfs files. Therefore, 21 you can write and use your personalized DAMON debugfs wrapper programs that 22 reads/writes the debugfs files instead of you. The `DAMON user space tool 23 <https://github.com/awslabs/damo>`_ is one example of such programs. It 24 supports both virtual and physical address spaces monitoring. Note that this 25 interface provides only simple :ref:`statistics <damos_stats>` for the 26 monitoring results. For detailed monitoring results, DAMON provides a 27 :ref:`tracepoint <tracepoint>`. 28- *Kernel Space Programming Interface.* 29 :doc:`This </vm/damon/api>` is for kernel space programmers. Using this, 30 users can utilize every feature of DAMON most flexibly and efficiently by 31 writing kernel space DAMON application programs for you. You can even extend 32 DAMON for various address spaces. For detail, please refer to the interface 33 :doc:`document </vm/damon/api>`. 34 35 36.. _debugfs_interface: 37 38debugfs Interface 39================= 40 41DAMON exports eight files, ``attrs``, ``target_ids``, ``init_regions``, 42``schemes``, ``monitor_on``, ``kdamond_pid``, ``mk_contexts`` and 43``rm_contexts`` under its debugfs directory, ``<debugfs>/damon/``. 44 45 46Attributes 47---------- 48 49Users can get and set the ``sampling interval``, ``aggregation interval``, 50``regions update interval``, and min/max number of monitoring target regions by 51reading from and writing to the ``attrs`` file. To know about the monitoring 52attributes in detail, please refer to the :doc:`/vm/damon/design`. For 53example, below commands set those values to 5 ms, 100 ms, 1,000 ms, 10 and 541000, and then check it again:: 55 56 # cd <debugfs>/damon 57 # echo 5000 100000 1000000 10 1000 > attrs 58 # cat attrs 59 5000 100000 1000000 10 1000 60 61 62Target IDs 63---------- 64 65Some types of address spaces supports multiple monitoring target. For example, 66the virtual memory address spaces monitoring can have multiple processes as the 67monitoring targets. Users can set the targets by writing relevant id values of 68the targets to, and get the ids of the current targets by reading from the 69``target_ids`` file. In case of the virtual address spaces monitoring, the 70values should be pids of the monitoring target processes. For example, below 71commands set processes having pids 42 and 4242 as the monitoring targets and 72check it again:: 73 74 # cd <debugfs>/damon 75 # echo 42 4242 > target_ids 76 # cat target_ids 77 42 4242 78 79Users can also monitor the physical memory address space of the system by 80writing a special keyword, "``paddr\n``" to the file. Because physical address 81space monitoring doesn't support multiple targets, reading the file will show a 82fake value, ``42``, as below:: 83 84 # cd <debugfs>/damon 85 # echo paddr > target_ids 86 # cat target_ids 87 42 88 89Note that setting the target ids doesn't start the monitoring. 90 91 92Initial Monitoring Target Regions 93--------------------------------- 94 95In case of the virtual address space monitoring, DAMON automatically sets and 96updates the monitoring target regions so that entire memory mappings of target 97processes can be covered. However, users can want to limit the monitoring 98region to specific address ranges, such as the heap, the stack, or specific 99file-mapped area. Or, some users can know the initial access pattern of their 100workloads and therefore want to set optimal initial regions for the 'adaptive 101regions adjustment'. 102 103In contrast, DAMON do not automatically sets and updates the monitoring target 104regions in case of physical memory monitoring. Therefore, users should set the 105monitoring target regions by themselves. 106 107In such cases, users can explicitly set the initial monitoring target regions 108as they want, by writing proper values to the ``init_regions`` file. Each line 109of the input should represent one region in below form.:: 110 111 <target id> <start address> <end address> 112 113The ``target id`` should already in ``target_ids`` file, and the regions should 114be passed in address order. For example, below commands will set a couple of 115address ranges, ``1-100`` and ``100-200`` as the initial monitoring target 116region of process 42, and another couple of address ranges, ``20-40`` and 117``50-100`` as that of process 4242.:: 118 119 # cd <debugfs>/damon 120 # echo "42 1 100 121 42 100 200 122 4242 20 40 123 4242 50 100" > init_regions 124 125Note that this sets the initial monitoring target regions only. In case of 126virtual memory monitoring, DAMON will automatically updates the boundary of the 127regions after one ``regions update interval``. Therefore, users should set the 128``regions update interval`` large enough in this case, if they don't want the 129update. 130 131 132Schemes 133------- 134 135For usual DAMON-based data access aware memory management optimizations, users 136would simply want the system to apply a memory management action to a memory 137region of a specific access pattern. DAMON receives such formalized operation 138schemes from the user and applies those to the target processes. 139 140Users can get and set the schemes by reading from and writing to ``schemes`` 141debugfs file. Reading the file also shows the statistics of each scheme. To 142the file, each of the schemes should be represented in each line in below 143form:: 144 145 <target access pattern> <action> <quota> <watermarks> 146 147You can disable schemes by simply writing an empty string to the file. 148 149Target Access Pattern 150~~~~~~~~~~~~~~~~~~~~~ 151 152The ``<target access pattern>`` is constructed with three ranges in below 153form:: 154 155 min-size max-size min-acc max-acc min-age max-age 156 157Specifically, bytes for the size of regions (``min-size`` and ``max-size``), 158number of monitored accesses per aggregate interval for access frequency 159(``min-acc`` and ``max-acc``), number of aggregate intervals for the age of 160regions (``min-age`` and ``max-age``) are specified. Note that the ranges are 161closed interval. 162 163Action 164~~~~~~ 165 166The ``<action>`` is a predefined integer for memory management actions, which 167DAMON will apply to the regions having the target access pattern. The 168supported numbers and their meanings are as below. 169 170 - 0: Call ``madvise()`` for the region with ``MADV_WILLNEED`` 171 - 1: Call ``madvise()`` for the region with ``MADV_COLD`` 172 - 2: Call ``madvise()`` for the region with ``MADV_PAGEOUT`` 173 - 3: Call ``madvise()`` for the region with ``MADV_HUGEPAGE`` 174 - 4: Call ``madvise()`` for the region with ``MADV_NOHUGEPAGE`` 175 - 5: Do nothing but count the statistics 176 177Quota 178~~~~~ 179 180Optimal ``target access pattern`` for each ``action`` is workload dependent, so 181not easy to find. Worse yet, setting a scheme of some action too aggressive 182can cause severe overhead. To avoid such overhead, users can limit time and 183size quota for the scheme via the ``<quota>`` in below form:: 184 185 <ms> <sz> <reset interval> <priority weights> 186 187This makes DAMON to try to use only up to ``<ms>`` milliseconds for applying 188the action to memory regions of the ``target access pattern`` within the 189``<reset interval>`` milliseconds, and to apply the action to only up to 190``<sz>`` bytes of memory regions within the ``<reset interval>``. Setting both 191``<ms>`` and ``<sz>`` zero disables the quota limits. 192 193When the quota limit is expected to be exceeded, DAMON prioritizes found memory 194regions of the ``target access pattern`` based on their size, access frequency, 195and age. For personalized prioritization, users can set the weights for the 196three properties in ``<priority weights>`` in below form:: 197 198 <size weight> <access frequency weight> <age weight> 199 200Watermarks 201~~~~~~~~~~ 202 203Some schemes would need to run based on current value of the system's specific 204metrics like free memory ratio. For such cases, users can specify watermarks 205for the condition.:: 206 207 <metric> <check interval> <high mark> <middle mark> <low mark> 208 209``<metric>`` is a predefined integer for the metric to be checked. The 210supported numbers and their meanings are as below. 211 212 - 0: Ignore the watermarks 213 - 1: System's free memory rate (per thousand) 214 215The value of the metric is checked every ``<check interval>`` microseconds. 216 217If the value is higher than ``<high mark>`` or lower than ``<low mark>``, the 218scheme is deactivated. If the value is lower than ``<mid mark>``, the scheme 219is activated. 220 221.. _damos_stats: 222 223Statistics 224~~~~~~~~~~ 225 226It also counts the total number and bytes of regions that each scheme is 227applied. This statistics can be used for online analysis or tuning of the 228schemes. 229 230The statistics can be shown by reading the ``schemes`` file. Reading the file 231will show each scheme you entered in each line, and the two numbers for the 232statistics will be added at the end of each line. 233 234Example 235~~~~~~~ 236 237Below commands applies a scheme saying "If a memory region of size in [4KiB, 2388KiB] is showing accesses per aggregate interval in [0, 5] for aggregate 239interval in [10, 20], page out the region. For the paging out, use only up to 24010ms per second, and also don't page out more than 1GiB per second. Under the 241limitation, page out memory regions having longer age first. Also, check the 242free memory rate of the system every 5 seconds, start the monitoring and paging 243out when the free memory rate becomes lower than 50%, but stop it if the free 244memory rate becomes larger than 60%, or lower than 30%".:: 245 246 # cd <debugfs>/damon 247 # scheme="4096 8192 0 5 10 20 2" # target access pattern and action 248 # scheme+=" 10 $((1024*1024*1024)) 1000" # quotas 249 # scheme+=" 0 0 100" # prioritization weights 250 # scheme+=" 1 5000000 600 500 300" # watermarks 251 # echo "$scheme" > schemes 252 253 254Turning On/Off 255-------------- 256 257Setting the files as described above doesn't incur effect unless you explicitly 258start the monitoring. You can start, stop, and check the current status of the 259monitoring by writing to and reading from the ``monitor_on`` file. Writing 260``on`` to the file starts the monitoring of the targets with the attributes. 261Writing ``off`` to the file stops those. DAMON also stops if every target 262process is terminated. Below example commands turn on, off, and check the 263status of DAMON:: 264 265 # cd <debugfs>/damon 266 # echo on > monitor_on 267 # echo off > monitor_on 268 # cat monitor_on 269 off 270 271Please note that you cannot write to the above-mentioned debugfs files while 272the monitoring is turned on. If you write to the files while DAMON is running, 273an error code such as ``-EBUSY`` will be returned. 274 275 276Monitoring Thread PID 277--------------------- 278 279DAMON does requested monitoring with a kernel thread called ``kdamond``. You 280can get the pid of the thread by reading the ``kdamond_pid`` file. When the 281monitoring is turned off, reading the file returns ``none``. :: 282 283 # cd <debugfs>/damon 284 # cat monitor_on 285 off 286 # cat kdamond_pid 287 none 288 # echo on > monitor_on 289 # cat kdamond_pid 290 18594 291 292 293Using Multiple Monitoring Threads 294--------------------------------- 295 296One ``kdamond`` thread is created for each monitoring context. You can create 297and remove monitoring contexts for multiple ``kdamond`` required use case using 298the ``mk_contexts`` and ``rm_contexts`` files. 299 300Writing the name of the new context to the ``mk_contexts`` file creates a 301directory of the name on the DAMON debugfs directory. The directory will have 302DAMON debugfs files for the context. :: 303 304 # cd <debugfs>/damon 305 # ls foo 306 # ls: cannot access 'foo': No such file or directory 307 # echo foo > mk_contexts 308 # ls foo 309 # attrs init_regions kdamond_pid schemes target_ids 310 311If the context is not needed anymore, you can remove it and the corresponding 312directory by putting the name of the context to the ``rm_contexts`` file. :: 313 314 # echo foo > rm_contexts 315 # ls foo 316 # ls: cannot access 'foo': No such file or directory 317 318Note that ``mk_contexts``, ``rm_contexts``, and ``monitor_on`` files are in the 319root directory only. 320 321 322.. _tracepoint: 323 324Tracepoint for Monitoring Results 325================================= 326 327DAMON provides the monitoring results via a tracepoint, 328``damon:damon_aggregated``. While the monitoring is turned on, you could 329record the tracepoint events and show results using tracepoint supporting tools 330like ``perf``. For example:: 331 332 # echo on > monitor_on 333 # perf record -e damon:damon_aggregated & 334 # sleep 5 335 # kill 9 $(pidof perf) 336 # echo off > monitor_on 337 # perf script 338