1.. SPDX-License-Identifier: GPL-2.0 2 3=============== 4Detailed Usages 5=============== 6 7DAMON provides below three interfaces for different users. 8 9- *DAMON user space tool.* 10 `This <https://github.com/awslabs/damo>`_ is for privileged people such as 11 system administrators who want a just-working human-friendly interface. 12 Using this, users can use the DAMON’s major features in a human-friendly way. 13 It may not be highly tuned for special cases, though. It supports both 14 virtual and physical address spaces monitoring. For more detail, please 15 refer to its `usage document 16 <https://github.com/awslabs/damo/blob/next/USAGE.md>`_. 17- *debugfs interface.* 18 :ref:`This <debugfs_interface>` is for privileged user space programmers who 19 want more optimized use of DAMON. Using this, users can use DAMON’s major 20 features by reading from and writing to special debugfs files. Therefore, 21 you can write and use your personalized DAMON debugfs wrapper programs that 22 reads/writes the debugfs files instead of you. The `DAMON user space tool 23 <https://github.com/awslabs/damo>`_ is one example of such programs. It 24 supports both virtual and physical address spaces monitoring. Note that this 25 interface provides only simple :ref:`statistics <damos_stats>` for the 26 monitoring results. For detailed monitoring results, DAMON provides a 27 :ref:`tracepoint <tracepoint>`. 28- *Kernel Space Programming Interface.* 29 :doc:`This </vm/damon/api>` is for kernel space programmers. Using this, 30 users can utilize every feature of DAMON most flexibly and efficiently by 31 writing kernel space DAMON application programs for you. You can even extend 32 DAMON for various address spaces. For detail, please refer to the interface 33 :doc:`document </vm/damon/api>`. 34 35 36.. _debugfs_interface: 37 38debugfs Interface 39================= 40 41DAMON exports eight files, ``attrs``, ``target_ids``, ``init_regions``, 42``schemes``, ``monitor_on``, ``kdamond_pid``, ``mk_contexts`` and 43``rm_contexts`` under its debugfs directory, ``<debugfs>/damon/``. 44 45 46Attributes 47---------- 48 49Users can get and set the ``sampling interval``, ``aggregation interval``, 50``regions update interval``, and min/max number of monitoring target regions by 51reading from and writing to the ``attrs`` file. To know about the monitoring 52attributes in detail, please refer to the :doc:`/vm/damon/design`. For 53example, below commands set those values to 5 ms, 100 ms, 1,000 ms, 10 and 541000, and then check it again:: 55 56 # cd <debugfs>/damon 57 # echo 5000 100000 1000000 10 1000 > attrs 58 # cat attrs 59 5000 100000 1000000 10 1000 60 61 62Target IDs 63---------- 64 65Some types of address spaces supports multiple monitoring target. For example, 66the virtual memory address spaces monitoring can have multiple processes as the 67monitoring targets. Users can set the targets by writing relevant id values of 68the targets to, and get the ids of the current targets by reading from the 69``target_ids`` file. In case of the virtual address spaces monitoring, the 70values should be pids of the monitoring target processes. For example, below 71commands set processes having pids 42 and 4242 as the monitoring targets and 72check it again:: 73 74 # cd <debugfs>/damon 75 # echo 42 4242 > target_ids 76 # cat target_ids 77 42 4242 78 79Users can also monitor the physical memory address space of the system by 80writing a special keyword, "``paddr\n``" to the file. Because physical address 81space monitoring doesn't support multiple targets, reading the file will show a 82fake value, ``42``, as below:: 83 84 # cd <debugfs>/damon 85 # echo paddr > target_ids 86 # cat target_ids 87 42 88 89Note that setting the target ids doesn't start the monitoring. 90 91 92Initial Monitoring Target Regions 93--------------------------------- 94 95In case of the virtual address space monitoring, DAMON automatically sets and 96updates the monitoring target regions so that entire memory mappings of target 97processes can be covered. However, users can want to limit the monitoring 98region to specific address ranges, such as the heap, the stack, or specific 99file-mapped area. Or, some users can know the initial access pattern of their 100workloads and therefore want to set optimal initial regions for the 'adaptive 101regions adjustment'. 102 103In contrast, DAMON do not automatically sets and updates the monitoring target 104regions in case of physical memory monitoring. Therefore, users should set the 105monitoring target regions by themselves. 106 107In such cases, users can explicitly set the initial monitoring target regions 108as they want, by writing proper values to the ``init_regions`` file. Each line 109of the input should represent one region in below form.:: 110 111 <target id> <start address> <end address> 112 113The ``target id`` should already in ``target_ids`` file, and the regions should 114be passed in address order. For example, below commands will set a couple of 115address ranges, ``1-100`` and ``100-200`` as the initial monitoring target 116region of process 42, and another couple of address ranges, ``20-40`` and 117``50-100`` as that of process 4242.:: 118 119 # cd <debugfs>/damon 120 # echo "42 1 100 121 42 100 200 122 4242 20 40 123 4242 50 100" > init_regions 124 125Note that this sets the initial monitoring target regions only. In case of 126virtual memory monitoring, DAMON will automatically updates the boundary of the 127regions after one ``regions update interval``. Therefore, users should set the 128``regions update interval`` large enough in this case, if they don't want the 129update. 130 131 132Schemes 133------- 134 135For usual DAMON-based data access aware memory management optimizations, users 136would simply want the system to apply a memory management action to a memory 137region of a specific access pattern. DAMON receives such formalized operation 138schemes from the user and applies those to the target processes. 139 140Users can get and set the schemes by reading from and writing to ``schemes`` 141debugfs file. Reading the file also shows the statistics of each scheme. To 142the file, each of the schemes should be represented in each line in below 143form:: 144 145 <target access pattern> <action> <quota> <watermarks> 146 147You can disable schemes by simply writing an empty string to the file. 148 149Target Access Pattern 150~~~~~~~~~~~~~~~~~~~~~ 151 152The ``<target access pattern>`` is constructed with three ranges in below 153form:: 154 155 min-size max-size min-acc max-acc min-age max-age 156 157Specifically, bytes for the size of regions (``min-size`` and ``max-size``), 158number of monitored accesses per aggregate interval for access frequency 159(``min-acc`` and ``max-acc``), number of aggregate intervals for the age of 160regions (``min-age`` and ``max-age``) are specified. Note that the ranges are 161closed interval. 162 163Action 164~~~~~~ 165 166The ``<action>`` is a predefined integer for memory management actions, which 167DAMON will apply to the regions having the target access pattern. The 168supported numbers and their meanings are as below. 169 170 - 0: Call ``madvise()`` for the region with ``MADV_WILLNEED`` 171 - 1: Call ``madvise()`` for the region with ``MADV_COLD`` 172 - 2: Call ``madvise()`` for the region with ``MADV_PAGEOUT`` 173 - 3: Call ``madvise()`` for the region with ``MADV_HUGEPAGE`` 174 - 4: Call ``madvise()`` for the region with ``MADV_NOHUGEPAGE`` 175 - 5: Do nothing but count the statistics 176 177Quota 178~~~~~ 179 180Optimal ``target access pattern`` for each ``action`` is workload dependent, so 181not easy to find. Worse yet, setting a scheme of some action too aggressive 182can cause severe overhead. To avoid such overhead, users can limit time and 183size quota for the scheme via the ``<quota>`` in below form:: 184 185 <ms> <sz> <reset interval> <priority weights> 186 187This makes DAMON to try to use only up to ``<ms>`` milliseconds for applying 188the action to memory regions of the ``target access pattern`` within the 189``<reset interval>`` milliseconds, and to apply the action to only up to 190``<sz>`` bytes of memory regions within the ``<reset interval>``. Setting both 191``<ms>`` and ``<sz>`` zero disables the quota limits. 192 193When the quota limit is expected to be exceeded, DAMON prioritizes found memory 194regions of the ``target access pattern`` based on their size, access frequency, 195and age. For personalized prioritization, users can set the weights for the 196three properties in ``<priority weights>`` in below form:: 197 198 <size weight> <access frequency weight> <age weight> 199 200Watermarks 201~~~~~~~~~~ 202 203Some schemes would need to run based on current value of the system's specific 204metrics like free memory ratio. For such cases, users can specify watermarks 205for the condition.:: 206 207 <metric> <check interval> <high mark> <middle mark> <low mark> 208 209``<metric>`` is a predefined integer for the metric to be checked. The 210supported numbers and their meanings are as below. 211 212 - 0: Ignore the watermarks 213 - 1: System's free memory rate (per thousand) 214 215The value of the metric is checked every ``<check interval>`` microseconds. 216 217If the value is higher than ``<high mark>`` or lower than ``<low mark>``, the 218scheme is deactivated. If the value is lower than ``<mid mark>``, the scheme 219is activated. 220 221.. _damos_stats: 222 223Statistics 224~~~~~~~~~~ 225 226It also counts the total number and bytes of regions that each scheme is tried 227to be applied, the two numbers for the regions that each scheme is successfully 228applied, and the total number of the quota limit exceeds. This statistics can 229be used for online analysis or tuning of the schemes. 230 231The statistics can be shown by reading the ``schemes`` file. Reading the file 232will show each scheme you entered in each line, and the five numbers for the 233statistics will be added at the end of each line. 234 235Example 236~~~~~~~ 237 238Below commands applies a scheme saying "If a memory region of size in [4KiB, 2398KiB] is showing accesses per aggregate interval in [0, 5] for aggregate 240interval in [10, 20], page out the region. For the paging out, use only up to 24110ms per second, and also don't page out more than 1GiB per second. Under the 242limitation, page out memory regions having longer age first. Also, check the 243free memory rate of the system every 5 seconds, start the monitoring and paging 244out when the free memory rate becomes lower than 50%, but stop it if the free 245memory rate becomes larger than 60%, or lower than 30%".:: 246 247 # cd <debugfs>/damon 248 # scheme="4096 8192 0 5 10 20 2" # target access pattern and action 249 # scheme+=" 10 $((1024*1024*1024)) 1000" # quotas 250 # scheme+=" 0 0 100" # prioritization weights 251 # scheme+=" 1 5000000 600 500 300" # watermarks 252 # echo "$scheme" > schemes 253 254 255Turning On/Off 256-------------- 257 258Setting the files as described above doesn't incur effect unless you explicitly 259start the monitoring. You can start, stop, and check the current status of the 260monitoring by writing to and reading from the ``monitor_on`` file. Writing 261``on`` to the file starts the monitoring of the targets with the attributes. 262Writing ``off`` to the file stops those. DAMON also stops if every target 263process is terminated. Below example commands turn on, off, and check the 264status of DAMON:: 265 266 # cd <debugfs>/damon 267 # echo on > monitor_on 268 # echo off > monitor_on 269 # cat monitor_on 270 off 271 272Please note that you cannot write to the above-mentioned debugfs files while 273the monitoring is turned on. If you write to the files while DAMON is running, 274an error code such as ``-EBUSY`` will be returned. 275 276 277Monitoring Thread PID 278--------------------- 279 280DAMON does requested monitoring with a kernel thread called ``kdamond``. You 281can get the pid of the thread by reading the ``kdamond_pid`` file. When the 282monitoring is turned off, reading the file returns ``none``. :: 283 284 # cd <debugfs>/damon 285 # cat monitor_on 286 off 287 # cat kdamond_pid 288 none 289 # echo on > monitor_on 290 # cat kdamond_pid 291 18594 292 293 294Using Multiple Monitoring Threads 295--------------------------------- 296 297One ``kdamond`` thread is created for each monitoring context. You can create 298and remove monitoring contexts for multiple ``kdamond`` required use case using 299the ``mk_contexts`` and ``rm_contexts`` files. 300 301Writing the name of the new context to the ``mk_contexts`` file creates a 302directory of the name on the DAMON debugfs directory. The directory will have 303DAMON debugfs files for the context. :: 304 305 # cd <debugfs>/damon 306 # ls foo 307 # ls: cannot access 'foo': No such file or directory 308 # echo foo > mk_contexts 309 # ls foo 310 # attrs init_regions kdamond_pid schemes target_ids 311 312If the context is not needed anymore, you can remove it and the corresponding 313directory by putting the name of the context to the ``rm_contexts`` file. :: 314 315 # echo foo > rm_contexts 316 # ls foo 317 # ls: cannot access 'foo': No such file or directory 318 319Note that ``mk_contexts``, ``rm_contexts``, and ``monitor_on`` files are in the 320root directory only. 321 322 323.. _tracepoint: 324 325Tracepoint for Monitoring Results 326================================= 327 328DAMON provides the monitoring results via a tracepoint, 329``damon:damon_aggregated``. While the monitoring is turned on, you could 330record the tracepoint events and show results using tracepoint supporting tools 331like ``perf``. For example:: 332 333 # echo on > monitor_on 334 # perf record -e damon:damon_aggregated & 335 # sleep 5 336 # kill 9 $(pidof perf) 337 # echo off > monitor_on 338 # perf script 339