1.. SPDX-License-Identifier: GPL-2.0 2 3=============== 4Detailed Usages 5=============== 6 7DAMON provides below three interfaces for different users. 8 9- *DAMON user space tool.* 10 `This <https://github.com/awslabs/damo>`_ is for privileged people such as 11 system administrators who want a just-working human-friendly interface. 12 Using this, users can use the DAMON’s major features in a human-friendly way. 13 It may not be highly tuned for special cases, though. It supports both 14 virtual and physical address spaces monitoring. For more detail, please 15 refer to its `usage document 16 <https://github.com/awslabs/damo/blob/next/USAGE.md>`_. 17- *debugfs interface.* 18 :ref:`This <debugfs_interface>` is for privileged user space programmers who 19 want more optimized use of DAMON. Using this, users can use DAMON’s major 20 features by reading from and writing to special debugfs files. Therefore, 21 you can write and use your personalized DAMON debugfs wrapper programs that 22 reads/writes the debugfs files instead of you. The `DAMON user space tool 23 <https://github.com/awslabs/damo>`_ is one example of such programs. It 24 supports both virtual and physical address spaces monitoring. 25- *Kernel Space Programming Interface.* 26 :doc:`This </vm/damon/api>` is for kernel space programmers. Using this, 27 users can utilize every feature of DAMON most flexibly and efficiently by 28 writing kernel space DAMON application programs for you. You can even extend 29 DAMON for various address spaces. For detail, please refer to the interface 30 :doc:`document </vm/damon/api>`. 31 32 33.. _debugfs_interface: 34 35debugfs Interface 36================= 37 38DAMON exports five files, ``attrs``, ``target_ids``, ``init_regions``, 39``schemes`` and ``monitor_on`` under its debugfs directory, 40``<debugfs>/damon/``. 41 42 43Attributes 44---------- 45 46Users can get and set the ``sampling interval``, ``aggregation interval``, 47``regions update interval``, and min/max number of monitoring target regions by 48reading from and writing to the ``attrs`` file. To know about the monitoring 49attributes in detail, please refer to the :doc:`/vm/damon/design`. For 50example, below commands set those values to 5 ms, 100 ms, 1,000 ms, 10 and 511000, and then check it again:: 52 53 # cd <debugfs>/damon 54 # echo 5000 100000 1000000 10 1000 > attrs 55 # cat attrs 56 5000 100000 1000000 10 1000 57 58 59Target IDs 60---------- 61 62Some types of address spaces supports multiple monitoring target. For example, 63the virtual memory address spaces monitoring can have multiple processes as the 64monitoring targets. Users can set the targets by writing relevant id values of 65the targets to, and get the ids of the current targets by reading from the 66``target_ids`` file. In case of the virtual address spaces monitoring, the 67values should be pids of the monitoring target processes. For example, below 68commands set processes having pids 42 and 4242 as the monitoring targets and 69check it again:: 70 71 # cd <debugfs>/damon 72 # echo 42 4242 > target_ids 73 # cat target_ids 74 42 4242 75 76Users can also monitor the physical memory address space of the system by 77writing a special keyword, "``paddr\n``" to the file. Because physical address 78space monitoring doesn't support multiple targets, reading the file will show a 79fake value, ``42``, as below:: 80 81 # cd <debugfs>/damon 82 # echo paddr > target_ids 83 # cat target_ids 84 42 85 86Note that setting the target ids doesn't start the monitoring. 87 88 89Initial Monitoring Target Regions 90--------------------------------- 91 92In case of the virtual address space monitoring, DAMON automatically sets and 93updates the monitoring target regions so that entire memory mappings of target 94processes can be covered. However, users can want to limit the monitoring 95region to specific address ranges, such as the heap, the stack, or specific 96file-mapped area. Or, some users can know the initial access pattern of their 97workloads and therefore want to set optimal initial regions for the 'adaptive 98regions adjustment'. 99 100In contrast, DAMON do not automatically sets and updates the monitoring target 101regions in case of physical memory monitoring. Therefore, users should set the 102monitoring target regions by themselves. 103 104In such cases, users can explicitly set the initial monitoring target regions 105as they want, by writing proper values to the ``init_regions`` file. Each line 106of the input should represent one region in below form.:: 107 108 <target id> <start address> <end address> 109 110The ``target id`` should already in ``target_ids`` file, and the regions should 111be passed in address order. For example, below commands will set a couple of 112address ranges, ``1-100`` and ``100-200`` as the initial monitoring target 113region of process 42, and another couple of address ranges, ``20-40`` and 114``50-100`` as that of process 4242.:: 115 116 # cd <debugfs>/damon 117 # echo "42 1 100 118 42 100 200 119 4242 20 40 120 4242 50 100" > init_regions 121 122Note that this sets the initial monitoring target regions only. In case of 123virtual memory monitoring, DAMON will automatically updates the boundary of the 124regions after one ``regions update interval``. Therefore, users should set the 125``regions update interval`` large enough in this case, if they don't want the 126update. 127 128 129Schemes 130------- 131 132For usual DAMON-based data access aware memory management optimizations, users 133would simply want the system to apply a memory management action to a memory 134region of a specific access pattern. DAMON receives such formalized operation 135schemes from the user and applies those to the target processes. 136 137Users can get and set the schemes by reading from and writing to ``schemes`` 138debugfs file. Reading the file also shows the statistics of each scheme. To 139the file, each of the schemes should be represented in each line in below 140form:: 141 142 <target access pattern> <action> <quota> <watermarks> 143 144You can disable schemes by simply writing an empty string to the file. 145 146Target Access Pattern 147~~~~~~~~~~~~~~~~~~~~~ 148 149The ``<target access pattern>`` is constructed with three ranges in below 150form:: 151 152 min-size max-size min-acc max-acc min-age max-age 153 154Specifically, bytes for the size of regions (``min-size`` and ``max-size``), 155number of monitored accesses per aggregate interval for access frequency 156(``min-acc`` and ``max-acc``), number of aggregate intervals for the age of 157regions (``min-age`` and ``max-age``) are specified. Note that the ranges are 158closed interval. 159 160Action 161~~~~~~ 162 163The ``<action>`` is a predefined integer for memory management actions, which 164DAMON will apply to the regions having the target access pattern. The 165supported numbers and their meanings are as below. 166 167 - 0: Call ``madvise()`` for the region with ``MADV_WILLNEED`` 168 - 1: Call ``madvise()`` for the region with ``MADV_COLD`` 169 - 2: Call ``madvise()`` for the region with ``MADV_PAGEOUT`` 170 - 3: Call ``madvise()`` for the region with ``MADV_HUGEPAGE`` 171 - 4: Call ``madvise()`` for the region with ``MADV_NOHUGEPAGE`` 172 - 5: Do nothing but count the statistics 173 174Quota 175~~~~~ 176 177Optimal ``target access pattern`` for each ``action`` is workload dependent, so 178not easy to find. Worse yet, setting a scheme of some action too aggressive 179can cause severe overhead. To avoid such overhead, users can limit time and 180size quota for the scheme via the ``<quota>`` in below form:: 181 182 <ms> <sz> <reset interval> <priority weights> 183 184This makes DAMON to try to use only up to ``<ms>`` milliseconds for applying 185the action to memory regions of the ``target access pattern`` within the 186``<reset interval>`` milliseconds, and to apply the action to only up to 187``<sz>`` bytes of memory regions within the ``<reset interval>``. Setting both 188``<ms>`` and ``<sz>`` zero disables the quota limits. 189 190When the quota limit is expected to be exceeded, DAMON prioritizes found memory 191regions of the ``target access pattern`` based on their size, access frequency, 192and age. For personalized prioritization, users can set the weights for the 193three properties in ``<priority weights>`` in below form:: 194 195 <size weight> <access frequency weight> <age weight> 196 197Watermarks 198~~~~~~~~~~ 199 200Some schemes would need to run based on current value of the system's specific 201metrics like free memory ratio. For such cases, users can specify watermarks 202for the condition.:: 203 204 <metric> <check interval> <high mark> <middle mark> <low mark> 205 206``<metric>`` is a predefined integer for the metric to be checked. The 207supported numbers and their meanings are as below. 208 209 - 0: Ignore the watermarks 210 - 1: System's free memory rate (per thousand) 211 212The value of the metric is checked every ``<check interval>`` microseconds. 213 214If the value is higher than ``<high mark>`` or lower than ``<low mark>``, the 215scheme is deactivated. If the value is lower than ``<mid mark>``, the scheme 216is activated. 217 218Statistics 219~~~~~~~~~~ 220 221It also counts the total number and bytes of regions that each scheme is 222applied. This statistics can be used for online analysis or tuning of the 223schemes. 224 225The statistics can be shown by reading the ``schemes`` file. Reading the file 226will show each scheme you entered in each line, and the two numbers for the 227statistics will be added at the end of each line. 228 229Example 230~~~~~~~ 231 232Below commands applies a scheme saying "If a memory region of size in [4KiB, 2338KiB] is showing accesses per aggregate interval in [0, 5] for aggregate 234interval in [10, 20], page out the region. For the paging out, use only up to 23510ms per second, and also don't page out more than 1GiB per second. Under the 236limitation, page out memory regions having longer age first. Also, check the 237free memory rate of the system every 5 seconds, start the monitoring and paging 238out when the free memory rate becomes lower than 50%, but stop it if the free 239memory rate becomes larger than 60%, or lower than 30%".:: 240 241 # cd <debugfs>/damon 242 # scheme="4096 8192 0 5 10 20 2" # target access pattern and action 243 # scheme+=" 10 $((1024*1024*1024)) 1000" # quotas 244 # scheme+=" 0 0 100" # prioritization weights 245 # scheme+=" 1 5000000 600 500 300" # watermarks 246 # echo "$scheme" > schemes 247 248 249Turning On/Off 250-------------- 251 252Setting the files as described above doesn't incur effect unless you explicitly 253start the monitoring. You can start, stop, and check the current status of the 254monitoring by writing to and reading from the ``monitor_on`` file. Writing 255``on`` to the file starts the monitoring of the targets with the attributes. 256Writing ``off`` to the file stops those. DAMON also stops if every target 257process is terminated. Below example commands turn on, off, and check the 258status of DAMON:: 259 260 # cd <debugfs>/damon 261 # echo on > monitor_on 262 # echo off > monitor_on 263 # cat monitor_on 264 off 265 266Please note that you cannot write to the above-mentioned debugfs files while 267the monitoring is turned on. If you write to the files while DAMON is running, 268an error code such as ``-EBUSY`` will be returned. 269 270 271Tracepoint for Monitoring Results 272================================= 273 274DAMON provides the monitoring results via a tracepoint, 275``damon:damon_aggregated``. While the monitoring is turned on, you could 276record the tracepoint events and show results using tracepoint supporting tools 277like ``perf``. For example:: 278 279 # echo on > monitor_on 280 # perf record -e damon:damon_aggregated & 281 # sleep 5 282 # kill 9 $(pidof perf) 283 # echo off > monitor_on 284 # perf script 285