devel/migration/qpl-compression.rst

4 The Intel Query Processing Library (Intel ``QPL``) is an open-source library to
8 The ``QPL`` compression relies on Intel In-Memory Analytics Accelerator(``IAA``)
9 and Shared Virtual Memory(``SVM``) technology, they are new features supported
13 For more ``QPL`` introduction, please refer to `QPL Introduction
21   +----------------+       +------------------+
22   | MultiFD Thread |       |accel-config tool |
23   +-------+--------+       +--------+---------+
27   +-------+--------+                | Setup IAA
29   +-------+---+----+                |
31           |   +-------------+-------+
33           |   Devices +-----+-----+
35           |           +-----+-----+
38           |           +-----+-----+
39           +-----------+IAA Devices|
40       Submit jobs     +-----------+
45 --------------------------
47 .. code-block:: shell
49   $git clone --recursive https://github.com/intel/qpl.git qpl
52   $cmake -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=/usr -DQPL_LIBRARY_TYPE=SHARED ..
53   $sudo cmake --build . --target install
55 For more details about ``QPL`` installation, please refer to `QPL Installation
59 ---------------------
62 On a ``SPR`` server, there can be a maximum of 8 ``IAA`` devices, with up to
65 By default, all ``IAA`` devices are disabled and need to be configured and
70 .. code-block:: shell
72   #lspci -d 8086:0cfe
73   6a:02.0 System peripheral: Intel Corporation Device 0cfe
74   6f:02.0 System peripheral: Intel Corporation Device 0cfe
75   74:02.0 System peripheral: Intel Corporation Device 0cfe
76   79:02.0 System peripheral: Intel Corporation Device 0cfe
77   e7:02.0 System peripheral: Intel Corporation Device 0cfe
78   ec:02.0 System peripheral: Intel Corporation Device 0cfe
79   f1:02.0 System peripheral: Intel Corporation Device 0cfe
80   f6:02.0 System peripheral: Intel Corporation Device 0cfe
85 The ``accel-config`` tool is used to enable ``IAA`` devices and configure
88 to a work queue via ``group`` attribute.
90 For ``accel-config`` installation, please refer to `accel-config installation
91 <https://github.com/intel/idxd-config>`_
95 .. code-block:: shell
97   #accel-config config-engine iax1/engine1.0 -g 0
98   #accel-config config-engine iax1/engine1.1 -g 0
99   #accel-config config-engine iax1/engine1.2 -g 0
100   #accel-config config-engine iax1/engine1.3 -g 0
101   #accel-config config-engine iax1/engine1.4 -g 0
102   #accel-config config-engine iax1/engine1.5 -g 0
103   #accel-config config-engine iax1/engine1.6 -g 0
104   #accel-config config-engine iax1/engine1.7 -g 0
105   #accel-config config-wq iax1/wq1.0 -g 0 -s 128 -p 10 -b 1 -t 128 -m shared -y user -n app1 -d user
106   #accel-config enable-device iax1
107   #accel-config enable-wq iax1/wq1.0
112 - The ``IAA`` device index is 1, use ``ls -lh /sys/bus/dsa/devices/iax*``
113   command to query the ``IAA`` device index.
115 - 8 engines and 1 work queue are configured in group 0, so all compression jobs
116   submitted to this work queue can be processed by all engines at the same time.
118 - Set work queue attributes including the work mode, work queue size and so on.
120 - Enable the ``IAA1`` device and work queue 1.0
124   Set work queue mode to shared mode, since ``QPL`` library only supports
127 For more detailed configuration, please refer to `IAA Configuration Samples
128 <https://github.com/intel/idxd-config/tree/stable/Documentation/accfg>`_
133 - Enabling ``IAA`` devices for Xeon platform, please refer to `IAA User Guide
134 …ttps://www.intel.com/content/www/us/en/content-details/780887/intel-in-memory-analytics-accelerato…
136 - ``IAA`` device driver is Intel Data Accelerator Driver (idxd), it is
139 - Add ``"intel_iommu=on,sm_on"`` parameter to kernel command line
142 Here is an easy way to verify ``IAA`` device driver and ``SVM`` with `iaa_test
143 <https://github.com/intel/idxd-config/tree/stable/test>`_
145 .. code-block:: shell
160 ``accel-config`` tool configuration cannot directly specify the ``IAA``
174   #chown -R qemu /dev/iax
176 Shared Virtual Memory(SVM) Introduction
179 An ability for an accelerator I/O device to operate in the same virtual
180 memory space of applications on host processors. It also implies the
181 ability to operate from pageable memory, avoiding functional requirements
182 to pin memory for DMA operations.
184 When using ``SVM`` technology, users do not need to reserve memory for the
185 ``IAA`` device and perform pin memory operation. The ``IAA`` device can
188 For more ``SVM`` technology, please refer to
193 How To Use QPL Compression In Migration
196 1 - Installation of ``QPL`` library and ``accel-config`` library if using IAA
198 2 - Configure and enable ``IAA`` devices and work queues via ``accel-config``
200 3 - Build ``QEMU`` with ``--enable-qpl`` parameter
202   E.g. configure --target-list=x86_64-softmmu --enable-kvm ``--enable-qpl``
204 4 - Enable ``QPL`` compression during migration
206   Set ``migrate_set_parameter multifd-compression qpl`` when migrating, the
227 The introduction for ``Z_SYNC_FLUSH``, please refer to `Zlib Manual
233 to add ``-mem-prealloc`` parameter to the destination boot parameters. This
237 The example of booting with ``-mem-prealloc`` parameter
239 .. code-block:: shell
241    $qemu-system-x86_64 --enable-kvm -cpu host --mem-prealloc ...
245 ``-mem-prealloc``, the ``svm_prq`` indicates the number of I/O page fault
248 .. code-block:: shell
256 …<0.1us   0.1us-1us    1us-10us  10us-100us   100us-1ms    1ms-10ms      >=10ms     min(us)     max…