115d9c3ceSAlex Bennée.. _vhost_user_proto: 215d9c3ceSAlex Bennée 3ed1be66bSMarc-André Lureau=================== 4ed1be66bSMarc-André LureauVhost-user Protocol 5ed1be66bSMarc-André Lureau=================== 64e0b15c2SPeter Maydell 74e0b15c2SPeter Maydell.. 84e0b15c2SPeter Maydell Copyright 2014 Virtual Open Systems Sarl. 94e0b15c2SPeter Maydell Copyright 2019 Intel Corporation 104e0b15c2SPeter Maydell Licence: This work is licensed under the terms of the GNU GPL, 11ed1be66bSMarc-André Lureau version 2 or later. See the COPYING file in the top-level 12ed1be66bSMarc-André Lureau directory. 13ed1be66bSMarc-André Lureau 14ed1be66bSMarc-André Lureau.. contents:: Table of Contents 15ed1be66bSMarc-André Lureau 16ed1be66bSMarc-André LureauIntroduction 17ed1be66bSMarc-André Lureau============ 18ed1be66bSMarc-André Lureau 19ed1be66bSMarc-André LureauThis protocol is aiming to complement the ``ioctl`` interface used to 20ed1be66bSMarc-André Lureaucontrol the vhost implementation in the Linux kernel. It implements 21ed1be66bSMarc-André Lureauthe control plane needed to establish virtqueue sharing with a user 22ed1be66bSMarc-André Lureauspace process on the same host. It uses communication over a Unix 23ed1be66bSMarc-André Lureaudomain socket to share file descriptors in the ancillary data of the 24ed1be66bSMarc-André Lureaumessage. 25ed1be66bSMarc-André Lureau 26bd59f2a1SPaolo BonziniThe protocol defines 2 sides of the communication, *front-end* and 27bd59f2a1SPaolo Bonzini*back-end*. The *front-end* is the application that shares its virtqueues, in 28bd59f2a1SPaolo Bonziniour case QEMU. The *back-end* is the consumer of the virtqueues. 29ed1be66bSMarc-André Lureau 30bd59f2a1SPaolo BonziniIn the current implementation QEMU is the *front-end*, and the *back-end* 31bd59f2a1SPaolo Bonziniis the external process consuming the virtio queues, for example a 32ed1be66bSMarc-André Lureausoftware Ethernet switch running in user space, such as Snabbswitch, 33bd59f2a1SPaolo Bonzinior a block device back-end processing read & write to a virtual 34bd59f2a1SPaolo Bonzinidisk. In order to facilitate interoperability between various back-end 35ed1be66bSMarc-André Lureauimplementations, it is recommended to follow the :ref:`Backend program 36ed1be66bSMarc-André Lureauconventions <backend_conventions>`. 37ed1be66bSMarc-André Lureau 38bd59f2a1SPaolo BonziniThe *front-end* and *back-end* can be either a client (i.e. connecting) or 39ed1be66bSMarc-André Lureauserver (listening) in the socket communication. 40ed1be66bSMarc-André Lureau 41b5108290SSergio LopezSupport for platforms other than Linux 42b5108290SSergio Lopez-------------------------------------- 43b5108290SSergio Lopez 44b5108290SSergio LopezWhile vhost-user was initially developed targeting Linux, nowadays it 45b5108290SSergio Lopezis supported on any platform that provides the following features: 46b5108290SSergio Lopez 47b5108290SSergio Lopez- A way for requesting shared memory represented by a file descriptor 48b5108290SSergio Lopez so it can be passed over a UNIX domain socket and then mapped by the 49b5108290SSergio Lopez other process. 50b5108290SSergio Lopez 51b5108290SSergio Lopez- AF_UNIX sockets with SCM_RIGHTS, so QEMU and the other process can 52b5108290SSergio Lopez exchange messages through it, including ancillary data when needed. 53b5108290SSergio Lopez 54b5108290SSergio Lopez- Either eventfd or pipe/pipe2. On platforms where eventfd is not 55b5108290SSergio Lopez available, QEMU will automatically fall back to pipe2 or, as a last 56b5108290SSergio Lopez resort, pipe. Each file descriptor will be used for receiving or 57b5108290SSergio Lopez sending events by reading or writing (respectively) an 8-byte value 58b5108290SSergio Lopez to the corresponding it. The 8-value itself has no meaning and 59b5108290SSergio Lopez should not be interpreted. 60b5108290SSergio Lopez 61ed1be66bSMarc-André LureauMessage Specification 62ed1be66bSMarc-André Lureau===================== 63ed1be66bSMarc-André Lureau 64ed1be66bSMarc-André Lureau.. Note:: All numbers are in the machine native byte order. 65ed1be66bSMarc-André Lureau 66ed1be66bSMarc-André LureauA vhost-user message consists of 3 header fields and a payload. 67ed1be66bSMarc-André Lureau 68ed1be66bSMarc-André Lureau+---------+-------+------+---------+ 69ed1be66bSMarc-André Lureau| request | flags | size | payload | 70ed1be66bSMarc-André Lureau+---------+-------+------+---------+ 71ed1be66bSMarc-André Lureau 72ed1be66bSMarc-André LureauHeader 73ed1be66bSMarc-André Lureau------ 74ed1be66bSMarc-André Lureau 75ed1be66bSMarc-André Lureau:request: 32-bit type of the request 76ed1be66bSMarc-André Lureau 77ed1be66bSMarc-André Lureau:flags: 32-bit bit field 78ed1be66bSMarc-André Lureau 79ed1be66bSMarc-André Lureau- Lower 2 bits are the version (currently 0x01) 80bd59f2a1SPaolo Bonzini- Bit 2 is the reply flag - needs to be sent on each reply from the back-end 81ed1be66bSMarc-André Lureau- Bit 3 is the need_reply flag - see :ref:`REPLY_ACK <reply_ack>` for 82ed1be66bSMarc-André Lureau details. 83ed1be66bSMarc-André Lureau 84ed1be66bSMarc-André Lureau:size: 32-bit size of the payload 85ed1be66bSMarc-André Lureau 86ed1be66bSMarc-André LureauPayload 87ed1be66bSMarc-André Lureau------- 88ed1be66bSMarc-André Lureau 89ed1be66bSMarc-André LureauDepending on the request type, **payload** can be: 90ed1be66bSMarc-André Lureau 91ed1be66bSMarc-André LureauA single 64-bit integer 92ed1be66bSMarc-André Lureau^^^^^^^^^^^^^^^^^^^^^^^ 93ed1be66bSMarc-André Lureau 94ed1be66bSMarc-André Lureau+-----+ 95ed1be66bSMarc-André Lureau| u64 | 96ed1be66bSMarc-André Lureau+-----+ 97ed1be66bSMarc-André Lureau 98ed1be66bSMarc-André Lureau:u64: a 64-bit unsigned integer 99ed1be66bSMarc-André Lureau 100ed1be66bSMarc-André LureauA vring state description 101ed1be66bSMarc-André Lureau^^^^^^^^^^^^^^^^^^^^^^^^^ 102ed1be66bSMarc-André Lureau 103ed1be66bSMarc-André Lureau+-------+-----+ 104ed1be66bSMarc-André Lureau| index | num | 105ed1be66bSMarc-André Lureau+-------+-----+ 106ed1be66bSMarc-André Lureau 107ed1be66bSMarc-André Lureau:index: a 32-bit index 108ed1be66bSMarc-André Lureau 109ed1be66bSMarc-André Lureau:num: a 32-bit number 110ed1be66bSMarc-André Lureau 111c7f21816SHanna CzenczekA vring descriptor index for split virtqueues 112c7f21816SHanna Czenczek^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 113c7f21816SHanna Czenczek 114c7f21816SHanna Czenczek+-------------+---------------------+ 115c7f21816SHanna Czenczek| vring index | index in avail ring | 116c7f21816SHanna Czenczek+-------------+---------------------+ 117c7f21816SHanna Czenczek 118c7f21816SHanna Czenczek:vring index: 32-bit index of the respective virtqueue 119c7f21816SHanna Czenczek 120c7f21816SHanna Czenczek:index in avail ring: 32-bit value, of which currently only the lower 16 121c7f21816SHanna Czenczek bits are used: 122c7f21816SHanna Czenczek 123c7f21816SHanna Czenczek - Bits 0–15: Index of the next *Available Ring* descriptor that the 124c7f21816SHanna Czenczek back-end will process. This is a free-running index that is not 125c7f21816SHanna Czenczek wrapped by the ring size. 126c7f21816SHanna Czenczek - Bits 16–31: Reserved (set to zero) 127c7f21816SHanna Czenczek 128c7f21816SHanna CzenczekVring descriptor indices for packed virtqueues 129c7f21816SHanna Czenczek^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 130c7f21816SHanna Czenczek 131c7f21816SHanna Czenczek+-------------+--------------------+ 132c7f21816SHanna Czenczek| vring index | descriptor indices | 133c7f21816SHanna Czenczek+-------------+--------------------+ 134c7f21816SHanna Czenczek 135c7f21816SHanna Czenczek:vring index: 32-bit index of the respective virtqueue 136c7f21816SHanna Czenczek 137c7f21816SHanna Czenczek:descriptor indices: 32-bit value: 138c7f21816SHanna Czenczek 139c7f21816SHanna Czenczek - Bits 0–14: Index of the next *Available Ring* descriptor that the 140c7f21816SHanna Czenczek back-end will process. This is a free-running index that is not 141c7f21816SHanna Czenczek wrapped by the ring size. 142c7f21816SHanna Czenczek - Bit 15: Driver (Available) Ring Wrap Counter 143c7f21816SHanna Czenczek - Bits 16–30: Index of the entry in the *Used Ring* where the back-end 144c7f21816SHanna Czenczek will place the next descriptor. This is a free-running index that 145c7f21816SHanna Czenczek is not wrapped by the ring size. 146c7f21816SHanna Czenczek - Bit 31: Device (Used) Ring Wrap Counter 147c7f21816SHanna Czenczek 148ed1be66bSMarc-André LureauA vring address description 149ed1be66bSMarc-André Lureau^^^^^^^^^^^^^^^^^^^^^^^^^^^ 150ed1be66bSMarc-André Lureau 151*17ae7ebeSAndrey Ignatov+-------+-------+------------+------+-----------+-----+ 152*17ae7ebeSAndrey Ignatov| index | flags | descriptor | used | available | log | 153*17ae7ebeSAndrey Ignatov+-------+-------+------------+------+-----------+-----+ 154ed1be66bSMarc-André Lureau 155ed1be66bSMarc-André Lureau:index: a 32-bit vring index 156ed1be66bSMarc-André Lureau 157ed1be66bSMarc-André Lureau:flags: a 32-bit vring flags 158ed1be66bSMarc-André Lureau 159ed1be66bSMarc-André Lureau:descriptor: a 64-bit ring address of the vring descriptor table 160ed1be66bSMarc-André Lureau 161ed1be66bSMarc-André Lureau:used: a 64-bit ring address of the vring used ring 162ed1be66bSMarc-André Lureau 163ed1be66bSMarc-André Lureau:available: a 64-bit ring address of the vring available ring 164ed1be66bSMarc-André Lureau 165ed1be66bSMarc-André Lureau:log: a 64-bit guest address for logging 166ed1be66bSMarc-André Lureau 167ed1be66bSMarc-André LureauNote that a ring address is an IOVA if ``VIRTIO_F_IOMMU_PLATFORM`` has 168ed1be66bSMarc-André Lureaubeen negotiated. Otherwise it is a user address. 169ed1be66bSMarc-André Lureau 17017c67f68SViresh KumarMemory region description 17117c67f68SViresh Kumar^^^^^^^^^^^^^^^^^^^^^^^^^ 172ed1be66bSMarc-André Lureau 173ed1be66bSMarc-André Lureau+---------------+------+--------------+-------------+ 174ed1be66bSMarc-André Lureau| guest address | size | user address | mmap offset | 175ed1be66bSMarc-André Lureau+---------------+------+--------------+-------------+ 176ed1be66bSMarc-André Lureau 177ed1be66bSMarc-André Lureau:guest address: a 64-bit guest address of the region 178ed1be66bSMarc-André Lureau 179ed1be66bSMarc-André Lureau:size: a 64-bit size 180ed1be66bSMarc-André Lureau 181ed1be66bSMarc-André Lureau:user address: a 64-bit user address 182ed1be66bSMarc-André Lureau 183ed1be66bSMarc-André Lureau:mmap offset: 64-bit offset where region starts in the mapped memory 184ed1be66bSMarc-André Lureau 185f21e95eeSViresh KumarWhen the ``VHOST_USER_PROTOCOL_F_XEN_MMAP`` protocol feature has been 186f21e95eeSViresh Kumarsuccessfully negotiated, the memory region description contains two extra 187f21e95eeSViresh Kumarfields at the end. 188f21e95eeSViresh Kumar 189f21e95eeSViresh Kumar+---------------+------+--------------+-------------+----------------+-------+ 190f21e95eeSViresh Kumar| guest address | size | user address | mmap offset | xen mmap flags | domid | 191f21e95eeSViresh Kumar+---------------+------+--------------+-------------+----------------+-------+ 192f21e95eeSViresh Kumar 193f21e95eeSViresh Kumar:xen mmap flags: 32-bit bit field 194f21e95eeSViresh Kumar 195f21e95eeSViresh Kumar- Bit 0 is set for Xen foreign memory mapping. 196f21e95eeSViresh Kumar- Bit 1 is set for Xen grant memory mapping. 197f21e95eeSViresh Kumar- Bit 8 is set if the memory region can not be mapped in advance, and memory 198f21e95eeSViresh Kumar areas within this region must be mapped / unmapped only when required by the 199f21e95eeSViresh Kumar back-end. The back-end shouldn't try to map the entire region at once, as the 200f21e95eeSViresh Kumar front-end may not allow it. The back-end should rather map only the required 201f21e95eeSViresh Kumar amount of memory at once and unmap it after it is used. 202f21e95eeSViresh Kumar 203f21e95eeSViresh Kumar:domid: a 32-bit Xen hypervisor specific domain id. 204f21e95eeSViresh Kumar 2053009edffSStefan HajnocziSingle memory region description 2063009edffSStefan Hajnoczi^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2073009edffSStefan Hajnoczi 20817c67f68SViresh Kumar+---------+--------+ 20917c67f68SViresh Kumar| padding | region | 21017c67f68SViresh Kumar+---------+--------+ 2113009edffSStefan Hajnoczi 2123009edffSStefan Hajnoczi:padding: 64-bit 2133009edffSStefan Hajnoczi 21417c67f68SViresh KumarA region is represented by Memory region description. 2153009edffSStefan Hajnoczi 21617c67f68SViresh KumarMultiple Memory regions description 21717c67f68SViresh Kumar^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2183009edffSStefan Hajnoczi 21917c67f68SViresh Kumar+-------------+---------+---------+-----+---------+ 22017c67f68SViresh Kumar| num regions | padding | region0 | ... | region7 | 22117c67f68SViresh Kumar+-------------+---------+---------+-----+---------+ 2223009edffSStefan Hajnoczi 22317c67f68SViresh Kumar:num regions: a 32-bit number of regions 22417c67f68SViresh Kumar 22517c67f68SViresh Kumar:padding: 32-bit 22617c67f68SViresh Kumar 22717c67f68SViresh KumarA region is represented by Memory region description. 2283009edffSStefan Hajnoczi 229ed1be66bSMarc-André LureauLog description 230ed1be66bSMarc-André Lureau^^^^^^^^^^^^^^^ 231ed1be66bSMarc-André Lureau 232ed1be66bSMarc-André Lureau+----------+------------+ 233ed1be66bSMarc-André Lureau| log size | log offset | 234ed1be66bSMarc-André Lureau+----------+------------+ 235ed1be66bSMarc-André Lureau 236ed1be66bSMarc-André Lureau:log size: size of area used for logging 237ed1be66bSMarc-André Lureau 238ed1be66bSMarc-André Lureau:log offset: offset from start of supplied file descriptor where 239ed1be66bSMarc-André Lureau logging starts (i.e. where guest address 0 would be 240ed1be66bSMarc-André Lureau logged) 241ed1be66bSMarc-André Lureau 242ed1be66bSMarc-André LureauAn IOTLB message 243ed1be66bSMarc-André Lureau^^^^^^^^^^^^^^^^ 244ed1be66bSMarc-André Lureau 245ed1be66bSMarc-André Lureau+------+------+--------------+-------------------+------+ 246ed1be66bSMarc-André Lureau| iova | size | user address | permissions flags | type | 247ed1be66bSMarc-André Lureau+------+------+--------------+-------------------+------+ 248ed1be66bSMarc-André Lureau 249ed1be66bSMarc-André Lureau:iova: a 64-bit I/O virtual address programmed by the guest 250ed1be66bSMarc-André Lureau 251ed1be66bSMarc-André Lureau:size: a 64-bit size 252ed1be66bSMarc-André Lureau 253ed1be66bSMarc-André Lureau:user address: a 64-bit user address 254ed1be66bSMarc-André Lureau 255ed1be66bSMarc-André Lureau:permissions flags: an 8-bit value: 256ed1be66bSMarc-André Lureau - 0: No access 257ed1be66bSMarc-André Lureau - 1: Read access 258ed1be66bSMarc-André Lureau - 2: Write access 259ed1be66bSMarc-André Lureau - 3: Read/Write access 260ed1be66bSMarc-André Lureau 261ed1be66bSMarc-André Lureau:type: an 8-bit IOTLB message type: 262ed1be66bSMarc-André Lureau - 1: IOTLB miss 263ed1be66bSMarc-André Lureau - 2: IOTLB update 264ed1be66bSMarc-André Lureau - 3: IOTLB invalidate 265ed1be66bSMarc-André Lureau - 4: IOTLB access fail 266ed1be66bSMarc-André Lureau 267ed1be66bSMarc-André LureauVirtio device config space 268ed1be66bSMarc-André Lureau^^^^^^^^^^^^^^^^^^^^^^^^^^ 269ed1be66bSMarc-André Lureau 270ed1be66bSMarc-André Lureau+--------+------+-------+---------+ 271ed1be66bSMarc-André Lureau| offset | size | flags | payload | 272ed1be66bSMarc-André Lureau+--------+------+-------+---------+ 273ed1be66bSMarc-André Lureau 274ed1be66bSMarc-André Lureau:offset: a 32-bit offset of virtio device's configuration space 275ed1be66bSMarc-André Lureau 276ed1be66bSMarc-André Lureau:size: a 32-bit configuration space access size in bytes 277ed1be66bSMarc-André Lureau 278ed1be66bSMarc-André Lureau:flags: a 32-bit value: 2799323e79fSPeter Maydell - 0: Vhost front-end messages used for writable fields 280bd59f2a1SPaolo Bonzini - 1: Vhost front-end messages used for live migration 281ed1be66bSMarc-André Lureau 282ed1be66bSMarc-André Lureau:payload: Size bytes array holding the contents of the virtio 283ed1be66bSMarc-André Lureau device's configuration space 284ed1be66bSMarc-André Lureau 285ed1be66bSMarc-André LureauVring area description 286ed1be66bSMarc-André Lureau^^^^^^^^^^^^^^^^^^^^^^ 287ed1be66bSMarc-André Lureau 288ed1be66bSMarc-André Lureau+-----+------+--------+ 289ed1be66bSMarc-André Lureau| u64 | size | offset | 290ed1be66bSMarc-André Lureau+-----+------+--------+ 291ed1be66bSMarc-André Lureau 292ed1be66bSMarc-André Lureau:u64: a 64-bit integer contains vring index and flags 293ed1be66bSMarc-André Lureau 294ed1be66bSMarc-André Lureau:size: a 64-bit size of this area 295ed1be66bSMarc-André Lureau 296ed1be66bSMarc-André Lureau:offset: a 64-bit offset of this area from the start of the 297ed1be66bSMarc-André Lureau supplied file descriptor 298ed1be66bSMarc-André Lureau 299ed1be66bSMarc-André LureauInflight description 300ed1be66bSMarc-André Lureau^^^^^^^^^^^^^^^^^^^^ 301ed1be66bSMarc-André Lureau 302ed1be66bSMarc-André Lureau+-----------+-------------+------------+------------+ 303ed1be66bSMarc-André Lureau| mmap size | mmap offset | num queues | queue size | 304ed1be66bSMarc-André Lureau+-----------+-------------+------------+------------+ 305ed1be66bSMarc-André Lureau 306ed1be66bSMarc-André Lureau:mmap size: a 64-bit size of area to track inflight I/O 307ed1be66bSMarc-André Lureau 308ed1be66bSMarc-André Lureau:mmap offset: a 64-bit offset of this area from the start 309ed1be66bSMarc-André Lureau of the supplied file descriptor 310ed1be66bSMarc-André Lureau 311ed1be66bSMarc-André Lureau:num queues: a 16-bit number of virtqueues 312ed1be66bSMarc-André Lureau 313ed1be66bSMarc-André Lureau:queue size: a 16-bit size of virtqueues 314ed1be66bSMarc-André Lureau 315d4eb5038SHanna CzenczekVhostUserShared 316d4eb5038SHanna Czenczek^^^^^^^^^^^^^^^ 317d4eb5038SHanna Czenczek 318d4eb5038SHanna Czenczek+------+ 319d4eb5038SHanna Czenczek| UUID | 320d4eb5038SHanna Czenczek+------+ 321d4eb5038SHanna Czenczek 322d4eb5038SHanna Czenczek:UUID: 16 bytes UUID, whose first three components (a 32-bit value, then 323d4eb5038SHanna Czenczek two 16-bit values) are stored in big endian. 324d4eb5038SHanna Czenczek 32501923309SHanna CzenczekDevice state transfer parameters 32601923309SHanna Czenczek^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 32701923309SHanna Czenczek 32801923309SHanna Czenczek+--------------------+-----------------+ 32901923309SHanna Czenczek| transfer direction | migration phase | 33001923309SHanna Czenczek+--------------------+-----------------+ 33101923309SHanna Czenczek 33201923309SHanna Czenczek:transfer direction: a 32-bit enum, describing the direction in which 33301923309SHanna Czenczek the state is transferred: 33401923309SHanna Czenczek 33501923309SHanna Czenczek - 0: Save: Transfer the state from the back-end to the front-end, 33601923309SHanna Czenczek which happens on the source side of migration 33701923309SHanna Czenczek - 1: Load: Transfer the state from the front-end to the back-end, 33801923309SHanna Czenczek which happens on the destination side of migration 33901923309SHanna Czenczek 34001923309SHanna Czenczek:migration phase: a 32-bit enum, describing the state in which the VM 34101923309SHanna Czenczek guest and devices are: 34201923309SHanna Czenczek 34301923309SHanna Czenczek - 0: Stopped (in the period after the transfer of memory-mapped 34401923309SHanna Czenczek regions before switch-over to the destination): The VM guest is 34501923309SHanna Czenczek stopped, and the vhost-user device is suspended (see 34601923309SHanna Czenczek :ref:`Suspended device state <suspended_device_state>`). 34701923309SHanna Czenczek 34801923309SHanna Czenczek In the future, additional phases might be added e.g. to allow 34901923309SHanna Czenczek iterative migration while the device is running. 35001923309SHanna Czenczek 351ed1be66bSMarc-André LureauC structure 352ed1be66bSMarc-André Lureau----------- 353ed1be66bSMarc-André Lureau 354ed1be66bSMarc-André LureauIn QEMU the vhost-user message is implemented with the following struct: 355ed1be66bSMarc-André Lureau 356ed1be66bSMarc-André Lureau.. code:: c 357ed1be66bSMarc-André Lureau 358ed1be66bSMarc-André Lureau typedef struct VhostUserMsg { 359ed1be66bSMarc-André Lureau VhostUserRequest request; 360ed1be66bSMarc-André Lureau uint32_t flags; 361ed1be66bSMarc-André Lureau uint32_t size; 362ed1be66bSMarc-André Lureau union { 363ed1be66bSMarc-André Lureau uint64_t u64; 364ed1be66bSMarc-André Lureau struct vhost_vring_state state; 365ed1be66bSMarc-André Lureau struct vhost_vring_addr addr; 366ed1be66bSMarc-André Lureau VhostUserMemory memory; 367ed1be66bSMarc-André Lureau VhostUserLog log; 368ed1be66bSMarc-André Lureau struct vhost_iotlb_msg iotlb; 369ed1be66bSMarc-André Lureau VhostUserConfig config; 370ed1be66bSMarc-André Lureau VhostUserVringArea area; 371ed1be66bSMarc-André Lureau VhostUserInflight inflight; 372ed1be66bSMarc-André Lureau }; 373ed1be66bSMarc-André Lureau } QEMU_PACKED VhostUserMsg; 374ed1be66bSMarc-André Lureau 375ed1be66bSMarc-André LureauCommunication 376ed1be66bSMarc-André Lureau============= 377ed1be66bSMarc-André Lureau 378ed1be66bSMarc-André LureauThe protocol for vhost-user is based on the existing implementation of 379ed1be66bSMarc-André Lureauvhost for the Linux Kernel. Most messages that can be sent via the 380ed1be66bSMarc-André LureauUnix domain socket implementing vhost-user have an equivalent ioctl to 381ed1be66bSMarc-André Lureauthe kernel implementation. 382ed1be66bSMarc-André Lureau 383bd59f2a1SPaolo BonziniThe communication consists of the *front-end* sending message requests and 384bd59f2a1SPaolo Bonzinithe *back-end* sending message replies. Most of the requests don't require 385ed1be66bSMarc-André Lureaureplies. Here is a list of the ones that do: 386ed1be66bSMarc-André Lureau 387ed1be66bSMarc-André Lureau* ``VHOST_USER_GET_FEATURES`` 388ed1be66bSMarc-André Lureau* ``VHOST_USER_GET_PROTOCOL_FEATURES`` 389ed1be66bSMarc-André Lureau* ``VHOST_USER_GET_VRING_BASE`` 390ed1be66bSMarc-André Lureau* ``VHOST_USER_SET_LOG_BASE`` (if ``VHOST_USER_PROTOCOL_F_LOG_SHMFD``) 391ed1be66bSMarc-André Lureau* ``VHOST_USER_GET_INFLIGHT_FD`` (if ``VHOST_USER_PROTOCOL_F_INFLIGHT_SHMFD``) 392ed1be66bSMarc-André Lureau 393ed1be66bSMarc-André Lureau.. seealso:: 394ed1be66bSMarc-André Lureau 395ed1be66bSMarc-André Lureau :ref:`REPLY_ACK <reply_ack>` 396ed1be66bSMarc-André Lureau The section on ``REPLY_ACK`` protocol extension. 397ed1be66bSMarc-André Lureau 398bd59f2a1SPaolo BonziniThere are several messages that the front-end sends with file descriptors passed 399ed1be66bSMarc-André Lureauin the ancillary data: 400ed1be66bSMarc-André Lureau 40131009d13SKevin Wolf* ``VHOST_USER_ADD_MEM_REG`` 402ed1be66bSMarc-André Lureau* ``VHOST_USER_SET_MEM_TABLE`` 403ed1be66bSMarc-André Lureau* ``VHOST_USER_SET_LOG_BASE`` (if ``VHOST_USER_PROTOCOL_F_LOG_SHMFD``) 404ed1be66bSMarc-André Lureau* ``VHOST_USER_SET_LOG_FD`` 405ed1be66bSMarc-André Lureau* ``VHOST_USER_SET_VRING_KICK`` 406ed1be66bSMarc-André Lureau* ``VHOST_USER_SET_VRING_CALL`` 407ed1be66bSMarc-André Lureau* ``VHOST_USER_SET_VRING_ERR`` 408e9ca9f33SMaxime Coquelin* ``VHOST_USER_SET_BACKEND_REQ_FD`` (previous name ``VHOST_USER_SET_SLAVE_REQ_FD``) 409ed1be66bSMarc-André Lureau* ``VHOST_USER_SET_INFLIGHT_FD`` (if ``VHOST_USER_PROTOCOL_F_INFLIGHT_SHMFD``) 41001923309SHanna Czenczek* ``VHOST_USER_SET_DEVICE_STATE_FD`` 411ed1be66bSMarc-André Lureau 412bd59f2a1SPaolo BonziniIf *front-end* is unable to send the full message or receives a wrong 413ed1be66bSMarc-André Lureaureply it will close the connection. An optional reconnection mechanism 414ed1be66bSMarc-André Lureaucan be implemented. 415ed1be66bSMarc-André Lureau 416bd59f2a1SPaolo BonziniIf *back-end* detects some error such as incompatible features, it may also 4173348e7e3SJohannes Bergclose the connection. This should only happen in exceptional circumstances. 4183348e7e3SJohannes Berg 419ed1be66bSMarc-André LureauAny protocol extensions are gated by protocol feature bits, which 420bd59f2a1SPaolo Bonziniallows full backwards compatibility on both front-end and back-end. As 421bd59f2a1SPaolo Bonziniolder back-ends don't support negotiating protocol features, a feature 422ed1be66bSMarc-André Lureaubit was dedicated for this purpose:: 423ed1be66bSMarc-André Lureau 424ed1be66bSMarc-André Lureau #define VHOST_USER_F_PROTOCOL_FEATURES 30 425ed1be66bSMarc-André Lureau 426fa997266SAlex BennéeNote that VHOST_USER_F_PROTOCOL_FEATURES is the UNUSED (30) feature 427fa997266SAlex Bennéebit defined in `VIRTIO 1.1 6.3 Legacy Interface: Reserved Feature Bits 428fa997266SAlex Bennée<https://docs.oasis-open.org/virtio/virtio/v1.1/cs01/virtio-v1.1-cs01.html#x1-4130003>`_. 429fa997266SAlex BennéeVIRTIO devices do not advertise this feature bit and therefore VIRTIO 430fa997266SAlex Bennéedrivers cannot negotiate it. 431fa997266SAlex Bennée 432fa997266SAlex BennéeThis reserved feature bit was reused by the vhost-user protocol to add 433fa997266SAlex Bennéevhost-user protocol feature negotiation in a backwards compatible 43437cbfcebSMichael S. Tsirkinfashion. Old vhost-user front-end and back-end implementations continue to 435fa997266SAlex Bennéework even though they are not aware of vhost-user protocol feature 436fa997266SAlex Bennéenegotiation. 437fa997266SAlex Bennée 4380647098dSPaolo BonziniRing states 4390647098dSPaolo Bonzini----------- 440ed1be66bSMarc-André Lureau 441eae69cc3SHanna CzenczekRings have two independent states: started/stopped, and enabled/disabled. 442ed1be66bSMarc-André Lureau 443eae69cc3SHanna Czenczek* While a ring is stopped, the back-end must not process the ring at 444eae69cc3SHanna Czenczek all, regardless of whether it is enabled or disabled. The 445eae69cc3SHanna Czenczek enabled/disabled state should still be tracked, though, so it can come 446eae69cc3SHanna Czenczek into effect once the ring is started. 447ed1be66bSMarc-André Lureau 448eae69cc3SHanna Czenczek* started and disabled: The back-end must process the ring without 4490647098dSPaolo Bonzini causing any side effects. For example, for a networking device, 450bd59f2a1SPaolo Bonzini in the disabled state the back-end must not supply any new RX packets, 4510647098dSPaolo Bonzini but must process and discard any TX packets. 452ed1be66bSMarc-André Lureau 453eae69cc3SHanna Czenczek* started and enabled: The back-end must process the ring normally, i.e. 454eae69cc3SHanna Czenczek process all requests and execute them. 4550647098dSPaolo Bonzini 456eae69cc3SHanna CzenczekEach ring is initialized in a stopped and disabled state. The back-end 457eae69cc3SHanna Czenczekmust start a ring upon receiving a kick (that is, detecting that file 458eae69cc3SHanna Czenczekdescriptor is readable) on the descriptor specified by 459eae69cc3SHanna Czenczek``VHOST_USER_SET_VRING_KICK`` or receiving the in-band message 460eae69cc3SHanna Czenczek``VHOST_USER_VRING_KICK`` if negotiated, and stop a ring upon receiving 461eae69cc3SHanna Czenczek``VHOST_USER_GET_VRING_BASE``. 4620647098dSPaolo Bonzini 4630647098dSPaolo BonziniRings can be enabled or disabled by ``VHOST_USER_SET_VRING_ENABLE``. 464ed1be66bSMarc-André Lureau 465eae69cc3SHanna CzenczekIn addition, upon receiving a ``VHOST_USER_SET_FEATURES`` message from 466eae69cc3SHanna Czenczekthe front-end without ``VHOST_USER_F_PROTOCOL_FEATURES`` set, the 467eae69cc3SHanna Czenczekback-end must enable all rings immediately. 468ed1be66bSMarc-André Lureau 469bd59f2a1SPaolo BonziniWhile processing the rings (whether they are enabled or not), the back-end 470ed1be66bSMarc-André Lureaumust support changing some configuration aspects on the fly. 471ed1be66bSMarc-André Lureau 472a6e76dd3SHanna Czenczek.. _suspended_device_state: 473a6e76dd3SHanna Czenczek 474a6e76dd3SHanna CzenczekSuspended device state 475a6e76dd3SHanna Czenczek^^^^^^^^^^^^^^^^^^^^^^ 476a6e76dd3SHanna Czenczek 477a6e76dd3SHanna CzenczekWhile all vrings are stopped, the device is *suspended*. In addition to 478a6e76dd3SHanna Czenczeknot processing any vring (because they are stopped), the device must: 479a6e76dd3SHanna Czenczek 480a6e76dd3SHanna Czenczek* not write to any guest memory regions, 481a6e76dd3SHanna Czenczek* not send any notifications to the guest, 482a6e76dd3SHanna Czenczek* not send any messages to the front-end, 483a6e76dd3SHanna Czenczek* still process and reply to messages from the front-end. 484a6e76dd3SHanna Czenczek 485ed1be66bSMarc-André LureauMultiple queue support 486ed1be66bSMarc-André Lureau---------------------- 487ed1be66bSMarc-André Lureau 488bd59f2a1SPaolo BonziniMany devices have a fixed number of virtqueues. In this case the front-end 489df98d7ccSStefan Hajnoczialready knows the number of available virtqueues without communicating with the 490bd59f2a1SPaolo Bonziniback-end. 491df98d7ccSStefan Hajnoczi 492df98d7ccSStefan HajnocziSome devices do not have a fixed number of virtqueues. Instead the maximum 493bd59f2a1SPaolo Bonzininumber of virtqueues is chosen by the back-end. The number can depend on host 494bd59f2a1SPaolo Bonziniresource availability or back-end implementation details. Such devices are called 495df98d7ccSStefan Hajnoczimultiple queue devices. 496df98d7ccSStefan Hajnoczi 497bd59f2a1SPaolo BonziniMultiple queue support allows the back-end to advertise the maximum number of 498bd59f2a1SPaolo Bonziniqueues. This is treated as a protocol extension, hence the back-end has to 4993ef4dff2SStefan Hajnocziimplement protocol features first. The multiple queues feature is supported 5003ef4dff2SStefan Hajnoczionly when the protocol feature ``VHOST_USER_PROTOCOL_F_MQ`` (bit 0) is set. 501ed1be66bSMarc-André Lureau 502bd59f2a1SPaolo BonziniThe max number of queues the back-end supports can be queried with message 503bd59f2a1SPaolo Bonzini``VHOST_USER_GET_QUEUE_NUM``. Front-end should stop when the number of requested 5043ef4dff2SStefan Hajnocziqueues is bigger than that. 505ed1be66bSMarc-André Lureau 506bd59f2a1SPaolo BonziniAs all queues share one connection, the front-end uses a unique index for each 5073ef4dff2SStefan Hajnocziqueue in the sent message to identify a specified queue. 5083ef4dff2SStefan Hajnoczi 509bd59f2a1SPaolo BonziniThe front-end enables queues by sending message ``VHOST_USER_SET_VRING_ENABLE``. 5103ef4dff2SStefan Hajnoczivhost-user-net has historically automatically enabled the first queue pair. 511ed1be66bSMarc-André Lureau 512bd59f2a1SPaolo BonziniBack-ends should always implement the ``VHOST_USER_PROTOCOL_F_MQ`` protocol 513df98d7ccSStefan Hajnoczifeature, even for devices with a fixed number of virtqueues, since it is simple 514df98d7ccSStefan Hajnoczito implement and offers a degree of introspection. 515df98d7ccSStefan Hajnoczi 516bd59f2a1SPaolo BonziniFront-ends must not rely on the ``VHOST_USER_PROTOCOL_F_MQ`` protocol feature for 517df98d7ccSStefan Hajnoczidevices with a fixed number of virtqueues. Only true multiqueue devices 518df98d7ccSStefan Hajnoczirequire this protocol feature. 519df98d7ccSStefan Hajnoczi 520ed1be66bSMarc-André LureauMigration 521ed1be66bSMarc-André Lureau--------- 522ed1be66bSMarc-André Lureau 523bd59f2a1SPaolo BonziniDuring live migration, the front-end may need to track the modifications 524bd59f2a1SPaolo Bonzinithe back-end makes to the memory mapped regions. The front-end should mark 525ed1be66bSMarc-André Lureauthe dirty pages in a log. Once it complies to this logging, it may 526ed1be66bSMarc-André Lureaudeclare the ``VHOST_F_LOG_ALL`` vhost feature. 527ed1be66bSMarc-André Lureau 528bd59f2a1SPaolo BonziniTo start/stop logging of data/used ring writes, the front-end may send 529ed1be66bSMarc-André Lureaumessages ``VHOST_USER_SET_FEATURES`` with ``VHOST_F_LOG_ALL`` and 530ed1be66bSMarc-André Lureau``VHOST_USER_SET_VRING_ADDR`` with ``VHOST_VRING_F_LOG`` in ring's 531ed1be66bSMarc-André Lureauflags set to 1/0, respectively. 532ed1be66bSMarc-André Lureau 533ed1be66bSMarc-André LureauAll the modifications to memory pointed by vring "descriptor" should 534ed1be66bSMarc-André Lureaube marked. Modifications to "used" vring should be marked if 535ed1be66bSMarc-André Lureau``VHOST_VRING_F_LOG`` is part of ring's flags. 536ed1be66bSMarc-André Lureau 537ed1be66bSMarc-André LureauDirty pages are of size:: 538ed1be66bSMarc-André Lureau 539ed1be66bSMarc-André Lureau #define VHOST_LOG_PAGE 0x1000 540ed1be66bSMarc-André Lureau 541ed1be66bSMarc-André LureauThe log memory fd is provided in the ancillary data of 542bd59f2a1SPaolo Bonzini``VHOST_USER_SET_LOG_BASE`` message when the back-end has 543ed1be66bSMarc-André Lureau``VHOST_USER_PROTOCOL_F_LOG_SHMFD`` protocol feature. 544ed1be66bSMarc-André Lureau 545ed1be66bSMarc-André LureauThe size of the log is supplied as part of ``VhostUserMsg`` which 546ed1be66bSMarc-André Lureaushould be large enough to cover all known guest addresses. Log starts 547ed1be66bSMarc-André Lureauat the supplied offset in the supplied file descriptor. The log 548ed1be66bSMarc-André Lureaucovers from address 0 to the maximum of guest regions. In pseudo-code, 549ed1be66bSMarc-André Lureauto mark page at ``addr`` as dirty:: 550ed1be66bSMarc-André Lureau 551ed1be66bSMarc-André Lureau page = addr / VHOST_LOG_PAGE 552ed1be66bSMarc-André Lureau log[page / 8] |= 1 << page % 8 553ed1be66bSMarc-André Lureau 554ed1be66bSMarc-André LureauWhere ``addr`` is the guest physical address. 555ed1be66bSMarc-André Lureau 556ed1be66bSMarc-André LureauUse atomic operations, as the log may be concurrently manipulated. 557ed1be66bSMarc-André Lureau 558ed1be66bSMarc-André LureauNote that when logging modifications to the used ring (when 559ed1be66bSMarc-André Lureau``VHOST_VRING_F_LOG`` is set for this ring), ``log_guest_addr`` should 560ed1be66bSMarc-André Lureaube used to calculate the log offset: the write to first byte of the 561ed1be66bSMarc-André Lureauused ring is logged at this offset from log start. Also note that this 562ed1be66bSMarc-André Lureauvalue might be outside the legal guest physical address range 563ed1be66bSMarc-André Lureau(i.e. does not have to be covered by the ``VhostUserMemory`` table), but 564ed1be66bSMarc-André Lureauthe bit offset of the last byte of the ring must fall within the size 565ed1be66bSMarc-André Lureausupplied by ``VhostUserLog``. 566ed1be66bSMarc-André Lureau 567ed1be66bSMarc-André Lureau``VHOST_USER_SET_LOG_FD`` is an optional message with an eventfd in 568bd59f2a1SPaolo Bonziniancillary data, it may be used to inform the front-end that the log has 569ed1be66bSMarc-André Lureaubeen modified. 570ed1be66bSMarc-André Lureau 571ed1be66bSMarc-André LureauOnce the source has finished migration, rings will be stopped by the 572a6e76dd3SHanna Czenczeksource (:ref:`Suspended device state <suspended_device_state>`). No 573a6e76dd3SHanna Czenczekfurther update must be done before rings are restarted. 574ed1be66bSMarc-André Lureau 575bd59f2a1SPaolo BonziniIn postcopy migration the back-end is started before all the memory has 576ed1be66bSMarc-André Lureaubeen received from the source host, and care must be taken to avoid 577bd59f2a1SPaolo Bonziniaccessing pages that have yet to be received. The back-end opens a 578ed1be66bSMarc-André Lureau'userfault'-fd and registers the memory with it; this fd is then 579bd59f2a1SPaolo Bonzinipassed back over to the front-end. The front-end services requests on the 580ed1be66bSMarc-André Lureauuserfaultfd for pages that are accessed and when the page is available 581ed1be66bSMarc-André Lureauit performs WAKE ioctl's on the userfaultfd to wake the stalled 582bd59f2a1SPaolo Bonziniback-end. The front-end indicates support for this via the 583ed1be66bSMarc-André Lureau``VHOST_USER_PROTOCOL_F_PAGEFAULT`` feature. 584ed1be66bSMarc-André Lureau 58501923309SHanna Czenczek.. _migrating_backend_state: 58601923309SHanna Czenczek 58701923309SHanna CzenczekMigrating back-end state 58801923309SHanna Czenczek^^^^^^^^^^^^^^^^^^^^^^^^ 58901923309SHanna Czenczek 59001923309SHanna CzenczekMigrating device state involves transferring the state from one 59101923309SHanna Czenczekback-end, called the source, to another back-end, called the 59201923309SHanna Czenczekdestination. After migration, the destination transparently resumes 59301923309SHanna Czenczekoperation without requiring the driver to re-initialize the device at 59401923309SHanna Czenczekthe VIRTIO level. If the migration fails, then the source can 59501923309SHanna Czenczektransparently resume operation until another migration attempt is made. 59601923309SHanna Czenczek 59701923309SHanna CzenczekGenerally, the front-end is connected to a virtual machine guest (which 59801923309SHanna Czenczekcontains the driver), which has its own state to transfer between source 59901923309SHanna Czenczekand destination, and therefore will have an implementation-specific 60001923309SHanna Czenczekmechanism to do so. The ``VHOST_USER_PROTOCOL_F_DEVICE_STATE`` feature 60101923309SHanna Czenczekprovides functionality to have the front-end include the back-end's 60201923309SHanna Czenczekstate in this transfer operation so the back-end does not need to 60301923309SHanna Czenczekimplement its own mechanism, and so the virtual machine may have its 60401923309SHanna Czenczekcomplete state, including vhost-user devices' states, contained within a 60501923309SHanna Czenczeksingle stream of data. 60601923309SHanna Czenczek 60701923309SHanna CzenczekTo do this, the back-end state is transferred from back-end to front-end 60801923309SHanna Czenczekon the source side, and vice versa on the destination side. This 60901923309SHanna Czenczektransfer happens over a channel that is negotiated using the 61001923309SHanna Czenczek``VHOST_USER_SET_DEVICE_STATE_FD`` message. This message has two 61101923309SHanna Czenczekparameters: 61201923309SHanna Czenczek 61301923309SHanna Czenczek* Direction of transfer: On the source, the data is saved, transferring 61401923309SHanna Czenczek it from the back-end to the front-end. On the destination, the data 61501923309SHanna Czenczek is loaded, transferring it from the front-end to the back-end. 61601923309SHanna Czenczek 61701923309SHanna Czenczek* Migration phase: Currently, the only supported phase is the period 61801923309SHanna Czenczek after the transfer of memory-mapped regions before switch-over to the 61901923309SHanna Czenczek destination, when both the source and destination devices are 62001923309SHanna Czenczek suspended (:ref:`Suspended device state <suspended_device_state>`). 62101923309SHanna Czenczek In the future, additional phases might be supported to allow iterative 62201923309SHanna Czenczek migration while the device is running. 62301923309SHanna Czenczek 62401923309SHanna CzenczekThe nature of the channel is implementation-defined, but it must 62501923309SHanna Czenczekgenerally behave like a pipe: The writing end will write all the data it 62601923309SHanna Czenczekhas into it, signalling the end of data by closing its end. The reading 62701923309SHanna Czenczekend must read all of this data (until encountering the end of file) and 62801923309SHanna Czenczekprocess it. 62901923309SHanna Czenczek 63001923309SHanna Czenczek* When saving, the writing end is the source back-end, and the reading 63101923309SHanna Czenczek end is the source front-end. After reading the state data from the 63201923309SHanna Czenczek channel, the source front-end must transfer it to the destination 63301923309SHanna Czenczek front-end through an implementation-defined mechanism. 63401923309SHanna Czenczek 63501923309SHanna Czenczek* When loading, the writing end is the destination front-end, and the 63601923309SHanna Czenczek reading end is the destination back-end. After reading the state data 63701923309SHanna Czenczek from the channel, the destination back-end must deserialize its 63801923309SHanna Czenczek internal state from that data and set itself up to allow the driver to 63901923309SHanna Czenczek seamlessly resume operation on the VIRTIO level. 64001923309SHanna Czenczek 64101923309SHanna CzenczekSeamlessly resuming operation means that the migration must be 64201923309SHanna Czenczektransparent to the guest driver, which operates on the VIRTIO level. 64301923309SHanna CzenczekThis driver will not perform any re-initialization steps, but continue 64401923309SHanna Czenczekto use the device as if no migration had occurred. The vhost-user 64501923309SHanna Czenczekfront-end, however, will re-initialize the vhost state on the 64601923309SHanna Czenczekdestination, following the usual protocol for establishing a connection 64701923309SHanna Czenczekto a vhost-user back-end: This includes, for example, setting up memory 64801923309SHanna Czenczekmappings and kick and call FDs as necessary, negotiating protocol 64901923309SHanna Czenczekfeatures, or setting the initial vring base indices (to the same value 65001923309SHanna Czenczekas on the source side, so that operation can resume). 65101923309SHanna Czenczek 65201923309SHanna CzenczekBoth on the source and on the destination side, after the respective 65301923309SHanna Czenczekfront-end has seen all data transferred (when the transfer FD has been 65401923309SHanna Czenczekclosed), it sends the ``VHOST_USER_CHECK_DEVICE_STATE`` message to 65501923309SHanna Czenczekverify that data transfer was successful in the back-end, too. The 65601923309SHanna Czenczekback-end responds once it knows whether the transfer and processing was 65701923309SHanna Czenczeksuccessful or not. 65801923309SHanna Czenczek 659ed1be66bSMarc-André LureauMemory access 660ed1be66bSMarc-André Lureau------------- 661ed1be66bSMarc-André Lureau 662bd59f2a1SPaolo BonziniThe front-end sends a list of vhost memory regions to the back-end using the 663ed1be66bSMarc-André Lureau``VHOST_USER_SET_MEM_TABLE`` message. Each region has two base 664ed1be66bSMarc-André Lureauaddresses: a guest address and a user address. 665ed1be66bSMarc-André Lureau 666ed1be66bSMarc-André LureauMessages contain guest addresses and/or user addresses to reference locations 667ed1be66bSMarc-André Lureauwithin the shared memory. The mapping of these addresses works as follows. 668ed1be66bSMarc-André Lureau 669ed1be66bSMarc-André LureauUser addresses map to the vhost memory region containing that user address. 670ed1be66bSMarc-André Lureau 671ed1be66bSMarc-André LureauWhen the ``VIRTIO_F_IOMMU_PLATFORM`` feature has not been negotiated: 672ed1be66bSMarc-André Lureau 673ed1be66bSMarc-André Lureau* Guest addresses map to the vhost memory region containing that guest 674ed1be66bSMarc-André Lureau address. 675ed1be66bSMarc-André Lureau 676ed1be66bSMarc-André LureauWhen the ``VIRTIO_F_IOMMU_PLATFORM`` feature has been negotiated: 677ed1be66bSMarc-André Lureau 678ed1be66bSMarc-André Lureau* Guest addresses are also called I/O virtual addresses (IOVAs). They are 679ed1be66bSMarc-André Lureau translated to user addresses via the IOTLB. 680ed1be66bSMarc-André Lureau 681ed1be66bSMarc-André Lureau* The vhost memory region guest address is not used. 682ed1be66bSMarc-André Lureau 683ed1be66bSMarc-André LureauIOMMU support 684ed1be66bSMarc-André Lureau------------- 685ed1be66bSMarc-André Lureau 686ed1be66bSMarc-André LureauWhen the ``VIRTIO_F_IOMMU_PLATFORM`` feature has been negotiated, the 687bd59f2a1SPaolo Bonzinifront-end sends IOTLB entries update & invalidation by sending 688bd59f2a1SPaolo Bonzini``VHOST_USER_IOTLB_MSG`` requests to the back-end with a ``struct 689ed1be66bSMarc-André Lureauvhost_iotlb_msg`` as payload. For update events, the ``iotlb`` payload 690ed1be66bSMarc-André Lureauhas to be filled with the update message type (2), the I/O virtual 691ed1be66bSMarc-André Lureauaddress, the size, the user virtual address, and the permissions 692ed1be66bSMarc-André Lureauflags. Addresses and size must be within vhost memory regions set via 693ed1be66bSMarc-André Lureauthe ``VHOST_USER_SET_MEM_TABLE`` request. For invalidation events, the 694ed1be66bSMarc-André Lureau``iotlb`` payload has to be filled with the invalidation message type 695bd59f2a1SPaolo Bonzini(3), the I/O virtual address and the size. On success, the back-end is 696ed1be66bSMarc-André Lureauexpected to reply with a zero payload, non-zero otherwise. 697ed1be66bSMarc-André Lureau 698bd59f2a1SPaolo BonziniThe back-end relies on the back-end communication channel (see :ref:`Back-end 699bd59f2a1SPaolo Bonzinicommunication <backend_communication>` section below) to send IOTLB miss 700e9ca9f33SMaxime Coquelinand access failure events, by sending ``VHOST_USER_BACKEND_IOTLB_MSG`` 701bd59f2a1SPaolo Bonzinirequests to the front-end with a ``struct vhost_iotlb_msg`` as 702ed1be66bSMarc-André Lureaupayload. For miss events, the iotlb payload has to be filled with the 703ed1be66bSMarc-André Lureaumiss message type (1), the I/O virtual address and the permissions 704ed1be66bSMarc-André Lureauflags. For access failure event, the iotlb payload has to be filled 705ed1be66bSMarc-André Lureauwith the access failure message type (4), the I/O virtual address and 706bd59f2a1SPaolo Bonzinithe permissions flags. For synchronization purpose, the back-end may 707bd59f2a1SPaolo Bonzinirely on the reply-ack feature, so the front-end may send a reply when 708ed1be66bSMarc-André Lureauoperation is completed if the reply-ack feature is negotiated and 709bd59f2a1SPaolo Bonziniback-ends requests a reply. For miss events, completed operation means 710bd59f2a1SPaolo Bonzinieither front-end sent an update message containing the IOTLB entry 711bd59f2a1SPaolo Bonzinicontaining requested address and permission, or front-end sent nothing if 712ed1be66bSMarc-André Lureauthe IOTLB miss message is invalid (invalid IOVA or permission). 713ed1be66bSMarc-André Lureau 714bd59f2a1SPaolo BonziniThe front-end isn't expected to take the initiative to send IOTLB update 715bd59f2a1SPaolo Bonzinimessages, as the back-end sends IOTLB miss messages for the guest virtual 716ed1be66bSMarc-André Lureaumemory areas it needs to access. 717ed1be66bSMarc-André Lureau 718bd59f2a1SPaolo Bonzini.. _backend_communication: 719ed1be66bSMarc-André Lureau 720bd59f2a1SPaolo BonziniBack-end communication 721bd59f2a1SPaolo Bonzini---------------------- 722ed1be66bSMarc-André Lureau 723bd59f2a1SPaolo BonziniAn optional communication channel is provided if the back-end declares 724e9ca9f33SMaxime Coquelin``VHOST_USER_PROTOCOL_F_BACKEND_REQ`` protocol feature, to allow the 725bd59f2a1SPaolo Bonziniback-end to make requests to the front-end. 726ed1be66bSMarc-André Lureau 727e9ca9f33SMaxime CoquelinThe fd is provided via ``VHOST_USER_SET_BACKEND_REQ_FD`` ancillary data. 728ed1be66bSMarc-André Lureau 729e9ca9f33SMaxime CoquelinA back-end may then send ``VHOST_USER_BACKEND_*`` messages to the front-end 730ed1be66bSMarc-André Lureauusing this fd communication channel. 731ed1be66bSMarc-André Lureau 732e9ca9f33SMaxime CoquelinIf ``VHOST_USER_PROTOCOL_F_BACKEND_SEND_FD`` protocol feature is 733bd59f2a1SPaolo Bonzininegotiated, back-end can send file descriptors (at most 8 descriptors in 734bd59f2a1SPaolo Bonzinieach message) to front-end via ancillary data using this fd communication 735ed1be66bSMarc-André Lureauchannel. 736ed1be66bSMarc-André Lureau 737ed1be66bSMarc-André LureauInflight I/O tracking 738ed1be66bSMarc-André Lureau--------------------- 739ed1be66bSMarc-André Lureau 740bd59f2a1SPaolo BonziniTo support reconnecting after restart or crash, back-end may need to 741ed1be66bSMarc-André Lureauresubmit inflight I/Os. If virtqueue is processed in order, we can 742ed1be66bSMarc-André Lureaueasily achieve that by getting the inflight descriptors from 743ed1be66bSMarc-André Lureaudescriptor table (split virtqueue) or descriptor ring (packed 744ed1be66bSMarc-André Lureauvirtqueue). However, it can't work when we process descriptors 745ed1be66bSMarc-André Lureauout-of-order because some entries which store the information of 746ed1be66bSMarc-André Lureauinflight descriptors in available ring (split virtqueue) or descriptor 747ac9574bcSStefan Weilring (packed virtqueue) might be overridden by new entries. To solve 748bd59f2a1SPaolo Bonzinithis problem, the back-end need to allocate an extra buffer to store this 749bd59f2a1SPaolo Bonziniinformation of inflight descriptors and share it with front-end for 750ed1be66bSMarc-André Lureaupersistent. ``VHOST_USER_GET_INFLIGHT_FD`` and 751ed1be66bSMarc-André Lureau``VHOST_USER_SET_INFLIGHT_FD`` are used to transfer this buffer 752bd59f2a1SPaolo Bonzinibetween front-end and back-end. And the format of this buffer is described 753ed1be66bSMarc-André Lureaubelow: 754ed1be66bSMarc-André Lureau 755ed1be66bSMarc-André Lureau+---------------+---------------+-----+---------------+ 756ed1be66bSMarc-André Lureau| queue0 region | queue1 region | ... | queueN region | 757ed1be66bSMarc-André Lureau+---------------+---------------+-----+---------------+ 758ed1be66bSMarc-André Lureau 759bd59f2a1SPaolo BonziniN is the number of available virtqueues. The back-end could get it from num 760ed1be66bSMarc-André Lureauqueues field of ``VhostUserInflight``. 761ed1be66bSMarc-André Lureau 762ed1be66bSMarc-André LureauFor split virtqueue, queue region can be implemented as: 763ed1be66bSMarc-André Lureau 764ed1be66bSMarc-André Lureau.. code:: c 765ed1be66bSMarc-André Lureau 766ed1be66bSMarc-André Lureau typedef struct DescStateSplit { 767ed1be66bSMarc-André Lureau /* Indicate whether this descriptor is inflight or not. 768ed1be66bSMarc-André Lureau * Only available for head-descriptor. */ 769ed1be66bSMarc-André Lureau uint8_t inflight; 770ed1be66bSMarc-André Lureau 771ed1be66bSMarc-André Lureau /* Padding */ 772ed1be66bSMarc-André Lureau uint8_t padding[5]; 773ed1be66bSMarc-André Lureau 774ed1be66bSMarc-André Lureau /* Maintain a list for the last batch of used descriptors. 775ed1be66bSMarc-André Lureau * Only available when batching is used for submitting */ 776ed1be66bSMarc-André Lureau uint16_t next; 777ed1be66bSMarc-André Lureau 778ed1be66bSMarc-André Lureau /* Used to preserve the order of fetching available descriptors. 779ed1be66bSMarc-André Lureau * Only available for head-descriptor. */ 780ed1be66bSMarc-André Lureau uint64_t counter; 781ed1be66bSMarc-André Lureau } DescStateSplit; 782ed1be66bSMarc-André Lureau 783ed1be66bSMarc-André Lureau typedef struct QueueRegionSplit { 784ed1be66bSMarc-André Lureau /* The feature flags of this region. Now it's initialized to 0. */ 785ed1be66bSMarc-André Lureau uint64_t features; 786ed1be66bSMarc-André Lureau 787ed1be66bSMarc-André Lureau /* The version of this region. It's 1 currently. 788ed1be66bSMarc-André Lureau * Zero value indicates an uninitialized buffer */ 789ed1be66bSMarc-André Lureau uint16_t version; 790ed1be66bSMarc-André Lureau 791bd59f2a1SPaolo Bonzini /* The size of DescStateSplit array. It's equal to the virtqueue size. 792bd59f2a1SPaolo Bonzini * The back-end could get it from queue size field of VhostUserInflight. */ 793ed1be66bSMarc-André Lureau uint16_t desc_num; 794ed1be66bSMarc-André Lureau 795ed1be66bSMarc-André Lureau /* The head of list that track the last batch of used descriptors. */ 796ed1be66bSMarc-André Lureau uint16_t last_batch_head; 797ed1be66bSMarc-André Lureau 798ed1be66bSMarc-André Lureau /* Store the idx value of used ring */ 799ed1be66bSMarc-André Lureau uint16_t used_idx; 800ed1be66bSMarc-André Lureau 801ed1be66bSMarc-André Lureau /* Used to track the state of each descriptor in descriptor table */ 802880a7817SPhilippe Mathieu-Daudé DescStateSplit desc[]; 803ed1be66bSMarc-André Lureau } QueueRegionSplit; 804ed1be66bSMarc-André Lureau 805ed1be66bSMarc-André LureauTo track inflight I/O, the queue region should be processed as follows: 806ed1be66bSMarc-André Lureau 807ed1be66bSMarc-André LureauWhen receiving available buffers from the driver: 808ed1be66bSMarc-André Lureau 809ed1be66bSMarc-André Lureau#. Get the next available head-descriptor index from available ring, ``i`` 810ed1be66bSMarc-André Lureau 811ed1be66bSMarc-André Lureau#. Set ``desc[i].counter`` to the value of global counter 812ed1be66bSMarc-André Lureau 813ed1be66bSMarc-André Lureau#. Increase global counter by 1 814ed1be66bSMarc-André Lureau 815ed1be66bSMarc-André Lureau#. Set ``desc[i].inflight`` to 1 816ed1be66bSMarc-André Lureau 817ed1be66bSMarc-André LureauWhen supplying used buffers to the driver: 818ed1be66bSMarc-André Lureau 819ed1be66bSMarc-André Lureau1. Get corresponding used head-descriptor index, i 820ed1be66bSMarc-André Lureau 821ed1be66bSMarc-André Lureau2. Set ``desc[i].next`` to ``last_batch_head`` 822ed1be66bSMarc-André Lureau 823ed1be66bSMarc-André Lureau3. Set ``last_batch_head`` to ``i`` 824ed1be66bSMarc-André Lureau 825ed1be66bSMarc-André Lureau#. Steps 1,2,3 may be performed repeatedly if batching is possible 826ed1be66bSMarc-André Lureau 827ed1be66bSMarc-André Lureau#. Increase the ``idx`` value of used ring by the size of the batch 828ed1be66bSMarc-André Lureau 829ed1be66bSMarc-André Lureau#. Set the ``inflight`` field of each ``DescStateSplit`` entry in the batch to 0 830ed1be66bSMarc-André Lureau 831ed1be66bSMarc-André Lureau#. Set ``used_idx`` to the ``idx`` value of used ring 832ed1be66bSMarc-André Lureau 833ed1be66bSMarc-André LureauWhen reconnecting: 834ed1be66bSMarc-André Lureau 835ed1be66bSMarc-André Lureau#. If the value of ``used_idx`` does not match the ``idx`` value of 836ed1be66bSMarc-André Lureau used ring (means the inflight field of ``DescStateSplit`` entries in 837ed1be66bSMarc-André Lureau last batch may be incorrect), 838ed1be66bSMarc-André Lureau 839ed1be66bSMarc-André Lureau a. Subtract the value of ``used_idx`` from the ``idx`` value of 840ed1be66bSMarc-André Lureau used ring to get last batch size of ``DescStateSplit`` entries 841ed1be66bSMarc-André Lureau 842ed1be66bSMarc-André Lureau #. Set the ``inflight`` field of each ``DescStateSplit`` entry to 0 in last batch 843ed1be66bSMarc-André Lureau list which starts from ``last_batch_head`` 844ed1be66bSMarc-André Lureau 845ed1be66bSMarc-André Lureau #. Set ``used_idx`` to the ``idx`` value of used ring 846ed1be66bSMarc-André Lureau 847ed1be66bSMarc-André Lureau#. Resubmit inflight ``DescStateSplit`` entries in order of their 848ed1be66bSMarc-André Lureau counter value 849ed1be66bSMarc-André Lureau 850ed1be66bSMarc-André LureauFor packed virtqueue, queue region can be implemented as: 851ed1be66bSMarc-André Lureau 852ed1be66bSMarc-André Lureau.. code:: c 853ed1be66bSMarc-André Lureau 854ed1be66bSMarc-André Lureau typedef struct DescStatePacked { 855ed1be66bSMarc-André Lureau /* Indicate whether this descriptor is inflight or not. 856ed1be66bSMarc-André Lureau * Only available for head-descriptor. */ 857ed1be66bSMarc-André Lureau uint8_t inflight; 858ed1be66bSMarc-André Lureau 859ed1be66bSMarc-André Lureau /* Padding */ 860ed1be66bSMarc-André Lureau uint8_t padding; 861ed1be66bSMarc-André Lureau 862ed1be66bSMarc-André Lureau /* Link to the next free entry */ 863ed1be66bSMarc-André Lureau uint16_t next; 864ed1be66bSMarc-André Lureau 865ed1be66bSMarc-André Lureau /* Link to the last entry of descriptor list. 866ed1be66bSMarc-André Lureau * Only available for head-descriptor. */ 867ed1be66bSMarc-André Lureau uint16_t last; 868ed1be66bSMarc-André Lureau 869ed1be66bSMarc-André Lureau /* The length of descriptor list. 870ed1be66bSMarc-André Lureau * Only available for head-descriptor. */ 871ed1be66bSMarc-André Lureau uint16_t num; 872ed1be66bSMarc-André Lureau 873ed1be66bSMarc-André Lureau /* Used to preserve the order of fetching available descriptors. 874ed1be66bSMarc-André Lureau * Only available for head-descriptor. */ 875ed1be66bSMarc-André Lureau uint64_t counter; 876ed1be66bSMarc-André Lureau 877ed1be66bSMarc-André Lureau /* The buffer id */ 878ed1be66bSMarc-André Lureau uint16_t id; 879ed1be66bSMarc-André Lureau 880ed1be66bSMarc-André Lureau /* The descriptor flags */ 881ed1be66bSMarc-André Lureau uint16_t flags; 882ed1be66bSMarc-André Lureau 883ed1be66bSMarc-André Lureau /* The buffer length */ 884ed1be66bSMarc-André Lureau uint32_t len; 885ed1be66bSMarc-André Lureau 886ed1be66bSMarc-André Lureau /* The buffer address */ 887ed1be66bSMarc-André Lureau uint64_t addr; 888ed1be66bSMarc-André Lureau } DescStatePacked; 889ed1be66bSMarc-André Lureau 890ed1be66bSMarc-André Lureau typedef struct QueueRegionPacked { 891ed1be66bSMarc-André Lureau /* The feature flags of this region. Now it's initialized to 0. */ 892ed1be66bSMarc-André Lureau uint64_t features; 893ed1be66bSMarc-André Lureau 894ed1be66bSMarc-André Lureau /* The version of this region. It's 1 currently. 895ed1be66bSMarc-André Lureau * Zero value indicates an uninitialized buffer */ 896ed1be66bSMarc-André Lureau uint16_t version; 897ed1be66bSMarc-André Lureau 898bd59f2a1SPaolo Bonzini /* The size of DescStatePacked array. It's equal to the virtqueue size. 899bd59f2a1SPaolo Bonzini * The back-end could get it from queue size field of VhostUserInflight. */ 900ed1be66bSMarc-André Lureau uint16_t desc_num; 901ed1be66bSMarc-André Lureau 902ed1be66bSMarc-André Lureau /* The head of free DescStatePacked entry list */ 903ed1be66bSMarc-André Lureau uint16_t free_head; 904ed1be66bSMarc-André Lureau 905ed1be66bSMarc-André Lureau /* The old head of free DescStatePacked entry list */ 906ed1be66bSMarc-André Lureau uint16_t old_free_head; 907ed1be66bSMarc-André Lureau 908ed1be66bSMarc-André Lureau /* The used index of descriptor ring */ 909ed1be66bSMarc-André Lureau uint16_t used_idx; 910ed1be66bSMarc-André Lureau 911ed1be66bSMarc-André Lureau /* The old used index of descriptor ring */ 912ed1be66bSMarc-André Lureau uint16_t old_used_idx; 913ed1be66bSMarc-André Lureau 914ed1be66bSMarc-André Lureau /* Device ring wrap counter */ 915ed1be66bSMarc-André Lureau uint8_t used_wrap_counter; 916ed1be66bSMarc-André Lureau 917ed1be66bSMarc-André Lureau /* The old device ring wrap counter */ 918ed1be66bSMarc-André Lureau uint8_t old_used_wrap_counter; 919ed1be66bSMarc-André Lureau 920ed1be66bSMarc-André Lureau /* Padding */ 921ed1be66bSMarc-André Lureau uint8_t padding[7]; 922ed1be66bSMarc-André Lureau 923ed1be66bSMarc-André Lureau /* Used to track the state of each descriptor fetched from descriptor ring */ 924880a7817SPhilippe Mathieu-Daudé DescStatePacked desc[]; 925ed1be66bSMarc-André Lureau } QueueRegionPacked; 926ed1be66bSMarc-André Lureau 927ed1be66bSMarc-André LureauTo track inflight I/O, the queue region should be processed as follows: 928ed1be66bSMarc-André Lureau 929ed1be66bSMarc-André LureauWhen receiving available buffers from the driver: 930ed1be66bSMarc-André Lureau 931ed1be66bSMarc-André Lureau#. Get the next available descriptor entry from descriptor ring, ``d`` 932ed1be66bSMarc-André Lureau 933ed1be66bSMarc-André Lureau#. If ``d`` is head descriptor, 934ed1be66bSMarc-André Lureau 935ed1be66bSMarc-André Lureau a. Set ``desc[old_free_head].num`` to 0 936ed1be66bSMarc-André Lureau 937ed1be66bSMarc-André Lureau #. Set ``desc[old_free_head].counter`` to the value of global counter 938ed1be66bSMarc-André Lureau 939ed1be66bSMarc-André Lureau #. Increase global counter by 1 940ed1be66bSMarc-André Lureau 941ed1be66bSMarc-André Lureau #. Set ``desc[old_free_head].inflight`` to 1 942ed1be66bSMarc-André Lureau 943ed1be66bSMarc-André Lureau#. If ``d`` is last descriptor, set ``desc[old_free_head].last`` to 944ed1be66bSMarc-André Lureau ``free_head`` 945ed1be66bSMarc-André Lureau 946ed1be66bSMarc-André Lureau#. Increase ``desc[old_free_head].num`` by 1 947ed1be66bSMarc-André Lureau 948ed1be66bSMarc-André Lureau#. Set ``desc[free_head].addr``, ``desc[free_head].len``, 949ed1be66bSMarc-André Lureau ``desc[free_head].flags``, ``desc[free_head].id`` to ``d.addr``, 950ed1be66bSMarc-André Lureau ``d.len``, ``d.flags``, ``d.id`` 951ed1be66bSMarc-André Lureau 952ed1be66bSMarc-André Lureau#. Set ``free_head`` to ``desc[free_head].next`` 953ed1be66bSMarc-André Lureau 954ed1be66bSMarc-André Lureau#. If ``d`` is last descriptor, set ``old_free_head`` to ``free_head`` 955ed1be66bSMarc-André Lureau 956ed1be66bSMarc-André LureauWhen supplying used buffers to the driver: 957ed1be66bSMarc-André Lureau 958ed1be66bSMarc-André Lureau1. Get corresponding used head-descriptor entry from descriptor ring, 959ed1be66bSMarc-André Lureau ``d`` 960ed1be66bSMarc-André Lureau 961ed1be66bSMarc-André Lureau2. Get corresponding ``DescStatePacked`` entry, ``e`` 962ed1be66bSMarc-André Lureau 963ed1be66bSMarc-André Lureau3. Set ``desc[e.last].next`` to ``free_head`` 964ed1be66bSMarc-André Lureau 965ed1be66bSMarc-André Lureau4. Set ``free_head`` to the index of ``e`` 966ed1be66bSMarc-André Lureau 967ed1be66bSMarc-André Lureau#. Steps 1,2,3,4 may be performed repeatedly if batching is possible 968ed1be66bSMarc-André Lureau 969ed1be66bSMarc-André Lureau#. Increase ``used_idx`` by the size of the batch and update 970ed1be66bSMarc-André Lureau ``used_wrap_counter`` if needed 971ed1be66bSMarc-André Lureau 972ed1be66bSMarc-André Lureau#. Update ``d.flags`` 973ed1be66bSMarc-André Lureau 974ed1be66bSMarc-André Lureau#. Set the ``inflight`` field of each head ``DescStatePacked`` entry 975ed1be66bSMarc-André Lureau in the batch to 0 976ed1be66bSMarc-André Lureau 977ed1be66bSMarc-André Lureau#. Set ``old_free_head``, ``old_used_idx``, ``old_used_wrap_counter`` 978ed1be66bSMarc-André Lureau to ``free_head``, ``used_idx``, ``used_wrap_counter`` 979ed1be66bSMarc-André Lureau 980ed1be66bSMarc-André LureauWhen reconnecting: 981ed1be66bSMarc-André Lureau 982ed1be66bSMarc-André Lureau#. If ``used_idx`` does not match ``old_used_idx`` (means the 983ed1be66bSMarc-André Lureau ``inflight`` field of ``DescStatePacked`` entries in last batch may 984ed1be66bSMarc-André Lureau be incorrect), 985ed1be66bSMarc-André Lureau 986ed1be66bSMarc-André Lureau a. Get the next descriptor ring entry through ``old_used_idx``, ``d`` 987ed1be66bSMarc-André Lureau 988ed1be66bSMarc-André Lureau #. Use ``old_used_wrap_counter`` to calculate the available flags 989ed1be66bSMarc-André Lureau 990ed1be66bSMarc-André Lureau #. If ``d.flags`` is not equal to the calculated flags value (means 991bd59f2a1SPaolo Bonzini back-end has submitted the buffer to guest driver before crash, so 992ed1be66bSMarc-André Lureau it has to commit the in-progres update), set ``old_free_head``, 993ed1be66bSMarc-André Lureau ``old_used_idx``, ``old_used_wrap_counter`` to ``free_head``, 994ed1be66bSMarc-André Lureau ``used_idx``, ``used_wrap_counter`` 995ed1be66bSMarc-André Lureau 996ed1be66bSMarc-André Lureau#. Set ``free_head``, ``used_idx``, ``used_wrap_counter`` to 997ed1be66bSMarc-André Lureau ``old_free_head``, ``old_used_idx``, ``old_used_wrap_counter`` 998ed1be66bSMarc-André Lureau (roll back any in-progress update) 999ed1be66bSMarc-André Lureau 1000ed1be66bSMarc-André Lureau#. Set the ``inflight`` field of each ``DescStatePacked`` entry in 1001ed1be66bSMarc-André Lureau free list to 0 1002ed1be66bSMarc-André Lureau 1003ed1be66bSMarc-André Lureau#. Resubmit inflight ``DescStatePacked`` entries in order of their 1004ed1be66bSMarc-André Lureau counter value 1005ed1be66bSMarc-André Lureau 10063348e7e3SJohannes BergIn-band notifications 10073348e7e3SJohannes Berg--------------------- 10083348e7e3SJohannes Berg 10093348e7e3SJohannes BergIn some limited situations (e.g. for simulation) it is desirable to 10103348e7e3SJohannes Berghave the kick, call and error (if used) signals done via in-band 10113348e7e3SJohannes Bergmessages instead of asynchronous eventfd notifications. This can be 10123348e7e3SJohannes Bergdone by negotiating the ``VHOST_USER_PROTOCOL_F_INBAND_NOTIFICATIONS`` 10133348e7e3SJohannes Bergprotocol feature. 10143348e7e3SJohannes Berg 10153348e7e3SJohannes BergNote that due to the fact that too many messages on the sockets can 10163348e7e3SJohannes Bergcause the sending application(s) to block, it is not advised to use 10173348e7e3SJohannes Bergthis feature unless absolutely necessary. It is also considered an 10183348e7e3SJohannes Bergerror to negotiate this feature without also negotiating 1019e9ca9f33SMaxime Coquelin``VHOST_USER_PROTOCOL_F_BACKEND_REQ`` and ``VHOST_USER_PROTOCOL_F_REPLY_ACK``, 1020bd59f2a1SPaolo Bonzinithe former is necessary for getting a message channel from the back-end 1021bd59f2a1SPaolo Bonzinito the front-end, while the latter needs to be used with the in-band 10223348e7e3SJohannes Bergnotification messages to block until they are processed, both to avoid 10233348e7e3SJohannes Bergblocking later and for proper processing (at least in the simulation 1024bd59f2a1SPaolo Bonziniuse case.) As it has no other way of signalling this error, the back-end 10253348e7e3SJohannes Bergshould close the connection as a response to a 10263348e7e3SJohannes Berg``VHOST_USER_SET_PROTOCOL_FEATURES`` message that sets the in-band 10273348e7e3SJohannes Bergnotifications feature flag without the other two. 10283348e7e3SJohannes Berg 1029ed1be66bSMarc-André LureauProtocol features 1030ed1be66bSMarc-André Lureau----------------- 1031ed1be66bSMarc-André Lureau 1032ed1be66bSMarc-André Lureau.. code:: c 1033ed1be66bSMarc-André Lureau 1034ed1be66bSMarc-André Lureau #define VHOST_USER_PROTOCOL_F_MQ 0 1035ed1be66bSMarc-André Lureau #define VHOST_USER_PROTOCOL_F_LOG_SHMFD 1 1036ed1be66bSMarc-André Lureau #define VHOST_USER_PROTOCOL_F_RARP 2 1037ed1be66bSMarc-André Lureau #define VHOST_USER_PROTOCOL_F_REPLY_ACK 3 1038ed1be66bSMarc-André Lureau #define VHOST_USER_PROTOCOL_F_MTU 4 1039e9ca9f33SMaxime Coquelin #define VHOST_USER_PROTOCOL_F_BACKEND_REQ 5 1040ed1be66bSMarc-André Lureau #define VHOST_USER_PROTOCOL_F_CROSS_ENDIAN 6 1041ed1be66bSMarc-André Lureau #define VHOST_USER_PROTOCOL_F_CRYPTO_SESSION 7 1042ed1be66bSMarc-André Lureau #define VHOST_USER_PROTOCOL_F_PAGEFAULT 8 1043ed1be66bSMarc-André Lureau #define VHOST_USER_PROTOCOL_F_CONFIG 9 1044e9ca9f33SMaxime Coquelin #define VHOST_USER_PROTOCOL_F_BACKEND_SEND_FD 10 1045ed1be66bSMarc-André Lureau #define VHOST_USER_PROTOCOL_F_HOST_NOTIFIER 11 1046ed1be66bSMarc-André Lureau #define VHOST_USER_PROTOCOL_F_INFLIGHT_SHMFD 12 1047d91d57e6SRaphael Norwitz #define VHOST_USER_PROTOCOL_F_RESET_DEVICE 13 10483348e7e3SJohannes Berg #define VHOST_USER_PROTOCOL_F_INBAND_NOTIFICATIONS 14 10496b0eff1aSRaphael Norwitz #define VHOST_USER_PROTOCOL_F_CONFIGURE_MEM_SLOTS 15 1050553dc662SMaxime Coquelin #define VHOST_USER_PROTOCOL_F_STATUS 16 1051f21e95eeSViresh Kumar #define VHOST_USER_PROTOCOL_F_XEN_MMAP 17 1052d4eb5038SHanna Czenczek #define VHOST_USER_PROTOCOL_F_SHARED_OBJECT 18 105301923309SHanna Czenczek #define VHOST_USER_PROTOCOL_F_DEVICE_STATE 19 1054ed1be66bSMarc-André Lureau 1055bd59f2a1SPaolo BonziniFront-end message types 1056bd59f2a1SPaolo Bonzini----------------------- 1057ed1be66bSMarc-André Lureau 1058ed1be66bSMarc-André Lureau``VHOST_USER_GET_FEATURES`` 1059ed1be66bSMarc-André Lureau :id: 1 1060ed1be66bSMarc-André Lureau :equivalent ioctl: ``VHOST_GET_FEATURES`` 106143725d48SPaolo Bonzini :request payload: N/A 106243725d48SPaolo Bonzini :reply payload: ``u64`` 1063ed1be66bSMarc-André Lureau 1064ed1be66bSMarc-André Lureau Get from the underlying vhost implementation the features bitmask. 1065bd59f2a1SPaolo Bonzini Feature bit ``VHOST_USER_F_PROTOCOL_FEATURES`` signals back-end support 1066ed1be66bSMarc-André Lureau for ``VHOST_USER_GET_PROTOCOL_FEATURES`` and 1067ed1be66bSMarc-André Lureau ``VHOST_USER_SET_PROTOCOL_FEATURES``. 1068ed1be66bSMarc-André Lureau 1069ed1be66bSMarc-André Lureau``VHOST_USER_SET_FEATURES`` 1070ed1be66bSMarc-André Lureau :id: 2 1071ed1be66bSMarc-André Lureau :equivalent ioctl: ``VHOST_SET_FEATURES`` 107243725d48SPaolo Bonzini :request payload: ``u64`` 107343725d48SPaolo Bonzini :reply payload: N/A 1074ed1be66bSMarc-André Lureau 1075ed1be66bSMarc-André Lureau Enable features in the underlying vhost implementation using a 1076ed1be66bSMarc-André Lureau bitmask. Feature bit ``VHOST_USER_F_PROTOCOL_FEATURES`` signals 1077bd59f2a1SPaolo Bonzini back-end support for ``VHOST_USER_GET_PROTOCOL_FEATURES`` and 1078ed1be66bSMarc-André Lureau ``VHOST_USER_SET_PROTOCOL_FEATURES``. 1079ed1be66bSMarc-André Lureau 1080ed1be66bSMarc-André Lureau``VHOST_USER_GET_PROTOCOL_FEATURES`` 1081ed1be66bSMarc-André Lureau :id: 15 1082ed1be66bSMarc-André Lureau :equivalent ioctl: ``VHOST_GET_FEATURES`` 108343725d48SPaolo Bonzini :request payload: N/A 108443725d48SPaolo Bonzini :reply payload: ``u64`` 1085ed1be66bSMarc-André Lureau 1086ed1be66bSMarc-André Lureau Get the protocol feature bitmask from the underlying vhost 1087ed1be66bSMarc-André Lureau implementation. Only legal if feature bit 1088ed1be66bSMarc-André Lureau ``VHOST_USER_F_PROTOCOL_FEATURES`` is present in 1089fa997266SAlex Bennée ``VHOST_USER_GET_FEATURES``. It does not need to be acknowledged by 1090fa997266SAlex Bennée ``VHOST_USER_SET_FEATURES``. 1091ed1be66bSMarc-André Lureau 1092ed1be66bSMarc-André Lureau.. Note:: 1093bd59f2a1SPaolo Bonzini Back-ends that report ``VHOST_USER_F_PROTOCOL_FEATURES`` must 1094ed1be66bSMarc-André Lureau support this message even before ``VHOST_USER_SET_FEATURES`` was 1095ed1be66bSMarc-André Lureau called. 1096ed1be66bSMarc-André Lureau 1097ed1be66bSMarc-André Lureau``VHOST_USER_SET_PROTOCOL_FEATURES`` 1098ed1be66bSMarc-André Lureau :id: 16 1099ed1be66bSMarc-André Lureau :equivalent ioctl: ``VHOST_SET_FEATURES`` 110043725d48SPaolo Bonzini :request payload: ``u64`` 110143725d48SPaolo Bonzini :reply payload: N/A 1102ed1be66bSMarc-André Lureau 1103ed1be66bSMarc-André Lureau Enable protocol features in the underlying vhost implementation. 1104ed1be66bSMarc-André Lureau 1105ed1be66bSMarc-André Lureau Only legal if feature bit ``VHOST_USER_F_PROTOCOL_FEATURES`` is present in 1106fa997266SAlex Bennée ``VHOST_USER_GET_FEATURES``. It does not need to be acknowledged by 1107fa997266SAlex Bennée ``VHOST_USER_SET_FEATURES``. 1108ed1be66bSMarc-André Lureau 1109ed1be66bSMarc-André Lureau.. Note:: 1110bd59f2a1SPaolo Bonzini Back-ends that report ``VHOST_USER_F_PROTOCOL_FEATURES`` must support 1111ed1be66bSMarc-André Lureau this message even before ``VHOST_USER_SET_FEATURES`` was called. 1112ed1be66bSMarc-André Lureau 1113ed1be66bSMarc-André Lureau``VHOST_USER_SET_OWNER`` 1114ed1be66bSMarc-André Lureau :id: 3 1115ed1be66bSMarc-André Lureau :equivalent ioctl: ``VHOST_SET_OWNER`` 111643725d48SPaolo Bonzini :request payload: N/A 111743725d48SPaolo Bonzini :reply payload: N/A 1118ed1be66bSMarc-André Lureau 1119bd59f2a1SPaolo Bonzini Issued when a new connection is established. It marks the sender 1120bd59f2a1SPaolo Bonzini as the front-end that owns of the session. This can be used on the *back-end* 1121ed1be66bSMarc-André Lureau as a "session start" flag. 1122ed1be66bSMarc-André Lureau 1123ed1be66bSMarc-André Lureau``VHOST_USER_RESET_OWNER`` 1124ed1be66bSMarc-André Lureau :id: 4 112543725d48SPaolo Bonzini :request payload: N/A 112643725d48SPaolo Bonzini :reply payload: N/A 1127ed1be66bSMarc-André Lureau 1128ed1be66bSMarc-André Lureau.. admonition:: Deprecated 1129ed1be66bSMarc-André Lureau 1130ed1be66bSMarc-André Lureau This is no longer used. Used to be sent to request disabling all 1131bd59f2a1SPaolo Bonzini rings, but some back-ends interpreted it to also discard connection 1132ed1be66bSMarc-André Lureau state (this interpretation would lead to bugs). It is recommended 1133bd59f2a1SPaolo Bonzini that back-ends either ignore this message, or use it to disable all 1134ed1be66bSMarc-André Lureau rings. 1135ed1be66bSMarc-André Lureau 1136ed1be66bSMarc-André Lureau``VHOST_USER_SET_MEM_TABLE`` 1137ed1be66bSMarc-André Lureau :id: 5 1138ed1be66bSMarc-André Lureau :equivalent ioctl: ``VHOST_SET_MEM_TABLE`` 113917c67f68SViresh Kumar :request payload: multiple memory regions description 114017c67f68SViresh Kumar :reply payload: (postcopy only) multiple memory regions description 1141ed1be66bSMarc-André Lureau 1142bd59f2a1SPaolo Bonzini Sets the memory map regions on the back-end so it can translate the 1143ed1be66bSMarc-André Lureau vring addresses. In the ancillary data there is an array of file 1144ed1be66bSMarc-André Lureau descriptors for each memory mapped region. The size and ordering of 1145ed1be66bSMarc-André Lureau the fds matches the number and ordering of memory regions. 1146ed1be66bSMarc-André Lureau 1147ed1be66bSMarc-André Lureau When ``VHOST_USER_POSTCOPY_LISTEN`` has been received, 1148ed1be66bSMarc-André Lureau ``SET_MEM_TABLE`` replies with the bases of the memory mapped 1149bd59f2a1SPaolo Bonzini regions to the front-end. The back-end must have mmap'd the regions but 1150ed1be66bSMarc-André Lureau not yet accessed them and should not yet generate a userfault 1151ed1be66bSMarc-André Lureau event. 1152ed1be66bSMarc-André Lureau 1153ed1be66bSMarc-André Lureau.. Note:: 1154ed1be66bSMarc-André Lureau ``NEED_REPLY_MASK`` is not set in this case. QEMU will then 1155ed1be66bSMarc-André Lureau reply back to the list of mappings with an empty 1156ed1be66bSMarc-André Lureau ``VHOST_USER_SET_MEM_TABLE`` as an acknowledgement; only upon 1157ed1be66bSMarc-André Lureau reception of this message may the guest start accessing the memory 1158ed1be66bSMarc-André Lureau and generating faults. 1159ed1be66bSMarc-André Lureau 1160ed1be66bSMarc-André Lureau``VHOST_USER_SET_LOG_BASE`` 1161ed1be66bSMarc-André Lureau :id: 6 1162ed1be66bSMarc-André Lureau :equivalent ioctl: ``VHOST_SET_LOG_BASE`` 116343725d48SPaolo Bonzini :request payload: u64 116443725d48SPaolo Bonzini :reply payload: N/A 1165ed1be66bSMarc-André Lureau 1166ed1be66bSMarc-André Lureau Sets logging shared memory space. 1167ed1be66bSMarc-André Lureau 1168bd59f2a1SPaolo Bonzini When the back-end has ``VHOST_USER_PROTOCOL_F_LOG_SHMFD`` protocol feature, 1169ed1be66bSMarc-André Lureau the log memory fd is provided in the ancillary data of 1170ed1be66bSMarc-André Lureau ``VHOST_USER_SET_LOG_BASE`` message, the size and offset of shared 1171ed1be66bSMarc-André Lureau memory area provided in the message. 1172ed1be66bSMarc-André Lureau 1173ed1be66bSMarc-André Lureau``VHOST_USER_SET_LOG_FD`` 1174ed1be66bSMarc-André Lureau :id: 7 1175ed1be66bSMarc-André Lureau :equivalent ioctl: ``VHOST_SET_LOG_FD`` 117643725d48SPaolo Bonzini :request payload: N/A 117743725d48SPaolo Bonzini :reply payload: N/A 1178ed1be66bSMarc-André Lureau 1179ed1be66bSMarc-André Lureau Sets the logging file descriptor, which is passed as ancillary data. 1180ed1be66bSMarc-André Lureau 1181ed1be66bSMarc-André Lureau``VHOST_USER_SET_VRING_NUM`` 1182ed1be66bSMarc-André Lureau :id: 8 1183ed1be66bSMarc-André Lureau :equivalent ioctl: ``VHOST_SET_VRING_NUM`` 118443725d48SPaolo Bonzini :request payload: vring state description 118543725d48SPaolo Bonzini :reply payload: N/A 1186ed1be66bSMarc-André Lureau 1187ed1be66bSMarc-André Lureau Set the size of the queue. 1188ed1be66bSMarc-André Lureau 1189ed1be66bSMarc-André Lureau``VHOST_USER_SET_VRING_ADDR`` 1190ed1be66bSMarc-André Lureau :id: 9 1191ed1be66bSMarc-André Lureau :equivalent ioctl: ``VHOST_SET_VRING_ADDR`` 119243725d48SPaolo Bonzini :request payload: vring address description 119343725d48SPaolo Bonzini :reply payload: N/A 1194ed1be66bSMarc-André Lureau 1195ed1be66bSMarc-André Lureau Sets the addresses of the different aspects of the vring. 1196ed1be66bSMarc-André Lureau 1197ed1be66bSMarc-André Lureau``VHOST_USER_SET_VRING_BASE`` 1198ed1be66bSMarc-André Lureau :id: 10 1199ed1be66bSMarc-André Lureau :equivalent ioctl: ``VHOST_SET_VRING_BASE`` 1200c7f21816SHanna Czenczek :request payload: vring descriptor index/indices 120143725d48SPaolo Bonzini :reply payload: N/A 1202ed1be66bSMarc-André Lureau 1203c7f21816SHanna Czenczek Sets the next index to use for descriptors in this vring: 1204c7f21816SHanna Czenczek 1205c7f21816SHanna Czenczek * For a split virtqueue, sets only the next descriptor index to 1206c7f21816SHanna Czenczek process in the *Available Ring*. The device is supposed to read the 1207c7f21816SHanna Czenczek next index in the *Used Ring* from the respective vring structure in 1208c7f21816SHanna Czenczek guest memory. 1209c7f21816SHanna Czenczek 1210c7f21816SHanna Czenczek * For a packed virtqueue, both indices are supplied, as they are not 1211c7f21816SHanna Czenczek explicitly available in memory. 1212c7f21816SHanna Czenczek 1213c7f21816SHanna Czenczek Consequently, the payload type is specific to the type of virt queue 1214c7f21816SHanna Czenczek (*a vring descriptor index for split virtqueues* vs. *vring descriptor 1215c7f21816SHanna Czenczek indices for packed virtqueues*). 1216ed1be66bSMarc-André Lureau 1217ed1be66bSMarc-André Lureau``VHOST_USER_GET_VRING_BASE`` 1218ed1be66bSMarc-André Lureau :id: 11 1219ed1be66bSMarc-André Lureau :equivalent ioctl: ``VHOST_USER_GET_VRING_BASE`` 122043725d48SPaolo Bonzini :request payload: vring state description 1221c7f21816SHanna Czenczek :reply payload: vring descriptor index/indices 1222ed1be66bSMarc-André Lureau 1223c7f21816SHanna Czenczek Stops the vring and returns the current descriptor index or indices: 1224c7f21816SHanna Czenczek 1225c7f21816SHanna Czenczek * For a split virtqueue, returns only the 16-bit next descriptor 1226c7f21816SHanna Czenczek index to process in the *Available Ring*. Note that this may 1227c7f21816SHanna Czenczek differ from the available ring index in the vring structure in 1228c7f21816SHanna Czenczek memory, which points to where the driver will put new available 1229c7f21816SHanna Czenczek descriptors. For the *Used Ring*, the device only needs the next 1230c7f21816SHanna Czenczek descriptor index at which to put new descriptors, which is the 1231c7f21816SHanna Czenczek value in the vring structure in memory, so this value is not 1232c7f21816SHanna Czenczek covered by this message. 1233c7f21816SHanna Czenczek 1234c7f21816SHanna Czenczek * For a packed virtqueue, neither index is explicitly available to 1235c7f21816SHanna Czenczek read from memory, so both indices (as maintained by the device) are 1236c7f21816SHanna Czenczek returned. 1237c7f21816SHanna Czenczek 1238c7f21816SHanna Czenczek Consequently, the payload type is specific to the type of virt queue 1239c7f21816SHanna Czenczek (*a vring descriptor index for split virtqueues* vs. *vring descriptor 1240c7f21816SHanna Czenczek indices for packed virtqueues*). 1241c7f21816SHanna Czenczek 1242a6e76dd3SHanna Czenczek When and as long as all of a device’s vrings are stopped, it is 1243a6e76dd3SHanna Czenczek *suspended*, see :ref:`Suspended device state 1244a6e76dd3SHanna Czenczek <suspended_device_state>`. 1245a6e76dd3SHanna Czenczek 1246c7f21816SHanna Czenczek The request payload’s *num* field is currently reserved and must be 1247c7f21816SHanna Czenczek set to 0. 1248ed1be66bSMarc-André Lureau 1249ed1be66bSMarc-André Lureau``VHOST_USER_SET_VRING_KICK`` 1250ed1be66bSMarc-André Lureau :id: 12 1251ed1be66bSMarc-André Lureau :equivalent ioctl: ``VHOST_SET_VRING_KICK`` 125243725d48SPaolo Bonzini :request payload: ``u64`` 125343725d48SPaolo Bonzini :reply payload: N/A 1254ed1be66bSMarc-André Lureau 1255ed1be66bSMarc-André Lureau Set the event file descriptor for adding buffers to the vring. It is 1256ed1be66bSMarc-André Lureau passed in the ancillary data. 1257ed1be66bSMarc-André Lureau 1258ed1be66bSMarc-André Lureau Bits (0-7) of the payload contain the vring index. Bit 8 is the 1259ed1be66bSMarc-André Lureau invalid FD flag. This flag is set when there is no file descriptor 1260ed1be66bSMarc-André Lureau in the ancillary data. This signals that polling should be used 12613348e7e3SJohannes Berg instead of waiting for the kick. Note that if the protocol feature 12623348e7e3SJohannes Berg ``VHOST_USER_PROTOCOL_F_INBAND_NOTIFICATIONS`` has been negotiated 12633348e7e3SJohannes Berg this message isn't necessary as the ring is also started on the 12643348e7e3SJohannes Berg ``VHOST_USER_VRING_KICK`` message, it may however still be used to 12653348e7e3SJohannes Berg set an event file descriptor (which will be preferred over the 12663348e7e3SJohannes Berg message) or to enable polling. 1267ed1be66bSMarc-André Lureau 1268ed1be66bSMarc-André Lureau``VHOST_USER_SET_VRING_CALL`` 1269ed1be66bSMarc-André Lureau :id: 13 1270ed1be66bSMarc-André Lureau :equivalent ioctl: ``VHOST_SET_VRING_CALL`` 127143725d48SPaolo Bonzini :request payload: ``u64`` 127243725d48SPaolo Bonzini :reply payload: N/A 1273ed1be66bSMarc-André Lureau 1274ed1be66bSMarc-André Lureau Set the event file descriptor to signal when buffers are used. It is 1275ed1be66bSMarc-André Lureau passed in the ancillary data. 1276ed1be66bSMarc-André Lureau 1277ed1be66bSMarc-André Lureau Bits (0-7) of the payload contain the vring index. Bit 8 is the 1278ed1be66bSMarc-André Lureau invalid FD flag. This flag is set when there is no file descriptor 1279ed1be66bSMarc-André Lureau in the ancillary data. This signals that polling will be used 12803348e7e3SJohannes Berg instead of waiting for the call. Note that if the protocol features 12813348e7e3SJohannes Berg ``VHOST_USER_PROTOCOL_F_INBAND_NOTIFICATIONS`` and 1282e9ca9f33SMaxime Coquelin ``VHOST_USER_PROTOCOL_F_BACKEND_REQ`` have been negotiated this message 1283e9ca9f33SMaxime Coquelin isn't necessary as the ``VHOST_USER_BACKEND_VRING_CALL`` message can be 12843348e7e3SJohannes Berg used, it may however still be used to set an event file descriptor 12853348e7e3SJohannes Berg or to enable polling. 1286ed1be66bSMarc-André Lureau 1287ed1be66bSMarc-André Lureau``VHOST_USER_SET_VRING_ERR`` 1288ed1be66bSMarc-André Lureau :id: 14 1289ed1be66bSMarc-André Lureau :equivalent ioctl: ``VHOST_SET_VRING_ERR`` 129043725d48SPaolo Bonzini :request payload: ``u64`` 129143725d48SPaolo Bonzini :reply payload: N/A 1292ed1be66bSMarc-André Lureau 1293ed1be66bSMarc-André Lureau Set the event file descriptor to signal when error occurs. It is 1294ed1be66bSMarc-André Lureau passed in the ancillary data. 1295ed1be66bSMarc-André Lureau 1296ed1be66bSMarc-André Lureau Bits (0-7) of the payload contain the vring index. Bit 8 is the 1297ed1be66bSMarc-André Lureau invalid FD flag. This flag is set when there is no file descriptor 12983348e7e3SJohannes Berg in the ancillary data. Note that if the protocol features 12993348e7e3SJohannes Berg ``VHOST_USER_PROTOCOL_F_INBAND_NOTIFICATIONS`` and 1300e9ca9f33SMaxime Coquelin ``VHOST_USER_PROTOCOL_F_BACKEND_REQ`` have been negotiated this message 1301e9ca9f33SMaxime Coquelin isn't necessary as the ``VHOST_USER_BACKEND_VRING_ERR`` message can be 13023348e7e3SJohannes Berg used, it may however still be used to set an event file descriptor 13033348e7e3SJohannes Berg (which will be preferred over the message). 1304ed1be66bSMarc-André Lureau 1305ed1be66bSMarc-André Lureau``VHOST_USER_GET_QUEUE_NUM`` 1306ed1be66bSMarc-André Lureau :id: 17 1307ed1be66bSMarc-André Lureau :equivalent ioctl: N/A 130843725d48SPaolo Bonzini :request payload: N/A 130943725d48SPaolo Bonzini :reply payload: u64 1310ed1be66bSMarc-André Lureau 1311bd59f2a1SPaolo Bonzini Query how many queues the back-end supports. 1312ed1be66bSMarc-André Lureau 1313ed1be66bSMarc-André Lureau This request should be sent only when ``VHOST_USER_PROTOCOL_F_MQ`` 1314ed1be66bSMarc-André Lureau is set in queried protocol features by 1315ed1be66bSMarc-André Lureau ``VHOST_USER_GET_PROTOCOL_FEATURES``. 1316ed1be66bSMarc-André Lureau 1317ed1be66bSMarc-André Lureau``VHOST_USER_SET_VRING_ENABLE`` 1318ed1be66bSMarc-André Lureau :id: 18 1319ed1be66bSMarc-André Lureau :equivalent ioctl: N/A 132043725d48SPaolo Bonzini :request payload: vring state description 132143725d48SPaolo Bonzini :reply payload: N/A 1322ed1be66bSMarc-André Lureau 1323bd59f2a1SPaolo Bonzini Signal the back-end to enable or disable corresponding vring. 1324ed1be66bSMarc-André Lureau 1325ed1be66bSMarc-André Lureau This request should be sent only when 1326ed1be66bSMarc-André Lureau ``VHOST_USER_F_PROTOCOL_FEATURES`` has been negotiated. 1327ed1be66bSMarc-André Lureau 1328ed1be66bSMarc-André Lureau``VHOST_USER_SEND_RARP`` 1329ed1be66bSMarc-André Lureau :id: 19 1330ed1be66bSMarc-André Lureau :equivalent ioctl: N/A 133143725d48SPaolo Bonzini :request payload: ``u64`` 133243725d48SPaolo Bonzini :reply payload: N/A 1333ed1be66bSMarc-André Lureau 1334bd59f2a1SPaolo Bonzini Ask vhost user back-end to broadcast a fake RARP to notify the migration 1335ed1be66bSMarc-André Lureau is terminated for guest that does not support GUEST_ANNOUNCE. 1336ed1be66bSMarc-André Lureau 1337ed1be66bSMarc-André Lureau Only legal if feature bit ``VHOST_USER_F_PROTOCOL_FEATURES`` is 1338ed1be66bSMarc-André Lureau present in ``VHOST_USER_GET_FEATURES`` and protocol feature bit 1339ed1be66bSMarc-André Lureau ``VHOST_USER_PROTOCOL_F_RARP`` is present in 1340ed1be66bSMarc-André Lureau ``VHOST_USER_GET_PROTOCOL_FEATURES``. The first 6 bytes of the 1341ed1be66bSMarc-André Lureau payload contain the mac address of the guest to allow the vhost user 1342bd59f2a1SPaolo Bonzini back-end to construct and broadcast the fake RARP. 1343ed1be66bSMarc-André Lureau 1344ed1be66bSMarc-André Lureau``VHOST_USER_NET_SET_MTU`` 1345ed1be66bSMarc-André Lureau :id: 20 1346ed1be66bSMarc-André Lureau :equivalent ioctl: N/A 134743725d48SPaolo Bonzini :request payload: ``u64`` 134843725d48SPaolo Bonzini :reply payload: N/A 1349ed1be66bSMarc-André Lureau 1350ed1be66bSMarc-André Lureau Set host MTU value exposed to the guest. 1351ed1be66bSMarc-André Lureau 1352ed1be66bSMarc-André Lureau This request should be sent only when ``VIRTIO_NET_F_MTU`` feature 1353ed1be66bSMarc-André Lureau has been successfully negotiated, ``VHOST_USER_F_PROTOCOL_FEATURES`` 1354ed1be66bSMarc-André Lureau is present in ``VHOST_USER_GET_FEATURES`` and protocol feature bit 1355ed1be66bSMarc-André Lureau ``VHOST_USER_PROTOCOL_F_NET_MTU`` is present in 1356ed1be66bSMarc-André Lureau ``VHOST_USER_GET_PROTOCOL_FEATURES``. 1357ed1be66bSMarc-André Lureau 1358bd59f2a1SPaolo Bonzini If ``VHOST_USER_PROTOCOL_F_REPLY_ACK`` is negotiated, the back-end must 1359ed1be66bSMarc-André Lureau respond with zero in case the specified MTU is valid, or non-zero 1360ed1be66bSMarc-André Lureau otherwise. 1361ed1be66bSMarc-André Lureau 1362e9ca9f33SMaxime Coquelin``VHOST_USER_SET_BACKEND_REQ_FD`` (previous name ``VHOST_USER_SET_SLAVE_REQ_FD``) 1363ed1be66bSMarc-André Lureau :id: 21 1364ed1be66bSMarc-André Lureau :equivalent ioctl: N/A 136543725d48SPaolo Bonzini :request payload: N/A 136643725d48SPaolo Bonzini :reply payload: N/A 1367ed1be66bSMarc-André Lureau 1368bd59f2a1SPaolo Bonzini Set the socket file descriptor for back-end initiated requests. It is passed 1369ed1be66bSMarc-André Lureau in the ancillary data. 1370ed1be66bSMarc-André Lureau 1371ed1be66bSMarc-André Lureau This request should be sent only when 1372ed1be66bSMarc-André Lureau ``VHOST_USER_F_PROTOCOL_FEATURES`` has been negotiated, and protocol 1373e9ca9f33SMaxime Coquelin feature bit ``VHOST_USER_PROTOCOL_F_BACKEND_REQ`` bit is present in 1374ed1be66bSMarc-André Lureau ``VHOST_USER_GET_PROTOCOL_FEATURES``. If 1375bd59f2a1SPaolo Bonzini ``VHOST_USER_PROTOCOL_F_REPLY_ACK`` is negotiated, the back-end must 1376ed1be66bSMarc-André Lureau respond with zero for success, non-zero otherwise. 1377ed1be66bSMarc-André Lureau 1378ed1be66bSMarc-André Lureau``VHOST_USER_IOTLB_MSG`` 1379ed1be66bSMarc-André Lureau :id: 22 1380ed1be66bSMarc-André Lureau :equivalent ioctl: N/A (equivalent to ``VHOST_IOTLB_MSG`` message type) 138143725d48SPaolo Bonzini :request payload: ``struct vhost_iotlb_msg`` 138243725d48SPaolo Bonzini :reply payload: ``u64`` 1383ed1be66bSMarc-André Lureau 1384ed1be66bSMarc-André Lureau Send IOTLB messages with ``struct vhost_iotlb_msg`` as payload. 1385ed1be66bSMarc-André Lureau 1386bd59f2a1SPaolo Bonzini The front-end sends such requests to update and invalidate entries in the 1387bd59f2a1SPaolo Bonzini device IOTLB. The back-end has to acknowledge the request with sending 1388ed1be66bSMarc-André Lureau zero as ``u64`` payload for success, non-zero otherwise. 1389ed1be66bSMarc-André Lureau 1390ed1be66bSMarc-André Lureau This request should be send only when ``VIRTIO_F_IOMMU_PLATFORM`` 1391ed1be66bSMarc-André Lureau feature has been successfully negotiated. 1392ed1be66bSMarc-André Lureau 1393ed1be66bSMarc-André Lureau``VHOST_USER_SET_VRING_ENDIAN`` 1394ed1be66bSMarc-André Lureau :id: 23 1395ed1be66bSMarc-André Lureau :equivalent ioctl: ``VHOST_SET_VRING_ENDIAN`` 139643725d48SPaolo Bonzini :request payload: vring state description 139743725d48SPaolo Bonzini :reply payload: N/A 1398ed1be66bSMarc-André Lureau 1399ed1be66bSMarc-André Lureau Set the endianness of a VQ for legacy devices. Little-endian is 1400ed1be66bSMarc-André Lureau indicated with state.num set to 0 and big-endian is indicated with 1401ed1be66bSMarc-André Lureau state.num set to 1. Other values are invalid. 1402ed1be66bSMarc-André Lureau 1403ed1be66bSMarc-André Lureau This request should be sent only when 1404ed1be66bSMarc-André Lureau ``VHOST_USER_PROTOCOL_F_CROSS_ENDIAN`` has been negotiated. 1405ed1be66bSMarc-André Lureau Backends that negotiated this feature should handle both 1406ed1be66bSMarc-André Lureau endiannesses and expect this message once (per VQ) during device 1407bd59f2a1SPaolo Bonzini configuration (ie. before the front-end starts the VQ). 1408ed1be66bSMarc-André Lureau 1409ed1be66bSMarc-André Lureau``VHOST_USER_GET_CONFIG`` 1410ed1be66bSMarc-André Lureau :id: 24 1411ed1be66bSMarc-André Lureau :equivalent ioctl: N/A 141243725d48SPaolo Bonzini :request payload: virtio device config space 141343725d48SPaolo Bonzini :reply payload: virtio device config space 1414ed1be66bSMarc-André Lureau 1415ed1be66bSMarc-André Lureau When ``VHOST_USER_PROTOCOL_F_CONFIG`` is negotiated, this message is 1416bd59f2a1SPaolo Bonzini submitted by the vhost-user front-end to fetch the contents of the 1417bd59f2a1SPaolo Bonzini virtio device configuration space, vhost-user back-end's payload size 1418bd59f2a1SPaolo Bonzini MUST match the front-end's request, vhost-user back-end uses zero length of 1419bd59f2a1SPaolo Bonzini payload to indicate an error to the vhost-user front-end. The vhost-user 1420bd59f2a1SPaolo Bonzini front-end may cache the contents to avoid repeated 1421ed1be66bSMarc-André Lureau ``VHOST_USER_GET_CONFIG`` calls. 1422ed1be66bSMarc-André Lureau 1423ed1be66bSMarc-André Lureau``VHOST_USER_SET_CONFIG`` 1424ed1be66bSMarc-André Lureau :id: 25 1425ed1be66bSMarc-André Lureau :equivalent ioctl: N/A 142643725d48SPaolo Bonzini :request payload: virtio device config space 142743725d48SPaolo Bonzini :reply payload: N/A 1428ed1be66bSMarc-André Lureau 1429ed1be66bSMarc-André Lureau When ``VHOST_USER_PROTOCOL_F_CONFIG`` is negotiated, this message is 1430bd59f2a1SPaolo Bonzini submitted by the vhost-user front-end when the Guest changes the virtio 1431ed1be66bSMarc-André Lureau device configuration space and also can be used for live migration 1432bd59f2a1SPaolo Bonzini on the destination host. The vhost-user back-end must check the flags 1433bd59f2a1SPaolo Bonzini field, and back-ends MUST NOT accept SET_CONFIG for read-only 1434ed1be66bSMarc-André Lureau configuration space fields unless the live migration bit is set. 1435ed1be66bSMarc-André Lureau 1436ed1be66bSMarc-André Lureau``VHOST_USER_CREATE_CRYPTO_SESSION`` 1437ed1be66bSMarc-André Lureau :id: 26 1438ed1be66bSMarc-André Lureau :equivalent ioctl: N/A 143943725d48SPaolo Bonzini :request payload: crypto session description 144043725d48SPaolo Bonzini :reply payload: crypto session description 1441ed1be66bSMarc-André Lureau 1442bd59f2a1SPaolo Bonzini Create a session for crypto operation. The back-end must return 1443ed1be66bSMarc-André Lureau the session id, 0 or positive for success, negative for failure. 1444ed1be66bSMarc-André Lureau This request should be sent only when 1445ed1be66bSMarc-André Lureau ``VHOST_USER_PROTOCOL_F_CRYPTO_SESSION`` feature has been 1446ed1be66bSMarc-André Lureau successfully negotiated. It's a required feature for crypto 1447ed1be66bSMarc-André Lureau devices. 1448ed1be66bSMarc-André Lureau 1449ed1be66bSMarc-André Lureau``VHOST_USER_CLOSE_CRYPTO_SESSION`` 1450ed1be66bSMarc-André Lureau :id: 27 1451ed1be66bSMarc-André Lureau :equivalent ioctl: N/A 145243725d48SPaolo Bonzini :request payload: ``u64`` 145343725d48SPaolo Bonzini :reply payload: N/A 1454ed1be66bSMarc-André Lureau 1455ed1be66bSMarc-André Lureau Close a session for crypto operation which was previously 1456ed1be66bSMarc-André Lureau created by ``VHOST_USER_CREATE_CRYPTO_SESSION``. 1457ed1be66bSMarc-André Lureau 1458ed1be66bSMarc-André Lureau This request should be sent only when 1459ed1be66bSMarc-André Lureau ``VHOST_USER_PROTOCOL_F_CRYPTO_SESSION`` feature has been 1460ed1be66bSMarc-André Lureau successfully negotiated. It's a required feature for crypto 1461ed1be66bSMarc-André Lureau devices. 1462ed1be66bSMarc-André Lureau 1463ed1be66bSMarc-André Lureau``VHOST_USER_POSTCOPY_ADVISE`` 1464ed1be66bSMarc-André Lureau :id: 28 146543725d48SPaolo Bonzini :request payload: N/A 146643725d48SPaolo Bonzini :reply payload: userfault fd 1467ed1be66bSMarc-André Lureau 1468bd59f2a1SPaolo Bonzini When ``VHOST_USER_PROTOCOL_F_PAGEFAULT`` is supported, the front-end 1469bd59f2a1SPaolo Bonzini advises back-end that a migration with postcopy enabled is underway, 1470bd59f2a1SPaolo Bonzini the back-end must open a userfaultfd for later use. Note that at this 1471ed1be66bSMarc-André Lureau stage the migration is still in precopy mode. 1472ed1be66bSMarc-André Lureau 1473ed1be66bSMarc-André Lureau``VHOST_USER_POSTCOPY_LISTEN`` 1474ed1be66bSMarc-André Lureau :id: 29 147543725d48SPaolo Bonzini :request payload: N/A 147643725d48SPaolo Bonzini :reply payload: N/A 1477ed1be66bSMarc-André Lureau 1478bd59f2a1SPaolo Bonzini The front-end advises back-end that a transition to postcopy mode has 1479bd59f2a1SPaolo Bonzini happened. The back-end must ensure that shared memory is registered 1480ed1be66bSMarc-André Lureau with userfaultfd to cause faulting of non-present pages. 1481ed1be66bSMarc-André Lureau 1482ed1be66bSMarc-André Lureau This is always sent sometime after a ``VHOST_USER_POSTCOPY_ADVISE``, 1483ed1be66bSMarc-André Lureau and thus only when ``VHOST_USER_PROTOCOL_F_PAGEFAULT`` is supported. 1484ed1be66bSMarc-André Lureau 1485ed1be66bSMarc-André Lureau``VHOST_USER_POSTCOPY_END`` 1486ed1be66bSMarc-André Lureau :id: 30 148743725d48SPaolo Bonzini :request payload: N/A 148843725d48SPaolo Bonzini :reply payload: ``u64`` 1489ed1be66bSMarc-André Lureau 1490bd59f2a1SPaolo Bonzini The front-end advises that postcopy migration has now completed. The back-end 149143725d48SPaolo Bonzini must disable the userfaultfd. The reply is an acknowledgement 1492ed1be66bSMarc-André Lureau only. 1493ed1be66bSMarc-André Lureau 1494ed1be66bSMarc-André Lureau When ``VHOST_USER_PROTOCOL_F_PAGEFAULT`` is supported, this message 1495ed1be66bSMarc-André Lureau is sent at the end of the migration, after 1496ed1be66bSMarc-André Lureau ``VHOST_USER_POSTCOPY_LISTEN`` was previously sent. 1497ed1be66bSMarc-André Lureau 1498ed1be66bSMarc-André Lureau The value returned is an error indication; 0 is success. 1499ed1be66bSMarc-André Lureau 1500ed1be66bSMarc-André Lureau``VHOST_USER_GET_INFLIGHT_FD`` 1501ed1be66bSMarc-André Lureau :id: 31 1502ed1be66bSMarc-André Lureau :equivalent ioctl: N/A 150343725d48SPaolo Bonzini :request payload: inflight description 150443725d48SPaolo Bonzini :reply payload: N/A 1505ed1be66bSMarc-André Lureau 1506ed1be66bSMarc-André Lureau When ``VHOST_USER_PROTOCOL_F_INFLIGHT_SHMFD`` protocol feature has 1507bd59f2a1SPaolo Bonzini been successfully negotiated, this message is submitted by the front-end to 1508bd59f2a1SPaolo Bonzini get a shared buffer from back-end. The shared buffer will be used to 1509bd59f2a1SPaolo Bonzini track inflight I/O by back-end. QEMU should retrieve a new one when vm 1510ed1be66bSMarc-André Lureau reset. 1511ed1be66bSMarc-André Lureau 1512ed1be66bSMarc-André Lureau``VHOST_USER_SET_INFLIGHT_FD`` 1513ed1be66bSMarc-André Lureau :id: 32 1514ed1be66bSMarc-André Lureau :equivalent ioctl: N/A 151543725d48SPaolo Bonzini :request payload: inflight description 151643725d48SPaolo Bonzini :reply payload: N/A 1517ed1be66bSMarc-André Lureau 1518ed1be66bSMarc-André Lureau When ``VHOST_USER_PROTOCOL_F_INFLIGHT_SHMFD`` protocol feature has 1519bd59f2a1SPaolo Bonzini been successfully negotiated, this message is submitted by the front-end to 1520bd59f2a1SPaolo Bonzini send the shared inflight buffer back to the back-end so that the back-end 1521bd59f2a1SPaolo Bonzini could get inflight I/O after a crash or restart. 1522ed1be66bSMarc-André Lureau 1523bd2e44feSMarc-André Lureau``VHOST_USER_GPU_SET_SOCKET`` 1524bd2e44feSMarc-André Lureau :id: 33 1525bd2e44feSMarc-André Lureau :equivalent ioctl: N/A 152643725d48SPaolo Bonzini :request payload: N/A 152743725d48SPaolo Bonzini :reply payload: N/A 1528bd2e44feSMarc-André Lureau 1529bd2e44feSMarc-André Lureau Sets the GPU protocol socket file descriptor, which is passed as 1530bd59f2a1SPaolo Bonzini ancillary data. The GPU protocol is used to inform the front-end of 1531bd2e44feSMarc-André Lureau rendering state and updates. See vhost-user-gpu.rst for details. 1532bd2e44feSMarc-André Lureau 1533d91d57e6SRaphael Norwitz``VHOST_USER_RESET_DEVICE`` 1534d91d57e6SRaphael Norwitz :id: 34 1535d91d57e6SRaphael Norwitz :equivalent ioctl: N/A 153643725d48SPaolo Bonzini :request payload: N/A 153743725d48SPaolo Bonzini :reply payload: N/A 1538d91d57e6SRaphael Norwitz 1539bd59f2a1SPaolo Bonzini Ask the vhost user back-end to disable all rings and reset all 1540d91d57e6SRaphael Norwitz internal device state to the initial state, ready to be 1541bd59f2a1SPaolo Bonzini reinitialized. The back-end retains ownership of the device 1542d91d57e6SRaphael Norwitz throughout the reset operation. 1543d91d57e6SRaphael Norwitz 1544d91d57e6SRaphael Norwitz Only valid if the ``VHOST_USER_PROTOCOL_F_RESET_DEVICE`` protocol 1545bd59f2a1SPaolo Bonzini feature is set by the back-end. 1546d91d57e6SRaphael Norwitz 15473348e7e3SJohannes Berg``VHOST_USER_VRING_KICK`` 15483348e7e3SJohannes Berg :id: 35 15493348e7e3SJohannes Berg :equivalent ioctl: N/A 155043725d48SPaolo Bonzini :request payload: vring state description 155143725d48SPaolo Bonzini :reply payload: N/A 15523348e7e3SJohannes Berg 15533348e7e3SJohannes Berg When the ``VHOST_USER_PROTOCOL_F_INBAND_NOTIFICATIONS`` protocol 15543348e7e3SJohannes Berg feature has been successfully negotiated, this message may be 1555bd59f2a1SPaolo Bonzini submitted by the front-end to indicate that a buffer was added to 15563348e7e3SJohannes Berg the vring instead of signalling it using the vring's kick file 1557bd59f2a1SPaolo Bonzini descriptor or having the back-end rely on polling. 15583348e7e3SJohannes Berg 15593348e7e3SJohannes Berg The state.num field is currently reserved and must be set to 0. 15603348e7e3SJohannes Berg 15616b0eff1aSRaphael Norwitz``VHOST_USER_GET_MAX_MEM_SLOTS`` 15626b0eff1aSRaphael Norwitz :id: 36 15636b0eff1aSRaphael Norwitz :equivalent ioctl: N/A 156443725d48SPaolo Bonzini :request payload: N/A 156543725d48SPaolo Bonzini :reply payload: u64 15666b0eff1aSRaphael Norwitz 15676b0eff1aSRaphael Norwitz When the ``VHOST_USER_PROTOCOL_F_CONFIGURE_MEM_SLOTS`` protocol 15686b0eff1aSRaphael Norwitz feature has been successfully negotiated, this message is submitted 1569bd59f2a1SPaolo Bonzini by the front-end to the back-end. The back-end should return the message with a 15706b0eff1aSRaphael Norwitz u64 payload containing the maximum number of memory slots for 1571bd59f2a1SPaolo Bonzini QEMU to expose to the guest. The value returned by the back-end 157227598393SRaphael Norwitz will be capped at the maximum number of ram slots which can be 157327598393SRaphael Norwitz supported by the target platform. 1574f1aeb14bSRaphael Norwitz 1575f1aeb14bSRaphael Norwitz``VHOST_USER_ADD_MEM_REG`` 1576f1aeb14bSRaphael Norwitz :id: 37 1577f1aeb14bSRaphael Norwitz :equivalent ioctl: N/A 157843725d48SPaolo Bonzini :request payload: N/A 157943725d48SPaolo Bonzini :reply payload: single memory region description 1580f1aeb14bSRaphael Norwitz 1581f1aeb14bSRaphael Norwitz When the ``VHOST_USER_PROTOCOL_F_CONFIGURE_MEM_SLOTS`` protocol 1582f1aeb14bSRaphael Norwitz feature has been successfully negotiated, this message is submitted 1583bd59f2a1SPaolo Bonzini by the front-end to the back-end. The message payload contains a memory 1584f1aeb14bSRaphael Norwitz region descriptor struct, describing a region of guest memory which 1585bd59f2a1SPaolo Bonzini the back-end device must map in. When the 1586f1aeb14bSRaphael Norwitz ``VHOST_USER_PROTOCOL_F_CONFIGURE_MEM_SLOTS`` protocol feature has 1587f1aeb14bSRaphael Norwitz been successfully negotiated, along with the 1588f1aeb14bSRaphael Norwitz ``VHOST_USER_REM_MEM_REG`` message, this message is used to set and 1589bd59f2a1SPaolo Bonzini update the memory tables of the back-end device. 1590f1aeb14bSRaphael Norwitz 159131009d13SKevin Wolf Exactly one file descriptor from which the memory is mapped is 159231009d13SKevin Wolf passed in the ancillary data. 159331009d13SKevin Wolf 1594bd59f2a1SPaolo Bonzini In postcopy mode (see ``VHOST_USER_POSTCOPY_LISTEN``), the back-end 1595bd59f2a1SPaolo Bonzini replies with the bases of the memory mapped region to the front-end. 159631009d13SKevin Wolf For further details on postcopy, see ``VHOST_USER_SET_MEM_TABLE``. 159731009d13SKevin Wolf They apply to ``VHOST_USER_ADD_MEM_REG`` accordingly. 159831009d13SKevin Wolf 1599f1aeb14bSRaphael Norwitz``VHOST_USER_REM_MEM_REG`` 1600f1aeb14bSRaphael Norwitz :id: 38 1601f1aeb14bSRaphael Norwitz :equivalent ioctl: N/A 160243725d48SPaolo Bonzini :request payload: N/A 160343725d48SPaolo Bonzini :reply payload: single memory region description 1604f1aeb14bSRaphael Norwitz 1605f1aeb14bSRaphael Norwitz When the ``VHOST_USER_PROTOCOL_F_CONFIGURE_MEM_SLOTS`` protocol 1606f1aeb14bSRaphael Norwitz feature has been successfully negotiated, this message is submitted 1607bd59f2a1SPaolo Bonzini by the front-end to the back-end. The message payload contains a memory 1608f1aeb14bSRaphael Norwitz region descriptor struct, describing a region of guest memory which 1609bd59f2a1SPaolo Bonzini the back-end device must unmap. When the 1610f1aeb14bSRaphael Norwitz ``VHOST_USER_PROTOCOL_F_CONFIGURE_MEM_SLOTS`` protocol feature has 1611f1aeb14bSRaphael Norwitz been successfully negotiated, along with the 1612f1aeb14bSRaphael Norwitz ``VHOST_USER_ADD_MEM_REG`` message, this message is used to set and 1613bd59f2a1SPaolo Bonzini update the memory tables of the back-end device. 16146b0eff1aSRaphael Norwitz 161531009d13SKevin Wolf The memory region to be removed is identified by its guest address, 161631009d13SKevin Wolf user address and size. The mmap offset is ignored. 161731009d13SKevin Wolf 161831009d13SKevin Wolf No file descriptors SHOULD be passed in the ancillary data. For 1619bd59f2a1SPaolo Bonzini compatibility with existing incorrect implementations, the back-end MAY 162031009d13SKevin Wolf accept messages with one file descriptor. If a file descriptor is 1621bd59f2a1SPaolo Bonzini passed, the back-end MUST close it without using it otherwise. 162231009d13SKevin Wolf 1623553dc662SMaxime Coquelin``VHOST_USER_SET_STATUS`` 1624553dc662SMaxime Coquelin :id: 39 1625553dc662SMaxime Coquelin :equivalent ioctl: VHOST_VDPA_SET_STATUS 162643725d48SPaolo Bonzini :request payload: ``u64`` 162743725d48SPaolo Bonzini :reply payload: N/A 1628553dc662SMaxime Coquelin 1629553dc662SMaxime Coquelin When the ``VHOST_USER_PROTOCOL_F_STATUS`` protocol feature has been 1630bd59f2a1SPaolo Bonzini successfully negotiated, this message is submitted by the front-end to 1631bd59f2a1SPaolo Bonzini notify the back-end with updated device status as defined in the Virtio 1632553dc662SMaxime Coquelin specification. 1633553dc662SMaxime Coquelin 1634553dc662SMaxime Coquelin``VHOST_USER_GET_STATUS`` 1635553dc662SMaxime Coquelin :id: 40 1636553dc662SMaxime Coquelin :equivalent ioctl: VHOST_VDPA_GET_STATUS 163743725d48SPaolo Bonzini :request payload: N/A 163843725d48SPaolo Bonzini :reply payload: ``u64`` 1639553dc662SMaxime Coquelin 1640553dc662SMaxime Coquelin When the ``VHOST_USER_PROTOCOL_F_STATUS`` protocol feature has been 1641bd59f2a1SPaolo Bonzini successfully negotiated, this message is submitted by the front-end to 1642bd59f2a1SPaolo Bonzini query the back-end for its device status as defined in the Virtio 1643553dc662SMaxime Coquelin specification. 1644553dc662SMaxime Coquelin 164516094766SAlbert Esteve``VHOST_USER_GET_SHARED_OBJECT`` 164616094766SAlbert Esteve :id: 41 164716094766SAlbert Esteve :equivalent ioctl: N/A 164816094766SAlbert Esteve :request payload: ``struct VhostUserShared`` 164916094766SAlbert Esteve :reply payload: dmabuf fd 165016094766SAlbert Esteve 165116094766SAlbert Esteve When the ``VHOST_USER_PROTOCOL_F_SHARED_OBJECT`` protocol 165216094766SAlbert Esteve feature has been successfully negotiated, and the UUID is found 165316094766SAlbert Esteve in the exporters cache, this message is submitted by the front-end 165416094766SAlbert Esteve to retrieve a given dma-buf fd from a given back-end, determined by 165516094766SAlbert Esteve the requested UUID. Back-end will reply passing the fd when the operation 165616094766SAlbert Esteve is successful, or no fd otherwise. 1657553dc662SMaxime Coquelin 165801923309SHanna Czenczek``VHOST_USER_SET_DEVICE_STATE_FD`` 165901923309SHanna Czenczek :id: 42 166001923309SHanna Czenczek :equivalent ioctl: N/A 166101923309SHanna Czenczek :request payload: device state transfer parameters 166201923309SHanna Czenczek :reply payload: ``u64`` 166301923309SHanna Czenczek 166401923309SHanna Czenczek Front-end and back-end negotiate a channel over which to transfer the 166501923309SHanna Czenczek back-end’s internal state during migration. Either side (front-end or 166601923309SHanna Czenczek back-end) may create the channel. The nature of this channel is not 166701923309SHanna Czenczek restricted or defined in this document, but whichever side creates it 166801923309SHanna Czenczek must create a file descriptor that is provided to the respectively 166901923309SHanna Czenczek other side, allowing access to the channel. This FD must behave as 167001923309SHanna Czenczek follows: 167101923309SHanna Czenczek 167201923309SHanna Czenczek * For the writing end, it must allow writing the whole back-end state 167301923309SHanna Czenczek sequentially. Closing the file descriptor signals the end of 167401923309SHanna Czenczek transfer. 167501923309SHanna Czenczek 167601923309SHanna Czenczek * For the reading end, it must allow reading the whole back-end state 167701923309SHanna Czenczek sequentially. The end of file signals the end of the transfer. 167801923309SHanna Czenczek 167901923309SHanna Czenczek For example, the channel may be a pipe, in which case the two ends of 168001923309SHanna Czenczek the pipe fulfill these requirements respectively. 168101923309SHanna Czenczek 168201923309SHanna Czenczek Initially, the front-end creates a channel along with such an FD. It 168301923309SHanna Czenczek passes the FD to the back-end as ancillary data of a 168401923309SHanna Czenczek ``VHOST_USER_SET_DEVICE_STATE_FD`` message. The back-end may create a 168501923309SHanna Czenczek different transfer channel, passing the respective FD back to the 168601923309SHanna Czenczek front-end as ancillary data of the reply. If so, the front-end must 168701923309SHanna Czenczek then discard its channel and use the one provided by the back-end. 168801923309SHanna Czenczek 168901923309SHanna Czenczek Whether the back-end should decide to use its own channel is decided 169001923309SHanna Czenczek based on efficiency: If the channel is a pipe, both ends will most 169101923309SHanna Czenczek likely need to copy data into and out of it. Any channel that allows 169201923309SHanna Czenczek for more efficient processing on at least one end, e.g. through 169301923309SHanna Czenczek zero-copy, is considered more efficient and thus preferred. If the 169401923309SHanna Czenczek back-end can provide such a channel, it should decide to use it. 169501923309SHanna Czenczek 169601923309SHanna Czenczek The request payload contains parameters for the subsequent data 169701923309SHanna Czenczek transfer, as described in the :ref:`Migrating back-end state 169801923309SHanna Czenczek <migrating_backend_state>` section. 169901923309SHanna Czenczek 170001923309SHanna Czenczek The value returned is both an indication for success, and whether a 170101923309SHanna Czenczek file descriptor for a back-end-provided channel is returned: Bits 0–7 170201923309SHanna Czenczek are 0 on success, and non-zero on error. Bit 8 is the invalid FD 170301923309SHanna Czenczek flag; this flag is set when there is no file descriptor returned. 170401923309SHanna Czenczek When this flag is not set, the front-end must use the returned file 170501923309SHanna Czenczek descriptor as its end of the transfer channel. The back-end must not 170601923309SHanna Czenczek both indicate an error and return a file descriptor. 170701923309SHanna Czenczek 170801923309SHanna Czenczek Using this function requires prior negotiation of the 170901923309SHanna Czenczek ``VHOST_USER_PROTOCOL_F_DEVICE_STATE`` feature. 171001923309SHanna Czenczek 171101923309SHanna Czenczek``VHOST_USER_CHECK_DEVICE_STATE`` 171201923309SHanna Czenczek :id: 43 171301923309SHanna Czenczek :equivalent ioctl: N/A 171401923309SHanna Czenczek :request payload: N/A 171501923309SHanna Czenczek :reply payload: ``u64`` 171601923309SHanna Czenczek 171701923309SHanna Czenczek After transferring the back-end’s internal state during migration (see 171801923309SHanna Czenczek the :ref:`Migrating back-end state <migrating_backend_state>` 171901923309SHanna Czenczek section), check whether the back-end was able to successfully fully 172001923309SHanna Czenczek process the state. 172101923309SHanna Czenczek 172201923309SHanna Czenczek The value returned indicates success or error; 0 is success, any 172301923309SHanna Czenczek non-zero value is an error. 172401923309SHanna Czenczek 172501923309SHanna Czenczek Using this function requires prior negotiation of the 172601923309SHanna Czenczek ``VHOST_USER_PROTOCOL_F_DEVICE_STATE`` feature. 172701923309SHanna Czenczek 1728bd59f2a1SPaolo BonziniBack-end message types 1729bd59f2a1SPaolo Bonzini---------------------- 1730ed1be66bSMarc-André Lureau 1731bd59f2a1SPaolo BonziniFor this type of message, the request is sent by the back-end and the reply 1732bd59f2a1SPaolo Bonziniis sent by the front-end. 173343725d48SPaolo Bonzini 1734e9ca9f33SMaxime Coquelin``VHOST_USER_BACKEND_IOTLB_MSG`` (previous name ``VHOST_USER_SLAVE_IOTLB_MSG``) 1735ed1be66bSMarc-André Lureau :id: 1 1736ed1be66bSMarc-André Lureau :equivalent ioctl: N/A (equivalent to ``VHOST_IOTLB_MSG`` message type) 173743725d48SPaolo Bonzini :request payload: ``struct vhost_iotlb_msg`` 173843725d48SPaolo Bonzini :reply payload: N/A 1739ed1be66bSMarc-André Lureau 1740ed1be66bSMarc-André Lureau Send IOTLB messages with ``struct vhost_iotlb_msg`` as payload. 1741bd59f2a1SPaolo Bonzini The back-end sends such requests to notify of an IOTLB miss, or an IOTLB 1742ed1be66bSMarc-André Lureau access failure. If ``VHOST_USER_PROTOCOL_F_REPLY_ACK`` is 1743bd59f2a1SPaolo Bonzini negotiated, and back-end set the ``VHOST_USER_NEED_REPLY`` flag, the front-end 1744ed1be66bSMarc-André Lureau must respond with zero when operation is successfully completed, or 1745ed1be66bSMarc-André Lureau non-zero otherwise. This request should be send only when 1746ed1be66bSMarc-André Lureau ``VIRTIO_F_IOMMU_PLATFORM`` feature has been successfully 1747ed1be66bSMarc-André Lureau negotiated. 1748ed1be66bSMarc-André Lureau 1749e9ca9f33SMaxime Coquelin``VHOST_USER_BACKEND_CONFIG_CHANGE_MSG`` (previous name ``VHOST_USER_SLAVE_CONFIG_CHANGE_MSG``) 1750ed1be66bSMarc-André Lureau :id: 2 1751ed1be66bSMarc-André Lureau :equivalent ioctl: N/A 175243725d48SPaolo Bonzini :request payload: N/A 175343725d48SPaolo Bonzini :reply payload: N/A 1754ed1be66bSMarc-André Lureau 1755ed1be66bSMarc-André Lureau When ``VHOST_USER_PROTOCOL_F_CONFIG`` is negotiated, vhost-user 1756bd59f2a1SPaolo Bonzini back-end sends such messages to notify that the virtio device's 1757ed1be66bSMarc-André Lureau configuration space has changed, for those host devices which can 1758ed1be66bSMarc-André Lureau support such feature, host driver can send ``VHOST_USER_GET_CONFIG`` 1759bd59f2a1SPaolo Bonzini message to the back-end to get the latest content. If 1760bd59f2a1SPaolo Bonzini ``VHOST_USER_PROTOCOL_F_REPLY_ACK`` is negotiated, and the back-end sets the 1761bd59f2a1SPaolo Bonzini ``VHOST_USER_NEED_REPLY`` flag, the front-end must respond with zero when 1762ed1be66bSMarc-André Lureau operation is successfully completed, or non-zero otherwise. 1763ed1be66bSMarc-André Lureau 1764e9ca9f33SMaxime Coquelin``VHOST_USER_BACKEND_VRING_HOST_NOTIFIER_MSG`` (previous name ``VHOST_USER_SLAVE_VRING_HOST_NOTIFIER_MSG``) 1765ed1be66bSMarc-André Lureau :id: 3 1766ed1be66bSMarc-André Lureau :equivalent ioctl: N/A 176743725d48SPaolo Bonzini :request payload: vring area description 176843725d48SPaolo Bonzini :reply payload: N/A 1769ed1be66bSMarc-André Lureau 1770ed1be66bSMarc-André Lureau Sets host notifier for a specified queue. The queue index is 1771ed1be66bSMarc-André Lureau contained in the ``u64`` field of the vring area description. The 1772ed1be66bSMarc-André Lureau host notifier is described by the file descriptor (typically it's a 1773ed1be66bSMarc-André Lureau VFIO device fd) which is passed as ancillary data and the size 1774ed1be66bSMarc-André Lureau (which is mmap size and should be the same as host page size) and 1775ed1be66bSMarc-André Lureau offset (which is mmap offset) carried in the vring area 1776ed1be66bSMarc-André Lureau description. QEMU can mmap the file descriptor based on the size and 1777ed1be66bSMarc-André Lureau offset to get a memory range. Registering a host notifier means 1778ed1be66bSMarc-André Lureau mapping this memory range to the VM as the specified queue's notify 1779bd59f2a1SPaolo Bonzini MMIO region. The back-end sends this request to tell QEMU to de-register 1780ed1be66bSMarc-André Lureau the existing notifier if any and register the new notifier if the 1781ed1be66bSMarc-André Lureau request is sent with a file descriptor. 1782ed1be66bSMarc-André Lureau 1783ed1be66bSMarc-André Lureau This request should be sent only when 1784ed1be66bSMarc-André Lureau ``VHOST_USER_PROTOCOL_F_HOST_NOTIFIER`` protocol feature has been 1785ed1be66bSMarc-André Lureau successfully negotiated. 1786ed1be66bSMarc-André Lureau 1787e9ca9f33SMaxime Coquelin``VHOST_USER_BACKEND_VRING_CALL`` (previous name ``VHOST_USER_SLAVE_VRING_CALL``) 17883348e7e3SJohannes Berg :id: 4 17893348e7e3SJohannes Berg :equivalent ioctl: N/A 179043725d48SPaolo Bonzini :request payload: vring state description 179143725d48SPaolo Bonzini :reply payload: N/A 17923348e7e3SJohannes Berg 17933348e7e3SJohannes Berg When the ``VHOST_USER_PROTOCOL_F_INBAND_NOTIFICATIONS`` protocol 17943348e7e3SJohannes Berg feature has been successfully negotiated, this message may be 1795bd59f2a1SPaolo Bonzini submitted by the back-end to indicate that a buffer was used from 17963348e7e3SJohannes Berg the vring instead of signalling this using the vring's call file 1797bd59f2a1SPaolo Bonzini descriptor or having the front-end relying on polling. 17983348e7e3SJohannes Berg 17993348e7e3SJohannes Berg The state.num field is currently reserved and must be set to 0. 18003348e7e3SJohannes Berg 1801e9ca9f33SMaxime Coquelin``VHOST_USER_BACKEND_VRING_ERR`` (previous name ``VHOST_USER_SLAVE_VRING_ERR``) 18023348e7e3SJohannes Berg :id: 5 18033348e7e3SJohannes Berg :equivalent ioctl: N/A 180443725d48SPaolo Bonzini :request payload: vring state description 180543725d48SPaolo Bonzini :reply payload: N/A 18063348e7e3SJohannes Berg 18073348e7e3SJohannes Berg When the ``VHOST_USER_PROTOCOL_F_INBAND_NOTIFICATIONS`` protocol 18083348e7e3SJohannes Berg feature has been successfully negotiated, this message may be 1809bd59f2a1SPaolo Bonzini submitted by the back-end to indicate that an error occurred on the 18103348e7e3SJohannes Berg specific vring, instead of signalling the error file descriptor 1811bd59f2a1SPaolo Bonzini set by the front-end via ``VHOST_USER_SET_VRING_ERR``. 18123348e7e3SJohannes Berg 18133348e7e3SJohannes Berg The state.num field is currently reserved and must be set to 0. 18143348e7e3SJohannes Berg 181516094766SAlbert Esteve``VHOST_USER_BACKEND_SHARED_OBJECT_ADD`` 181616094766SAlbert Esteve :id: 6 181716094766SAlbert Esteve :equivalent ioctl: N/A 181816094766SAlbert Esteve :request payload: ``struct VhostUserShared`` 181916094766SAlbert Esteve :reply payload: N/A 182016094766SAlbert Esteve 182116094766SAlbert Esteve When the ``VHOST_USER_PROTOCOL_F_SHARED_OBJECT`` protocol 182216094766SAlbert Esteve feature has been successfully negotiated, this message can be submitted 182316094766SAlbert Esteve by the backends to add themselves as exporters to the virtio shared lookup 182416094766SAlbert Esteve table. The back-end device gets associated with a UUID in the shared table. 182516094766SAlbert Esteve The back-end is responsible of keeping its own table with exported dma-buf fds. 182616094766SAlbert Esteve When another back-end tries to import the resource associated with the UUID, 182716094766SAlbert Esteve it will send a message to the front-end, which will act as a proxy to the 182816094766SAlbert Esteve exporter back-end. If ``VHOST_USER_PROTOCOL_F_REPLY_ACK`` is negotiated, and 182916094766SAlbert Esteve the back-end sets the ``VHOST_USER_NEED_REPLY`` flag, the front-end must 183016094766SAlbert Esteve respond with zero when operation is successfully completed, or non-zero 183116094766SAlbert Esteve otherwise. 183216094766SAlbert Esteve 183316094766SAlbert Esteve``VHOST_USER_BACKEND_SHARED_OBJECT_REMOVE`` 183416094766SAlbert Esteve :id: 7 183516094766SAlbert Esteve :equivalent ioctl: N/A 183616094766SAlbert Esteve :request payload: ``struct VhostUserShared`` 183716094766SAlbert Esteve :reply payload: N/A 183816094766SAlbert Esteve 183916094766SAlbert Esteve When the ``VHOST_USER_PROTOCOL_F_SHARED_OBJECT`` protocol 184016094766SAlbert Esteve feature has been successfully negotiated, this message can be submitted 184116094766SAlbert Esteve by the backend to remove themselves from to the virtio-dmabuf shared 184216094766SAlbert Esteve table API. The shared table will remove the back-end device associated with 184316094766SAlbert Esteve the UUID. If ``VHOST_USER_PROTOCOL_F_REPLY_ACK`` is negotiated, and the 184416094766SAlbert Esteve back-end sets the ``VHOST_USER_NEED_REPLY`` flag, the front-end must respond 184516094766SAlbert Esteve with zero when operation is successfully completed, or non-zero otherwise. 184616094766SAlbert Esteve 184716094766SAlbert Esteve``VHOST_USER_BACKEND_SHARED_OBJECT_LOOKUP`` 184816094766SAlbert Esteve :id: 8 184916094766SAlbert Esteve :equivalent ioctl: N/A 185016094766SAlbert Esteve :request payload: ``struct VhostUserShared`` 185116094766SAlbert Esteve :reply payload: dmabuf fd and ``u64`` 185216094766SAlbert Esteve 185316094766SAlbert Esteve When the ``VHOST_USER_PROTOCOL_F_SHARED_OBJECT`` protocol 185416094766SAlbert Esteve feature has been successfully negotiated, this message can be submitted 185516094766SAlbert Esteve by the backends to retrieve a given dma-buf fd from the virtio-dmabuf 185616094766SAlbert Esteve shared table given a UUID. Frontend will reply passing the fd and a zero 185716094766SAlbert Esteve when the operation is successful, or non-zero otherwise. Note that if the 185816094766SAlbert Esteve operation fails, no fd is sent to the backend. 185916094766SAlbert Esteve 1860ed1be66bSMarc-André Lureau.. _reply_ack: 1861ed1be66bSMarc-André Lureau 1862ed1be66bSMarc-André LureauVHOST_USER_PROTOCOL_F_REPLY_ACK 1863ed1be66bSMarc-André Lureau------------------------------- 1864ed1be66bSMarc-André Lureau 1865ed1be66bSMarc-André LureauThe original vhost-user specification only demands replies for certain 1866ed1be66bSMarc-André Lureaucommands. This differs from the vhost protocol implementation where 1867bd59f2a1SPaolo Bonzinicommands are sent over an ``ioctl()`` call and block until the back-end 1868ed1be66bSMarc-André Lureauhas completed. 1869ed1be66bSMarc-André Lureau 1870ed1be66bSMarc-André LureauWith this protocol extension negotiated, the sender (QEMU) can set the 1871ed1be66bSMarc-André Lureau``need_reply`` [Bit 3] flag to any command. This indicates that the 1872bd59f2a1SPaolo Bonziniback-end MUST respond with a Payload ``VhostUserMsg`` indicating success 1873ed1be66bSMarc-André Lureauor failure. The payload should be set to zero on success or non-zero 1874ed1be66bSMarc-André Lureauon failure, unless the message already has an explicit reply body. 1875ed1be66bSMarc-André Lureau 187643725d48SPaolo BonziniThe reply payload gives QEMU a deterministic indication of the result 1877ed1be66bSMarc-André Lureauof the command. Today, QEMU is expected to terminate the main vhost-user 1878ed1be66bSMarc-André Lureauloop upon receiving such errors. In future, qemu could be taught to be more 1879ed1be66bSMarc-André Lureauresilient for selective requests. 1880ed1be66bSMarc-André Lureau 1881bd59f2a1SPaolo BonziniFor the message types that already solicit a reply from the back-end, 1882ed1be66bSMarc-André Lureauthe presence of ``VHOST_USER_PROTOCOL_F_REPLY_ACK`` or need_reply bit 1883ed1be66bSMarc-André Lureaubeing set brings no behavioural change. (See the Communication_ 1884ed1be66bSMarc-André Lureausection for details.) 1885ed1be66bSMarc-André Lureau 1886ed1be66bSMarc-André Lureau.. _backend_conventions: 1887ed1be66bSMarc-André Lureau 1888ed1be66bSMarc-André LureauBackend program conventions 1889ed1be66bSMarc-André Lureau=========================== 1890ed1be66bSMarc-André Lureau 1891bd59f2a1SPaolo Bonzinivhost-user back-ends can provide various devices & services and may 1892ed1be66bSMarc-André Lureauneed to be configured manually depending on the use case. However, it 1893ed1be66bSMarc-André Lureauis a good idea to follow the conventions listed here when 1894ed1be66bSMarc-André Lureaupossible. Users, QEMU or libvirt, can then rely on some common 189576ca4b58Szhaolichangbehaviour to avoid heterogeneous configuration and management of the 1896bd59f2a1SPaolo Bonziniback-end programs and facilitate interoperability. 1897ed1be66bSMarc-André Lureau 1898bd59f2a1SPaolo BonziniEach back-end installed on a host system should come with at least one 1899ed1be66bSMarc-André LureauJSON file that conforms to the vhost-user.json schema. Each file 1900bd59f2a1SPaolo Bonziniinforms the management applications about the back-end type, and binary 1901ed1be66bSMarc-André Lureaulocation. In addition, it defines rules for management apps for 1902bd59f2a1SPaolo Bonzinipicking the highest priority back-end when multiple match the search 1903ed1be66bSMarc-André Lureaucriteria (see ``@VhostUserBackend`` documentation in the schema file). 1904ed1be66bSMarc-André Lureau 1905bd59f2a1SPaolo BonziniIf the back-end is not capable of enabling a requested feature on the 1906ed1be66bSMarc-André Lureauhost (such as 3D acceleration with virgl), or the initialization 1907bd59f2a1SPaolo Bonzinifailed, the back-end should fail to start early and exit with a status 1908ed1be66bSMarc-André Lureau!= 0. It may also print a message to stderr for further details. 1909ed1be66bSMarc-André Lureau 1910bd59f2a1SPaolo BonziniThe back-end program must not daemonize itself, but it may be 1911ed1be66bSMarc-André Lureaudaemonized by the management layer. It may also have a restricted 1912ed1be66bSMarc-André Lureauaccess to the system. 1913ed1be66bSMarc-André Lureau 1914ed1be66bSMarc-André LureauFile descriptors 0, 1 and 2 will exist, and have regular 1915ed1be66bSMarc-André Lureaustdin/stdout/stderr usage (they may have been redirected to /dev/null 1916ed1be66bSMarc-André Lureauby the management layer, or to a log handler). 1917ed1be66bSMarc-André Lureau 1918bd59f2a1SPaolo BonziniThe back-end program must end (as quickly and cleanly as possible) when 1919ed1be66bSMarc-André Lureauthe SIGTERM signal is received. Eventually, it may receive SIGKILL by 1920ed1be66bSMarc-André Lureauthe management layer after a few seconds. 1921ed1be66bSMarc-André Lureau 1922ed1be66bSMarc-André LureauThe following command line options have an expected behaviour. They 1923ed1be66bSMarc-André Lureauare mandatory, unless explicitly said differently: 1924ed1be66bSMarc-André Lureau 1925ed1be66bSMarc-André Lureau--socket-path=PATH 1926ed1be66bSMarc-André Lureau 1927ed1be66bSMarc-André Lureau This option specify the location of the vhost-user Unix domain socket. 1928ed1be66bSMarc-André Lureau It is incompatible with --fd. 1929ed1be66bSMarc-André Lureau 1930ed1be66bSMarc-André Lureau--fd=FDNUM 1931ed1be66bSMarc-André Lureau 1932bd59f2a1SPaolo Bonzini When this argument is given, the back-end program is started with the 1933ed1be66bSMarc-André Lureau vhost-user socket as file descriptor FDNUM. It is incompatible with 1934ed1be66bSMarc-André Lureau --socket-path. 1935ed1be66bSMarc-André Lureau 1936ed1be66bSMarc-André Lureau--print-capabilities 1937ed1be66bSMarc-André Lureau 1938bd59f2a1SPaolo Bonzini Output to stdout the back-end capabilities in JSON format, and then 1939ed1be66bSMarc-André Lureau exit successfully. Other options and arguments should be ignored, and 1940bd59f2a1SPaolo Bonzini the back-end program should not perform its normal function. The 1941ed1be66bSMarc-André Lureau capabilities can be reported dynamically depending on the host 1942ed1be66bSMarc-André Lureau capabilities. 1943ed1be66bSMarc-André Lureau 1944ed1be66bSMarc-André LureauThe JSON output is described in the ``vhost-user.json`` schema, by 1945ed1be66bSMarc-André Lureau```@VHostUserBackendCapabilities``. Example: 1946ed1be66bSMarc-André Lureau 1947ed1be66bSMarc-André Lureau.. code:: json 1948ed1be66bSMarc-André Lureau 1949ed1be66bSMarc-André Lureau { 1950ed1be66bSMarc-André Lureau "type": "foo", 1951ed1be66bSMarc-André Lureau "features": [ 1952ed1be66bSMarc-André Lureau "feature-a", 1953ed1be66bSMarc-André Lureau "feature-b" 1954ed1be66bSMarc-André Lureau ] 1955ed1be66bSMarc-André Lureau } 1956ed1be66bSMarc-André Lureau 1957ed1be66bSMarc-André Lureauvhost-user-input 1958ed1be66bSMarc-André Lureau---------------- 1959ed1be66bSMarc-André Lureau 1960ed1be66bSMarc-André LureauCommand line options: 1961ed1be66bSMarc-André Lureau 1962ed1be66bSMarc-André Lureau--evdev-path=PATH 1963ed1be66bSMarc-André Lureau 1964ed1be66bSMarc-André Lureau Specify the linux input device. 1965ed1be66bSMarc-André Lureau 1966ed1be66bSMarc-André Lureau (optional) 1967ed1be66bSMarc-André Lureau 1968ed1be66bSMarc-André Lureau--no-grab 1969ed1be66bSMarc-André Lureau 1970ed1be66bSMarc-André Lureau Do no request exclusive access to the input device. 1971ed1be66bSMarc-André Lureau 1972ed1be66bSMarc-André Lureau (optional) 1973ed1be66bSMarc-André Lureau 1974ed1be66bSMarc-André Lureauvhost-user-gpu 1975ed1be66bSMarc-André Lureau-------------- 1976ed1be66bSMarc-André Lureau 1977ed1be66bSMarc-André LureauCommand line options: 1978ed1be66bSMarc-André Lureau 1979ed1be66bSMarc-André Lureau--render-node=PATH 1980ed1be66bSMarc-André Lureau 1981ed1be66bSMarc-André Lureau Specify the GPU DRM render node. 1982ed1be66bSMarc-André Lureau 1983ed1be66bSMarc-André Lureau (optional) 1984ed1be66bSMarc-André Lureau 1985ed1be66bSMarc-André Lureau--virgl 1986ed1be66bSMarc-André Lureau 1987ed1be66bSMarc-André Lureau Enable virgl rendering support. 1988ed1be66bSMarc-André Lureau 1989ed1be66bSMarc-André Lureau (optional) 19906620801fSMicky Yun Chan 19916620801fSMicky Yun Chanvhost-user-blk 19926620801fSMicky Yun Chan-------------- 19936620801fSMicky Yun Chan 19946620801fSMicky Yun ChanCommand line options: 19956620801fSMicky Yun Chan 19966620801fSMicky Yun Chan--blk-file=PATH 19976620801fSMicky Yun Chan 19986620801fSMicky Yun Chan Specify block device or file path. 19996620801fSMicky Yun Chan 20006620801fSMicky Yun Chan (optional) 20016620801fSMicky Yun Chan 20026620801fSMicky Yun Chan--read-only 20036620801fSMicky Yun Chan 20046620801fSMicky Yun Chan Enable read-only. 20056620801fSMicky Yun Chan 20066620801fSMicky Yun Chan (optional) 2007