devel/migration/vfio.rst

5 Migration of virtual machine involves saving the state for each device that
12 accommodate VFIO devices that have a large amount of data that needs to be
22 and recommends that the initial bytes are sent and loaded in the destination
24 guarantee that and thus, can potentially reduce downtime even further.
26 To support migration of multiple devices that might do P2P transactions between
33 All the devices that support P2P migration are first transitioned to the P2P
48 * A ``save_setup`` function that sets up migration on the source.
50 * A ``load_setup`` function that sets the VFIO device on the destination in
53 * A ``state_pending_estimate`` function that reports an estimate of the
54   remaining pre-copy data that the vendor driver has yet to save for the VFIO
57 * A ``state_pending_exact`` function that reads pending_bytes from the vendor
58   driver, which indicates the amount of data that the vendor driver has yet to
61 * An ``is_active_iterate`` function that indicates ``save_live_iterate`` is
64 * A ``save_live_iterate`` function that reads the VFIO device's data from the
67 * A ``switchover_ack_needed`` function that checks if the VFIO device uses
70 * A ``switchover_start`` function that in the multifd mode starts a thread that
78 * A ``save_complete`` function that sets the VFIO device in _STOP_COPY
80   vendor driver indicates that no data remains.  In the multifd mode it
83 * A ``save_complete_precopy_thread`` function that in the multifd mode
87   driver indicates that no data remains.
88   After that, it saves the device config space and queues it for multifd
92 * A ``load_state`` function that loads the config section and the data
93   sections that are generated by the save functions above.
95 * A ``load_state_buffer`` function that loads the device state and the device
96   config that arrived via multifd channels.
99 * ``cleanup`` functions for both save and load that perform any migration
145 dirty pages continuously, then it understands that even in stop-and-copy phase,
157 phase of migration. In that case, the unmap ioctl returns any dirty pages in
158 that range and QEMU reports corresponding guest physical pages dirty. During
167 Below is the state change flow during live migration for a VFIO device that
168 supports both precopy and P2P migration. The flow for devices that don't
169 support it is similar, except that the relevant states for precopy and P2P are
219      For each device, .load_state() is called for that device section data
248 AUTO, which means that VFIO device state transfer via multifd channels is
249 attempted in configurations that otherwise support it.
253 This means that a malicious QEMU source could theoretically cause the target
264 Some host platforms (like ARM64) require that VFIO device config is loaded only
268 that actually require it.