devel/migration/vfio.rst

2 VFIO device migration
5 Migration of virtual machine involves saving the state for each device that
10 Migration of VFIO devices consists of two phases: the optional pre-copy phase,
13 transferred. The iterative pre-copy phase of migration allows for the guest to
20 enabling "switchover-ack" migration capability.
21 VFIO migration uAPI defines "initial bytes" as part of its pre-copy data stream
23 before stopping the source VM. Enabling this migration capability will
26 To support migration of multiple devices that might do P2P transactions between
27 themselves, VFIO migration uAPI defines an intermediate P2P quiescent state.
33 All the devices that support P2P migration are first transitioned to the P2P
34 quiescent state and only then are they stopped or started. This makes migration
38 Thus, multiple VFIO devices migration is allowed only if all the devices
39 support P2P migration. Single VFIO device migration is allowed regardless of
40 P2P migration support.
42 A detailed description of the UAPI for VFIO device migration can be found in
48 * A ``save_setup`` function that sets up migration on the source.
68   "switchover-ack" migration capability when this capability is enabled.
79 * ``cleanup`` functions for both save and load that perform any migration
83 The VFIO migration code uses a VM state change handler to change the VFIO
87 Similarly, a migration state change handler is used to trigger a transition of
88 the VFIO device state when certain changes of the migration state occur. For
90 migration failed or was canceled.
137 phase of migration. In that case, the unmap ioctl returns any dirty pages in
141 mapped ranges. If device dirty tracking is enabled with vIOMMU, live migration
144 Flow of state changes during Live migration
147 Below is the state change flow during live migration for a VFIO device that
148 supports both precopy and P2P migration. The flow for devices that don't
151 The values in the parentheses represent the VM state, the migration state, and
154 Live migration save path
163             Migration thread then calls each device's .save_setup()
172        On migration completion, the vCPUs and the VFIO device are stopped
188 Live migration resume path
193              Incoming migration calls .load_setup() for each device
208 Postcopy migration is currently not supported for VFIO devices.