1======================= 2Virtio device migration 3======================= 4 5Copyright 2015 IBM Corp. 6 7This work is licensed under the terms of the GNU GPL, version 2 or later. See 8the COPYING file in the top-level directory. 9 10Saving and restoring the state of virtio devices is a bit of a twisty maze, 11for several reasons: 12 13- state is distributed between several parts: 14 15 - virtio core, for common fields like features, number of queues, ... 16 17 - virtio transport (pci, ccw, ...), for the different proxy devices and 18 transport specific state (msix vectors, indicators, ...) 19 20 - virtio device (net, blk, ...), for the different device types and their 21 state (mac address, request queue, ...) 22 23- most fields are saved via the stream interface; subsequently, subsections 24 have been added to make cross-version migration possible 25 26This file attempts to document the current procedure and point out some 27caveats. 28 29Save state procedure 30==================== 31 32:: 33 34 virtio core virtio transport virtio device 35 ----------- ---------------- ------------- 36 37 save() function registered 38 via VMState wrapper on 39 device class 40 virtio_save() <---------- 41 ------> save_config() 42 - save proxy device 43 - save transport-specific 44 device fields 45 - save common device 46 fields 47 - save common virtqueue 48 fields 49 ------> save_queue() 50 - save transport-specific 51 virtqueue fields 52 ------> save_device() 53 - save device-specific 54 fields 55 - save subsections 56 - device endianness, 57 if changed from 58 default endianness 59 - 64 bit features, if 60 any high feature bit 61 is set 62 - virtio-1 virtqueue 63 fields, if VERSION_1 64 is set 65 66Load state procedure 67==================== 68 69:: 70 71 virtio core virtio transport virtio device 72 ----------- ---------------- ------------- 73 74 load() function registered 75 via VMState wrapper on 76 device class 77 virtio_load() <---------- 78 ------> load_config() 79 - load proxy device 80 - load transport-specific 81 device fields 82 - load common device 83 fields 84 - load common virtqueue 85 fields 86 ------> load_queue() 87 - load transport-specific 88 virtqueue fields 89 - notify guest 90 ------> load_device() 91 - load device-specific 92 fields 93 - load subsections 94 - device endianness 95 - 64 bit features 96 - virtio-1 virtqueue 97 fields 98 - sanitize endianness 99 - sanitize features 100 - virtqueue index sanity 101 check 102 - feature-dependent setup 103 104Implications of this setup 105========================== 106 107Devices need to be careful in their state processing during load: The 108load_device() procedure is invoked by the core before subsections have 109been loaded. Any code that depends on information transmitted in subsections 110therefore has to be invoked in the device's load() function _after_ 111virtio_load() returned (like e.g. code depending on features). 112 113Any extension of the state being migrated should be done in subsections 114added to the core for compatibility reasons. If transport or device specific 115state is added, core needs to invoke a callback from the new subsection. 116