1..
2    Copyright (C) 2017 Red Hat Inc.
3
4    This work is licensed under the terms of the GNU GPL, version 2 or
5    later.  See the COPYING file in the top-level directory.
6
7.. _Live Block Operations:
8
9============================
10Live Block Device Operations
11============================
12
13QEMU Block Layer currently (as of QEMU 2.9) supports four major kinds of
14live block device jobs -- stream, commit, mirror, and backup.  These can
15be used to manipulate disk image chains to accomplish certain tasks,
16namely: live copy data from backing files into overlays; shorten long
17disk image chains by merging data from overlays into backing files; live
18synchronize data from a disk image chain (including current active disk)
19to another target image; and point-in-time (and incremental) backups of
20a block device.  Below is a description of the said block (QMP)
21primitives, and some (non-exhaustive list of) examples to illustrate
22their use.
23
24.. note::
25    The file ``qapi/block-core.json`` in the QEMU source tree has the
26    canonical QEMU API (QAPI) schema documentation for the QMP
27    primitives discussed here.
28
29.. todo (kashyapc):: Remove the ".. contents::" directive when Sphinx is
30                     integrated.
31
32.. contents::
33
34Disk image backing chain notation
35---------------------------------
36
37A simple disk image chain.  (This can be created live using QMP
38``blockdev-snapshot-sync``, or offline via ``qemu-img``)::
39
40                   (Live QEMU)
41                        |
42                        .
43                        V
44
45            [A] <----- [B]
46
47    (backing file)    (overlay)
48
49The arrow can be read as: Image [A] is the backing file of disk image
50[B].  And live QEMU is currently writing to image [B], consequently, it
51is also referred to as the "active layer".
52
53There are two kinds of terminology that are common when referring to
54files in a disk image backing chain:
55
56(1) Directional: 'base' and 'top'.  Given the simple disk image chain
57    above, image [A] can be referred to as 'base', and image [B] as
58    'top'.  (This terminology can be seen in the QAPI schema file,
59    block-core.json.)
60
61(2) Relational: 'backing file' and 'overlay'.  Again, taking the same
62    simple disk image chain from the above, disk image [A] is referred
63    to as the backing file, and image [B] as overlay.
64
65   Throughout this document, we will use the relational terminology.
66
67.. important::
68    The overlay files can generally be any format that supports a
69    backing file, although QCOW2 is the preferred format and the one
70    used in this document.
71
72
73Brief overview of live block QMP primitives
74-------------------------------------------
75
76The following are the four different kinds of live block operations that
77QEMU block layer supports.
78
79(1) ``block-stream``: Live copy of data from backing files into overlay
80    files.
81
82    .. note:: Once the 'stream' operation has finished, three things to
83              note:
84
85                (a) QEMU rewrites the backing chain to remove
86                    reference to the now-streamed and redundant backing
87                    file;
88
89                (b) the streamed file *itself* won't be removed by QEMU,
90                    and must be explicitly discarded by the user;
91
92                (c) the streamed file remains valid -- i.e. further
93                    overlays can be created based on it.  Refer the
94                    ``block-stream`` section further below for more
95                    details.
96
97(2) ``block-commit``: Live merge of data from overlay files into backing
98    files (with the optional goal of removing the overlay file from the
99    chain).  Since QEMU 2.0, this includes "active ``block-commit``"
100    (i.e. merge the current active layer into the base image).
101
102    .. note:: Once the 'commit' operation has finished, there are three
103              things to note here as well:
104
105                (a) QEMU rewrites the backing chain to remove reference
106                    to now-redundant overlay images that have been
107                    committed into a backing file;
108
109                (b) the committed file *itself* won't be removed by QEMU
110                    -- it ought to be manually removed;
111
112                (c) however, unlike in the case of ``block-stream``, the
113                    intermediate images will be rendered invalid -- i.e.
114                    no more further overlays can be created based on
115                    them.  Refer the ``block-commit`` section further
116                    below for more details.
117
118(3) ``drive-mirror`` (and ``blockdev-mirror``): Synchronize a running
119    disk to another image.
120
121(4) ``blockdev-backup`` (and the deprecated ``drive-backup``):
122    Point-in-time (live) copy of a block device to a destination.
123
124
125.. _`Interacting with a QEMU instance`:
126
127Interacting with a QEMU instance
128--------------------------------
129
130To show some example invocations of command-line, we will use the
131following invocation of QEMU, with a QMP server running over UNIX
132socket:
133
134.. parsed-literal::
135
136  $ |qemu_system| -display none -no-user-config -nodefaults \\
137    -m 512 -blockdev \\
138    node-name=node-A,driver=qcow2,file.driver=file,file.node-name=file,file.filename=./a.qcow2 \\
139    -device virtio-blk,drive=node-A,id=virtio0 \\
140    -monitor stdio -qmp unix:/tmp/qmp-sock,server=on,wait=off
141
142The ``-blockdev`` command-line option, used above, is available from
143QEMU 2.9 onwards.  In the above invocation, notice the ``node-name``
144parameter that is used to refer to the disk image a.qcow2 ('node-A') --
145this is a cleaner way to refer to a disk image (as opposed to referring
146to it by spelling out file paths).  So, we will continue to designate a
147``node-name`` to each further disk image created (either via
148``blockdev-snapshot-sync``, or ``blockdev-add``) as part of the disk
149image chain, and continue to refer to the disks using their
150``node-name`` (where possible, because ``block-commit`` does not yet, as
151of QEMU 2.9, accept ``node-name`` parameter) when performing various
152block operations.
153
154To interact with the QEMU instance launched above, we will use the
155``qmp-shell`` utility (located at: ``qemu/scripts/qmp``, as part of the
156QEMU source directory), which takes key-value pairs for QMP commands.
157Invoke it as below (which will also print out the complete raw JSON
158syntax for reference -- examples in the following sections)::
159
160    $ ./qmp-shell -v -p /tmp/qmp-sock
161    (QEMU)
162
163.. note::
164    In the event we have to repeat a certain QMP command, we will: for
165    the first occurrence of it, show the ``qmp-shell`` invocation, *and*
166    the corresponding raw JSON QMP syntax; but for subsequent
167    invocations, present just the ``qmp-shell`` syntax, and omit the
168    equivalent JSON output.
169
170
171Example disk image chain
172------------------------
173
174We will use the below disk image chain (and occasionally spelling it
175out where appropriate) when discussing various primitives::
176
177    [A] <-- [B] <-- [C] <-- [D]
178
179Where [A] is the original base image; [B] and [C] are intermediate
180overlay images; image [D] is the active layer -- i.e. live QEMU is
181writing to it.  (The rule of thumb is: live QEMU will always be pointing
182to the rightmost image in a disk image chain.)
183
184The above image chain can be created by invoking
185``blockdev-snapshot-sync`` commands as following (which shows the
186creation of overlay image [B]) using the ``qmp-shell`` (our invocation
187also prints the raw JSON invocation of it)::
188
189    (QEMU) blockdev-snapshot-sync node-name=node-A snapshot-file=b.qcow2 snapshot-node-name=node-B format=qcow2
190    {
191        "execute": "blockdev-snapshot-sync",
192        "arguments": {
193            "node-name": "node-A",
194            "snapshot-file": "b.qcow2",
195            "format": "qcow2",
196            "snapshot-node-name": "node-B"
197        }
198    }
199
200Here, "node-A" is the name QEMU internally uses to refer to the base
201image [A] -- it is the backing file, based on which the overlay image,
202[B], is created.
203
204To create the rest of the overlay images, [C], and [D] (omitting the raw
205JSON output for brevity)::
206
207    (QEMU) blockdev-snapshot-sync node-name=node-B snapshot-file=c.qcow2 snapshot-node-name=node-C format=qcow2
208    (QEMU) blockdev-snapshot-sync node-name=node-C snapshot-file=d.qcow2 snapshot-node-name=node-D format=qcow2
209
210
211A note on points-in-time vs file names
212--------------------------------------
213
214In our disk image chain::
215
216    [A] <-- [B] <-- [C] <-- [D]
217
218We have *three* points in time and an active layer:
219
220- Point 1: Guest state when [B] was created is contained in file [A]
221- Point 2: Guest state when [C] was created is contained in [A] + [B]
222- Point 3: Guest state when [D] was created is contained in
223  [A] + [B] + [C]
224- Active layer: Current guest state is contained in [A] + [B] + [C] +
225  [D]
226
227Therefore, be aware with naming choices:
228
229- Naming a file after the time it is created is misleading -- the
230  guest data for that point in time is *not* contained in that file
231  (as explained earlier)
232- Rather, think of files as a *delta* from the backing file
233
234
235Live block streaming --- ``block-stream``
236-----------------------------------------
237
238The ``block-stream`` command allows you to do live copy data from backing
239files into overlay images.
240
241Given our original example disk image chain from earlier::
242
243    [A] <-- [B] <-- [C] <-- [D]
244
245The disk image chain can be shortened in one of the following different
246ways (not an exhaustive list).
247
248.. _`Case-1`:
249
250(1) Merge everything into the active layer: I.e. copy all contents from
251    the base image, [A], and overlay images, [B] and [C], into [D],
252    *while* the guest is running.  The resulting chain will be a
253    standalone image, [D] -- with contents from [A], [B] and [C] merged
254    into it (where live QEMU writes go to)::
255
256        [D]
257
258.. _`Case-2`:
259
260(2) Taking the same example disk image chain mentioned earlier, merge
261    only images [B] and [C] into [D], the active layer.  The result will
262    be contents of images [B] and [C] will be copied into [D], and the
263    backing file pointer of image [D] will be adjusted to point to image
264    [A].  The resulting chain will be::
265
266        [A] <-- [D]
267
268.. _`Case-3`:
269
270(3) Intermediate streaming (available since QEMU 2.8): Starting afresh
271    with the original example disk image chain, with a total of four
272    images, it is possible to copy contents from image [B] into image
273    [C].  Once the copy is finished, image [B] can now be (optionally)
274    discarded; and the backing file pointer of image [C] will be
275    adjusted to point to [A].  I.e. after performing "intermediate
276    streaming" of [B] into [C], the resulting image chain will be (where
277    live QEMU is writing to [D])::
278
279        [A] <-- [C] <-- [D]
280
281
282QMP invocation for ``block-stream``
283~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
284
285For `Case-1`_, to merge contents of all the backing files into the
286active layer, where 'node-D' is the current active image (by default
287``block-stream`` will flatten the entire chain); ``qmp-shell`` (and its
288corresponding JSON output)::
289
290    (QEMU) block-stream device=node-D job-id=job0
291    {
292        "execute": "block-stream",
293        "arguments": {
294            "device": "node-D",
295            "job-id": "job0"
296        }
297    }
298
299For `Case-2`_, merge contents of the images [B] and [C] into [D], where
300image [D] ends up referring to image [A] as its backing file::
301
302    (QEMU) block-stream device=node-D base-node=node-A job-id=job0
303
304And for `Case-3`_, of "intermediate" streaming", merge contents of
305images [B] into [C], where [C] ends up referring to [A] as its backing
306image::
307
308    (QEMU) block-stream device=node-C base-node=node-A job-id=job0
309
310Progress of a ``block-stream`` operation can be monitored via the QMP
311command::
312
313    (QEMU) query-block-jobs
314    {
315        "execute": "query-block-jobs",
316        "arguments": {}
317    }
318
319
320Once the ``block-stream`` operation has completed, QEMU will emit an
321event, ``BLOCK_JOB_COMPLETED``.  The intermediate overlays remain valid,
322and can now be (optionally) discarded, or retained to create further
323overlays based on them.  Finally, the ``block-stream`` jobs can be
324restarted at anytime.
325
326
327Live block commit --- ``block-commit``
328--------------------------------------
329
330The ``block-commit`` command lets you merge live data from overlay
331images into backing file(s).  Since QEMU 2.0, this includes "live active
332commit" (i.e. it is possible to merge the "active layer", the right-most
333image in a disk image chain where live QEMU will be writing to, into the
334base image).  This is analogous to ``block-stream``, but in the opposite
335direction.
336
337Again, starting afresh with our example disk image chain, where live
338QEMU is writing to the right-most image in the chain, [D]::
339
340    [A] <-- [B] <-- [C] <-- [D]
341
342The disk image chain can be shortened in one of the following ways:
343
344.. _`block-commit_Case-1`:
345
346(1) Commit content from only image [B] into image [A].  The resulting
347    chain is the following, where image [C] is adjusted to point at [A]
348    as its new backing file::
349
350        [A] <-- [C] <-- [D]
351
352(2) Commit content from images [B] and [C] into image [A].  The
353    resulting chain, where image [D] is adjusted to point to image [A]
354    as its new backing file::
355
356        [A] <-- [D]
357
358.. _`block-commit_Case-3`:
359
360(3) Commit content from images [B], [C], and the active layer [D] into
361    image [A].  The resulting chain (in this case, a consolidated single
362    image)::
363
364        [A]
365
366(4) Commit content from image only image [C] into image [B].  The
367    resulting chain::
368
369	[A] <-- [B] <-- [D]
370
371(5) Commit content from image [C] and the active layer [D] into image
372    [B].  The resulting chain::
373
374	[A] <-- [B]
375
376
377QMP invocation for ``block-commit``
378~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
379
380For :ref:`Case-1 <block-commit_Case-1>`, to merge contents only from
381image [B] into image [A], the invocation is as follows::
382
383    (QEMU) block-commit device=node-D base=a.qcow2 top=b.qcow2 job-id=job0
384    {
385        "execute": "block-commit",
386        "arguments": {
387            "device": "node-D",
388            "job-id": "job0",
389            "top": "b.qcow2",
390            "base": "a.qcow2"
391        }
392    }
393
394Once the above ``block-commit`` operation has completed, a
395``BLOCK_JOB_COMPLETED`` event will be issued, and no further action is
396required.  As the end result, the backing file of image [C] is adjusted
397to point to image [A], and the original 4-image chain will end up being
398transformed to::
399
400    [A] <-- [C] <-- [D]
401
402.. note::
403    The intermediate image [B] is invalid (as in: no more further
404    overlays based on it can be created).
405
406    Reasoning: An intermediate image after a 'stream' operation still
407    represents that old point-in-time, and may be valid in that context.
408    However, an intermediate image after a 'commit' operation no longer
409    represents any point-in-time, and is invalid in any context.
410
411
412However, :ref:`Case-3 <block-commit_Case-3>` (also called: "active
413``block-commit``") is a *two-phase* operation: In the first phase, the
414content from the active overlay, along with the intermediate overlays,
415is copied into the backing file (also called the base image).  In the
416second phase, adjust the said backing file as the current active image
417-- possible via issuing the command ``block-job-complete``.  Optionally,
418the ``block-commit`` operation can be cancelled by issuing the command
419``block-job-cancel``, but be careful when doing this.
420
421Once the ``block-commit`` operation has completed, the event
422``BLOCK_JOB_READY`` will be emitted, signalling that the synchronization
423has finished.  Now the job can be gracefully completed by issuing the
424command ``block-job-complete`` -- until such a command is issued, the
425'commit' operation remains active.
426
427The following is the flow for :ref:`Case-3 <block-commit_Case-3>` to
428convert a disk image chain such as this::
429
430    [A] <-- [B] <-- [C] <-- [D]
431
432Into::
433
434    [A]
435
436Where content from all the subsequent overlays, [B], and [C], including
437the active layer, [D], is committed back to [A] -- which is where live
438QEMU is performing all its current writes).
439
440Start the "active ``block-commit``" operation::
441
442    (QEMU) block-commit device=node-D base=a.qcow2 top=d.qcow2 job-id=job0
443    {
444        "execute": "block-commit",
445        "arguments": {
446            "device": "node-D",
447            "job-id": "job0",
448            "top": "d.qcow2",
449            "base": "a.qcow2"
450        }
451    }
452
453
454Once the synchronization has completed, the event ``BLOCK_JOB_READY`` will
455be emitted.
456
457Then, optionally query for the status of the active block operations.
458We can see the 'commit' job is now ready to be completed, as indicated
459by the line *"ready": true*::
460
461    (QEMU) query-block-jobs
462    {
463        "execute": "query-block-jobs",
464        "arguments": {}
465    }
466    {
467        "return": [
468            {
469                "busy": false,
470                "type": "commit",
471                "len": 1376256,
472                "paused": false,
473                "ready": true,
474                "io-status": "ok",
475                "offset": 1376256,
476                "device": "job0",
477                "speed": 0
478            }
479        ]
480    }
481
482Gracefully complete the 'commit' block device job::
483
484    (QEMU) block-job-complete device=job0
485    {
486        "execute": "block-job-complete",
487        "arguments": {
488            "device": "job0"
489        }
490    }
491    {
492        "return": {}
493    }
494
495Finally, once the above job is completed, an event
496``BLOCK_JOB_COMPLETED`` will be emitted.
497
498.. note::
499    The invocation for rest of the cases (2, 4, and 5), discussed in the
500    previous section, is omitted for brevity.
501
502
503Live disk synchronization --- ``drive-mirror`` and ``blockdev-mirror``
504----------------------------------------------------------------------
505
506Synchronize a running disk image chain (all or part of it) to a target
507image.
508
509Again, given our familiar disk image chain::
510
511    [A] <-- [B] <-- [C] <-- [D]
512
513The ``drive-mirror`` (and its newer equivalent ``blockdev-mirror``)
514allows you to copy data from the entire chain into a single target image
515(which can be located on a different host), [E].
516
517.. note::
518
519    When you cancel an in-progress 'mirror' job *before* the source and
520    target are synchronized, ``block-job-cancel`` will emit the event
521    ``BLOCK_JOB_CANCELLED``.  However, note that if you cancel a
522    'mirror' job *after* it has indicated (via the event
523    ``BLOCK_JOB_READY``) that the source and target have reached
524    synchronization, then the event emitted by ``block-job-cancel``
525    changes to ``BLOCK_JOB_COMPLETED``.
526
527    Besides the 'mirror' job, the "active ``block-commit``" is the only
528    other block device job that emits the event ``BLOCK_JOB_READY``.
529    The rest of the block device jobs ('stream', "non-active
530    ``block-commit``", and 'backup') end automatically.
531
532So there are two possible actions to take, after a 'mirror' job has
533emitted the event ``BLOCK_JOB_READY``, indicating that the source and
534target have reached synchronization:
535
536(1) Issuing the command ``block-job-cancel`` (after it emits the event
537    ``BLOCK_JOB_COMPLETED``) will create a point-in-time (which is at
538    the time of *triggering* the cancel command) copy of the entire disk
539    image chain (or only the top-most image, depending on the ``sync``
540    mode), contained in the target image [E]. One use case for this is
541    live VM migration with non-shared storage.
542
543(2) Issuing the command ``block-job-complete`` (after it emits the event
544    ``BLOCK_JOB_COMPLETED``) will adjust the guest device (i.e. live
545    QEMU) to point to the target image, [E], causing all the new writes
546    from this point on to happen there.
547
548About synchronization modes: The synchronization mode determines
549*which* part of the disk image chain will be copied to the target.
550Currently, there are four different kinds:
551
552(1) ``full`` -- Synchronize the content of entire disk image chain to
553    the target
554
555(2) ``top`` -- Synchronize only the contents of the top-most disk image
556    in the chain to the target
557
558(3) ``none`` -- Synchronize only the new writes from this point on.
559
560    .. note:: In the case of ``blockdev-backup`` (or deprecated
561              ``drive-backup``), the behavior of ``none``
562              synchronization mode is different.  Normally, a
563              ``backup`` job consists of two parts: Anything that is
564              overwritten by the guest is first copied out to the
565              backup, and in the background the whole image is copied
566              from start to end. With ``sync=none``, it's only the
567              first part.
568
569(4) ``incremental`` -- Synchronize content that is described by the
570    dirty bitmap
571
572.. note::
573    Refer to the :doc:`bitmaps` document in the QEMU source
574    tree to learn about the detailed workings of the ``incremental``
575    synchronization mode.
576
577
578QMP invocation for ``drive-mirror``
579~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
580
581To copy the contents of the entire disk image chain, from [A] all the
582way to [D], to a new target (``drive-mirror`` will create the destination
583file, if it doesn't already exist), call it [E]::
584
585    (QEMU) drive-mirror device=node-D target=e.qcow2 sync=full job-id=job0
586    {
587        "execute": "drive-mirror",
588        "arguments": {
589            "device": "node-D",
590            "job-id": "job0",
591            "target": "e.qcow2",
592            "sync": "full"
593        }
594    }
595
596The ``"sync": "full"``, from the above, means: copy the *entire* chain
597to the destination.
598
599Following the above, querying for active block jobs will show that a
600'mirror' job is "ready" to be completed (and QEMU will also emit an
601event, ``BLOCK_JOB_READY``)::
602
603    (QEMU) query-block-jobs
604    {
605        "execute": "query-block-jobs",
606        "arguments": {}
607    }
608    {
609        "return": [
610            {
611                "busy": false,
612                "type": "mirror",
613                "len": 21757952,
614                "paused": false,
615                "ready": true,
616                "io-status": "ok",
617                "offset": 21757952,
618                "device": "job0",
619                "speed": 0
620            }
621        ]
622    }
623
624And, as noted in the previous section, there are two possible actions
625at this point:
626
627(a) Create a point-in-time snapshot by ending the synchronization.  The
628    point-in-time is at the time of *ending* the sync.  (The result of
629    the following being: the target image, [E], will be populated with
630    content from the entire chain, [A] to [D])::
631
632        (QEMU) block-job-cancel device=job0
633        {
634            "execute": "block-job-cancel",
635            "arguments": {
636                "device": "job0"
637            }
638        }
639
640(b) Or, complete the operation and pivot the live QEMU to the target
641    copy::
642
643        (QEMU) block-job-complete device=job0
644
645In either of the above cases, if you once again run the
646``query-block-jobs`` command, there should not be any active block
647operation.
648
649Comparing 'commit' and 'mirror': In both then cases, the overlay images
650can be discarded.  However, with 'commit', the *existing* base image
651will be modified (by updating it with contents from overlays); while in
652the case of 'mirror', a *new* target image is populated with the data
653from the disk image chain.
654
655
656QMP invocation for live storage migration with ``drive-mirror`` + NBD
657~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
658
659Live storage migration (without shared storage setup) is one of the most
660common use-cases that takes advantage of the ``drive-mirror`` primitive
661and QEMU's built-in Network Block Device (NBD) server.  Here's a quick
662walk-through of this setup.
663
664Given the disk image chain::
665
666    [A] <-- [B] <-- [C] <-- [D]
667
668Instead of copying content from the entire chain, synchronize *only* the
669contents of the *top*-most disk image (i.e. the active layer), [D], to a
670target, say, [TargetDisk].
671
672.. important::
673    The destination host must already have the contents of the backing
674    chain, involving images [A], [B], and [C], visible via other means
675    -- whether by ``cp``, ``rsync``, or by some storage array-specific
676    command.)
677
678Sometimes, this is also referred to as "shallow copy" -- because only
679the "active layer", and not the rest of the image chain, is copied to
680the destination.
681
682.. note::
683    In this example, for the sake of simplicity, we'll be using the same
684    ``localhost`` as both source and destination.
685
686As noted earlier, on the destination host the contents of the backing
687chain -- from images [A] to [C] -- are already expected to exist in some
688form (e.g. in a file called, ``Contents-of-A-B-C.qcow2``).  Now, on the
689destination host, let's create a target overlay image (with the image
690``Contents-of-A-B-C.qcow2`` as its backing file), to which the contents
691of image [D] (from the source QEMU) will be mirrored to::
692
693    $ qemu-img create -f qcow2 -b ./Contents-of-A-B-C.qcow2 \
694        -F qcow2 ./target-disk.qcow2
695
696And start the destination QEMU (we already have the source QEMU running
697-- discussed in the section: `Interacting with a QEMU instance`_)
698instance, with the following invocation.  (As noted earlier, for
699simplicity's sake, the destination QEMU is started on the same host, but
700it could be located elsewhere):
701
702.. parsed-literal::
703
704  $ |qemu_system| -display none -no-user-config -nodefaults \\
705    -m 512 -blockdev \\
706    node-name=node-TargetDisk,driver=qcow2,file.driver=file,file.node-name=file,file.filename=./target-disk.qcow2 \\
707    -device virtio-blk,drive=node-TargetDisk,id=virtio0 \\
708    -S -monitor stdio -qmp unix:./qmp-sock2,server=on,wait=off \\
709    -incoming tcp:localhost:6666
710
711Given the disk image chain on source QEMU::
712
713    [A] <-- [B] <-- [C] <-- [D]
714
715On the destination host, it is expected that the contents of the chain
716``[A] <-- [B] <-- [C]`` are *already* present, and therefore copy *only*
717the content of image [D].
718
719(1) [On *destination* QEMU] As part of the first step, start the
720    built-in NBD server on a given host (local host, represented by
721    ``::``)and port::
722
723        (QEMU) nbd-server-start addr={"type":"inet","data":{"host":"::","port":"49153"}}
724        {
725            "execute": "nbd-server-start",
726            "arguments": {
727                "addr": {
728                    "data": {
729                        "host": "::",
730                        "port": "49153"
731                    },
732                    "type": "inet"
733                }
734            }
735        }
736
737(2) [On *destination* QEMU] And export the destination disk image using
738    QEMU's built-in NBD server::
739
740        (QEMU) nbd-server-add device=node-TargetDisk writable=true
741        {
742            "execute": "nbd-server-add",
743            "arguments": {
744                "device": "node-TargetDisk"
745            }
746        }
747
748(3) [On *source* QEMU] Then, invoke ``drive-mirror`` (NB: since we're
749    running ``drive-mirror`` with ``mode=existing`` (meaning:
750    synchronize to a pre-created file, therefore 'existing', file on the
751    target host), with the synchronization mode as 'top' (``"sync:
752    "top"``)::
753
754        (QEMU) drive-mirror device=node-D target=nbd:localhost:49153:exportname=node-TargetDisk sync=top mode=existing job-id=job0
755        {
756            "execute": "drive-mirror",
757            "arguments": {
758                "device": "node-D",
759                "mode": "existing",
760                "job-id": "job0",
761                "target": "nbd:localhost:49153:exportname=node-TargetDisk",
762                "sync": "top"
763            }
764        }
765
766(4) [On *source* QEMU] Once ``drive-mirror`` copies the entire data, and the
767    event ``BLOCK_JOB_READY`` is emitted, issue ``block-job-cancel`` to
768    gracefully end the synchronization, from source QEMU::
769
770        (QEMU) block-job-cancel device=job0
771        {
772            "execute": "block-job-cancel",
773            "arguments": {
774                "device": "job0"
775            }
776        }
777
778(5) [On *destination* QEMU] Then, stop the NBD server::
779
780        (QEMU) nbd-server-stop
781        {
782            "execute": "nbd-server-stop",
783            "arguments": {}
784        }
785
786(6) [On *destination* QEMU] Finally, resume the guest vCPUs by issuing the
787    QMP command ``cont``::
788
789        (QEMU) cont
790        {
791            "execute": "cont",
792            "arguments": {}
793        }
794
795.. note::
796    Higher-level libraries (e.g. libvirt) automate the entire above
797    process (although note that libvirt does not allow same-host
798    migrations to localhost for other reasons).
799
800
801Notes on ``blockdev-mirror``
802~~~~~~~~~~~~~~~~~~~~~~~~~~~~
803
804The ``blockdev-mirror`` command is equivalent in core functionality to
805``drive-mirror``, except that it operates at node-level in a BDS graph.
806
807Also: for ``blockdev-mirror``, the 'target' image needs to be explicitly
808created (using ``qemu-img``) and attach it to live QEMU via
809``blockdev-add``, which assigns a name to the to-be created target node.
810
811E.g. the sequence of actions to create a point-in-time backup of an
812entire disk image chain, to a target, using ``blockdev-mirror`` would be:
813
814(0) Create the QCOW2 overlays, to arrive at a backing chain of desired
815    depth
816
817(1) Create the target image (using ``qemu-img``), say, ``e.qcow2``
818
819(2) Attach the above created file (``e.qcow2``), run-time, using
820    ``blockdev-add`` to QEMU
821
822(3) Perform ``blockdev-mirror`` (use ``"sync": "full"`` to copy the
823    entire chain to the target).  And notice the event
824    ``BLOCK_JOB_READY``
825
826(4) Optionally, query for active block jobs, there should be a 'mirror'
827    job ready to be completed
828
829(5) Gracefully complete the 'mirror' block device job, and notice the
830    event ``BLOCK_JOB_COMPLETED``
831
832(6) Shutdown the guest by issuing the QMP ``quit`` command so that
833    caches are flushed
834
835(7) Then, finally, compare the contents of the disk image chain, and
836    the target copy with ``qemu-img compare``.  You should notice:
837    "Images are identical"
838
839
840QMP invocation for ``blockdev-mirror``
841~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
842
843Given the disk image chain::
844
845    [A] <-- [B] <-- [C] <-- [D]
846
847To copy the contents of the entire disk image chain, from [A] all the
848way to [D], to a new target, call it [E].  The following is the flow.
849
850Create the overlay images, [B], [C], and [D]::
851
852    (QEMU) blockdev-snapshot-sync node-name=node-A snapshot-file=b.qcow2 snapshot-node-name=node-B format=qcow2
853    (QEMU) blockdev-snapshot-sync node-name=node-B snapshot-file=c.qcow2 snapshot-node-name=node-C format=qcow2
854    (QEMU) blockdev-snapshot-sync node-name=node-C snapshot-file=d.qcow2 snapshot-node-name=node-D format=qcow2
855
856Create the target image, [E]::
857
858    $ qemu-img create -f qcow2 e.qcow2 39M
859
860Add the above created target image to QEMU, via ``blockdev-add``::
861
862    (QEMU) blockdev-add driver=qcow2 node-name=node-E file={"driver":"file","filename":"e.qcow2"}
863    {
864        "execute": "blockdev-add",
865        "arguments": {
866            "node-name": "node-E",
867            "driver": "qcow2",
868            "file": {
869                "driver": "file",
870                "filename": "e.qcow2"
871            }
872        }
873    }
874
875Perform ``blockdev-mirror``, and notice the event ``BLOCK_JOB_READY``::
876
877    (QEMU) blockdev-mirror device=node-B target=node-E sync=full job-id=job0
878    {
879        "execute": "blockdev-mirror",
880        "arguments": {
881            "device": "node-D",
882            "job-id": "job0",
883            "target": "node-E",
884            "sync": "full"
885        }
886    }
887
888Query for active block jobs, there should be a 'mirror' job ready::
889
890    (QEMU) query-block-jobs
891    {
892        "execute": "query-block-jobs",
893        "arguments": {}
894    }
895    {
896        "return": [
897            {
898                "busy": false,
899                "type": "mirror",
900                "len": 21561344,
901                "paused": false,
902                "ready": true,
903                "io-status": "ok",
904                "offset": 21561344,
905                "device": "job0",
906                "speed": 0
907            }
908        ]
909    }
910
911Gracefully complete the block device job operation, and notice the
912event ``BLOCK_JOB_COMPLETED``::
913
914    (QEMU) block-job-complete device=job0
915    {
916        "execute": "block-job-complete",
917        "arguments": {
918            "device": "job0"
919        }
920    }
921    {
922        "return": {}
923    }
924
925Shutdown the guest, by issuing the ``quit`` QMP command::
926
927    (QEMU) quit
928    {
929        "execute": "quit",
930        "arguments": {}
931    }
932
933
934Live disk backup --- ``blockdev-backup`` and the deprecated ``drive-backup``
935----------------------------------------------------------------------------
936
937The ``blockdev-backup`` (and the deprecated ``drive-backup``) allows
938you to create a point-in-time snapshot.
939
940In this case, the point-in-time is when you *start* the
941``blockdev-backup`` (or deprecated ``drive-backup``) command.
942
943
944QMP invocation for ``drive-backup``
945~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
946
947Note that ``drive-backup`` command is deprecated since QEMU 6.2 and
948will be removed in future.
949
950Yet again, starting afresh with our example disk image chain::
951
952    [A] <-- [B] <-- [C] <-- [D]
953
954To create a target image [E], with content populated from image [A] to
955[D], from the above chain, the following is the syntax.  (If the target
956image does not exist, ``drive-backup`` will create it)::
957
958    (QEMU) drive-backup device=node-D sync=full target=e.qcow2 job-id=job0
959    {
960        "execute": "drive-backup",
961        "arguments": {
962            "device": "node-D",
963            "job-id": "job0",
964            "sync": "full",
965            "target": "e.qcow2"
966        }
967    }
968
969Once the above ``drive-backup`` has completed, a ``BLOCK_JOB_COMPLETED`` event
970will be issued, indicating the live block device job operation has
971completed, and no further action is required.
972
973
974Moving from the deprecated ``drive-backup`` to newer ``blockdev-backup``
975~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
976
977``blockdev-backup`` differs from ``drive-backup`` in how you specify
978the backup target. With ``blockdev-backup`` you can't specify filename
979as a target.  Instead you use ``node-name`` of existing block node,
980which you may add by ``blockdev-add`` or ``blockdev-create`` commands.
981Correspondingly, ``blockdev-backup`` doesn't have ``mode`` and
982``format`` arguments which don't apply to an existing block node. See
983following sections for details and examples.
984
985
986Notes on ``blockdev-backup``
987~~~~~~~~~~~~~~~~~~~~~~~~~~~~
988
989The ``blockdev-backup`` command operates at node-level in a Block Driver
990State (BDS) graph.
991
992E.g. the sequence of actions to create a point-in-time backup
993of an entire disk image chain, to a target, using ``blockdev-backup``
994would be:
995
996(0) Create the QCOW2 overlays, to arrive at a backing chain of desired
997    depth
998
999(1) Create the target image (using ``qemu-img``), say, ``e.qcow2``
1000
1001(2) Attach the above created file (``e.qcow2``), run-time, using
1002    ``blockdev-add`` to QEMU
1003
1004(3) Perform ``blockdev-backup`` (use ``"sync": "full"`` to copy the
1005    entire chain to the target).  And notice the event
1006    ``BLOCK_JOB_COMPLETED``
1007
1008(4) Shutdown the guest, by issuing the QMP ``quit`` command, so that
1009    caches are flushed
1010
1011(5) Then, finally, compare the contents of the disk image chain, and
1012    the target copy with ``qemu-img compare``.  You should notice:
1013    "Images are identical"
1014
1015The following section shows an example QMP invocation for
1016``blockdev-backup``.
1017
1018QMP invocation for ``blockdev-backup``
1019~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1020
1021Given a disk image chain of depth 1 where image [B] is the active
1022overlay (live QEMU is writing to it)::
1023
1024    [A] <-- [B]
1025
1026The following is the procedure to copy the content from the entire chain
1027to a target image (say, [E]), which has the full content from [A] and
1028[B].
1029
1030Create the overlay [B]::
1031
1032    (QEMU) blockdev-snapshot-sync node-name=node-A snapshot-file=b.qcow2 snapshot-node-name=node-B format=qcow2
1033    {
1034        "execute": "blockdev-snapshot-sync",
1035        "arguments": {
1036            "node-name": "node-A",
1037            "snapshot-file": "b.qcow2",
1038            "format": "qcow2",
1039            "snapshot-node-name": "node-B"
1040        }
1041    }
1042
1043
1044Create a target image that will contain the copy::
1045
1046    $ qemu-img create -f qcow2 e.qcow2 39M
1047
1048Then add it to QEMU via ``blockdev-add``::
1049
1050    (QEMU) blockdev-add driver=qcow2 node-name=node-E file={"driver":"file","filename":"e.qcow2"}
1051    {
1052        "execute": "blockdev-add",
1053        "arguments": {
1054            "node-name": "node-E",
1055            "driver": "qcow2",
1056            "file": {
1057                "driver": "file",
1058                "filename": "e.qcow2"
1059            }
1060        }
1061    }
1062
1063Then invoke ``blockdev-backup`` to copy the contents from the entire
1064image chain, consisting of images [A] and [B] to the target image
1065'e.qcow2'::
1066
1067    (QEMU) blockdev-backup device=node-B target=node-E sync=full job-id=job0
1068    {
1069        "execute": "blockdev-backup",
1070        "arguments": {
1071            "device": "node-B",
1072            "job-id": "job0",
1073            "target": "node-E",
1074            "sync": "full"
1075        }
1076    }
1077
1078Once the above 'backup' operation has completed, the event,
1079``BLOCK_JOB_COMPLETED`` will be emitted, signalling successful
1080completion.
1081
1082Next, query for any active block device jobs (there should be none)::
1083
1084    (QEMU) query-block-jobs
1085    {
1086        "execute": "query-block-jobs",
1087        "arguments": {}
1088    }
1089
1090Shutdown the guest::
1091
1092    (QEMU) quit
1093    {
1094            "execute": "quit",
1095                "arguments": {}
1096    }
1097            "return": {}
1098    }
1099
1100.. note::
1101    The above step is really important; if forgotten, an error, "Failed
1102    to get shared "write" lock on e.qcow2", will be thrown when you do
1103    ``qemu-img compare`` to verify the integrity of the disk image
1104    with the backup content.
1105
1106
1107The end result will be the image 'e.qcow2' containing a
1108point-in-time backup of the disk image chain -- i.e. contents from
1109images [A] and [B] at the time the ``blockdev-backup`` command was
1110initiated.
1111
1112One way to confirm the backup disk image contains the identical content
1113with the disk image chain is to compare the backup and the contents of
1114the chain, you should see "Images are identical".  (NB: this is assuming
1115QEMU was launched with ``-S`` option, which will not start the CPUs at
1116guest boot up)::
1117
1118    $ qemu-img compare b.qcow2 e.qcow2
1119    Warning: Image size mismatch!
1120    Images are identical.
1121
1122NOTE: The "Warning: Image size mismatch!" is expected, as we created the
1123target image (e.qcow2) with 39M size.
1124