1.. 2 Copyright 2015 John Snow <jsnow@redhat.com> and Red Hat, Inc. 3 All rights reserved. 4 5 This file is licensed via The FreeBSD Documentation License, the full 6 text of which is included at the end of this document. 7 8==================================== 9Dirty Bitmaps and Incremental Backup 10==================================== 11 12- Dirty Bitmaps are objects that track which data needs to be backed up 13 for the next incremental backup. 14 15- Dirty bitmaps can be created at any time and attached to any node 16 (not just complete drives). 17 18.. contents:: 19 20Dirty Bitmap Names 21------------------ 22 23- A dirty bitmap's name is unique to the node, but bitmaps attached to 24 different nodes can share the same name. 25 26- Dirty bitmaps created for internal use by QEMU may be anonymous and 27 have no name, but any user-created bitmaps must have a name. There 28 can be any number of anonymous bitmaps per node. 29 30- The name of a user-created bitmap must not be empty (""). 31 32Bitmap Modes 33------------ 34 35- A bitmap can be "frozen," which means that it is currently in-use by 36 a backup operation and cannot be deleted, renamed, written to, reset, 37 etc. 38 39- The normal operating mode for a bitmap is "active." 40 41Basic QMP Usage 42--------------- 43 44Supported Commands 45~~~~~~~~~~~~~~~~~~ 46 47- ``block-dirty-bitmap-add`` 48- ``block-dirty-bitmap-remove`` 49- ``block-dirty-bitmap-clear`` 50 51Creation 52~~~~~~~~ 53 54- To create a new bitmap, enabled, on the drive with id=drive0: 55 56.. code:: json 57 58 { "execute": "block-dirty-bitmap-add", 59 "arguments": { 60 "node": "drive0", 61 "name": "bitmap0" 62 } 63 } 64 65- This bitmap will have a default granularity that matches the cluster 66 size of its associated drive, if available, clamped to between [4KiB, 67 64KiB]. The current default for qcow2 is 64KiB. 68 69- To create a new bitmap that tracks changes in 32KiB segments: 70 71.. code:: json 72 73 { "execute": "block-dirty-bitmap-add", 74 "arguments": { 75 "node": "drive0", 76 "name": "bitmap0", 77 "granularity": 32768 78 } 79 } 80 81Deletion 82~~~~~~~~ 83 84- Bitmaps that are frozen cannot be deleted. 85 86- Deleting the bitmap does not impact any other bitmaps attached to the 87 same node, nor does it affect any backups already created from this 88 node. 89 90- Because bitmaps are only unique to the node to which they are 91 attached, you must specify the node/drive name here, too. 92 93.. code:: json 94 95 { "execute": "block-dirty-bitmap-remove", 96 "arguments": { 97 "node": "drive0", 98 "name": "bitmap0" 99 } 100 } 101 102Resetting 103~~~~~~~~~ 104 105- Resetting a bitmap will clear all information it holds. 106 107- An incremental backup created from an empty bitmap will copy no data, 108 as if nothing has changed. 109 110.. code:: json 111 112 { "execute": "block-dirty-bitmap-clear", 113 "arguments": { 114 "node": "drive0", 115 "name": "bitmap0" 116 } 117 } 118 119Transactions 120------------ 121 122Justification 123~~~~~~~~~~~~~ 124 125Bitmaps can be safely modified when the VM is paused or halted by using 126the basic QMP commands. For instance, you might perform the following 127actions: 128 1291. Boot the VM in a paused state. 1302. Create a full drive backup of drive0. 1313. Create a new bitmap attached to drive0. 1324. Resume execution of the VM. 1335. Incremental backups are ready to be created. 134 135At this point, the bitmap and drive backup would be correctly in sync, 136and incremental backups made from this point forward would be correctly 137aligned to the full drive backup. 138 139This is not particularly useful if we decide we want to start 140incremental backups after the VM has been running for a while, for which 141we will need to perform actions such as the following: 142 1431. Boot the VM and begin execution. 1442. Using a single transaction, perform the following operations: 145 146 - Create ``bitmap0``. 147 - Create a full drive backup of ``drive0``. 148 1493. Incremental backups are now ready to be created. 150 151Supported Bitmap Transactions 152~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 153 154- ``block-dirty-bitmap-add`` 155- ``block-dirty-bitmap-clear`` 156 157The usages are identical to their respective QMP commands, but see below 158for examples. 159 160Example: New Incremental Backup 161~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 162 163As outlined in the justification, perhaps we want to create a new 164incremental backup chain attached to a drive. 165 166.. code:: json 167 168 { "execute": "transaction", 169 "arguments": { 170 "actions": [ 171 {"type": "block-dirty-bitmap-add", 172 "data": {"node": "drive0", "name": "bitmap0"} }, 173 {"type": "drive-backup", 174 "data": {"device": "drive0", "target": "/path/to/full_backup.img", 175 "sync": "full", "format": "qcow2"} } 176 ] 177 } 178 } 179 180Example: New Incremental Backup Anchor Point 181~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 182 183Maybe we just want to create a new full backup with an existing bitmap 184and want to reset the bitmap to track the new chain. 185 186.. code:: json 187 188 { "execute": "transaction", 189 "arguments": { 190 "actions": [ 191 {"type": "block-dirty-bitmap-clear", 192 "data": {"node": "drive0", "name": "bitmap0"} }, 193 {"type": "drive-backup", 194 "data": {"device": "drive0", "target": "/path/to/new_full_backup.img", 195 "sync": "full", "format": "qcow2"} } 196 ] 197 } 198 } 199 200Incremental Backups 201------------------- 202 203The star of the show. 204 205**Nota Bene!** Only incremental backups of entire drives are supported 206for now. So despite the fact that you can attach a bitmap to any 207arbitrary node, they are only currently useful when attached to the root 208node. This is because drive-backup only supports drives/devices instead 209of arbitrary nodes. 210 211Example: First Incremental Backup 212~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 213 2141. Create a full backup and sync it to the dirty bitmap, as in the 215 transactional examples above; or with the VM offline, manually create 216 a full copy and then create a new bitmap before the VM begins 217 execution. 218 219 - Let's assume the full backup is named ``full_backup.img``. 220 - Let's assume the bitmap you created is ``bitmap0`` attached to 221 ``drive0``. 222 2232. Create a destination image for the incremental backup that utilizes 224 the full backup as a backing image. 225 226 - Let's assume the new incremental image is named 227 ``incremental.0.img``. 228 229 .. code:: bash 230 231 $ qemu-img create -f qcow2 incremental.0.img -b full_backup.img -F qcow2 232 2333. Issue the incremental backup command: 234 235 .. code:: json 236 237 { "execute": "drive-backup", 238 "arguments": { 239 "device": "drive0", 240 "bitmap": "bitmap0", 241 "target": "incremental.0.img", 242 "format": "qcow2", 243 "sync": "incremental", 244 "mode": "existing" 245 } 246 } 247 248Example: Second Incremental Backup 249~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 250 2511. Create a new destination image for the incremental backup that points 252 to the previous one, e.g.: ``incremental.1.img`` 253 254 .. code:: bash 255 256 $ qemu-img create -f qcow2 incremental.1.img -b incremental.0.img -F qcow2 257 2582. Issue a new incremental backup command. The only difference here is 259 that we have changed the target image below. 260 261 .. code:: json 262 263 { "execute": "drive-backup", 264 "arguments": { 265 "device": "drive0", 266 "bitmap": "bitmap0", 267 "target": "incremental.1.img", 268 "format": "qcow2", 269 "sync": "incremental", 270 "mode": "existing" 271 } 272 } 273 274Errors 275------ 276 277- In the event of an error that occurs after a backup job is 278 successfully launched, either by a direct QMP command or a QMP 279 transaction, the user will receive a ``BLOCK_JOB_COMPLETE`` event with 280 a failure message, accompanied by a ``BLOCK_JOB_ERROR`` event. 281 282- In the case of an event being cancelled, the user will receive a 283 ``BLOCK_JOB_CANCELLED`` event instead of a pair of COMPLETE and ERROR 284 events. 285 286- In either case, the incremental backup data contained within the 287 bitmap is safely rolled back, and the data within the bitmap is not 288 lost. The image file created for the failed attempt can be safely 289 deleted. 290 291- Once the underlying problem is fixed (e.g. more storage space is 292 freed up), you can simply retry the incremental backup command with 293 the same bitmap. 294 295Example 296~~~~~~~ 297 2981. Create a target image: 299 300 .. code:: bash 301 302 $ qemu-img create -f qcow2 incremental.0.img -b full_backup.img -F qcow2 303 3042. Attempt to create an incremental backup via QMP: 305 306 .. code:: json 307 308 { "execute": "drive-backup", 309 "arguments": { 310 "device": "drive0", 311 "bitmap": "bitmap0", 312 "target": "incremental.0.img", 313 "format": "qcow2", 314 "sync": "incremental", 315 "mode": "existing" 316 } 317 } 318 3193. Receive an event notifying us of failure: 320 321 .. code:: json 322 323 { "timestamp": { "seconds": 1424709442, "microseconds": 844524 }, 324 "data": { "speed": 0, "offset": 0, "len": 67108864, 325 "error": "No space left on device", 326 "device": "drive1", "type": "backup" }, 327 "event": "BLOCK_JOB_COMPLETED" } 328 3294. Delete the failed incremental, and re-create the image. 330 331 .. code:: bash 332 333 $ rm incremental.0.img 334 $ qemu-img create -f qcow2 incremental.0.img -b full_backup.img -F qcow2 335 3365. Retry the command after fixing the underlying problem, such as 337 freeing up space on the backup volume: 338 339 .. code:: json 340 341 { "execute": "drive-backup", 342 "arguments": { 343 "device": "drive0", 344 "bitmap": "bitmap0", 345 "target": "incremental.0.img", 346 "format": "qcow2", 347 "sync": "incremental", 348 "mode": "existing" 349 } 350 } 351 3526. Receive confirmation that the job completed successfully: 353 354 .. code:: json 355 356 { "timestamp": { "seconds": 1424709668, "microseconds": 526525 }, 357 "data": { "device": "drive1", "type": "backup", 358 "speed": 0, "len": 67108864, "offset": 67108864}, 359 "event": "BLOCK_JOB_COMPLETED" } 360 361Partial Transactional Failures 362~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 363 364- Sometimes, a transaction will succeed in launching and return 365 success, but then later the backup jobs themselves may fail. It is 366 possible that a management application may have to deal with a 367 partial backup failure after a successful transaction. 368 369- If multiple backup jobs are specified in a single transaction, when 370 one of them fails, it will not interact with the other backup jobs in 371 any way. 372 373- The job(s) that succeeded will clear the dirty bitmap associated with 374 the operation, but the job(s) that failed will not. It is not "safe" 375 to delete any incremental backups that were created successfully in 376 this scenario, even though others failed. 377 378Example 379^^^^^^^ 380 381- QMP example highlighting two backup jobs: 382 383 .. code:: json 384 385 { "execute": "transaction", 386 "arguments": { 387 "actions": [ 388 { "type": "drive-backup", 389 "data": { "device": "drive0", "bitmap": "bitmap0", 390 "format": "qcow2", "mode": "existing", 391 "sync": "incremental", "target": "d0-incr-1.qcow2" } }, 392 { "type": "drive-backup", 393 "data": { "device": "drive1", "bitmap": "bitmap1", 394 "format": "qcow2", "mode": "existing", 395 "sync": "incremental", "target": "d1-incr-1.qcow2" } }, 396 ] 397 } 398 } 399 400- QMP example response, highlighting one success and one failure: 401 402 - Acknowledgement that the Transaction was accepted and jobs were 403 launched: 404 405 .. code:: json 406 407 { "return": {} } 408 409 - Later, QEMU sends notice that the first job was completed: 410 411 .. code:: json 412 413 { "timestamp": { "seconds": 1447192343, "microseconds": 615698 }, 414 "data": { "device": "drive0", "type": "backup", 415 "speed": 0, "len": 67108864, "offset": 67108864 }, 416 "event": "BLOCK_JOB_COMPLETED" 417 } 418 419 - Later yet, QEMU sends notice that the second job has failed: 420 421 .. code:: json 422 423 { "timestamp": { "seconds": 1447192399, "microseconds": 683015 }, 424 "data": { "device": "drive1", "action": "report", 425 "operation": "read" }, 426 "event": "BLOCK_JOB_ERROR" } 427 428 .. code:: json 429 430 { "timestamp": { "seconds": 1447192399, "microseconds": 431 685853 }, "data": { "speed": 0, "offset": 0, "len": 67108864, 432 "error": "Input/output error", "device": "drive1", "type": 433 "backup" }, "event": "BLOCK_JOB_COMPLETED" } 434 435- In the above example, ``d0-incr-1.qcow2`` is valid and must be kept, 436 but ``d1-incr-1.qcow2`` is invalid and should be deleted. If a VM-wide 437 incremental backup of all drives at a point-in-time is to be made, 438 new backups for both drives will need to be made, taking into account 439 that a new incremental backup for drive0 needs to be based on top of 440 ``d0-incr-1.qcow2``. 441 442Grouped Completion Mode 443~~~~~~~~~~~~~~~~~~~~~~~ 444 445- While jobs launched by transactions normally complete or fail on 446 their own, it is possible to instruct them to complete or fail 447 together as a group. 448 449- QMP transactions take an optional properties structure that can 450 affect the semantics of the transaction. 451 452- The "completion-mode" transaction property can be either "individual" 453 which is the default, legacy behavior described above, or "grouped," 454 a new behavior detailed below. 455 456- Delayed Completion: In grouped completion mode, no jobs will report 457 success until all jobs are ready to report success. 458 459- Grouped failure: If any job fails in grouped completion mode, all 460 remaining jobs will be cancelled. Any incremental backups will 461 restore their dirty bitmap objects as if no backup command was ever 462 issued. 463 464 - Regardless of if QEMU reports a particular incremental backup job 465 as CANCELLED or as an ERROR, the in-memory bitmap will be 466 restored. 467 468Example 469^^^^^^^ 470 471- Here's the same example scenario from above with the new property: 472 473 .. code:: json 474 475 { "execute": "transaction", 476 "arguments": { 477 "actions": [ 478 { "type": "drive-backup", 479 "data": { "device": "drive0", "bitmap": "bitmap0", 480 "format": "qcow2", "mode": "existing", 481 "sync": "incremental", "target": "d0-incr-1.qcow2" } }, 482 { "type": "drive-backup", 483 "data": { "device": "drive1", "bitmap": "bitmap1", 484 "format": "qcow2", "mode": "existing", 485 "sync": "incremental", "target": "d1-incr-1.qcow2" } }, 486 ], 487 "properties": { 488 "completion-mode": "grouped" 489 } 490 } 491 } 492 493- QMP example response, highlighting a failure for ``drive2``: 494 495 - Acknowledgement that the Transaction was accepted and jobs were 496 launched: 497 498 .. code:: json 499 500 { "return": {} } 501 502 - Later, QEMU sends notice that the second job has errored out, but 503 that the first job was also cancelled: 504 505 .. code:: json 506 507 { "timestamp": { "seconds": 1447193702, "microseconds": 632377 }, 508 "data": { "device": "drive1", "action": "report", 509 "operation": "read" }, 510 "event": "BLOCK_JOB_ERROR" } 511 512 .. code:: json 513 514 { "timestamp": { "seconds": 1447193702, "microseconds": 640074 }, 515 "data": { "speed": 0, "offset": 0, "len": 67108864, 516 "error": "Input/output error", 517 "device": "drive1", "type": "backup" }, 518 "event": "BLOCK_JOB_COMPLETED" } 519 520 .. code:: json 521 522 { "timestamp": { "seconds": 1447193702, "microseconds": 640163 }, 523 "data": { "device": "drive0", "type": "backup", "speed": 0, 524 "len": 67108864, "offset": 16777216 }, 525 "event": "BLOCK_JOB_CANCELLED" } 526 527.. raw:: html 528 529 <!-- 530 The FreeBSD Documentation License 531 532 Redistribution and use in source (Markdown) and 'compiled' forms (SGML, HTML, 533 PDF, PostScript, RTF and so forth) with or without modification, are permitted 534 provided that the following conditions are met: 535 536 Redistributions of source code (Markdown) must retain the above copyright 537 notice, this list of conditions and the following disclaimer of this file 538 unmodified. 539 540 Redistributions in compiled form (transformed to other DTDs, converted to PDF, 541 PostScript, RTF and other formats) must reproduce the above copyright notice, 542 this list of conditions and the following disclaimer in the documentation and/or 543 other materials provided with the distribution. 544 545 THIS DOCUMENTATION IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" 546 AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE 547 IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE 548 DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE 549 FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL 550 DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR 551 SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER 552 CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, 553 OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF 554 THIS DOCUMENTATION, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 555 --> 556