Merge remote-tracking branch 'remotes/kwolf/tags/for-upstream' into staging

Block layer patches

- Fail gracefully when blockdev-snapshot creates loops
- ide: Fix IDENTIFY DEVICE for disks > 128 GiB

Merge remote-tracking branch 'remotes/kwolf/tags/for-upstream' into staging

Block layer patches

- Fail gracefully when blockdev-snapshot creates loops
- ide: Fix IDENTIFY DEVICE for disks > 128 GiB
- file-posix: Fix return value translation for AIO discards
- file-posix: add 'aio-max-batch' option
- rbd: implement bdrv_co_block_status
- Code cleanups and build fixes

# gpg: Signature made Tue 02 Nov 2021 12:04:02 PM EDT
# gpg: using RSA key DC3DEB159A9AF95D3D7456FE7F09B272C88F2FD6
# gpg: issuer "kwolf@redhat.com"
# gpg: Good signature from "Kevin Wolf <kwolf@redhat.com>" [full]

* remotes/kwolf/tags/for-upstream:
block/nvme: Extract nvme_free_queue() from nvme_free_queue_pair()
block/nvme: Display CQ/SQ pointer in nvme_free_queue_pair()
block/nvme: Automatically free qemu_memalign() with QEMU_AUTO_VFREE
block-backend: Silence clang -m32 compiler warning
linux-aio: add `dev_max_batch` parameter to laio_io_unplug()
linux-aio: add `dev_max_batch` parameter to laio_co_submit()
file-posix: add `aio-max-batch` option
block/export/fuse.c: fix musl build
ide: Cap LBA28 capacity announcement to 2^28-1
block/rbd: implement bdrv_co_block_status
block: Fail gracefully when blockdev-snapshot creates loops
block/file-posix: Fix return value translation for AIO discards

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>

show more ...

block: Fail gracefully when blockdev-snapshot creates loops

Using blockdev-snapshot to append a node as an overlay to itself, or to
any of its parents, causes crashes. Catch the condition and return

block: Fail gracefully when blockdev-snapshot creates loops

Using blockdev-snapshot to append a node as an overlay to itself, or to
any of its parents, causes crashes. Catch the condition and return an
error for these cases instead.

Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=1824363
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Message-Id: <20211018134714.48438-1-kwolf@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>

show more ...

Merge remote-tracking branch 'remotes/kevin/tags/for-upstream' into staging

Block layer patches:

- qemu-storage-daemon: add --pidfile option
- qemu-storage-daemon: CLI error messages include the op

Merge remote-tracking branch 'remotes/kevin/tags/for-upstream' into staging

Block layer patches:

- qemu-storage-daemon: add --pidfile option
- qemu-storage-daemon: CLI error messages include the option name now
- vhost-user-blk export: Misc fixes
- docs: Improvements for qemu-storage-daemon documentation
- parallels: load bitmap extension
- backup-top: Don't crash on post-finalize accesses
- Improve error messages related to node-name options
- iotests improvements

# gpg: Signature made Mon 08 Mar 2021 17:01:41 GMT
# gpg: using RSA key DC3DEB159A9AF95D3D7456FE7F09B272C88F2FD6
# gpg: issuer "kwolf@redhat.com"
# gpg: Good signature from "Kevin Wolf <kwolf@redhat.com>" [full]
# Primary key fingerprint: DC3D EB15 9A9A F95D 3D74 56FE 7F09 B272 C88F 2FD6

* remotes/kevin/tags/for-upstream: (30 commits)
blockdev: Clarify error messages pertaining to 'node-name'
block: Clarify error messages pertaining to 'node-name'
docs: qsd: Explain --export nbd,name=... default
MAINTAINERS: update parallels block driver
iotests: add parallels-read-bitmap test
iotests.py: add unarchive_sample_image() helper
parallels: support bitmap extension for read-only mode
block/parallels: BDRVParallelsState: add cluster_size field
parallels.txt: fix bitmap L1 table description
qcow2-bitmap: make bytes_covered_by_bitmap_cluster() public
block/export: port virtio-blk read/write range check
block/export: port virtio-blk discard/write zeroes input validation
block/export: fix vhost-user-blk export sector number calculation
block/export: use VIRTIO_BLK_SECTOR_BITS
block/export: fix blk_size double byteswap
libqtest: add qtest_remove_abrt_handler()
libqtest: add qtest_kill_qemu()
libqtest: add qtest_socket_server()
vhost-user-blk: fix blkcfg->num_queues endianness
docs: replace insecure /tmp examples in qsd docs
...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

block: Clarify error messages pertaining to 'node-name'

Some error messages contain ambiguous representations of the 'node-name'
parameter. This can be particularly confusing when exchanging QMP
mes

block: Clarify error messages pertaining to 'node-name'

Some error messages contain ambiguous representations of the 'node-name'
parameter. This can be particularly confusing when exchanging QMP
messages (C = client, S = server):

C: {"execute": "block_resize", "arguments": { "device": "my_file", "size": 26843545600 }}
S: {"error": {"class": "GenericError", "desc": "Cannot find device=my_file nor node_name="}}
^^^^^^^^^

This error message suggests one could send a message with a key called
'node_name':

C: {"execute": "block_resize", "arguments": { "node_name": "my_file", "size": 26843545600 }}
^^^^^^^^^

but using the underscore is actually incorrect, the parameter should be
'node-name':

S: {"error": {"class": "GenericError", "desc": "Parameter 'node_name' is unexpected"}}

This behavior was uncovered in bz1651437, but I ended up going down a
rabbit hole looking for other areas where this miscommunication might
occur and changing those accordingly as well.

Fixes: https://bugzilla.redhat.com/1651437
Signed-off-by: Connor Kuehl <ckuehl@redhat.com>
Message-Id: <20210305151929.1947331-2-ckuehl@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>

show more ...

Merge remote-tracking branch 'remotes/maxreitz/tags/pull-block-2020-12-18' into staging

Block patches:
- New block filter: preallocate (which, on writes beyond an image file's
end, allocates big c

Merge remote-tracking branch 'remotes/maxreitz/tags/pull-block-2020-12-18' into staging

Block patches:
- New block filter: preallocate (which, on writes beyond an image file's
end, allocates big chunks of data so that such post-EOF writes will
occur less frequently)
- write-zeroes and block-status support for Quorum
- Implementation of truncate for the nvme block driver similarly to the
existing implementations for host block devices and iscsi devices
- Block layer refactoring: Drop the tighten_restrictions concept in the
block permission functions
- iotest fixes

# gpg: Signature made Fri 18 Dec 2020 14:45:30 GMT
# gpg: using RSA key 91BEB60A30DB3E8857D11829F407DB0061D5CF40
# gpg: issuer "mreitz@redhat.com"
# gpg: Good signature from "Max Reitz <mreitz@redhat.com>" [full]
# Primary key fingerprint: 91BE B60A 30DB 3E88 57D1 1829 F407 DB00 61D5 CF40

* remotes/maxreitz/tags/pull-block-2020-12-18: (30 commits)
iotests: Fix _send_qemu_cmd with bash 5.1
iotests/102: Pass $QEMU_HANDLE to _send_qemu_cmd
block/nvme: Implement fake truncate() coroutine
quorum: Implement bdrv_co_pwrite_zeroes()
quorum: Implement bdrv_co_block_status()
scripts/simplebench: add bench_prealloc.py
simplebench/results_to_text: make executable
simplebench/results_to_text: add difference line to the table
simplebench/results_to_text: improve view of the table
simplebench: move results_to_text() into separate file
simplebench: rename ascii() to results_to_text()
scripts/simplebench: use standard deviation for +- error
scripts/simplebench: support iops
scripts/simplebench: fix grammar: s/successed/succeeded/
iotests: add 298 to test new preallocate filter driver
iotests.py: execute_setup_common(): add required_fmts argument
iotests: qemu_io_silent: support --image-opts
qemu-io: add preallocate mode parameter for truncate command
block: introduce preallocate filter
block: bdrv_check_perm(): process children anyway
...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

iotests: Fix _send_qemu_cmd with bash 5.1

With bash 5.1, the output of the following script changes:

a=("double space")
a=${a[@]:0:1}
echo "$a"

from "double space" to "double space", i.e.

iotests: Fix _send_qemu_cmd with bash 5.1

With bash 5.1, the output of the following script changes:

a=("double space")
a=${a[@]:0:1}
echo "$a"

from "double space" to "double space", i.e. all white space is
preserved as-is. This is probably what we actually want here (judging
from the "...to accommodate pathnames with spaces" comment), but before
5.1, we would have to quote the ${} slice to get the same behavior.

In any case, without quoting, the reference output of many iotests is
different between bash 5.1 and pre-5.1, which is not very good. The
output of 5.1 is what we want, so whatever we do to get pre-5.1 to the
same result, it means we have to fix the reference output of basically
all tests that invoke _send_qemu_cmd (except the ones that only use
single spaces in the commands they invoke).

Instead of quoting the ${} slice (cmd="${$@: 1:...}"), we can also just
not use array slicing and replace the whole thing with a simple "cmd=$1;
shift", which works because all callers quote the whole $cmd argument
anyway.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20201217153803.101231-3-mreitz@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>

show more ...

Merge remote-tracking branch 'remotes/maxreitz/tags/pull-block-2020-08-26' into staging

Block patches:
- qcow2 subclusters (extended L2 entries)

# gpg: Signature made Wed 26 Aug 2020 08:37:04 BST
#

Merge remote-tracking branch 'remotes/maxreitz/tags/pull-block-2020-08-26' into staging

Block patches:
- qcow2 subclusters (extended L2 entries)

# gpg: Signature made Wed 26 Aug 2020 08:37:04 BST
# gpg: using RSA key 91BEB60A30DB3E8857D11829F407DB0061D5CF40
# gpg: issuer "mreitz@redhat.com"
# gpg: Good signature from "Max Reitz <mreitz@redhat.com>" [full]
# Primary key fingerprint: 91BE B60A 30DB 3E88 57D1 1829 F407 DB00 61D5 CF40

* remotes/maxreitz/tags/pull-block-2020-08-26: (34 commits)
iotests: Add tests for qcow2 images with extended L2 entries
qcow2: Assert that expand_zero_clusters_in_l1() does not support subclusters
qcow2: Allow preallocation and backing files if extended_l2 is set
qcow2: Add the 'extended_l2' option and the QCOW2_INCOMPAT_EXTL2 bit
qcow2: Add prealloc field to QCowL2Meta
qcow2: Add subcluster support to qcow2_measure()
qcow2: Add subcluster support to qcow2_co_pwrite_zeroes()
qcow2: Add subcluster support to handle_alloc_space()
qcow2: Clear the L2 bitmap when allocating a compressed cluster
qcow2: Update L2 bitmap in qcow2_alloc_cluster_link_l2()
qcow2: Add subcluster support to check_refcounts_l2()
qcow2: Add subcluster support to discard_in_l2_slice()
qcow2: Add subcluster support to zero_in_l2_slice()
qcow2: Add subcluster support to qcow2_get_host_offset()
qcow2: Add subcluster support to calculate_l2_meta()
qcow2: Handle QCOW2_SUBCLUSTER_UNALLOCATED_ALLOC
qcow2: Replace QCOW2_CLUSTER_* with QCOW2_SUBCLUSTER_*
qcow2: Add cluster type parameter to qcow2_get_host_offset()
qcow2: Add qcow2_cluster_is_allocated()
qcow2: Add qcow2_get_subcluster_range_type()
...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

qcow2: Add the 'extended_l2' option and the QCOW2_INCOMPAT_EXTL2 bit

Now that the implementation of subclusters is complete we can finally
add the necessary options to create and read images with th

qcow2: Add the 'extended_l2' option and the QCOW2_INCOMPAT_EXTL2 bit

Now that the implementation of subclusters is complete we can finally
add the necessary options to create and read images with this feature,
which we call "extended L2 entries".

Signed-off-by: Alberto Garcia <berto@igalia.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <6476caaa73216bd05b7bb2d504a20415e1665176.1594396418.git.berto@igalia.com>
[mreitz: %s/5\.1/5.2/; fixed 302's and 303's reference output]
Signed-off-by: Max Reitz <mreitz@redhat.com>

show more ...

Merge remote-tracking branch 'remotes/kevin/tags/for-upstream' into staging

Block layer patches:

- file-posix: Mitigate file fragmentation with extent size hints
- Tighten qemu-img rules on missing

Merge remote-tracking branch 'remotes/kevin/tags/for-upstream' into staging

Block layer patches:

- file-posix: Mitigate file fragmentation with extent size hints
- Tighten qemu-img rules on missing backing format
- qemu-img map: Don't limit block status request size
- Fix crash with virtio-scsi and iothreads

# gpg: Signature made Tue 14 Jul 2020 14:24:19 BST
# gpg: using RSA key DC3DEB159A9AF95D3D7456FE7F09B272C88F2FD6
# gpg: issuer "kwolf@redhat.com"
# gpg: Good signature from "Kevin Wolf <kwolf@redhat.com>" [full]
# Primary key fingerprint: DC3D EB15 9A9A F95D 3D74 56FE 7F09 B272 C88F 2FD6

* remotes/kevin/tags/for-upstream:
block: Avoid stale pointer dereference in blk_get_aio_context()
qemu-img: Deprecate use of -b without -F
block: Add support to warn on backing file change without format
iotests: Specify explicit backing format where sensible
qcow2: Deprecate use of qemu-img amend to change backing file
block: Error if backing file fails during creation without -u
qcow: Tolerate backing_fmt=
vmdk: Add trivial backing_fmt support
sheepdog: Add trivial backing_fmt support
block: Finish deprecation of 'qemu-img convert -n -o'
qemu-img: Flush stdout before before potential stderr messages
file-posix: Mitigate file fragmentation with extent size hints
iotests/059: Filter out disk size with more standard filter
qemu-img map: Don't limit block status request size
iotests: Simplify _filter_img_create() a bit

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

iotests: Specify explicit backing format where sensible

There are many existing qcow2 images that specify a backing file but
no format. This has been the source of CVEs in the past, but has
become

iotests: Specify explicit backing format where sensible

There are many existing qcow2 images that specify a backing file but
no format. This has been the source of CVEs in the past, but has
become more prominent of a problem now that libvirt has switched to
-blockdev. With older -drive, at least the probing was always done by
qemu (so the only risk of a changed format between successive boots of
a guest was if qemu was upgraded and probed differently). But with
newer -blockdev, libvirt must specify a format; if libvirt guesses raw
where the image was formatted, this results in data corruption visible
to the guest; conversely, if libvirt guesses qcow2 where qemu was
using raw, this can result in potential security holes, so modern
libvirt instead refuses to use images without explicit backing format.

The change in libvirt to reject images without explicit backing format
has pointed out that a number of tools have been far too reliant on
probing in the past. It's time to set a better example in our own
iotests of properly setting this parameter.

iotest calls to create, rebase, and convert are all impacted to some
degree. It's a bit annoying that we are inconsistent on command line
- while all of those accept -o backing_file=...,backing_fmt=..., the
shortcuts are different: create and rebase have -b and -F, while
convert has -B but no -F. (amend has no shortcuts, but the previous
patch just deprecated the use of amend to change backing chains).

Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <20200706203954.341758-9-eblake@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>

show more ...

Merge remote-tracking branch 'remotes/maxreitz/tags/pull-block-2020-07-06' into staging

Block patches for 5.1:
- LUKS keyslot amendment
(+ patches to make the iotests pass on non-Linux systems, an

Merge remote-tracking branch 'remotes/maxreitz/tags/pull-block-2020-07-06' into staging

Block patches for 5.1:
- LUKS keyslot amendment
(+ patches to make the iotests pass on non-Linux systems, and to keep
the tests passing for qcow v1, and to skip LUKS tests (including
qcow2 LUKS) when the built qemu does not support it)
- Refactoring in the block layer: Drop the basically unnecessary
unallocated_blocks_are_zero field from BlockDriverInfo
- Fix qcow2 preallocation when the image size is not a multiple of the
cluster size
- Fix in block-copy code

# gpg: Signature made Mon 06 Jul 2020 11:02:53 BST
# gpg: using RSA key 91BEB60A30DB3E8857D11829F407DB0061D5CF40
# gpg: issuer "mreitz@redhat.com"
# gpg: Good signature from "Max Reitz <mreitz@redhat.com>" [full]
# Primary key fingerprint: 91BE B60A 30DB 3E88 57D1 1829 F407 DB00 61D5 CF40

* remotes/maxreitz/tags/pull-block-2020-07-06: (31 commits)
qed: Simplify backing reads
block: drop unallocated_blocks_are_zero
block/vhdx: drop unallocated_blocks_are_zero
block/file-posix: drop unallocated_blocks_are_zero
block/iscsi: drop unallocated_blocks_are_zero
block/crypto: drop unallocated_blocks_are_zero
block/vpc: return ZERO block-status when appropriate
block/vdi: return ZERO block-status when appropriate
block: inline bdrv_unallocated_blocks_are_zero()
qemu-img: convert: don't use unallocated_blocks_are_zero
iotests: add tests for blockdev-amend
block/qcow2: implement blockdev-amend
block/crypto: implement blockdev-amend
block/core: add generic infrastructure for x-blockdev-amend qmp command
iotests: qemu-img tests for luks key management
block/qcow2: extend qemu-img amend interface with crypto options
block/crypto: implement the encryption key management
block/crypto: rename two functions
block/amend: refactor qcow2 amend options
block/amend: separate amend and create options for qemu-img
...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

block/amend: refactor qcow2 amend options

Some qcow2 create options can't be used for amend.
Remove them from the qcow2 create options and add generic logic to detect
such options in qemu-img

Signe

block/amend: refactor qcow2 amend options

Some qcow2 create options can't be used for amend.
Remove them from the qcow2 create options and add generic logic to detect
such options in qemu-img

Signed-off-by: Maxim Levitsky <mlevitsk@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
[mreitz: Dropped some iotests reference output hunks that became
unnecessary thanks to
"iotests: Make _filter_img_create more active"]
Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20200625125548.870061-12-mreitz@redhat.com>

show more ...

Merge remote-tracking branch 'remotes/maxreitz/tags/pull-block-2020-05-13' into staging

Block patches:
- zstd compression for qcow2
- Fix use-after-free

# gpg: Signature made Wed 13 May 2020 15:14:

Merge remote-tracking branch 'remotes/maxreitz/tags/pull-block-2020-05-13' into staging

Block patches:
- zstd compression for qcow2
- Fix use-after-free

# gpg: Signature made Wed 13 May 2020 15:14:06 BST
# gpg: using RSA key 91BEB60A30DB3E8857D11829F407DB0061D5CF40
# gpg: issuer "mreitz@redhat.com"
# gpg: Good signature from "Max Reitz <mreitz@redhat.com>" [full]
# Primary key fingerprint: 91BE B60A 30DB 3E88 57D1 1829 F407 DB00 61D5 CF40

* remotes/maxreitz/tags/pull-block-2020-05-13:
block/block-copy: fix use-after-free of task pointer
iotests: 287: add qcow2 compression type test
qcow2: add zstd cluster compression
qcow2: rework the cluster compression routine
qcow2: introduce compression type feature

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

qcow2: introduce compression type feature

The patch adds some preparation parts for incompatible compression type
feature to qcow2 allowing the use different compression methods for
image clusters (

qcow2: introduce compression type feature

The patch adds some preparation parts for incompatible compression type
feature to qcow2 allowing the use different compression methods for
image clusters (de)compressing.

It is implied that the compression type is set on the image creation and
can be changed only later by image conversion, thus compression type
defines the only compression algorithm used for the image, and thus,
for all image clusters.

The goal of the feature is to add support of other compression methods
to qcow2. For example, ZSTD which is more effective on compression than ZLIB.

The default compression is ZLIB. Images created with ZLIB compression type
are backward compatible with older qemu versions.

Adding of the compression type breaks a number of tests because now the
compression type is reported on image creation and there are some changes
in the qcow2 header in size and offsets.

The tests are fixed in the following ways:
* filter out compression_type for many tests
* fix header size, feature table size and backing file offset
affected tests: 031, 036, 061, 080
header_size +=8: 1 byte compression type
7 bytes padding
feature_table += 48: incompatible feature compression type
backing_file_offset += 56 (8 + 48 -> header_change + feature_table_change)
* add "compression type" for test output matching when it isn't filtered
affected tests: 049, 060, 061, 065, 082, 085, 144, 182, 185, 198, 206,
242, 255, 274, 280

Signed-off-by: Denis Plotnikov <dplotnikov@virtuozzo.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
QAPI part:
Acked-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20200507082521.29210-2-dplotnikov@virtuozzo.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>

show more ...

Merge remote-tracking branch 'remotes/kevin/tags/for-upstream' into staging

Block layer patches:

- Relax restrictions for blockdev-snapshot (allows libvirt to do live
storage migration with block

Merge remote-tracking branch 'remotes/kevin/tags/for-upstream' into staging

Block layer patches:

- Relax restrictions for blockdev-snapshot (allows libvirt to do live
storage migration with blockdev-mirror)
- luks: Delete created files when block_crypto_co_create_opts_luks fails
- Fix memleaks in qmp_object_add

# gpg: Signature made Wed 11 Mar 2020 15:38:59 GMT
# gpg: using RSA key 7F09B272C88F2FD6
# gpg: Good signature from "Kevin Wolf <kwolf@redhat.com>" [full]
# Primary key fingerprint: DC3D EB15 9A9A F95D 3D74 56FE 7F09 B272 C88F 2FD6

* remotes/kevin/tags/for-upstream:
qemu-iotests: adding LUKS cleanup for non-UTF8 secret error
crypto.c: cleanup created file when block_crypto_co_create_opts_luks fails
block.c: adding bdrv_co_delete_file
block: introducing 'bdrv_co_delete_file' interface
tests/qemu-iotests: Fix socket_scm_helper build path
qapi: Add '@allow-write-only-overlay' feature for 'blockdev-snapshot'
iotests: Add iothread cases to 155
block: Fix cross-AioContext blockdev-snapshot
iotests: Test mirror with temporarily disabled target backing file
iotests: Fix run_job() with use_log=False
block: Relax restrictions for blockdev-snapshot
block: Make bdrv_get_cumulative_perm() public
qom-qmp-cmds: fix two memleaks in qmp_object_add

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

block: Relax restrictions for blockdev-snapshot

blockdev-snapshot returned an error if the overlay was already in use,
which it defined as having any BlockBackend parent. This is in fact both
too st

block: Relax restrictions for blockdev-snapshot

blockdev-snapshot returned an error if the overlay was already in use,
which it defined as having any BlockBackend parent. This is in fact both
too strict (some parents can tolerate the change of visible data caused
by attaching a backing file) and too loose (some non-BlockBackend
parents may not be happy with it).

One important use case that is prevented by the too strict check is live
storage migration with blockdev-mirror. Here, the target node is
usually opened without a backing file so that the active layer is
mirrored while its backing chain can be copied in the background.

The backing chain should be attached to the mirror target node when
finalising the job, just before switching the users of the source node
to the new copy (at which point the mirror job still has a reference to
the node). drive-mirror did this automatically, but with blockdev-mirror
this is the job of the QMP client, so it needs a way to do this.

blockdev-snapshot is the obvious way, so this patch makes it work in
this scenario. The new condition is that no parent uses CONSISTENT_READ
permissions. This will ensure that the operation will still be blocked
when the node is attached to the guest device, so blockdev-snapshot
remains safe.

(For the sake of completeness, x-blockdev-reopen can be used to achieve
the same, however it is a big hammer, performs the graph change
completely unchecked and is still experimental. So even with the option
of using x-blockdev-reopen, there are reasons why blockdev-snapshot
should be able to perform this operation.)

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Message-Id: <20200310113831.27293-3-kwolf@redhat.com>
Reviewed-by: Peter Krempa <pkrempa@redhat.com>
Tested-by: Peter Krempa <pkrempa@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>

show more ...

Merge remote-tracking branch 'remotes/maxreitz/tags/pull-block-2020-01-06' into staging

Block patches:
- Minor fixes and tests from the freeze period (too minor to be included
in 4.2)
- Allow many

Merge remote-tracking branch 'remotes/maxreitz/tags/pull-block-2020-01-06' into staging

Block patches:
- Minor fixes and tests from the freeze period (too minor to be included
in 4.2)
- Allow many bash iotests to test qcow2's external data file feature
- Add compress filter driver
- Fix Python iotests after 6f6e1698a6
- Fix for the backup job

# gpg: Signature made Mon 06 Jan 2020 14:33:06 GMT
# gpg: using RSA key 91BEB60A30DB3E8857D11829F407DB0061D5CF40
# gpg: issuer "mreitz@redhat.com"
# gpg: Good signature from "Max Reitz <mreitz@redhat.com>" [full]
# Primary key fingerprint: 91BE B60A 30DB 3E88 57D1 1829 F407 DB00 61D5 CF40

* remotes/maxreitz/tags/pull-block-2020-01-06: (34 commits)
backup-top: Begin drain earlier
tests/qemu-iotests: Update tests to recent desugarized -accel option
tests/qemu-iotests: add case to write compressed data of multiple clusters
qcow2: Allow writing compressed data of multiple clusters
block: introduce compress filter driver
iotests: Allow check -o data_file
iotests: Disable data_file where it cannot be used
iotests: Make 198 work with data_file
iotests: Make 137 work with data_file
iotests: Make 110 work with data_file
iotests: Make 091 work with data_file
iotests: Avoid cp/mv of test images
iotests: Use _rm_test_img for deleting test images
iotests: Avoid qemu-img create
iotests: Drop IMGOPTS use in 267
iotests: Replace IMGOPTS='' by --no-opts
iotests: Replace IMGOPTS= by -o
iotests: Inject space into -ocompat=0.10 in 051
iotests: Add -o and --no-opts to _make_test_img
iotests: Let _make_test_img parse its parameters
...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

iotests: Avoid cp/mv of test images

This will not work with external data files, so try to get tests working
without it as far as possible.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by:

iotests: Avoid cp/mv of test images

This will not work with external data files, so try to get tests working
without it as far as possible.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Maxim Levitsky <mlevitsk@redhat.com>
Message-id: 20191107163708.833192-17-mreitz@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>

show more ...

Merge remote-tracking branch 'remotes/ericb/tags/pull-nbd-2019-11-19' into staging

nbd patches for 2019-11-19

- iotests: more tests of NBD reconnect, various test output improvements
- nbd: fix spe

Merge remote-tracking branch 'remotes/ericb/tags/pull-nbd-2019-11-19' into staging

nbd patches for 2019-11-19

- iotests: more tests of NBD reconnect, various test output improvements
- nbd: fix spec compliance issue with long strings
- slience a Coverity warning on coroutines

# gpg: Signature made Tue 19 Nov 2019 03:06:41 GMT
# gpg: using RSA key 71C2CC22B1C4602927D2F3AAA7A16B4A2527436A
# gpg: Good signature from "Eric Blake <eblake@redhat.com>" [full]
# gpg: aka "Eric Blake (Free Software Programmer) <ebb9@byu.net>" [full]
# gpg: aka "[jpeg image of size 6874]" [full]
# Primary key fingerprint: 71C2 CC22 B1C4 6029 27D2 F3AA A7A1 6B4A 2527 436A

* remotes/ericb/tags/pull-nbd-2019-11-19:
tests: More iotest 223 improvements
iotests: Include QMP input in .out files
iotests: Switch nbd tests to use Unix rather than TCP
iotests: Fix 173
MAINTAINERS: add more bitmap-related to Dirty Bitmaps section
nbd: Don't send oversize strings
bitmap: Enforce maximum bitmap name length
nbd/server: Prefer heap over stack for parsing client names
qemu-coroutine-sleep: Silence Coverity warning
iotests: Test NBD client reconnection

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

iotests: Include QMP input in .out files

We generally include relevant HMP input in .out files, by virtue of
the fact that HMP echoes its input. But QMP does not, so we have to
explicitly inject it

iotests: Include QMP input in .out files

We generally include relevant HMP input in .out files, by virtue of
the fact that HMP echoes its input. But QMP does not, so we have to
explicitly inject it in the output stream (appropriately filtered to
keep the tests passing), in order to make it easier to read .out files
to see what behavior is being tested (especially true where the output
file is a sequence of {'return': {}}).

Suggested-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <20191114213415.23499-4-eblake@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>

show more ...

Merge remote-tracking branch 'remotes/maxreitz/tags/pull-block-2019-06-14' into staging

Block patches:
- Allow blockdev-backup from nodes that are not in qemu's main AIO
context to newly added nod

Merge remote-tracking branch 'remotes/maxreitz/tags/pull-block-2019-06-14' into staging

Block patches:
- Allow blockdev-backup from nodes that are not in qemu's main AIO
context to newly added nodes
- Add salvaging mode to qemu-img convert
- Minor fixes to tests, documentation, and for less Valgrind annoyance

# gpg: Signature made Fri 14 Jun 2019 14:38:11 BST
# gpg: using RSA key 91BEB60A30DB3E8857D11829F407DB0061D5CF40
# gpg: issuer "mreitz@redhat.com"
# gpg: Good signature from "Max Reitz <mreitz@redhat.com>" [full]
# Primary key fingerprint: 91BE B60A 30DB 3E88 57D1 1829 F407 DB00 61D5 CF40

* remotes/maxreitz/tags/pull-block-2019-06-14:
iotests: Test qemu-img convert -C --salvage
iotests: Test qemu-img convert --salvage
blkdebug: Inject errors on .bdrv_co_block_status()
blkdebug: Add "none" event
blkdebug: Add @iotype error option
qemu-img: Add salvaging mode to convert
qemu-img: Move quiet into ImgConvertState
blockdev: Overlays are not snapshots
qapi/block-core: Overlays are not snapshots
qemu-img: Fix options leakage in img_rebase()
iotests: restrict 254 to support only qcow2
hw/block/fdc: floppy command FIFO memory initialization
iotests: Fix intermittent failure in 219
iotests: Filter 175's allocation information
event_match: always match on None value
iotests: add iotest 256 for testing blockdev-backup across iothread contexts
iotests.py: rewrite run_job to be pickier
QEMUMachine: add events_wait method
iotests.py: do not use infinite waits
blockdev-backup: don't check aio_context too early

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

blockdev: Overlays are not snapshots

There are error messages which refer to an overlay node as the snapshot.
That is wrong, those are two different things.

Signed-off-by: Max Reitz <mreitz@redhat.

blockdev: Overlays are not snapshots

There are error messages which refer to an overlay node as the snapshot.
That is wrong, those are two different things.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id: 20190603202236.1342-3-mreitz@redhat.com
Reviewed-by: John Snow <jsnow@redhat.com>
Reviewed-by: Alberto Garcia <berto@igalia.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>

show more ...

Merge remote-tracking branch 'remotes/maxreitz/tags/pull-block-2017-07-11' into staging

Block layer patches

# gpg: Signature made Tue 11 Jul 2017 17:05:56 BST
# gpg: using RSA key 0x

Merge remote-tracking branch 'remotes/maxreitz/tags/pull-block-2017-07-11' into staging

Block layer patches

# gpg: Signature made Tue 11 Jul 2017 17:05:56 BST
# gpg: using RSA key 0xF407DB0061D5CF40
# gpg: Good signature from "Max Reitz <mreitz@redhat.com>"
# Primary key fingerprint: 91BE B60A 30DB 3E88 57D1 1829 F407 DB00 61D5 CF40

* remotes/maxreitz/tags/pull-block-2017-07-11: (85 commits)
iotests: Add preallocated growth test for qcow2
iotests: Add preallocated resize test for raw
block/qcow2: falloc/full preallocating growth
block/qcow2: Rename "fail_block" to just "fail"
block/qcow2: Add qcow2_refcount_area()
block/qcow2: Metadata preallocation for truncate
block/qcow2: Lock s->lock in preallocate()
block/qcow2: Generalize preallocate()
block/file-posix: Preallocation for truncate
block/file-posix: Generalize raw_regular_truncate
block/file-posix: Extract raw_regular_truncate()
block/file-posix: Small fixes in raw_create()
qemu-img: Expose PreallocMode for resizing
block: Add PreallocMode to blk_truncate()
block: Add PreallocMode to bdrv_truncate()
block: Add PreallocMode to BD.bdrv_truncate()
iotests: add test 178 for qemu-img measure
qemu-iotests: support per-format golden output files
qemu-img: add measure subcommand
qcow2: add bdrv_measure() support
...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

block: deprecate "encryption=on" in favor of "encrypt.format=aes"

Historically the qcow & qcow2 image formats supported a property
"encryption=on" to enable their built-in AES encryption. We'll
soon

block: deprecate "encryption=on" in favor of "encrypt.format=aes"

Historically the qcow & qcow2 image formats supported a property
"encryption=on" to enable their built-in AES encryption. We'll
soon be supporting LUKS for qcow2, so need a more general purpose
way to enable encryption, with a choice of formats.

This introduces an "encrypt.format" option, which will later be
joined by a number of other "encrypt.XXX" options. The use of
a "encrypt." prefix instead of "encrypt-" is done to facilitate
mapping to a nested QAPI schema at later date.

e.g. the preferred syntax is now

qemu-img create -f qcow2 -o encrypt.format=aes demo.qcow2

Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
Message-id: 20170623162419.26068-8-berrange@redhat.com
Reviewed-by: Alberto Garcia <berto@igalia.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>

show more ...

Merge remote-tracking branch 'kwolf/tags/for-upstream' into staging

Block layer patches

# gpg: Signature made Thu 11 May 2017 10:31:37 AM EDT
# gpg: using RSA key 0x7F09B272C88F2FD6

Merge remote-tracking branch 'kwolf/tags/for-upstream' into staging

Block layer patches

# gpg: Signature made Thu 11 May 2017 10:31:37 AM EDT
# gpg: using RSA key 0x7F09B272C88F2FD6
# gpg: Good signature from "Kevin Wolf <kwolf@redhat.com>"
# Primary key fingerprint: DC3D EB15 9A9A F95D 3D74 56FE 7F09 B272 C88F 2FD6

* kwolf/tags/for-upstream: (58 commits)
MAINTAINERS: Add qemu-progress to the block layer
qcow2: Discard/zero clusters by byte count
qcow2: Assert that cluster operations are aligned
qcow2: Optimize write zero of unaligned tail cluster
iotests: Add test 179 to cover write zeroes with unmap
iotests: Improve _filter_qemu_img_map
qcow2: Optimize zero_single_l2() to minimize L2 churn
qcow2: Make distinction between zero cluster types obvious
qcow2: Name typedef for cluster type
qcow2: Correctly report status of preallocated zero clusters
block: Update comments on BDRV_BLOCK_* meanings
qcow2: Use consistent switch indentation
qcow2: Nicer variable names in qcow2_update_snapshot_refcount()
tests: Add coverage for recent block geometry fixes
blkdebug: Add ability to override unmap geometries
blkdebug: Simplify override logic
blkdebug: Add pass-through write_zero and discard support
blkdebug: Refactor error injection
blkdebug: Sanity check block layer guarantees
qemu-io: Switch 'map' output to byte-based reporting
...

Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>

show more ...