Merge tag 'for-upstream' of https://repo.or.cz/qemu/kevin into staging

Block layer patches (CVE-2024-4467)

- Don't open qcow2 data files in 'qemu-img info'
- Disallow protocol prefixes for qcow2 da

Merge tag 'for-upstream' of https://repo.or.cz/qemu/kevin into staging

Block layer patches (CVE-2024-4467)

- Don't open qcow2 data files in 'qemu-img info'
- Disallow protocol prefixes for qcow2 data files, VMDK extent files and
other child nodes that are neither 'file' nor 'backing'

# -----BEGIN PGP SIGNATURE-----
#
# iQJFBAABCAAvFiEE3D3rFZqa+V09dFb+fwmycsiPL9YFAmaEKQwRHGt3b2xmQHJl
# ZGhhdC5jb20ACgkQfwmycsiPL9YgMA/+OeQf0veFb02ZNqf907Etz8/DvnqbiWUN
# 0aT5z5x8ilZQIiEDbFtLKgF3A/WO7phyCKk1q1dbRNbc1ZaWFW7mTaJM2ew++EuB
# fq0mnskLt/GVSqTReO4od7flsssp3sEDxs74yuyNITIUqui4we9WK2lLRiAv3aco
# 2NbyNeMHJxIW+QlOO3R62i24yjQaLyg/YekmiIK8itQkpKuI80fiVgor5W3RR0P0
# 71AVSHC0Edv5eavmiRqmQ+pfSI8tlINsN1s5jvxge6XpVTaL8NHsgH3LVv1R3Qtx
# Uo9hp6lQboAfc4I06gf+fcsYSBRiGCwA/J+JsWusX4FLaaTNHLt5eJAEJhfZlioj
# wgTqpy2ImRu5lcuLjLWRu4cLapPLI6CSwf4/lG9/szmRA/1UtOKpquKeTuCwMl9Y
# XEVoNDzo7GpfSb7YONo7fU7kq00OuEEAn0he7eNd2UU+Ao9Abi7JvY+fKx71FHo3
# k24SQVhVJihV1IEC4psCtaQm2bB/jdMr0jB44zHLtmqeUMLrrVf64cSAntp+2KRa
# sINBXA5OeblGKQ7FoAzc5NNNveSdF1ioRCvKB3MlHzI+efzRS7+I3wwh2Uz1Uwfo
# sivg+dAXQQBKVXn8UbfznFyEKueT0RW5CUbfeEqGQ/ocw7iTrXABsX+tjcktxl8Q
# zrHZNoAz6Ds=
# =7LWn
# -----END PGP SIGNATURE-----
# gpg: Signature made Tue 02 Jul 2024 09:21:32 AM PDT
# gpg: using RSA key DC3DEB159A9AF95D3D7456FE7F09B272C88F2FD6
# gpg: issuer "kwolf@redhat.com"
# gpg: Good signature from "Kevin Wolf <kwolf@redhat.com>" [full]

* tag 'for-upstream' of https://repo.or.cz/qemu/kevin:
block: Parse filenames only when explicitly requested
iotests/270: Don't store data-file with json: prefix in image
iotests/244: Don't store data-file with protocol in image
qcow2: Don't open data_file with BDRV_O_NO_IO

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>

show more ...

qcow2: Don't open data_file with BDRV_O_NO_IO

One use case for 'qemu-img info' is verifying that untrusted images
don't reference an unwanted external file, be it as a backing file or an
external da

qcow2: Don't open data_file with BDRV_O_NO_IO

One use case for 'qemu-img info' is verifying that untrusted images
don't reference an unwanted external file, be it as a backing file or an
external data file. To make sure that calling 'qemu-img info' can't
already have undesired side effects with a malicious image, just don't
open the data file at all with BDRV_O_NO_IO. If nothing ever tries to do
I/O, we don't need to have it open.

This changes the output of iotests case 061, which used 'qemu-img info'
to show that opening an image with an invalid data file fails. After
this patch, it succeeds. Replace this part of the test with a qemu-io
call, but keep the final 'qemu-img info' to show that the invalid data
file is correctly displayed in the output.

Fixes: CVE-2024-4467
Cc: qemu-stable@nongnu.org
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Hanna Czenczek <hreitz@redhat.com>

show more ...

Merge remote-tracking branch 'remotes/hreitz-gitlab/tags/pull-block-2022-02-01' into staging

Block patches:
- Add support to the iotests to test qcow2's zstd compression mode
- Fix post-migration bl

Merge remote-tracking branch 'remotes/hreitz-gitlab/tags/pull-block-2022-02-01' into staging

Block patches:
- Add support to the iotests to test qcow2's zstd compression mode
- Fix post-migration block node permissions
- iotests fixes (051 and mirror-ready-cancel-error)
- Remove an outdated comment

# gpg: Signature made Tue 01 Feb 2022 13:34:54 GMT
# gpg: using RSA key CB62D7A0EE3829E45F004D34A1FA40D098019CDF
# gpg: issuer "hreitz@redhat.com"
# gpg: Good signature from "Hanna Reitz <hreitz@redhat.com>" [marginal]
# gpg: WARNING: This key is not certified with sufficiently trusted signatures!
# gpg: It is not certain that the signature belongs to the owner.
# Primary key fingerprint: CB62 D7A0 EE38 29E4 5F00 4D34 A1FA 40D0 9801 9CDF

* remotes/hreitz-gitlab/tags/pull-block-2022-02-01: (24 commits)
block.h: remove outdated comment
iotests/migration-permissions: New test
block-backend: Retain permissions after migration
iotests: declare lack of support for compresion_type in IMGOPTS
iotest 214: explicit compression type
iotests 60: more accurate set dirty bit in qcow2 header
iotests: bash tests: filter compression type
iotest 39: use _qcow2_dump_header
iotests: massive use _qcow2_dump_header
iotests/common.rc: introduce _qcow2_dump_header helper
qcow2: simple case support for downgrading of qcow2 images with zstd
iotest 302: use img_info_log() helper
iotests.py: filter compression type out
iotests.py: filter out successful output of qemu-img create
iotest 065: explicit compression type
iotest 303: explicit compression type
iotests.py: rewrite default luks support in qemu_img
iotests: drop qemu_img_verbose() helper
iotests.py: qemu_img*("create"): support IMGOPTS='compression_type=zstd'
iotests: specify some unsupported_imgopts for python iotests
...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

iotests: bash tests: filter compression type

We want iotests pass with both the default zlib compression and with
IMGOPTS='compression_type=zstd'.

Actually the only test that is interested in real

iotests: bash tests: filter compression type

We want iotests pass with both the default zlib compression and with
IMGOPTS='compression_type=zstd'.

Actually the only test that is interested in real compression type in
test output is 287 (test for qcow2 compression type), so implement
specific option for it.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Hanna Reitz <hreitz@redhat.com>
Message-Id: <20211223160144.1097696-17-vsementsov@virtuozzo.com>
Signed-off-by: Hanna Reitz <hreitz@redhat.com>

show more ...

Merge remote-tracking branch 'remotes/kevin/tags/for-upstream' into staging

Block layer patches

- Make blockdev-reopen stable
- Remove deprecated qemu-img backing file without format
- rbd: Convert

Merge remote-tracking branch 'remotes/kevin/tags/for-upstream' into staging

Block layer patches

- Make blockdev-reopen stable
- Remove deprecated qemu-img backing file without format
- rbd: Convert to coroutines and add write zeroes support
- rbd: Updated MAINTAINERS
- export/fuse: Allow other users access to the export
- vhost-user: Fix backends without multiqueue support
- Fix drive-backup transaction endless drained section

# gpg: Signature made Fri 09 Jul 2021 13:49:22 BST
# gpg: using RSA key DC3DEB159A9AF95D3D7456FE7F09B272C88F2FD6
# gpg: issuer "kwolf@redhat.com"
# gpg: Good signature from "Kevin Wolf <kwolf@redhat.com>" [full]
# Primary key fingerprint: DC3D EB15 9A9A F95D 3D74 56FE 7F09 B272 C88F 2FD6

* remotes/kevin/tags/for-upstream: (28 commits)
block: Make blockdev-reopen stable API
iotests: Test reopening multiple devices at the same time
block: Support multiple reopening with x-blockdev-reopen
block: Acquire AioContexts during bdrv_reopen_multiple()
block: Add bdrv_reopen_queue_free()
qcow2: Fix dangling pointer after reopen for 'file'
qemu-img: Improve error for rebase without backing format
qemu-img: Require -F with -b backing image
qcow2: Prohibit backing file changes in 'qemu-img amend'
blockdev: fix drive-backup transaction endless drained section
vhost-user: Fix backends without multiqueue support
MAINTAINERS: add block/rbd.c reviewer
block/rbd: fix type of task->complete
iotests/fuse-allow-other: Test allow-other
iotests/308: Test +w on read-only FUSE exports
export/fuse: Let permissions be adjustable
export/fuse: Give SET_ATTR_SIZE its own branch
export/fuse: Add allow-other option
export/fuse: Pass default_permissions for mount
util/uri: do not check argument of uri_free()
...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

qcow2: Prohibit backing file changes in 'qemu-img amend'

This was deprecated back in bc5ee6da7 (qcow2: Deprecate use of
qemu-img amend to change backing file), and no one in the meantime has
given a

qcow2: Prohibit backing file changes in 'qemu-img amend'

This was deprecated back in bc5ee6da7 (qcow2: Deprecate use of
qemu-img amend to change backing file), and no one in the meantime has
given any reasons why it should be supported. Time to make change
attempts a hard error (but for convenience, specifying the _same_
backing chain is not forbidden). Update a couple of iotests to match.

Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <20210503213600.569128-2-eblake@redhat.com>
Reviewed-by: Connor Kuehl <ckuehl@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>

show more ...

Merge remote-tracking branch 'remotes/maxreitz/tags/pull-block-2020-08-26' into staging

Block patches:
- qcow2 subclusters (extended L2 entries)

# gpg: Signature made Wed 26 Aug 2020 08:37:04 BST
#

Merge remote-tracking branch 'remotes/maxreitz/tags/pull-block-2020-08-26' into staging

Block patches:
- qcow2 subclusters (extended L2 entries)

# gpg: Signature made Wed 26 Aug 2020 08:37:04 BST
# gpg: using RSA key 91BEB60A30DB3E8857D11829F407DB0061D5CF40
# gpg: issuer "mreitz@redhat.com"
# gpg: Good signature from "Max Reitz <mreitz@redhat.com>" [full]
# Primary key fingerprint: 91BE B60A 30DB 3E88 57D1 1829 F407 DB00 61D5 CF40

* remotes/maxreitz/tags/pull-block-2020-08-26: (34 commits)
iotests: Add tests for qcow2 images with extended L2 entries
qcow2: Assert that expand_zero_clusters_in_l1() does not support subclusters
qcow2: Allow preallocation and backing files if extended_l2 is set
qcow2: Add the 'extended_l2' option and the QCOW2_INCOMPAT_EXTL2 bit
qcow2: Add prealloc field to QCowL2Meta
qcow2: Add subcluster support to qcow2_measure()
qcow2: Add subcluster support to qcow2_co_pwrite_zeroes()
qcow2: Add subcluster support to handle_alloc_space()
qcow2: Clear the L2 bitmap when allocating a compressed cluster
qcow2: Update L2 bitmap in qcow2_alloc_cluster_link_l2()
qcow2: Add subcluster support to check_refcounts_l2()
qcow2: Add subcluster support to discard_in_l2_slice()
qcow2: Add subcluster support to zero_in_l2_slice()
qcow2: Add subcluster support to qcow2_get_host_offset()
qcow2: Add subcluster support to calculate_l2_meta()
qcow2: Handle QCOW2_SUBCLUSTER_UNALLOCATED_ALLOC
qcow2: Replace QCOW2_CLUSTER_* with QCOW2_SUBCLUSTER_*
qcow2: Add cluster type parameter to qcow2_get_host_offset()
qcow2: Add qcow2_cluster_is_allocated()
qcow2: Add qcow2_get_subcluster_range_type()
...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

qcow2: Assert that expand_zero_clusters_in_l1() does not support subclusters

This function is only used by qcow2_expand_zero_clusters() to
downgrade a qcow2 image to a previous version. This would r

qcow2: Assert that expand_zero_clusters_in_l1() does not support subclusters

This function is only used by qcow2_expand_zero_clusters() to
downgrade a qcow2 image to a previous version. This would require
transforming all extended L2 entries into normal L2 entries but this
is not a simple task and there are no plans to implement this at the
moment.

Signed-off-by: Alberto Garcia <berto@igalia.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <15e65112b4144381b4d8c0bdf8fb76b0d813e3d1.1594396418.git.berto@igalia.com>
[mreitz: Fixed comment style]
Signed-off-by: Max Reitz <mreitz@redhat.com>

show more ...

qcow2: Add the 'extended_l2' option and the QCOW2_INCOMPAT_EXTL2 bit

Now that the implementation of subclusters is complete we can finally
add the necessary options to create and read images with th

qcow2: Add the 'extended_l2' option and the QCOW2_INCOMPAT_EXTL2 bit

Now that the implementation of subclusters is complete we can finally
add the necessary options to create and read images with this feature,
which we call "extended L2 entries".

Signed-off-by: Alberto Garcia <berto@igalia.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <6476caaa73216bd05b7bb2d504a20415e1665176.1594396418.git.berto@igalia.com>
[mreitz: %s/5\.1/5.2/; fixed 302's and 303's reference output]
Signed-off-by: Max Reitz <mreitz@redhat.com>

show more ...

Merge remote-tracking branch 'remotes/kevin/tags/for-upstream' into staging

Block layer patches:

- file-posix: Mitigate file fragmentation with extent size hints
- Tighten qemu-img rules on missing

Merge remote-tracking branch 'remotes/kevin/tags/for-upstream' into staging

Block layer patches:

- file-posix: Mitigate file fragmentation with extent size hints
- Tighten qemu-img rules on missing backing format
- qemu-img map: Don't limit block status request size
- Fix crash with virtio-scsi and iothreads

# gpg: Signature made Tue 14 Jul 2020 14:24:19 BST
# gpg: using RSA key DC3DEB159A9AF95D3D7456FE7F09B272C88F2FD6
# gpg: issuer "kwolf@redhat.com"
# gpg: Good signature from "Kevin Wolf <kwolf@redhat.com>" [full]
# Primary key fingerprint: DC3D EB15 9A9A F95D 3D74 56FE 7F09 B272 C88F 2FD6

* remotes/kevin/tags/for-upstream:
block: Avoid stale pointer dereference in blk_get_aio_context()
qemu-img: Deprecate use of -b without -F
block: Add support to warn on backing file change without format
iotests: Specify explicit backing format where sensible
qcow2: Deprecate use of qemu-img amend to change backing file
block: Error if backing file fails during creation without -u
qcow: Tolerate backing_fmt=
vmdk: Add trivial backing_fmt support
sheepdog: Add trivial backing_fmt support
block: Finish deprecation of 'qemu-img convert -n -o'
qemu-img: Flush stdout before before potential stderr messages
file-posix: Mitigate file fragmentation with extent size hints
iotests/059: Filter out disk size with more standard filter
qemu-img map: Don't limit block status request size
iotests: Simplify _filter_img_create() a bit

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

iotests: Specify explicit backing format where sensible

There are many existing qcow2 images that specify a backing file but
no format. This has been the source of CVEs in the past, but has
become

iotests: Specify explicit backing format where sensible

There are many existing qcow2 images that specify a backing file but
no format. This has been the source of CVEs in the past, but has
become more prominent of a problem now that libvirt has switched to
-blockdev. With older -drive, at least the probing was always done by
qemu (so the only risk of a changed format between successive boots of
a guest was if qemu was upgraded and probed differently). But with
newer -blockdev, libvirt must specify a format; if libvirt guesses raw
where the image was formatted, this results in data corruption visible
to the guest; conversely, if libvirt guesses qcow2 where qemu was
using raw, this can result in potential security holes, so modern
libvirt instead refuses to use images without explicit backing format.

The change in libvirt to reject images without explicit backing format
has pointed out that a number of tools have been far too reliant on
probing in the past. It's time to set a better example in our own
iotests of properly setting this parameter.

iotest calls to create, rebase, and convert are all impacted to some
degree. It's a bit annoying that we are inconsistent on command line
- while all of those accept -o backing_file=...,backing_fmt=..., the
shortcuts are different: create and rebase have -b and -F, while
convert has -B but no -F. (amend has no shortcuts, but the previous
patch just deprecated the use of amend to change backing chains).

Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <20200706203954.341758-9-eblake@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>

show more ...

qcow2: Deprecate use of qemu-img amend to change backing file

The use of 'qemu-img amend' to change qcow2 backing files is not
tested very well. In particular, our implementation has a bug where
if

qcow2: Deprecate use of qemu-img amend to change backing file

The use of 'qemu-img amend' to change qcow2 backing files is not
tested very well. In particular, our implementation has a bug where
if a new backing file is provided without a format, then the prior
format is blindly reused, even if this results in data corruption, but
this is not caught by iotests.

There are also situations where amending other options needs access to
the original backing file (for example, on a downgrade to a v2 image,
knowing whether a v3 zero cluster must be allocated or may be left
unallocated depends on knowing whether the backing file already reads
as zero), but the command line does not have a nice way to tell us
both the backing file to use for opening the image as well as the
backing file to install after the operation is complete.

Even if we do allow changing the backing file, it is redundant with
the existing ability to change backing files via 'qemu-img rebase -u'.
It is time to deprecate this support (leaving the existing behavior
intact, even if it is buggy), and at a point in the future, require
the use of only 'qemu-img rebase' for adjusting backing chain
relations, saving 'qemu-img amend' for changes unrelated to the
backing chain.

Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <20200706203954.341758-8-eblake@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>

show more ...

Merge remote-tracking branch 'remotes/maxreitz/tags/pull-block-2020-07-06' into staging

Block patches for 5.1:
- LUKS keyslot amendment
(+ patches to make the iotests pass on non-Linux systems, an

Merge remote-tracking branch 'remotes/maxreitz/tags/pull-block-2020-07-06' into staging

Block patches for 5.1:
- LUKS keyslot amendment
(+ patches to make the iotests pass on non-Linux systems, and to keep
the tests passing for qcow v1, and to skip LUKS tests (including
qcow2 LUKS) when the built qemu does not support it)
- Refactoring in the block layer: Drop the basically unnecessary
unallocated_blocks_are_zero field from BlockDriverInfo
- Fix qcow2 preallocation when the image size is not a multiple of the
cluster size
- Fix in block-copy code

# gpg: Signature made Mon 06 Jul 2020 11:02:53 BST
# gpg: using RSA key 91BEB60A30DB3E8857D11829F407DB0061D5CF40
# gpg: issuer "mreitz@redhat.com"
# gpg: Good signature from "Max Reitz <mreitz@redhat.com>" [full]
# Primary key fingerprint: 91BE B60A 30DB 3E88 57D1 1829 F407 DB00 61D5 CF40

* remotes/maxreitz/tags/pull-block-2020-07-06: (31 commits)
qed: Simplify backing reads
block: drop unallocated_blocks_are_zero
block/vhdx: drop unallocated_blocks_are_zero
block/file-posix: drop unallocated_blocks_are_zero
block/iscsi: drop unallocated_blocks_are_zero
block/crypto: drop unallocated_blocks_are_zero
block/vpc: return ZERO block-status when appropriate
block/vdi: return ZERO block-status when appropriate
block: inline bdrv_unallocated_blocks_are_zero()
qemu-img: convert: don't use unallocated_blocks_are_zero
iotests: add tests for blockdev-amend
block/qcow2: implement blockdev-amend
block/crypto: implement blockdev-amend
block/core: add generic infrastructure for x-blockdev-amend qmp command
iotests: qemu-img tests for luks key management
block/qcow2: extend qemu-img amend interface with crypto options
block/crypto: implement the encryption key management
block/crypto: rename two functions
block/amend: refactor qcow2 amend options
block/amend: separate amend and create options for qemu-img
...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

block/amend: refactor qcow2 amend options

Some qcow2 create options can't be used for amend.
Remove them from the qcow2 create options and add generic logic to detect
such options in qemu-img

Signe

block/amend: refactor qcow2 amend options

Some qcow2 create options can't be used for amend.
Remove them from the qcow2 create options and add generic logic to detect
such options in qemu-img

Signed-off-by: Maxim Levitsky <mlevitsk@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
[mreitz: Dropped some iotests reference output hunks that became
unnecessary thanks to
"iotests: Make _filter_img_create more active"]
Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20200625125548.870061-12-mreitz@redhat.com>

show more ...

Merge remote-tracking branch 'remotes/ericb/tags/pull-bitmaps-2020-06-09' into staging

bitmaps patches for 2020-06-09

- documenation fix
- various improvements to qcow2.py program used in iotests

Merge remote-tracking branch 'remotes/ericb/tags/pull-bitmaps-2020-06-09' into staging

bitmaps patches for 2020-06-09

- documenation fix
- various improvements to qcow2.py program used in iotests

# gpg: Signature made Tue 09 Jun 2020 21:50:35 BST
# gpg: using RSA key 71C2CC22B1C4602927D2F3AAA7A16B4A2527436A
# gpg: Good signature from "Eric Blake <eblake@redhat.com>" [full]
# gpg: aka "Eric Blake (Free Software Programmer) <ebb9@byu.net>" [full]
# gpg: aka "[jpeg image of size 6874]" [full]
# Primary key fingerprint: 71C2 CC22 B1C4 6029 27D2 F3AA A7A1 6B4A 2527 436A

* remotes/ericb/tags/pull-bitmaps-2020-06-09:
iotests: Fix 291 across more file systems
qcow2_format.py: dump bitmaps header extension
qcow2: QcowHeaderExtension print names for extension magics
qcow2_format: refactor QcowHeaderExtension as a subclass of Qcow2Struct
qcow2_format.py: QcowHeaderExtension: add dump method
qcow2_format.py: add field-formatting class
qcow2_format.py: separate generic functionality of structure classes
qcow2_format.py: use strings to specify c-type of struct fields
qcow2_format.py: use modern string formatting
qcow2_format.py: use tuples instead of lists for fields
qcow2_format.py: drop new line printing at end of dump()
qcow2.py: move qcow2 format classes to separate module
qcow2.py: add licensing blurb
qcow2.py: python style fixes
qemu-img: Fix doc typo for 'bitmap' subcommand

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

qcow2: QcowHeaderExtension print names for extension magics

Suggested-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com>
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
M

qcow2: QcowHeaderExtension print names for extension magics

Suggested-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com>
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20200606081806.23897-13-vsementsov@virtuozzo.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Eric Blake <eblake@redhat.com>

show more ...

Merge remote-tracking branch 'remotes/maxreitz/tags/pull-block-2020-05-13' into staging

Block patches:
- zstd compression for qcow2
- Fix use-after-free

# gpg: Signature made Wed 13 May 2020 15:14:

Merge remote-tracking branch 'remotes/maxreitz/tags/pull-block-2020-05-13' into staging

Block patches:
- zstd compression for qcow2
- Fix use-after-free

# gpg: Signature made Wed 13 May 2020 15:14:06 BST
# gpg: using RSA key 91BEB60A30DB3E8857D11829F407DB0061D5CF40
# gpg: issuer "mreitz@redhat.com"
# gpg: Good signature from "Max Reitz <mreitz@redhat.com>" [full]
# Primary key fingerprint: 91BE B60A 30DB 3E88 57D1 1829 F407 DB00 61D5 CF40

* remotes/maxreitz/tags/pull-block-2020-05-13:
block/block-copy: fix use-after-free of task pointer
iotests: 287: add qcow2 compression type test
qcow2: add zstd cluster compression
qcow2: rework the cluster compression routine
qcow2: introduce compression type feature

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

qcow2: introduce compression type feature

The patch adds some preparation parts for incompatible compression type
feature to qcow2 allowing the use different compression methods for
image clusters (

qcow2: introduce compression type feature

The patch adds some preparation parts for incompatible compression type
feature to qcow2 allowing the use different compression methods for
image clusters (de)compressing.

It is implied that the compression type is set on the image creation and
can be changed only later by image conversion, thus compression type
defines the only compression algorithm used for the image, and thus,
for all image clusters.

The goal of the feature is to add support of other compression methods
to qcow2. For example, ZSTD which is more effective on compression than ZLIB.

The default compression is ZLIB. Images created with ZLIB compression type
are backward compatible with older qemu versions.

Adding of the compression type breaks a number of tests because now the
compression type is reported on image creation and there are some changes
in the qcow2 header in size and offsets.

The tests are fixed in the following ways:
* filter out compression_type for many tests
* fix header size, feature table size and backing file offset
affected tests: 031, 036, 061, 080
header_size +=8: 1 byte compression type
7 bytes padding
feature_table += 48: incompatible feature compression type
backing_file_offset += 56 (8 + 48 -> header_change + feature_table_change)
* add "compression type" for test output matching when it isn't filtered
affected tests: 049, 060, 061, 065, 082, 085, 144, 182, 185, 198, 206,
242, 255, 274, 280

Signed-off-by: Denis Plotnikov <dplotnikov@virtuozzo.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
QAPI part:
Acked-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20200507082521.29210-2-dplotnikov@virtuozzo.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>

show more ...

Merge remote-tracking branch 'remotes/maxreitz/tags/pull-block-2020-05-05' into staging

Block patches:
- Asynchronous copying for block-copy (i.e., the backup job)
- Allow resizing of qcow2 images w

Merge remote-tracking branch 'remotes/maxreitz/tags/pull-block-2020-05-05' into staging

Block patches:
- Asynchronous copying for block-copy (i.e., the backup job)
- Allow resizing of qcow2 images when they have internal snapshots
- iotests: Logging improvements for Python tests
- iotest 153 fix, and block comment cleanups

# gpg: Signature made Tue 05 May 2020 13:56:58 BST
# gpg: using RSA key 91BEB60A30DB3E8857D11829F407DB0061D5CF40
# gpg: issuer "mreitz@redhat.com"
# gpg: Good signature from "Max Reitz <mreitz@redhat.com>" [full]
# Primary key fingerprint: 91BE B60A 30DB 3E88 57D1 1829 F407 DB00 61D5 CF40

* remotes/maxreitz/tags/pull-block-2020-05-05: (24 commits)
block/block-copy: use aio-task-pool API
block/block-copy: refactor task creation
block/block-copy: add state pointer to BlockCopyTask
block/block-copy: alloc task on each iteration
block/block-copy: rename in-flight requests to tasks
Fix iotest 153
block: Comment cleanups
qcow2: Tweak comment about bitmaps vs. resize
qcow2: Allow resize of images with internal snapshots
block: Add blk_new_with_bs() helper
iotests: use python logging for iotests.log()
iotests: Mark verify functions as private
iotest 258: use script_main
iotests: add script_initialize
iotests: add hmp helper with logging
iotests: limit line length to 79 chars
iotests: touch up log function signature
iotests: drop pre-Python 3.4 compatibility code
iotests: alphabetize standard imports
iotests: add pylintrc file
...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

qcow2: Allow resize of images with internal snapshots

We originally refused to allow resize of images with internal
snapshots because the v2 image format did not require the tracking of
snapshot siz

qcow2: Allow resize of images with internal snapshots

We originally refused to allow resize of images with internal
snapshots because the v2 image format did not require the tracking of
snapshot size, making it impossible to safely revert to a snapshot
with a different size than the current view of the image. But the
snapshot size tracking was rectified in v3, and our recent fixes to
qemu-img amend (see 0a85af35) guarantee that we always have a valid
snapshot size. Thus, we no longer need to artificially limit image
resizes, but it does become one more thing that would prevent a
downgrade back to v2. And now that we support different-sized
snapshots, it's also easy to fix reverting to a snapshot to apply the
new size.

Upgrade iotest 61 to cover this (we previously had NO coverage of
refusal to resize while snapshots exist). Note that the amend process
can fail but still have effects: in particular, since we break things
into upgrade, resize, downgrade, a failure during resize does not roll
back changes made during upgrade, nor does failure in downgrade roll
back a resize. But this situation is pre-existing even without this
patch; and without journaling, the best we could do is minimize the
chance of partial failure by collecting all changes prior to doing any
writes - which adds a lot of complexity but could still fail with EIO.
On the other hand, we are careful that even if we have partial
modification but then fail, the image is left viable (that is, we are
careful to sequence things so that after each successful cluster
write, there may be transient leaked clusters but no corrupt
metadata). And complicating the code to make it more transaction-like
is not worth the effort: a user can always request multiple 'qemu-img
amend' changing one thing each, if they need finer-grained control
over detecting the first failure than what they get by letting qemu
decide how to sequence multiple changes.

Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20200428192648.749066-3-eblake@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>

show more ...

Merge remote-tracking branch 'remotes/maxreitz/tags/pull-block-2020-03-26' into staging

Block patches for 5.0-rc1:
- Fix qemu-img convert with a host device or iscsi target
- Use-after-free fix in m

Merge remote-tracking branch 'remotes/maxreitz/tags/pull-block-2020-03-26' into staging

Block patches for 5.0-rc1:
- Fix qemu-img convert with a host device or iscsi target
- Use-after-free fix in mirror
- Some minor qcow2 fixes
- Minor sheepdog fix
- Minor qemu-img check report fix

# gpg: Signature made Thu 26 Mar 2020 14:28:26 GMT
# gpg: using RSA key 91BEB60A30DB3E8857D11829F407DB0061D5CF40
# gpg: issuer "mreitz@redhat.com"
# gpg: Good signature from "Max Reitz <mreitz@redhat.com>" [full]
# Primary key fingerprint: 91BE B60A 30DB 3E88 57D1 1829 F407 DB00 61D5 CF40

* remotes/maxreitz/tags/pull-block-2020-03-26:
iotests/138: Test leaks/corruptions fixed report
iotests: Add poke_file_[bl]e functions
qemu-img: Fix check's leak/corruption fix report
sheepdog: Consistently set bdrv_has_zero_init_truncate
qcow2: Avoid feature name extension on small cluster size
qcow2: List autoclear bit names in header
qcow2: Comment typo fixes
block: trickle down the fallback image creation function use to the block drivers
block: pass BlockDriver reference to the .bdrv_co_create
block/mirror: fix use after free of local_err

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

qcow2: List autoclear bit names in header

The feature table is supposed to advertise the name of all feature
bits that we support; however, we forgot to update the table for
autoclear bits. While a

qcow2: List autoclear bit names in header

The feature table is supposed to advertise the name of all feature
bits that we support; however, we forgot to update the table for
autoclear bits. While at it, move the table to read-only memory in
code, and tweak the qcow2 spec to name the second autoclear bit.
Update iotests that are affected by the longer header length.

Fixes: 88ddffae
Fixes: 93c24936
Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20200324174233.1622067-3-eblake@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>

show more ...

Merge remote-tracking branch 'remotes/maxreitz/tags/pull-block-2020-01-06' into staging

Block patches:
- Minor fixes and tests from the freeze period (too minor to be included
in 4.2)
- Allow many

Merge remote-tracking branch 'remotes/maxreitz/tags/pull-block-2020-01-06' into staging

Block patches:
- Minor fixes and tests from the freeze period (too minor to be included
in 4.2)
- Allow many bash iotests to test qcow2's external data file feature
- Add compress filter driver
- Fix Python iotests after 6f6e1698a6
- Fix for the backup job

# gpg: Signature made Mon 06 Jan 2020 14:33:06 GMT
# gpg: using RSA key 91BEB60A30DB3E8857D11829F407DB0061D5CF40
# gpg: issuer "mreitz@redhat.com"
# gpg: Good signature from "Max Reitz <mreitz@redhat.com>" [full]
# Primary key fingerprint: 91BE B60A 30DB 3E88 57D1 1829 F407 DB00 61D5 CF40

* remotes/maxreitz/tags/pull-block-2020-01-06: (34 commits)
backup-top: Begin drain earlier
tests/qemu-iotests: Update tests to recent desugarized -accel option
tests/qemu-iotests: add case to write compressed data of multiple clusters
qcow2: Allow writing compressed data of multiple clusters
block: introduce compress filter driver
iotests: Allow check -o data_file
iotests: Disable data_file where it cannot be used
iotests: Make 198 work with data_file
iotests: Make 137 work with data_file
iotests: Make 110 work with data_file
iotests: Make 091 work with data_file
iotests: Avoid cp/mv of test images
iotests: Use _rm_test_img for deleting test images
iotests: Avoid qemu-img create
iotests: Drop IMGOPTS use in 267
iotests: Replace IMGOPTS='' by --no-opts
iotests: Replace IMGOPTS= by -o
iotests: Inject space into -ocompat=0.10 in 051
iotests: Add -o and --no-opts to _make_test_img
iotests: Let _make_test_img parse its parameters
...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

iotests/qcow2.py: Split feature fields into bits

Print the feature fields as a set of bits so that filtering is easier.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Maxim Levitsky <mle

iotests/qcow2.py: Split feature fields into bits

Print the feature fields as a set of bits so that filtering is easier.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Maxim Levitsky <mlevitsk@redhat.com>
Message-id: 20191107163708.833192-4-mreitz@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>

show more ...

Merge remote-tracking branch 'remotes/kevin/tags/for-upstream' into staging

Block layer patches:

- qcow2: Allow overwriting multiple compressed clusters at once for
better performance
- nfs: add

Merge remote-tracking branch 'remotes/kevin/tags/for-upstream' into staging

Block layer patches:

- qcow2: Allow overwriting multiple compressed clusters at once for
better performance
- nfs: add support for nfs_umount
- file-posix: write_zeroes fixes
- qemu-io, blockdev-create, pr-manager: Fix crashes and memory leaks
- qcow2: Fix the calculation of the maximum L2 cache size
- vpc: Fix return code for vpc_co_create()
- blockjob: Code cleanup
- iotests improvements (e.g. for use with valgrind)

# gpg: Signature made Fri 13 Sep 2019 11:19:19 BST
# gpg: using RSA key 7F09B272C88F2FD6
# gpg: Good signature from "Kevin Wolf <kwolf@redhat.com>" [full]
# Primary key fingerprint: DC3D EB15 9A9A F95D 3D74 56FE 7F09 B272 C88F 2FD6

* remotes/kevin/tags/for-upstream: (23 commits)
qcow2: Stop overwriting compressed clusters one by one
block/create: Do not abort if a block driver is not available
qemu-io: Don't leak pattern file in error path
iotests: extend sleeping time under Valgrind
iotests: extended timeout under Valgrind
iotests: Valgrind fails with nonexistent directory
iotests: Add casenotrun report to bash tests
iotests: exclude killed processes from running under Valgrind
iotests: allow Valgrind checking all QEMU processes
block/nfs: add support for nfs_umount
block/nfs: tear down aio before nfs_close
iotests: skip 232 when run tests as root
iotests: Test blockdev-create for vpc
iotests: Restrict nbd Python tests to nbd
iotests: Restrict file Python tests to file
iotests: Add supported protocols to execute_test()
vpc: Return 0 from vpc_co_create() on success
file-posix: Fix has_write_zeroes after NO_FALLBACK
pr-manager: Fix invalid g_free() crash bug
iotests: Test reverse sub-cluster qcow2 writes
...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...