Merge remote-tracking branch 'remotes/maxreitz/tags/pull-block-2021-03-30' into staging

Block patches for 6.0-rc1:
- Mark the qcow2 cache clean timer as external to fix record/replay
- Fix the mirro

Merge remote-tracking branch 'remotes/maxreitz/tags/pull-block-2021-03-30' into staging

Block patches for 6.0-rc1:
- Mark the qcow2 cache clean timer as external to fix record/replay
- Fix the mirror filter node's permissions so that an external process
cannot grab an image while it is used as the mirror source
- Add documentation about FUSE exports to the storage daemon
- When creating a qcow2 image with the data-file-raw option, all
metadata structures should be preallocated
- iotest fixes

# gpg: Signature made Tue 30 Mar 2021 13:38:40 BST
# gpg: using RSA key 91BEB60A30DB3E8857D11829F407DB0061D5CF40
# gpg: issuer "mreitz@redhat.com"
# gpg: Good signature from "Max Reitz <mreitz@redhat.com>" [full]
# Primary key fingerprint: 91BE B60A 30DB 3E88 57D1 1829 F407 DB00 61D5 CF40

* remotes/maxreitz/tags/pull-block-2021-03-30:
iotests/244: Test preallocation for data-file-raw
qcow2: Force preallocation with data-file-raw
qsd: Document FUSE exports
block/mirror: Fix mirror_top's permissions
iotests/046: Filter request length
qcow2: use external virtual timers
iotests/116: Fix reference output
iotests: fix 051.out expected output after error text touchups
iotests: Fix typo in iotest 051

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

iotests/046: Filter request length

For its concurrent requests, 046 has always filtered the offset,
probably because concurrent requests may settle in any order. However,
it did not filter the requ

iotests/046: Filter request length

For its concurrent requests, 046 has always filtered the offset,
probably because concurrent requests may settle in any order. However,
it did not filter the request length, and so if requests with different
lengths settle in an unexpected order (notably the longer request before
the shorter request), the test fails (for no good reason).

Filter the length, too.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20200918153323.108932-1-mreitz@redhat.com>

show more ...

Merge remote-tracking branch 'remotes/kevin/tags/for-upstream' into staging

Block layer patches:

- Support for FUSE exports
- Fix deadlock in bdrv_co_yield_to_drain()
- Use lock guard macros
- Some

Merge remote-tracking branch 'remotes/kevin/tags/for-upstream' into staging

Block layer patches:

- Support for FUSE exports
- Fix deadlock in bdrv_co_yield_to_drain()
- Use lock guard macros
- Some preparational patches for 64 bit block layer
- file-posix: Fix request extension to INT64_MAX in raw_do_pwrite_zeroes()

# gpg: Signature made Fri 11 Dec 2020 17:06:19 GMT
# gpg: using RSA key DC3DEB159A9AF95D3D7456FE7F09B272C88F2FD6
# gpg: issuer "kwolf@redhat.com"
# gpg: Good signature from "Kevin Wolf <kwolf@redhat.com>" [full]
# Primary key fingerprint: DC3D EB15 9A9A F95D 3D74 56FE 7F09 B272 C88F 2FD6

* remotes/kevin/tags/for-upstream: (34 commits)
block: Fix deadlock in bdrv_co_yield_to_drain()
block: Fix locking in qmp_block_resize()
block: Simplify qmp_block_resize() error paths
block: introduce BDRV_MAX_LENGTH
block/io: bdrv_check_byte_request(): drop bdrv_is_inserted()
block/io: bdrv_refresh_limits(): use ERRP_GUARD
block/file-posix: fix workaround in raw_do_pwrite_zeroes()
can-host: Fix crash when 'canbus' property is not set
iotests/221: Discard image before qemu-img map
file-posix: check the use_lock before setting the file lock
iotests/308: Add test for FUSE exports
iotests: Enable fuse for many tests
iotests: Allow testing FUSE exports
iotests: Give access to the qemu-storage-daemon
storage-daemon: Call bdrv_close_all() on exit
iotests/287: Clean up subshell test image
iotests: Let _make_test_img guess $TEST_IMG_FILE
iotests: Restrict some Python tests to file
iotests/091: Use _cleanup_qemu instad of "wait"
iotests: Derive image names from $TEST_IMG
...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

iotests/046: Avoid renaming images

This generally does not work on non-file protocols. It is better to
create the image with the final name from the start, and most tests do
this already. Let 046

iotests/046: Avoid renaming images

This generally does not work on non-file protocols. It is better to
create the image with the final name from the start, and most tests do
this already. Let 046 follow suit.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Message-Id: <20201027190600.192171-11-mreitz@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>

show more ...

Merge remote-tracking branch 'remotes/kevin/tags/for-upstream' into staging

Block layer patches:

- file-posix: Mitigate file fragmentation with extent size hints
- Tighten qemu-img rules on missing

Merge remote-tracking branch 'remotes/kevin/tags/for-upstream' into staging

Block layer patches:

- file-posix: Mitigate file fragmentation with extent size hints
- Tighten qemu-img rules on missing backing format
- qemu-img map: Don't limit block status request size
- Fix crash with virtio-scsi and iothreads

# gpg: Signature made Tue 14 Jul 2020 14:24:19 BST
# gpg: using RSA key DC3DEB159A9AF95D3D7456FE7F09B272C88F2FD6
# gpg: issuer "kwolf@redhat.com"
# gpg: Good signature from "Kevin Wolf <kwolf@redhat.com>" [full]
# Primary key fingerprint: DC3D EB15 9A9A F95D 3D74 56FE 7F09 B272 C88F 2FD6

* remotes/kevin/tags/for-upstream:
block: Avoid stale pointer dereference in blk_get_aio_context()
qemu-img: Deprecate use of -b without -F
block: Add support to warn on backing file change without format
iotests: Specify explicit backing format where sensible
qcow2: Deprecate use of qemu-img amend to change backing file
block: Error if backing file fails during creation without -u
qcow: Tolerate backing_fmt=
vmdk: Add trivial backing_fmt support
sheepdog: Add trivial backing_fmt support
block: Finish deprecation of 'qemu-img convert -n -o'
qemu-img: Flush stdout before before potential stderr messages
file-posix: Mitigate file fragmentation with extent size hints
iotests/059: Filter out disk size with more standard filter
qemu-img map: Don't limit block status request size
iotests: Simplify _filter_img_create() a bit

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

iotests: Specify explicit backing format where sensible

There are many existing qcow2 images that specify a backing file but
no format. This has been the source of CVEs in the past, but has
become

iotests: Specify explicit backing format where sensible

There are many existing qcow2 images that specify a backing file but
no format. This has been the source of CVEs in the past, but has
become more prominent of a problem now that libvirt has switched to
-blockdev. With older -drive, at least the probing was always done by
qemu (so the only risk of a changed format between successive boots of
a guest was if qemu was upgraded and probed differently). But with
newer -blockdev, libvirt must specify a format; if libvirt guesses raw
where the image was formatted, this results in data corruption visible
to the guest; conversely, if libvirt guesses qcow2 where qemu was
using raw, this can result in potential security holes, so modern
libvirt instead refuses to use images without explicit backing format.

The change in libvirt to reject images without explicit backing format
has pointed out that a number of tools have been far too reliant on
probing in the past. It's time to set a better example in our own
iotests of properly setting this parameter.

iotest calls to create, rebase, and convert are all impacted to some
degree. It's a bit annoying that we are inconsistent on command line
- while all of those accept -o backing_file=...,backing_fmt=..., the
shortcuts are different: create and rebase have -b and -F, while
convert has -B but no -F. (amend has no shortcuts, but the previous
patch just deprecated the use of amend to change backing chains).

Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <20200706203954.341758-9-eblake@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>

show more ...

Merge remote-tracking branch 'remotes/maxreitz/tags/pull-block-2020-04-07' into staging

Block patches for 5.0-rc2:
- Fix double QLIST_REMOVE() and potential request object leak in
xen-block
- Prev

Merge remote-tracking branch 'remotes/maxreitz/tags/pull-block-2020-04-07' into staging

Block patches for 5.0-rc2:
- Fix double QLIST_REMOVE() and potential request object leak in
xen-block
- Prevent a potential assertion failure in qcow2's code for compressed
clusters by rejecting invalid (unaligned) requests with -EIO
- Prevent discards on qcow2 v2 images from making backing data reappear
- Make qemu-img convert report I/O error locations by byte offsets
consistently
- Fix for potential I/O test errors (accidental globbing due to missing
quotes)

# gpg: Signature made Tue 07 Apr 2020 13:30:01 BST
# gpg: using RSA key 91BEB60A30DB3E8857D11829F407DB0061D5CF40
# gpg: issuer "mreitz@redhat.com"
# gpg: Good signature from "Max Reitz <mreitz@redhat.com>" [full]
# Primary key fingerprint: 91BE B60A 30DB 3E88 57D1 1829 F407 DB00 61D5 CF40

* remotes/maxreitz/tags/pull-block-2020-04-07:
xen-block: Fix double qlist remove and request leak
iotests/common.pattern: Quote echos
qcow2: Check request size in qcow2_co_pwritev_compressed_part()
qemu-img: Report convert errors by bytes, not sectors
qcow2: Forbid discard in qcow2 v2 images with backing files

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

qcow2: Forbid discard in qcow2 v2 images with backing files

A discard request deallocates the selected clusters so they read back
as zeroes. This is done by clearing the cluster offset field and
set

qcow2: Forbid discard in qcow2 v2 images with backing files

A discard request deallocates the selected clusters so they read back
as zeroes. This is done by clearing the cluster offset field and
setting QCOW_OFLAG_ZERO in the L2 entry.

This flag is however only supported when qcow_version >= 3. In older
images the cluster is simply deallocated, exposing any possible stale
data from the backing file.

Since discard is an advisory operation it's safer to simply forbid it
in this scenario.

Note that we are adding this check to qcow2_co_pdiscard() and not to
qcow2_cluster_discard() or discard_in_l2_slice() because the last
two are also used by qcow2_snapshot_create() to discard the clusters
used by the VM state. In this case there's no risk of exposing stale
data to the guest and we really want that the clusters are always
discarded.

Signed-off-by: Alberto Garcia <berto@igalia.com>
Message-Id: <20200331114345.29993-1-berto@igalia.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>

show more ...

Merge remote-tracking branch 'remotes/mjt/tags/pull-trivial-patches-2015-09-11' into staging

trivial patches for 2015-09-11

# gpg: Signature made Fri 11 Sep 2015 12:02:43 BST using RSA key ID A4C3D

Merge remote-tracking branch 'remotes/mjt/tags/pull-trivial-patches-2015-09-11' into staging

trivial patches for 2015-09-11

# gpg: Signature made Fri 11 Sep 2015 12:02:43 BST using RSA key ID A4C3D7DB
# gpg: Good signature from "Michael Tokarev <mjt@tls.msk.ru>"
# gpg: aka "Michael Tokarev <mjt@corpit.ru>"
# gpg: aka "Michael Tokarev <mjt@debian.org>"

* remotes/mjt/tags/pull-trivial-patches-2015-09-11: (26 commits)
virtio-vga: enable for i386
hw/arm/spitz: Remove meaningless blank Property
hw/gpio/zaurus: Remove meaningless blank Property
hw/virtio/virtio-pci: Remove meaningless blank Property
hw/s390x/s390-virtio-bus: Remove meaningless blank Property
typofixes - v4
qapi-schema: remove legacy<> from doc
disas/microblaze: Remove unused code
help: dd missing newline
Target-ppc: Remove unnecessary variable
baum: Fix build with debugging enabled
linux-user: Fix warnings caused by missing 'static' attribute
opts: produce valid command line in qemu_opts_print
docs: fix a qga/qapi-schema.json comment
trivial: remove trailing newline from error_report
maint: avoid useless "if (foo) free(foo)" pattern
maint: avoid useless "if (foo) free(foo)" pattern
maint: remove unused include for strings.h
maint: remove unused include for signal.h
maint: remove unused include for dirent.h
...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

opts: produce valid command line in qemu_opts_print

This will let us print options in a format that the user would actually
write it on the command line (foo=bar,baz=asd,etc=def), without
prepending

opts: produce valid command line in qemu_opts_print

This will let us print options in a format that the user would actually
write it on the command line (foo=bar,baz=asd,etc=def), without
prepending a spurious comma at the beginning of the list, or quoting
values unnecessarily. This patch provides the following changes:
* write and id=, if the option has an id
* do not print separator before the first element
* do not quote string arguments
* properly escape commas (,) for QEMU

Signed-off-by: Kővágó, Zoltán <DirtY.iCE.hu@gmail.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Signed-off-by: Michael Tokarev <mjt@tls.msk.ru>

show more ...

Merge remote-tracking branch 'remotes/stefanha/tags/block-pull-request' into staging

# gpg: Signature made Fri 12 Dec 2014 17:09:56 GMT using RSA key ID 81AB73C8
# gpg: Good signature from "Stefan H

Merge remote-tracking branch 'remotes/stefanha/tags/block-pull-request' into staging

# gpg: Signature made Fri 12 Dec 2014 17:09:56 GMT using RSA key ID 81AB73C8
# gpg: Good signature from "Stefan Hajnoczi <stefanha@redhat.com>"
# gpg: aka "Stefan Hajnoczi <stefanha@gmail.com>"

* remotes/stefanha/tags/block-pull-request:
linux-aio: simplify removal of completed iocbs from the list
linux-aio: drop return code from laio_io_unplug and ioq_submit
linux-aio: rename LaioQueue idx field to "n"
linux-aio: track whether the queue is blocked
linux-aio: queue requests that cannot be submitted
block: drop unused bdrv_clear_incoming_migration_all() prototype
block: Don't add trailing space in "Formating..." message
qemu-iotests: Remove traling whitespaces in *.out
block: vhdx - set .bdrv_has_zero_init to bdrv_has_zero_init_1
iotests: Fix test 039
iotests: Filter for "Killed" in qemu-io output
qemu-io: Add sigraise command
block: vhdx - change .vhdx_create default block state to ZERO
block: vhdx - update PAYLOAD_BLOCK_UNMAPPED value to match 1.00 spec
block: vhdx - remove redundant comments
block/rbd: fix memory leak
iotests: Add test for vmdk JSON file names
vmdk: Fix error for JSON descriptor file names
block migration: fix return value

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

qemu-iotests: Remove traling whitespaces in *.out

This is simply:

$ cd tests/qemu-iotests; sed -i -e 's/ *$//' *.out

Signed-off-by: Fam Zheng <famz@redhat.com>
Reviewed-by: Eric Blake <eblake@re

qemu-iotests: Remove traling whitespaces in *.out

This is simply:

$ cd tests/qemu-iotests; sed -i -e 's/ *$//' *.out

Signed-off-by: Fam Zheng <famz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Message-id: 1418110684-19528-2-git-send-email-famz@redhat.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>

show more ...

Merge remote-tracking branch 'kwolf/tags/for-anthony' into staging

Block patches for 2.0 (flushing block-next)

# gpg: Signature made Fri 29 Nov 2013 08:43:18 AM PST using RSA key ID C88F2FD6
# gpg:

Merge remote-tracking branch 'kwolf/tags/for-anthony' into staging

Block patches for 2.0 (flushing block-next)

# gpg: Signature made Fri 29 Nov 2013 08:43:18 AM PST using RSA key ID C88F2FD6
# gpg: Can't check signature: public key not found

# By Peter Lieven (17) and others
# Via Kevin Wolf
* kwolf/tags/for-anthony: (41 commits)
qemu-iotests: Add sample image and test for VMDK version 3
vmdk: Allow read only open of VMDK version 3
qemu-iotests: Filter out 'qemu-io> ' prompt
qemu-iotests: Filter qemu-io output in 025
block: Use BDRV_O_NO_BACKING where appropriate
qemu-iotests: Test snapshot mode
block: Enable BDRV_O_SNAPSHOT with driver-specific options
qemu-iotests: Make test case 030, 040 and 055 deterministic
qemu-iotest: Add pause_drive and resume_drive methods
blkdebug: add "remove_break" command
qemu-iotests: Drop local version of cancel_and_wait from 040
sheepdog: support user-defined redundancy option
sheepdog: refactor do_sd_create()
qdict: Optimise qdict_do_flatten()
qdict: Fix memory leak in qdict_do_flatten()
MAINTAINERS: add sheepdog development mailing list
COW: Extend checking allocated bits to beyond one sector
COW: Speed up writes
qapi: Change BlockDirtyInfo to list
block: per caller dirty bitmap
...

Message-id: 1385743555-27888-1-git-send-email-kwolf@redhat.com
Signed-off-by: Anthony Liguori <aliguori@amazon.com>

show more ...

qemu-iotests: Filter out 'qemu-io> ' prompt

This removes "qemu-io> " prompt from qemu-io output in _filter_qemu_io,
and updates all the output files with the following command:

cd tests/qemu-iotes

qemu-iotests: Filter out 'qemu-io> ' prompt

This removes "qemu-io> " prompt from qemu-io output in _filter_qemu_io,
and updates all the output files with the following command:

cd tests/qemu-iotests && sed -i "s/qemu-io> //g" *.out

Signed-off-by: Fam Zheng <famz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>

show more ...

Merge remote-tracking branch 'stefanha/block' into staging

# By Kevin Wolf (22) and Peter Lieven (1)
# Via Stefan Hajnoczi
* stefanha/block: (23 commits)
block: Fix direct use of protocols as driv

Merge remote-tracking branch 'stefanha/block' into staging

# By Kevin Wolf (22) and Peter Lieven (1)
# Via Stefan Hajnoczi
* stefanha/block: (23 commits)
block: Fix direct use of protocols as driver for bdrv_open()
qcow2: Gather clusters in a looping loop
qcow2: Move cluster gathering to a non-looping loop
qcow2: Allow requests with multiple l2metas
qcow2: Use byte granularity in qcow2_alloc_cluster_offset()
qcow2: Prepare handle_alloc/copied() for byte granularity
qcow2: handle_copied(): Implement non-zero host_offset
qcow2: handle_copied(): Get rid of keep_clusters parameter
qcow2: handle_copied(): Get rid of nb_clusters parameter
qcow2: Factor out handle_copied()
qcow2: Clean up handle_alloc()
qcow2: Finalise interface of handle_alloc()
qcow2: handle_alloc(): Get rid of keep_clusters parameter
qcow2: handle_alloc(): Get rid of nb_clusters parameter
qcow2: Factor out handle_alloc()
qcow2: Decouple cluster allocation from cluster reuse code
qcow2: Change handle_dependency to byte granularity
qcow2: Improve check for overlapping allocations
qcow2: Handle dependencies earlier
qcow2: Remove bogus unlock of s->lock
...

show more ...

qemu-iotests: More concurrent allocation scenarios

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>

Merge branch 'master' of git://git.qemu.org/qemu into prep-up

Conflicts:
hw/Makefile.objs
hw/ppc_prep.c

Signed-off-by: Andreas Färber <andreas.faerber@web.de>

Merge remote-tracking branch 'kwolf/for-anthony' into staging

* kwolf/for-anthony: (43 commits)
qcow2: Factor out handle_dependencies()
qcow2: Execute run_dependent_requests() without lock
qco

Merge remote-tracking branch 'kwolf/for-anthony' into staging

* kwolf/for-anthony: (43 commits)
qcow2: Factor out handle_dependencies()
qcow2: Execute run_dependent_requests() without lock
qcow2: Enable dirty flag in qcow2_alloc_cluster_link_l2
qcow2: Allocate l2meta only for cluster allocations
qcow2: Drop l2meta.cluster_offset
qcow2: Allocate l2meta dynamically
qcow2: Introduce Qcow2COWRegion
qcow2: Round QCowL2Meta.offset down to cluster boundary
atapi: reset cdrom tray statuses on ide_reset
qemu-iotests: Test concurrent cluster allocations
qcow2: Move BLKDBG_EVENT out of the lock
qemu-io: Add AIO debugging commands
blkdebug: Implement suspend/resume of AIO requests
blkdebug: Factor out remove_rule()
blkdebug: Allow usage without config file
create new function: qemu_opt_set_number
use qemu_opts_create_nofail
introduce qemu_opts_create_nofail function
qemu-option: qemu_opt_set_bool(): fix code duplication
qemu-option: qemu_opts_validate(): fix duplicated code
...

Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>

show more ...

qemu-iotests: Test concurrent cluster allocations

This adds some first tests for qcow2's dependency handling when two
parallel write requests access the same cluster.

Signed-off-by: Kevin Wolf <kwo

qemu-iotests: Test concurrent cluster allocations

This adds some first tests for qcow2's dependency handling when two
parallel write requests access the same cluster.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>

show more ...