Merge remote-tracking branch 'remotes/kevin/tags/for-upstream' into staging

Block layer patches:

- file-posix: Mitigate file fragmentation with extent size hints
- Tighten qemu-img rules on missing

Merge remote-tracking branch 'remotes/kevin/tags/for-upstream' into staging

Block layer patches:

- file-posix: Mitigate file fragmentation with extent size hints
- Tighten qemu-img rules on missing backing format
- qemu-img map: Don't limit block status request size
- Fix crash with virtio-scsi and iothreads

# gpg: Signature made Tue 14 Jul 2020 14:24:19 BST
# gpg: using RSA key DC3DEB159A9AF95D3D7456FE7F09B272C88F2FD6
# gpg: issuer "kwolf@redhat.com"
# gpg: Good signature from "Kevin Wolf <kwolf@redhat.com>" [full]
# Primary key fingerprint: DC3D EB15 9A9A F95D 3D74 56FE 7F09 B272 C88F 2FD6

* remotes/kevin/tags/for-upstream:
block: Avoid stale pointer dereference in blk_get_aio_context()
qemu-img: Deprecate use of -b without -F
block: Add support to warn on backing file change without format
iotests: Specify explicit backing format where sensible
qcow2: Deprecate use of qemu-img amend to change backing file
block: Error if backing file fails during creation without -u
qcow: Tolerate backing_fmt=
vmdk: Add trivial backing_fmt support
sheepdog: Add trivial backing_fmt support
block: Finish deprecation of 'qemu-img convert -n -o'
qemu-img: Flush stdout before before potential stderr messages
file-posix: Mitigate file fragmentation with extent size hints
iotests/059: Filter out disk size with more standard filter
qemu-img map: Don't limit block status request size
iotests: Simplify _filter_img_create() a bit

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

iotests: Specify explicit backing format where sensible

There are many existing qcow2 images that specify a backing file but
no format. This has been the source of CVEs in the past, but has
become

iotests: Specify explicit backing format where sensible

There are many existing qcow2 images that specify a backing file but
no format. This has been the source of CVEs in the past, but has
become more prominent of a problem now that libvirt has switched to
-blockdev. With older -drive, at least the probing was always done by
qemu (so the only risk of a changed format between successive boots of
a guest was if qemu was upgraded and probed differently). But with
newer -blockdev, libvirt must specify a format; if libvirt guesses raw
where the image was formatted, this results in data corruption visible
to the guest; conversely, if libvirt guesses qcow2 where qemu was
using raw, this can result in potential security holes, so modern
libvirt instead refuses to use images without explicit backing format.

The change in libvirt to reject images without explicit backing format
has pointed out that a number of tools have been far too reliant on
probing in the past. It's time to set a better example in our own
iotests of properly setting this parameter.

iotest calls to create, rebase, and convert are all impacted to some
degree. It's a bit annoying that we are inconsistent on command line
- while all of those accept -o backing_file=...,backing_fmt=..., the
shortcuts are different: create and rebase have -b and -F, while
convert has -B but no -F. (amend has no shortcuts, but the previous
patch just deprecated the use of amend to change backing chains).

Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <20200706203954.341758-9-eblake@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>

show more ...

Merge remote-tracking branch 'remotes/kevin/tags/for-upstream' into staging

Block layer patches:

- iotests: Fix output of qemu-io related tests
- Don't ignore bdrv_set_aio_context() for nodes with

Merge remote-tracking branch 'remotes/kevin/tags/for-upstream' into staging

Block layer patches:

- iotests: Fix output of qemu-io related tests
- Don't ignore bdrv_set_aio_context() for nodes with bs->drv = NUL
- vmdk: Set vmdk parent backing_format to vmdk
- qcow2: Preallocation fixes (especially for external data files)
- Add linear-buffer-based APIs (as wrappers around qiov-based ones)
- Various code cleanups and small corner case fixes

# gpg: Signature made Tue 30 Apr 2019 16:35:09 BST
# gpg: using RSA key 7F09B272C88F2FD6
# gpg: Good signature from "Kevin Wolf <kwolf@redhat.com>" [full]
# Primary key fingerprint: DC3D EB15 9A9A F95D 3D74 56FE 7F09 B272 C88F 2FD6

* remotes/kevin/tags/for-upstream: (27 commits)
block/qed: add missed coroutine_fn markers
iotests: Check that images are in read-only mode after block-commit
commit: Make base read-only if there is an early failure
qemu-img: use buffer-based io
block/stream: use buffer-based io
block/commit: use buffer-based io
block/backup: use buffer-based io
block/parallels: use buffer-based io
block/qed: use buffer-based io
block/qcow: use buffer-based io
block/qcow2: use buffer-based io
block: introduce byte-based io helpers
qcow2: Fix error handling in the compression code
qcow2: Fix qcow2_make_empty() with external data file
qemu-img: Make create hint at protocol options
iotests: Perform the correct test in 082
qcow2: Fix full preallocation with external data file
qcow2: Add errp to preallocate_co()
qcow2: Avoid COW during metadata preallocation
qemu-img: Saner printing of large file sizes
...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

qemu-img: Saner printing of large file sizes

Disk sizes close to INT64_MAX cause overflow, for some pretty
ridiculous output:

$ ./nbdkit -U - memory size=$((2**63 - 512)) --run 'qemu-img info $nb

qemu-img: Saner printing of large file sizes

Disk sizes close to INT64_MAX cause overflow, for some pretty
ridiculous output:

$ ./nbdkit -U - memory size=$((2**63 - 512)) --run 'qemu-img info $nbd'
image: nbd+unix://?socket=/tmp/nbdkitHSAzNz/socket
file format: raw
virtual size: -8388607T (9223372036854775296 bytes)
disk size: unavailable

But there's no reason to have two separate implementations of integer
to human-readable abbreviation, where one has overflow and stops at
'T', while the other avoids overflow and goes all the way to 'E'. With
this patch, the output now claims 8EiB instead of -8388607T, which
really is the correct rounding of largest file size supported by qemu
(we could go 511 bytes larger if we used byte-accurate sizing instead
of rounding up to the next sector boundary, but that wouldn't change
the human-readable result).

Quite a few iotests need updates to expected output to match.

Reported-by: Richard W.M. Jones <rjones@redhat.com>
Signed-off-by: Eric Blake <eblake@redhat.com>
Tested-by: Richard W.M. Jones <rjones@redhat.com>
Reviewed-by: Alberto Garcia <berto@igalia.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Tested-by: Max Reitz <mreitz@redhat.com>

show more ...

Merge remote-tracking branch 'remotes/kevin/tags/for-upstream' into staging

Block layer patches

# gpg: Signature made Fri 18 Dec 2015 13:41:03 GMT using RSA key ID C88F2FD6
# gpg: Good signature fr

Merge remote-tracking branch 'remotes/kevin/tags/for-upstream' into staging

Block layer patches

# gpg: Signature made Fri 18 Dec 2015 13:41:03 GMT using RSA key ID C88F2FD6
# gpg: Good signature from "Kevin Wolf <kwolf@redhat.com>"

* remotes/kevin/tags/for-upstream: (48 commits)
block/qapi: allow best-effort query
qemu-img: abort when full_backing_filename not present
block/qapi: explicitly warn if !has_full_backing_filename
block/qapi: always report full_backing_filename
block/qapi: do not redundantly print "actual path"
qemu-iotests: s390x: fix test 068
qemu-iotests: s390x: fix test 051
qemu-iotests: refine common.config
block: fix bdrv_ioctl called from coroutine
block: use drained section around bdrv_snapshot_delete
iotests: Update comments for bdrv_swap() in 094
block: Remove prototype of bdrv_swap from header
raw-posix: Make aio=native option binding
qcow2: insert assert into qcow2_get_specific_info()
iotests: Extend test 112 for qemu-img amend
qcow2: Point to amend function in check
qcow2: Invoke refcount order amendment function
qcow2: Add function for refcount order amendment
qcow2: Use intermediate helper CB for amend
qcow2: Split upgrade/downgrade paths for amend
...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

Merge remote-tracking branch 'mreitz/tags/pull-block-for-kevin-2015-12-18' into queue-block

block-next patches from before the 2.5.0 release.

# gpg: Signature made Fri Dec 18 14:38:44 2015 CET usin

Merge remote-tracking branch 'mreitz/tags/pull-block-for-kevin-2015-12-18' into queue-block

block-next patches from before the 2.5.0 release.

# gpg: Signature made Fri Dec 18 14:38:44 2015 CET using RSA key ID E838ACAD
# gpg: Good signature from "Max Reitz <mreitz@redhat.com>"

* mreitz/tags/pull-block-for-kevin-2015-12-18:
block/qapi: allow best-effort query
qemu-img: abort when full_backing_filename not present
block/qapi: explicitly warn if !has_full_backing_filename
block/qapi: always report full_backing_filename
block/qapi: do not redundantly print "actual path"
qemu-iotests: s390x: fix test 068
qemu-iotests: s390x: fix test 051
qemu-iotests: refine common.config

Signed-off-by: Kevin Wolf <kwolf@redhat.com>

show more ...

block/qapi: always report full_backing_filename

Always report full_backing_filename, even if it's the same as
backing_filename. In the next patch, full_backing_filename may be
omitted if it cannot b

block/qapi: always report full_backing_filename

Always report full_backing_filename, even if it's the same as
backing_filename. In the next patch, full_backing_filename may be
omitted if it cannot be generated instead of allowing e.g. drive_query
to abort if it runs into this scenario.

The presence or absence of the "full" field becomes useful information.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-id: 1450122916-4706-3-git-send-email-jsnow@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>

show more ...

Merge remote-tracking branch 'remotes/mjt/tags/pull-trivial-patches-2015-09-11' into staging

trivial patches for 2015-09-11

# gpg: Signature made Fri 11 Sep 2015 12:02:43 BST using RSA key ID A4C3D

Merge remote-tracking branch 'remotes/mjt/tags/pull-trivial-patches-2015-09-11' into staging

trivial patches for 2015-09-11

# gpg: Signature made Fri 11 Sep 2015 12:02:43 BST using RSA key ID A4C3D7DB
# gpg: Good signature from "Michael Tokarev <mjt@tls.msk.ru>"
# gpg: aka "Michael Tokarev <mjt@corpit.ru>"
# gpg: aka "Michael Tokarev <mjt@debian.org>"

* remotes/mjt/tags/pull-trivial-patches-2015-09-11: (26 commits)
virtio-vga: enable for i386
hw/arm/spitz: Remove meaningless blank Property
hw/gpio/zaurus: Remove meaningless blank Property
hw/virtio/virtio-pci: Remove meaningless blank Property
hw/s390x/s390-virtio-bus: Remove meaningless blank Property
typofixes - v4
qapi-schema: remove legacy<> from doc
disas/microblaze: Remove unused code
help: dd missing newline
Target-ppc: Remove unnecessary variable
baum: Fix build with debugging enabled
linux-user: Fix warnings caused by missing 'static' attribute
opts: produce valid command line in qemu_opts_print
docs: fix a qga/qapi-schema.json comment
trivial: remove trailing newline from error_report
maint: avoid useless "if (foo) free(foo)" pattern
maint: avoid useless "if (foo) free(foo)" pattern
maint: remove unused include for strings.h
maint: remove unused include for signal.h
maint: remove unused include for dirent.h
...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

opts: produce valid command line in qemu_opts_print

This will let us print options in a format that the user would actually
write it on the command line (foo=bar,baz=asd,etc=def), without
prepending

opts: produce valid command line in qemu_opts_print

This will let us print options in a format that the user would actually
write it on the command line (foo=bar,baz=asd,etc=def), without
prepending a spurious comma at the beginning of the list, or quoting
values unnecessarily. This patch provides the following changes:
* write and id=, if the option has an id
* do not print separator before the first element
* do not quote string arguments
* properly escape commas (,) for QEMU

Signed-off-by: Kővágó, Zoltán <DirtY.iCE.hu@gmail.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Signed-off-by: Michael Tokarev <mjt@tls.msk.ru>

show more ...

Merge remote-tracking branch 'remotes/stefanha/tags/block-pull-request' into staging

# gpg: Signature made Fri 12 Dec 2014 17:09:56 GMT using RSA key ID 81AB73C8
# gpg: Good signature from "Stefan H

Merge remote-tracking branch 'remotes/stefanha/tags/block-pull-request' into staging

# gpg: Signature made Fri 12 Dec 2014 17:09:56 GMT using RSA key ID 81AB73C8
# gpg: Good signature from "Stefan Hajnoczi <stefanha@redhat.com>"
# gpg: aka "Stefan Hajnoczi <stefanha@gmail.com>"

* remotes/stefanha/tags/block-pull-request:
linux-aio: simplify removal of completed iocbs from the list
linux-aio: drop return code from laio_io_unplug and ioq_submit
linux-aio: rename LaioQueue idx field to "n"
linux-aio: track whether the queue is blocked
linux-aio: queue requests that cannot be submitted
block: drop unused bdrv_clear_incoming_migration_all() prototype
block: Don't add trailing space in "Formating..." message
qemu-iotests: Remove traling whitespaces in *.out
block: vhdx - set .bdrv_has_zero_init to bdrv_has_zero_init_1
iotests: Fix test 039
iotests: Filter for "Killed" in qemu-io output
qemu-io: Add sigraise command
block: vhdx - change .vhdx_create default block state to ZERO
block: vhdx - update PAYLOAD_BLOCK_UNMAPPED value to match 1.00 spec
block: vhdx - remove redundant comments
block/rbd: fix memory leak
iotests: Add test for vmdk JSON file names
vmdk: Fix error for JSON descriptor file names
block migration: fix return value

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

qemu-iotests: Remove traling whitespaces in *.out

This is simply:

$ cd tests/qemu-iotests; sed -i -e 's/ *$//' *.out

Signed-off-by: Fam Zheng <famz@redhat.com>
Reviewed-by: Eric Blake <eblake@re

qemu-iotests: Remove traling whitespaces in *.out

This is simply:

$ cd tests/qemu-iotests; sed -i -e 's/ *$//' *.out

Signed-off-by: Fam Zheng <famz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Message-id: 1418110684-19528-2-git-send-email-famz@redhat.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>

show more ...

Merge branch 'master' of git://git.qemu.org/qemu into prep-up

Conflicts:
hw/Makefile.objs
hw/ppc_prep.c

Signed-off-by: Andreas Färber <andreas.faerber@web.de>

Merge remote-tracking branch 'origin/master' into threadpool

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>

Merge remote-tracking branch 'kwolf/for-anthony' into staging

* kwolf/for-anthony: (32 commits)
osdep: Less restrictive F_SEFL in qemu_dup_flags()
qemu-iotests: add testcases for mirroring on-so

Merge remote-tracking branch 'kwolf/for-anthony' into staging

* kwolf/for-anthony: (32 commits)
osdep: Less restrictive F_SEFL in qemu_dup_flags()
qemu-iotests: add testcases for mirroring on-source-error/on-target-error
qmp: add pull_event function
mirror: add support for on-source-error/on-target-error
iostatus: forward block_job_iostatus_reset to block job
qemu-iotests: add mirroring test case
mirror: implement completion
qmp: add drive-mirror command
mirror: introduce mirror job
block: introduce BLOCK_JOB_READY event
block: add block-job-complete
block: rename block_job_complete to block_job_completed
block: export dirty bitmap information in query-block
block: introduce new dirty bitmap functionality
block: add bdrv_open_backing_file
block: add bdrv_query_stats
block: add bdrv_query_info
qemu-config: Add new -add-fd command line option
monitor: Prevent removing fd from set during init
monitor: Enable adding an inherited fd to an fd set
...

Conflicts:
vl.c

Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>

show more ...

qemu-iotests: Add 043 backing file chain infinite loop test

This new test verifies that qemu-img info --backing-chain safely aborts
when an image file has a backing file infinite loop.

Signed-off-b

qemu-iotests: Add 043 backing file chain infinite loop test

This new test verifies that qemu-img info --backing-chain safely aborts
when an image file has a backing file infinite loop.

Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>

show more ...