Merge remote-tracking branch 'remotes/kevin/tags/for-upstream' into staging

Block layer patches:

- file-posix: Mitigate file fragmentation with extent size hints
- Tighten qemu-img rules on missing

Merge remote-tracking branch 'remotes/kevin/tags/for-upstream' into staging

Block layer patches:

- file-posix: Mitigate file fragmentation with extent size hints
- Tighten qemu-img rules on missing backing format
- qemu-img map: Don't limit block status request size
- Fix crash with virtio-scsi and iothreads

# gpg: Signature made Tue 14 Jul 2020 14:24:19 BST
# gpg: using RSA key DC3DEB159A9AF95D3D7456FE7F09B272C88F2FD6
# gpg: issuer "kwolf@redhat.com"
# gpg: Good signature from "Kevin Wolf <kwolf@redhat.com>" [full]
# Primary key fingerprint: DC3D EB15 9A9A F95D 3D74 56FE 7F09 B272 C88F 2FD6

* remotes/kevin/tags/for-upstream:
block: Avoid stale pointer dereference in blk_get_aio_context()
qemu-img: Deprecate use of -b without -F
block: Add support to warn on backing file change without format
iotests: Specify explicit backing format where sensible
qcow2: Deprecate use of qemu-img amend to change backing file
block: Error if backing file fails during creation without -u
qcow: Tolerate backing_fmt=
vmdk: Add trivial backing_fmt support
sheepdog: Add trivial backing_fmt support
block: Finish deprecation of 'qemu-img convert -n -o'
qemu-img: Flush stdout before before potential stderr messages
file-posix: Mitigate file fragmentation with extent size hints
iotests/059: Filter out disk size with more standard filter
qemu-img map: Don't limit block status request size
iotests: Simplify _filter_img_create() a bit

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

iotests: Specify explicit backing format where sensible

There are many existing qcow2 images that specify a backing file but
no format. This has been the source of CVEs in the past, but has
become

iotests: Specify explicit backing format where sensible

There are many existing qcow2 images that specify a backing file but
no format. This has been the source of CVEs in the past, but has
become more prominent of a problem now that libvirt has switched to
-blockdev. With older -drive, at least the probing was always done by
qemu (so the only risk of a changed format between successive boots of
a guest was if qemu was upgraded and probed differently). But with
newer -blockdev, libvirt must specify a format; if libvirt guesses raw
where the image was formatted, this results in data corruption visible
to the guest; conversely, if libvirt guesses qcow2 where qemu was
using raw, this can result in potential security holes, so modern
libvirt instead refuses to use images without explicit backing format.

The change in libvirt to reject images without explicit backing format
has pointed out that a number of tools have been far too reliant on
probing in the past. It's time to set a better example in our own
iotests of properly setting this parameter.

iotest calls to create, rebase, and convert are all impacted to some
degree. It's a bit annoying that we are inconsistent on command line
- while all of those accept -o backing_file=...,backing_fmt=..., the
shortcuts are different: create and rebase have -b and -F, while
convert has -B but no -F. (amend has no shortcuts, but the previous
patch just deprecated the use of amend to change backing chains).

Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <20200706203954.341758-9-eblake@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>

show more ...

Merge remote-tracking branch 'remotes/kevin/tags/for-upstream' into staging

Block layer patches:

- iotests: Fix output of qemu-io related tests
- Don't ignore bdrv_set_aio_context() for nodes with

Merge remote-tracking branch 'remotes/kevin/tags/for-upstream' into staging

Block layer patches:

- iotests: Fix output of qemu-io related tests
- Don't ignore bdrv_set_aio_context() for nodes with bs->drv = NUL
- vmdk: Set vmdk parent backing_format to vmdk
- qcow2: Preallocation fixes (especially for external data files)
- Add linear-buffer-based APIs (as wrappers around qiov-based ones)
- Various code cleanups and small corner case fixes

# gpg: Signature made Tue 30 Apr 2019 16:35:09 BST
# gpg: using RSA key 7F09B272C88F2FD6
# gpg: Good signature from "Kevin Wolf <kwolf@redhat.com>" [full]
# Primary key fingerprint: DC3D EB15 9A9A F95D 3D74 56FE 7F09 B272 C88F 2FD6

* remotes/kevin/tags/for-upstream: (27 commits)
block/qed: add missed coroutine_fn markers
iotests: Check that images are in read-only mode after block-commit
commit: Make base read-only if there is an early failure
qemu-img: use buffer-based io
block/stream: use buffer-based io
block/commit: use buffer-based io
block/backup: use buffer-based io
block/parallels: use buffer-based io
block/qed: use buffer-based io
block/qcow: use buffer-based io
block/qcow2: use buffer-based io
block: introduce byte-based io helpers
qcow2: Fix error handling in the compression code
qcow2: Fix qcow2_make_empty() with external data file
qemu-img: Make create hint at protocol options
iotests: Perform the correct test in 082
qcow2: Fix full preallocation with external data file
qcow2: Add errp to preallocate_co()
qcow2: Avoid COW during metadata preallocation
qemu-img: Saner printing of large file sizes
...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

tests/qemu-iotests: Fix output of qemu-io related tests

One of the recent commits changed the way qemu-io prints out its
errors and warnings - they are now prefixed with the program name.
We've got

tests/qemu-iotests: Fix output of qemu-io related tests

One of the recent commits changed the way qemu-io prints out its
errors and warnings - they are now prefixed with the program name.
We've got to adapt the iotests accordingly to prevent that they
are failing.

Fixes: 99e98d7c9fc1a1639fad ("qemu-io: Use error_[gs]et_progname()")
Signed-off-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>

show more ...

Merge remote-tracking branch 'remotes/armbru/tags/pull-error-2016-01-13' into staging

Error reporting patches for 2016-01-13

# gpg: Signature made Wed 13 Jan 2016 14:21:48 GMT using RSA key ID EB91

Merge remote-tracking branch 'remotes/armbru/tags/pull-error-2016-01-13' into staging

Error reporting patches for 2016-01-13

# gpg: Signature made Wed 13 Jan 2016 14:21:48 GMT using RSA key ID EB918653
# gpg: Good signature from "Markus Armbruster <armbru@redhat.com>"
# gpg: aka "Markus Armbruster <armbru@pond.sub.org>"

* remotes/armbru/tags/pull-error-2016-01-13: (41 commits)
checkpatch: Detect newlines in error_report and other error functions
error: Consistently name Error * objects err, and not errp
s390/sclp: Simplify control flow in sclp_realize()
hw/s390x: Rename local variables Error *l_err to just err
error: Clean up errors with embedded newlines (again)
vhdx: Fix "log that needs to be replayed" error message
pci-assign: Clean up "Failed to assign" error messages
vmdk: Clean up "Invalid extent lines" error message
vmdk: Clean up control flow in vmdk_parse_extents() a bit
error: Strip trailing '\n' from error string arguments (again)
qemu-io qemu-nbd: Use error_report() etc. instead of fprintf()
migration: Use error_reportf_err() instead of monitor_printf()
spapr: Use error_reportf_err()
error: Use error_prepend() where it makes obvious sense
error: Use error_reportf_err() where it makes obvious sense
error: Don't decorate original error message when adding to it
error: New error_prepend(), error_reportf_err()
test-throttle: Simplify qemu_init_main_loop() error handling
qemu-nbd: Clean up "Failed to load snapshot" error message
block: Clean up "Could not create temporary overlay" error message
...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

qemu-io qemu-nbd: Use error_report() etc. instead of fprintf()

Just three instances left.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id

qemu-io qemu-nbd: Use error_report() etc. instead of fprintf()

Just three instances left.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <1450452927-8346-16-git-send-email-armbru@redhat.com>

show more ...

Merge remote-tracking branch 'remotes/mjt/tags/pull-trivial-patches-2015-09-11' into staging

trivial patches for 2015-09-11

# gpg: Signature made Fri 11 Sep 2015 12:02:43 BST using RSA key ID A4C3D

Merge remote-tracking branch 'remotes/mjt/tags/pull-trivial-patches-2015-09-11' into staging

trivial patches for 2015-09-11

# gpg: Signature made Fri 11 Sep 2015 12:02:43 BST using RSA key ID A4C3D7DB
# gpg: Good signature from "Michael Tokarev <mjt@tls.msk.ru>"
# gpg: aka "Michael Tokarev <mjt@corpit.ru>"
# gpg: aka "Michael Tokarev <mjt@debian.org>"

* remotes/mjt/tags/pull-trivial-patches-2015-09-11: (26 commits)
virtio-vga: enable for i386
hw/arm/spitz: Remove meaningless blank Property
hw/gpio/zaurus: Remove meaningless blank Property
hw/virtio/virtio-pci: Remove meaningless blank Property
hw/s390x/s390-virtio-bus: Remove meaningless blank Property
typofixes - v4
qapi-schema: remove legacy<> from doc
disas/microblaze: Remove unused code
help: dd missing newline
Target-ppc: Remove unnecessary variable
baum: Fix build with debugging enabled
linux-user: Fix warnings caused by missing 'static' attribute
opts: produce valid command line in qemu_opts_print
docs: fix a qga/qapi-schema.json comment
trivial: remove trailing newline from error_report
maint: avoid useless "if (foo) free(foo)" pattern
maint: avoid useless "if (foo) free(foo)" pattern
maint: remove unused include for strings.h
maint: remove unused include for signal.h
maint: remove unused include for dirent.h
...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

opts: produce valid command line in qemu_opts_print

This will let us print options in a format that the user would actually
write it on the command line (foo=bar,baz=asd,etc=def), without
prepending

opts: produce valid command line in qemu_opts_print

This will let us print options in a format that the user would actually
write it on the command line (foo=bar,baz=asd,etc=def), without
prepending a spurious comma at the beginning of the list, or quoting
values unnecessarily. This patch provides the following changes:
* write and id=, if the option has an id
* do not print separator before the first element
* do not quote string arguments
* properly escape commas (,) for QEMU

Signed-off-by: Kővágó, Zoltán <DirtY.iCE.hu@gmail.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Signed-off-by: Michael Tokarev <mjt@tls.msk.ru>

show more ...

Merge remote-tracking branch 'remotes/stefanha/tags/block-pull-request' into staging

# gpg: Signature made Fri 12 Dec 2014 17:09:56 GMT using RSA key ID 81AB73C8
# gpg: Good signature from "Stefan H

Merge remote-tracking branch 'remotes/stefanha/tags/block-pull-request' into staging

# gpg: Signature made Fri 12 Dec 2014 17:09:56 GMT using RSA key ID 81AB73C8
# gpg: Good signature from "Stefan Hajnoczi <stefanha@redhat.com>"
# gpg: aka "Stefan Hajnoczi <stefanha@gmail.com>"

* remotes/stefanha/tags/block-pull-request:
linux-aio: simplify removal of completed iocbs from the list
linux-aio: drop return code from laio_io_unplug and ioq_submit
linux-aio: rename LaioQueue idx field to "n"
linux-aio: track whether the queue is blocked
linux-aio: queue requests that cannot be submitted
block: drop unused bdrv_clear_incoming_migration_all() prototype
block: Don't add trailing space in "Formating..." message
qemu-iotests: Remove traling whitespaces in *.out
block: vhdx - set .bdrv_has_zero_init to bdrv_has_zero_init_1
iotests: Fix test 039
iotests: Filter for "Killed" in qemu-io output
qemu-io: Add sigraise command
block: vhdx - change .vhdx_create default block state to ZERO
block: vhdx - update PAYLOAD_BLOCK_UNMAPPED value to match 1.00 spec
block: vhdx - remove redundant comments
block/rbd: fix memory leak
iotests: Add test for vmdk JSON file names
vmdk: Fix error for JSON descriptor file names
block migration: fix return value

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

qemu-iotests: Remove traling whitespaces in *.out

This is simply:

$ cd tests/qemu-iotests; sed -i -e 's/ *$//' *.out

Signed-off-by: Fam Zheng <famz@redhat.com>
Reviewed-by: Eric Blake <eblake@re

qemu-iotests: Remove traling whitespaces in *.out

This is simply:

$ cd tests/qemu-iotests; sed -i -e 's/ *$//' *.out

Signed-off-by: Fam Zheng <famz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Message-id: 1418110684-19528-2-git-send-email-famz@redhat.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>

show more ...

Merge remote-tracking branch 'kwolf/tags/for-anthony' into staging

Block fixes for 1.7.0

# gpg: Signature made Fri 15 Nov 2013 09:51:25 AM PST using RSA key ID C88F2FD6
# gpg: Can't check signature

Merge remote-tracking branch 'kwolf/tags/for-anthony' into staging

Block fixes for 1.7.0

# gpg: Signature made Fri 15 Nov 2013 09:51:25 AM PST using RSA key ID C88F2FD6
# gpg: Can't check signature: public key not found

# By Max Reitz (3) and others
# Via Kevin Wolf
* kwolf/tags/for-anthony:
block: Fail if requested driver is not available
MAINTAINERS: add block driver sub-maintainers
qemu-img: Fix overwriting 'ret' before using
qemu-iotests: Test qcow2 count_contiguous_clusters()
qcow2: fix possible corruption when reading multiple clusters
qmp: access the local QemuOptsLists for drive option
MAINTAINERS: add block tree repo URLs
qemu-iotests: Extend 041 for unbacked mirroring
block/drive-mirror: Check for NULL backing_hd
qapi-schema: Update description for NewImageMode
block: Print its file name if backing file opening failed

Message-id: 1384537999-5972-1-git-send-email-kwolf@redhat.com
Signed-off-by: Anthony Liguori <aliguori@amazon.com>

show more ...

block: Print its file name if backing file opening failed

If backing file doesn't exist, the error message is confusing and
misleading:

$ qemu /tmp/a.qcow2
qemu: could not open disk image /

block: Print its file name if backing file opening failed

If backing file doesn't exist, the error message is confusing and
misleading:

$ qemu /tmp/a.qcow2
qemu: could not open disk image /tmp/a.qcow2: Could not open file: No
such file or directory

But...

$ ls /tmp/a.qcow2
/tmp/a.qcow2

$ qemu-img info /tmp/a.qcow2
image: /tmp/a.qcow2
file format: qcow2
virtual size: 8.0G (8589934592 bytes)
disk size: 196K
cluster_size: 65536
backing file: /tmp/b.qcow2

Because...

$ ls /tmp/b.qcow2
ls: cannot access /tmp/b.qcow2: No such file or directory

This is not intuitive. It's better to have the missing file's name in
the error message. With this patch:

$ qemu-io -c 'read 0 512' /tmp/a.qcow2
qemu-io: can't open device /tmp/a.qcow2: Could not open backing
file: Could not open '/stor/vm/arch.raw': No such file or directory
no file open, try 'help open'

Which is a little bit better.

Signed-off-by: Fam Zheng <famz@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>

show more ...

Merge remote-tracking branch 'kwolf/tags/for-anthony' into staging

Block patches for 1.7.0-rc0 (v2)

# gpg: Signature made Thu 31 Oct 2013 04:44:39 PM CET using RSA key ID C88F2FD6
# gpg: Can't chec

Merge remote-tracking branch 'kwolf/tags/for-anthony' into staging

Block patches for 1.7.0-rc0 (v2)

# gpg: Signature made Thu 31 Oct 2013 04:44:39 PM CET using RSA key ID C88F2FD6
# gpg: Can't check signature: public key not found

* kwolf/tags/for-anthony: (30 commits)
vmdk: Implment bdrv_get_specific_info
qapi: Add optional field 'compressed' to ImageInfo
qemu-iotests: prefill some data to test image
sheepdog: check simultaneous create in resend_aioreq
sheepdog: cancel aio requests if possible
sheepdog: make add_aio_request and send_aioreq void functions
sheepdog: try to reconnect to sheepdog after network error
coroutine: add co_aio_sleep_ns() to allow sleep in block drivers
sheepdog: reload inode outside of resend_aioreq
sheepdog: handle vdi objects in resend_aio_req
sheepdog: check return values of qemu_co_recv/send correctly
qemu-iotests: Test case for backing file deletion
qemu-iotests: drop duplicated "create_image"
qemu-iotests: Fix 051 reference output
block: Avoid unecessary drv->bdrv_getlength() calls
block: Disable BDRV_O_COPY_ON_READ for the backing file
ahci: fix win7 hang on boot
sheepdog: pass copy_policy in the request
sheepdog: explicitly set copies as type uint8_t
block: Don't copy backing file name on error
...

Message-id: 1383064269-27720-1-git-send-email-kwolf@redhat.com
Signed-off-by: Anthony Liguori <anthony@codemonkey.ws>

show more ...

qemu-iotests: Test case for backing file deletion

Add a test case for trying to open an image file where it is impossible
to open its backing file (in this case, because it was deleted). When
doing

qemu-iotests: Test case for backing file deletion

Add a test case for trying to open an image file where it is impossible
to open its backing file (in this case, because it was deleted). When
doing this, qemu (or qemu-io in this case) should not crash but rather
print an appropriate error message.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Fam Zheng <famz@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>

show more ...