Merge remote-tracking branch 'remotes/maxreitz/tags/pull-block-2019-07-15' into staging

Block patches for 4.1-rc1:
- Fixes for the NVMe block driver, the gluster block driver, and for
running mult

Merge remote-tracking branch 'remotes/maxreitz/tags/pull-block-2019-07-15' into staging

Block patches for 4.1-rc1:
- Fixes for the NVMe block driver, the gluster block driver, and for
running multiple block jobs concurrently on a single chain

# gpg: Signature made Mon 15 Jul 2019 14:51:43 BST
# gpg: using RSA key 91BEB60A30DB3E8857D11829F407DB0061D5CF40
# gpg: issuer "mreitz@redhat.com"
# gpg: Good signature from "Max Reitz <mreitz@redhat.com>" [full]
# Primary key fingerprint: 91BE B60A 30DB 3E88 57D1 1829 F407 DB00 61D5 CF40

* remotes/maxreitz/tags/pull-block-2019-07-15:
gluster: fix .bdrv_reopen_prepare when backing file is a JSON object
iotests: Add read-only test case to 030
iotests: Add new case to 030
iotests: Add @use_log to VM.run_job()
iotests: Compare error messages in 030
iotests: Fix throttling in 030
block: Deep-clear inherits_from
block/stream: Swap backing file change order
block/stream: Fix error path
block: Add BDS.never_freeze
nvme: Set number of queues later in nvme_init()

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

iotests: Add read-only test case to 030

This tests that the stream job exits cleanly (without abort) when the
top node is read-only and cannot be reopened read/write.

Signed-off-by: Max Reitz <mrei

iotests: Add read-only test case to 030

This tests that the stream job exits cleanly (without abort) when the
top node is read-only and cannot be reopened read/write.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-id: 20190703172813.6868-12-mreitz@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>

show more ...

iotests: Add new case to 030

We recently removed the dependency of the stream job on its base node.
That makes it OK to use a commit filter node there. Test that.

Signed-off-by: Max Reitz <mreitz@

iotests: Add new case to 030

We recently removed the dependency of the stream job on its base node.
That makes it OK to use a commit filter node there. Test that.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Tested-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com>
Reviewed-by: Alberto Garcia <berto@igalia.com>
Message-id: 20190703172813.6868-11-mreitz@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>

show more ...

Merge remote-tracking branch 'remotes/kevin/tags/for-upstream' into staging

Block layer patches:

- file-posix: Ignore unlock failure instead of crashing
- gluster: Limit the transfer size to 512 Mi

Merge remote-tracking branch 'remotes/kevin/tags/for-upstream' into staging

Block layer patches:

- file-posix: Ignore unlock failure instead of crashing
- gluster: Limit the transfer size to 512 MiB
- stream: Fix backing chain freezing
- qemu-img: Enable BDRV_REQ_MAY_UNMAP for zero writes in convert
- iotests fixes

# gpg: Signature made Tue 02 Apr 2019 13:47:43 BST
# gpg: using RSA key 7F09B272C88F2FD6
# gpg: Good signature from "Kevin Wolf <kwolf@redhat.com>" [full]
# Primary key fingerprint: DC3D EB15 9A9A F95D 3D74 56FE 7F09 B272 C88F 2FD6

* remotes/kevin/tags/for-upstream:
tests/qemu-iotests/235: Allow fallback to tcg
block: test block-stream with a base node that is used by block-commit
block: freeze the backing chain earlier in stream_start()
block: continue until base is found in bdrv_freeze_backing_chain() et al
block/file-posix: do not fail on unlock bytes
tests/qemu-iotests: Remove redundant COPYING file
block/gluster: limit the transfer size to 512 MiB
qemu-img: Enable BDRV_REQ_MAY_UNMAP in convert
iotests: Fix test 200 on s390x without virtio-pci

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

block: test block-stream with a base node that is used by block-commit

The base node of a block-stream operation indicates the first image
from the backing chain starting from which no data is copie

block: test block-stream with a base node that is used by block-commit

The base node of a block-stream operation indicates the first image
from the backing chain starting from which no data is copied to the
top node.

The block-stream job allows others to use that base image, so a second
block-stream job could be writing to it at the same time. An important
restriction is that the base image must not disappear while the stream
job is ongoing. stream_start() freezes the backing chain from top to
base with that purpose but it does it too late in the code so there is
a race condition there.

This bug was fixed in the previous commit, and this patch contains an
iotest for this scenario.

Signed-off-by: Alberto Garcia <berto@igalia.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>

show more ...

Merge remote-tracking branch 'remotes/kevin/tags/for-upstream' into staging

Block layer patches

# gpg: Signature made Fri 09 Mar 2018 15:09:20 GMT
# gpg: using RSA key 7F09B272C88F2F

Merge remote-tracking branch 'remotes/kevin/tags/for-upstream' into staging

Block layer patches

# gpg: Signature made Fri 09 Mar 2018 15:09:20 GMT
# gpg: using RSA key 7F09B272C88F2FD6
# gpg: Good signature from "Kevin Wolf <kwolf@redhat.com>"
# Primary key fingerprint: DC3D EB15 9A9A F95D 3D74 56FE 7F09 B272 C88F 2FD6

* remotes/kevin/tags/for-upstream: (56 commits)
qemu-iotests: fix 203 migration completion race
iotests: Tweak 030 in order to trigger a race condition with parallel jobs
iotests: Skip test for ENOMEM error
iotests: Mark all tests executable
iotests: Test creating overlay when guest running
qemu-iotests: Test ssh image creation over QMP
qemu-iotests: Test qcow2 over file image creation with QMP
block: Fail bdrv_truncate() with negative size
file-posix: Fix no-op bdrv_truncate() with falloc preallocation
ssh: Support .bdrv_co_create
ssh: Pass BlockdevOptionsSsh to connect_to_ssh()
ssh: QAPIfy host-key-check option
ssh: Use QAPI BlockdevOptionsSsh object
sheepdog: Support .bdrv_co_create
sheepdog: QAPIfy "redundancy" create option
nfs: Support .bdrv_co_create
nfs: Use QAPI options in nfs_client_open()
rbd: Use qemu_rbd_connect() in qemu_rbd_do_create()
rbd: Assign s->snap/image_name in qemu_rbd_open()
rbd: Support .bdrv_co_create
...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

Merge remote-tracking branch 'mreitz/tags/pull-block-2018-03-09' into queue-block

Block patches

# gpg: Signature made Fri Mar 9 15:40:32 2018 CET
# gpg: using RSA key F407DB0061D5CF

Merge remote-tracking branch 'mreitz/tags/pull-block-2018-03-09' into queue-block

Block patches

# gpg: Signature made Fri Mar 9 15:40:32 2018 CET
# gpg: using RSA key F407DB0061D5CF40
# gpg: Good signature from "Max Reitz <mreitz@redhat.com>"
# Primary key fingerprint: 91BE B60A 30DB 3E88 57D1 1829 F407 DB00 61D5 CF40

* mreitz/tags/pull-block-2018-03-09:
qemu-iotests: fix 203 migration completion race
iotests: Tweak 030 in order to trigger a race condition with parallel jobs
iotests: Skip test for ENOMEM error
iotests: Mark all tests executable
iotests: Test creating overlay when guest running

Signed-off-by: Kevin Wolf <kwolf@redhat.com>

show more ...

iotests: Tweak 030 in order to trigger a race condition with parallel jobs

This patch tweaks TestParallelOps in iotest 030 so it allocates data
in smaller regions (256KB/512KB instead of 512KB/1MB)

iotests: Tweak 030 in order to trigger a race condition with parallel jobs

This patch tweaks TestParallelOps in iotest 030 so it allocates data
in smaller regions (256KB/512KB instead of 512KB/1MB) and the
block-stream job in test_stream_commit() only needs to copy data that
is at the very end of the image.

This way when the block-stream job is awakened it will finish right
away without any chance of being stopped by block_job_sleep_ns(). This
triggers the bug that was fixed by 3d5d319e1221082974711af1d09d82f and
1a63a907507fbbcfaee3f622907ec24 and is therefore a more useful test
case for parallel block jobs.

After this patch the aforementiond bug can also be reproduced with the
test_stream_parallel() test case.

Since with this change the stream job in test_stream_commit() finishes
early, this patch introduces a similar test case where both jobs are
slowed down so they can actually run in parallel.

Signed-off-by: Alberto Garcia <berto@igalia.com>
Cc: John Snow <jsnow@redhat.com>
Message-id: 20180306130121.30243-1-berto@igalia.com
Signed-off-by: Max Reitz <mreitz@redhat.com>

show more ...

Merge remote-tracking branch 'kwolf/tags/for-upstream' into staging

Block layer patches

# gpg: Signature made Mon 29 May 2017 03:34:59 PM BST
# gpg: using RSA key 0x7F09B272C88F2FD6

Merge remote-tracking branch 'kwolf/tags/for-upstream' into staging

Block layer patches

# gpg: Signature made Mon 29 May 2017 03:34:59 PM BST
# gpg: using RSA key 0x7F09B272C88F2FD6
# gpg: Good signature from "Kevin Wolf <kwolf@redhat.com>"
# Primary key fingerprint: DC3D EB15 9A9A F95D 3D74 56FE 7F09 B272 C88F 2FD6

* kwolf/tags/for-upstream:
block/file-*: *_parse_filename() and colons
block: Fix backing paths for filenames with colons
block: Tweak error message related to qemu-img amend
qemu-img: Fix leakage of options on error
qemu-img: copy *key-secret opts when opening newly created files
qemu-img: introduce --target-image-opts for 'convert' command
qemu-img: fix --image-opts usage with dd command
qemu-img: add support for --object with 'dd' command
qemu-img: Fix documentation of convert
qcow2: remove extra local_error variable
mirror: Drop permissions on s->target on completion
nvme: Add support for Controller Memory Buffers
iotests: 147: Don't test inet6 if not available
qemu-iotests: Test streaming with missing job ID
stream: fix crash in stream_start() when block_job_create() fails

Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>

show more ...

qemu-iotests: Test streaming with missing job ID

This adds a small test for the image streaming error path for failing
block_job_create(), which would have found the null pointer dereference
in comm

qemu-iotests: Test streaming with missing job ID

This adds a small test for the image streaming error path for failing
block_job_create(), which would have found the null pointer dereference
in commit a170a91f.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Alberto Garcia <berto@igalia.com>
Reviewed-by: Kashyap Chamarthy <kchamart@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Jeff Cody <jcody@redhat.com>

show more ...

Merge remote-tracking branch 'remotes/kevin/tags/for-upstream' into staging

Block layer patches

# gpg: Signature made Mon 31 Oct 2016 16:10:07 GMT
# gpg: using RSA key 0x7F09B272C88F

Merge remote-tracking branch 'remotes/kevin/tags/for-upstream' into staging

Block layer patches

# gpg: Signature made Mon 31 Oct 2016 16:10:07 GMT
# gpg: using RSA key 0x7F09B272C88F2FD6
# gpg: Good signature from "Kevin Wolf <kwolf@redhat.com>"
# Primary key fingerprint: DC3D EB15 9A9A F95D 3D74 56FE 7F09 B272 C88F 2FD6

* remotes/kevin/tags/for-upstream: (29 commits)
qapi: allow blockdev-add for NFS
block/nfs: Introduce runtime_opts in NFS
block: Mention replication in BlockdevDriver enum docs
qemu-iotests: test 'offset' and 'size' options in raw driver
raw_bsd: add offset and size options
qemu-iotests: Test the 'base-node' parameter of 'block-stream'
block: Add 'base-node' parameter to the 'block-stream' command
qemu-iotests: Test streaming to a Quorum child
qemu-iotests: Add iotests.supports_quorum()
qemu-iotests: Test block-stream and block-commit in parallel
qemu-iotests: Test overlapping stream and commit operations
qemu-iotests: Test block-stream operations in parallel
qemu-iotests: Test streaming to an intermediate layer
docs: Document how to stream to an intermediate layer
block: Add QMP support for streaming to an intermediate layer
block: Support streaming to an intermediate layer
block: Block all intermediate nodes in commit_active_start()
block: Block all nodes involved in the block-commit operation
block: Check blockers in all nodes involved in a block-commit job
block: Use block_job_add_bdrv() in backup_start()
...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

qemu-iotests: Test the 'base-node' parameter of 'block-stream'

The block-stream command has traditionally used the 'base' parameter
to indicate the image to copy the data from. This test checks that

qemu-iotests: Test the 'base-node' parameter of 'block-stream'

The block-stream command has traditionally used the 'base' parameter
to indicate the image to copy the data from. This test checks that the
'base-node' parameter can also be used for the same purpose.

Signed-off-by: Alberto Garcia <berto@igalia.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>

show more ...

qemu-iotests: Test streaming to a Quorum child

Quorum children are special in the sense that they're not directly
attached to a block backend but they're not used as backing images
either. However t

qemu-iotests: Test streaming to a Quorum child

Quorum children are special in the sense that they're not directly
attached to a block backend but they're not used as backing images
either. However the intermediate block streaming code supports
streaming to them. This is a test case for that scenario.

Signed-off-by: Alberto Garcia <berto@igalia.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>

show more ...

qemu-iotests: Test block-stream and block-commit in parallel

As with test_stream_parallel(), we allow mixing block-stream and
block-commit operations in the same backing chain as long as there's
no

qemu-iotests: Test block-stream and block-commit in parallel

As with test_stream_parallel(), we allow mixing block-stream and
block-commit operations in the same backing chain as long as there's
no overlap among the involved nodes.

Signed-off-by: Alberto Garcia <berto@igalia.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>

show more ...

qemu-iotests: Test overlapping stream and commit operations

These test cases check that it's not possible to perform two
block-stream or block-commit operations if there are nodes involved in
both.

qemu-iotests: Test overlapping stream and commit operations

These test cases check that it's not possible to perform two
block-stream or block-commit operations if there are nodes involved in
both.

Signed-off-by: Alberto Garcia <berto@igalia.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>

show more ...

qemu-iotests: Test block-stream operations in parallel

This test case checks that it's possible to launch several stream
operations in parallel in the same snapshot chain, each one involving
a diffe

qemu-iotests: Test block-stream operations in parallel

This test case checks that it's possible to launch several stream
operations in parallel in the same snapshot chain, each one involving
a different set of nodes.

Signed-off-by: Alberto Garcia <berto@igalia.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>

show more ...

qemu-iotests: Test streaming to an intermediate layer

This adds test_stream_intermediate(), similar to test_stream() but
streams to the intermediate image instead.

Signed-off-by: Alberto Garcia <be

qemu-iotests: Test streaming to an intermediate layer

This adds test_stream_intermediate(), similar to test_stream() but
streams to the intermediate image instead.

Signed-off-by: Alberto Garcia <berto@igalia.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>

show more ...

Merge remote-tracking branch 'remotes/cody/tags/block-pull-request' into staging

# gpg: Signature made Tue 29 Mar 2016 01:48:09 BST using RSA key ID C0DE3057
# gpg: Good signature from "Jeffrey Cody

Merge remote-tracking branch 'remotes/cody/tags/block-pull-request' into staging

# gpg: Signature made Tue 29 Mar 2016 01:48:09 BST using RSA key ID C0DE3057
# gpg: Good signature from "Jeffrey Cody <jcody@redhat.com>"
# gpg: aka "Jeffrey Cody <jeff@codyprime.org>"
# gpg: aka "Jeffrey Cody <codyprime@gmail.com>"

* remotes/cody/tags/block-pull-request:
qemu-iotests: add no-op streaming test
qemu-iotests: fix test_stream_partial()
block: never cancel a streaming job without running stream_complete()

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

qemu-iotests: add no-op streaming test

This patch tests that in a partial block-stream operation, no data is
ever copied from the base image.

Signed-off-by: Alberto Garcia <berto@igalia.com>
Review

qemu-iotests: add no-op streaming test

This patch tests that in a partial block-stream operation, no data is
ever copied from the base image.

Signed-off-by: Alberto Garcia <berto@igalia.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-id: 5272a2aa57bc0b3f981f8b3e0c813e58a88c974b.1458566441.git.berto@igalia.com
Signed-off-by: Jeff Cody <jcody@redhat.com>

show more ...

Merge branch 'master' of git://git.qemu.org/qemu into prep-up

Conflicts:
hw/Makefile.objs
hw/ppc_prep.c

Signed-off-by: Andreas Färber <andreas.faerber@web.de>

Merge remote-tracking branch 'kwolf/for-anthony' into staging

* kwolf/for-anthony: (30 commits)
qemu-iotests: add tests for streaming error handling
qemu-iotests: map underscore to dash in QMP a

Merge remote-tracking branch 'kwolf/for-anthony' into staging

* kwolf/for-anthony: (30 commits)
qemu-iotests: add tests for streaming error handling
qemu-iotests: map underscore to dash in QMP argument names
blkdebug: process all set_state rules in the old state
stream: add on-error argument
block: introduce block job error
iostatus: reorganize io error code
iostatus: change is_read to a bool
iostatus: move BlockdevOnError declaration to QAPI
iostatus: rename BlockErrorAction, BlockQMPEventAction
qemu-iotests: add test for pausing a streaming operation
qmp: add block-job-pause and block-job-resume
block: add support for job pause/resume
qmp: add 'busy' member to BlockJobInfo
block: add block_job_query
block: move job APIs to separate files
block: fix documentation of block_job_cancel_sync
qerror/block: introduce QERR_BLOCK_JOB_NOT_ACTIVE
qemu-iotests: add initial tests for live block commit
QAPI: add command for live block commit, 'block-commit'
block: helper function, to find the base image of a chain
...

show more ...

qemu-iotests: add tests for streaming error handling

Add a test for each of report/ignore/stop. The tests use blkdebug
to generate an error in the middle of a script. The error is
recoverable (onc

qemu-iotests: add tests for streaming error handling

Add a test for each of report/ignore/stop. The tests use blkdebug
to generate an error in the middle of a script. The error is
recoverable (once = "on") so that we can test resuming a job after
stopping for an error.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>

show more ...

qemu-iotests: add test for pausing a streaming operation

These check that a paused streaming job does not advance its offset.

Sometimes the new test fails; the map is different between the source
a

qemu-iotests: add test for pausing a streaming operation

These check that a paused streaming job does not advance its offset.

Sometimes the new test fails; the map is different between the source
and the destination of the streaming because qemu-io does not always
pack adjacent clusters that have the same allocated/unallocated state.
However, this also happens with the existing test_stream testcase, and
is better fixed in qemu-io.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>

show more ...

Merge remote-tracking branch 'kwolf/for-anthony' into staging

* kwolf/for-anthony:
qemu-iotests: add backing file smaller than image test case
stream: complete early if end of backing file is re

Merge remote-tracking branch 'kwolf/for-anthony' into staging

* kwolf/for-anthony:
qemu-iotests: add backing file smaller than image test case
stream: complete early if end of backing file is reached
qed: refuse unaligned zero writes with a backing file

show more ...

qemu-iotests: add backing file smaller than image test case

This new test case checks that streaming completes successfully when the
backing file is smaller than the image file.

Signed-off-by: Stef

qemu-iotests: add backing file smaller than image test case

This new test case checks that streaming completes successfully when the
backing file is smaller than the image file.

Signed-off-by: Stefan Hajnoczi <stefanha@linux.vnet.ibm.com>
Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>

show more ...