Merge remote-tracking branch 'remotes/kevin/tags/for-upstream' into staging

Block layer patches:

- Remove deprecated -drive options for geometry/serial/addr
- luks: Allow shared writers if the pare

Merge remote-tracking branch 'remotes/kevin/tags/for-upstream' into staging

Block layer patches:

- Remove deprecated -drive options for geometry/serial/addr
- luks: Allow shared writers if the parents allow them (share-rw=on)
- qemu-img: Fix error when trying to convert to encrypted target image
- mirror: Fail gracefully for source == target
- I/O throttling: Fix behaviour during drain (always ignore the limits)
- bdrv_reopen() related fixes for bs->options/explicit_options content
- Documentation improvements

# gpg: Signature made Wed 15 Aug 2018 12:11:43 BST
# gpg: using RSA key 7F09B272C88F2FD6
# gpg: Good signature from "Kevin Wolf <kwolf@redhat.com>"
# Primary key fingerprint: DC3D EB15 9A9A F95D 3D74 56FE 7F09 B272 C88F 2FD6

* remotes/kevin/tags/for-upstream: (21 commits)
qapi: block: Remove mentions of error types which were removed
block: Simplify append_open_options()
block: Update bs->options if bdrv_reopen() succeeds
block: Simplify bdrv_reopen_abort()
block: Remove children options from bs->{options,explicit_options}
qdict: Make qdict_extract_subqdict() accept dst = NULL
block: drop empty .bdrv_close handlers
block: make .bdrv_close optional
qemu-img: fix regression copying secrets during convert
mirror: Fail gracefully for source == target
qapi/block: Document restrictions for node names
block: Remove dead deprecation warning code
block: Remove deprecated -drive option serial
block: Remove deprecated -drive option addr
block: Remove deprecated -drive geometry options
luks: Allow share-rw=on
throttle-groups: Don't allow timers without throttled requests
qemu-iotests: Update 093 to improve the draining test
throttle-groups: Skip the round-robin if a member is being drained
qemu-iotests: Test removing a throttle group member with a pending timer
...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

qemu-iotests: Test removing a throttle group member with a pending timer

A throttle group can have several members, and each one of them can
have several pending requests in the queue.

The requests

qemu-iotests: Test removing a throttle group member with a pending timer

A throttle group can have several members, and each one of them can
have several pending requests in the queue.

The requests are processed in a round-robin fashion, so the algorithm
decides the drive that is going to run the next request and sets a
timer in it. Once the timer fires and the throttled request is run
then the next drive from the group is selected and a new timer is set.

If the user tried to remove a drive from a group and that drive had a
timer set then the code was not taking care of setting up a new timer
in one of the remaining members of the group, freezing their I/O.

This problem was fixed in 6fccbb475bc6effc313ee9481726a1748b6dae57,
and this patch adds a new test case that reproduces this exact
scenario.

Signed-off-by: Alberto Garcia <berto@igalia.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>

show more ...

Merge remote-tracking branch 'remotes/stefanha/tags/block-pull-request' into staging

Pull request

The following disk I/O throttling fixes solve recent bugs.

# gpg: Signature made Tue 14 Nov 2017 1

Merge remote-tracking branch 'remotes/stefanha/tags/block-pull-request' into staging

Pull request

The following disk I/O throttling fixes solve recent bugs.

# gpg: Signature made Tue 14 Nov 2017 10:37:12 GMT
# gpg: using RSA key 0x9CA4ABB381AB73C8
# gpg: Good signature from "Stefan Hajnoczi <stefanha@redhat.com>"
# gpg: aka "Stefan Hajnoczi <stefanha@gmail.com>"
# Primary key fingerprint: 8695 A8BF D3F9 7CDA AC35 775A 9CA4 ABB3 81AB 73C8

* remotes/stefanha/tags/block-pull-request:
qemu-iotests: Test I/O limits with removable media
block: Leave valid throttle timers when removing a BDS from a backend
block: Check for inserted BlockDriverState in blk_io_limits_disable()
throttle-groups: drain before detaching ThrottleState
block: all I/O should be completed before removing throttle timers.

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

qemu-iotests: Test I/O limits with removable media

This test hotplugs a CD drive to a VM and checks that I/O limits can
be set only when the drive has media inserted and that they are kept
when the

qemu-iotests: Test I/O limits with removable media

This test hotplugs a CD drive to a VM and checks that I/O limits can
be set only when the drive has media inserted and that they are kept
when the media is replaced.

This also tests the removal of a device with valid I/O limits set but
no media inserted. This involves deleting and disabling the limits
of a BlockBackend without BlockDriverState, a scenario that has been
crashing until the fixes from the last couple of patches.

[Python PEP8 fixup: "Don't use spaces are the = sign when used to
indicate a keyword argument or a default parameter value"
--Stefan]

Signed-off-by: Alberto Garcia <berto@igalia.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-id: 071eb397118ed207c5a7f01d58766e415ee18d6a.1510339534.git.berto@igalia.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>

show more ...

Merge remote-tracking branch 'remotes/kevin/tags/for-upstream' into staging

Block layer patches

# gpg: Signature made Mon 24 Oct 2016 17:02:47 BST
# gpg: using RSA key 0x7F09B272C88F

Merge remote-tracking branch 'remotes/kevin/tags/for-upstream' into staging

Block layer patches

# gpg: Signature made Mon 24 Oct 2016 17:02:47 BST
# gpg: using RSA key 0x7F09B272C88F2FD6
# gpg: Good signature from "Kevin Wolf <kwolf@redhat.com>"
# Primary key fingerprint: DC3D EB15 9A9A F95D 3D74 56FE 7F09 B272 C88F 2FD6

* remotes/kevin/tags/for-upstream: (23 commits)
block/replication: Clarify 'top-id' parameter usage
block: More operations for meta dirty bitmap
tests: Add test code for hbitmap serialization
block: BdrvDirtyBitmap serialization interface
hbitmap: serialization
block: Assert that bdrv_release_dirty_bitmap succeeded
block: Add two dirty bitmap getters
block: Support meta dirty bitmap
tests: Add test code for meta bitmap
HBitmap: Introduce "meta" bitmap to track bit changes
block: Hide HBitmap in block dirty bitmap interface
quorum: do not allocate multiple iovecs for FIFO strategy
quorum: change child_iter to children_read
iotests: Do not rely on unavailable domains in 162
iotests: Remove raciness from 162
qemu-nbd: Add --fork option
qemu-iotests: Test I/O in a single drive from a throttling group
throttle: Correct access to wrong BlockBackendPublic structures
qapi: fix memory leak in bdrv_image_info_specific_dump
block: improve error handling in raw_open
...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

qemu-iotests: Test I/O in a single drive from a throttling group

iotest 093 contains a test that creates a throttling group with
several drives and performs I/O in all of them. This patch adds a new

qemu-iotests: Test I/O in a single drive from a throttling group

iotest 093 contains a test that creates a throttling group with
several drives and performs I/O in all of them. This patch adds a new
test that creates a similar setup but only performs I/O in one of the
drives at the same time.

This is useful to test that the round robin algorithm is behaving
properly in these scenarios, and is specifically written using the
regression introduced in 27ccdd52598290f0f8b58be56e as an example.

Signed-off-by: Alberto Garcia <berto@igalia.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>

show more ...

Merge remote-tracking branch 'remotes/kevin/tags/for-upstream' into staging

Block layer patches

# gpg: Signature made Wed 13 Jul 2016 12:46:17 BST
# gpg: using RSA key 0x7F09B272C88F

Merge remote-tracking branch 'remotes/kevin/tags/for-upstream' into staging

Block layer patches

# gpg: Signature made Wed 13 Jul 2016 12:46:17 BST
# gpg: using RSA key 0x7F09B272C88F2FD6
# gpg: Good signature from "Kevin Wolf <kwolf@redhat.com>"
# Primary key fingerprint: DC3D EB15 9A9A F95D 3D74 56FE 7F09 B272 C88F 2FD6

* remotes/kevin/tags/for-upstream: (34 commits)
iotests: Make 157 actually format-agnostic
vvfat: Fix qcow write target driver specification
hmp: show all of snapshot info on every block dev in output of 'info snapshots'
hmp: use snapshot name to determine whether a snapshot is 'fully available'
qemu-iotests: Test naming of throttling groups
blockdev: Fix regression with the default naming of throttling groups
vmdk: fix metadata write regression
Improve block job rate limiting for small bandwidth values
qcow2: Fix qcow2_get_cluster_offset()
qemu-io: Use correct range limitations
qcow2: Avoid making the L1 table too big
qemu-img: Use strerror() for generic resize error
block: Remove BB options from blockdev-add
qemu-iotests: Test setting WCE with qdev
block/qdev: Allow configuring rerror/werror with qdev properties
commit: Fix use of error handling policy
block/qdev: Allow configuring WCE with qdev properties
block/qdev: Allow node name for drive properties
coroutine: move entry argument to qemu_coroutine_create
test-coroutine: prepare for the next patch
...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

Merge remote-tracking branch 'mreitz/tags/pull-block-for-kevin-2016-07-13' into queue-block

Block patches (v2) for the block queue.

# gpg: Signature made Wed Jul 13 13:41:53 2016 CEST
# gpg:

Merge remote-tracking branch 'mreitz/tags/pull-block-for-kevin-2016-07-13' into queue-block

Block patches (v2) for the block queue.

# gpg: Signature made Wed Jul 13 13:41:53 2016 CEST
# gpg: using RSA key 0x3BB14202E838ACAD
# gpg: Good signature from "Max Reitz <mreitz@redhat.com>"
# Primary key fingerprint: 91BE B60A 30DB 3E88 57D1 1829 F407 DB00 61D5 CF40
# Subkey fingerprint: 58B3 81CE 2DC8 9CF9 9730 EE64 3BB1 4202 E838 ACAD

* mreitz/tags/pull-block-for-kevin-2016-07-13:
iotests: Make 157 actually format-agnostic
vvfat: Fix qcow write target driver specification
hmp: show all of snapshot info on every block dev in output of 'info snapshots'
hmp: use snapshot name to determine whether a snapshot is 'fully available'
qemu-iotests: Test naming of throttling groups
blockdev: Fix regression with the default naming of throttling groups
vmdk: fix metadata write regression
Improve block job rate limiting for small bandwidth values
qcow2: Fix qcow2_get_cluster_offset()
qemu-io: Use correct range limitations
qcow2: Avoid making the L1 table too big
qemu-img: Use strerror() for generic resize error

Signed-off-by: Kevin Wolf <kwolf@redhat.com>

show more ...

qemu-iotests: Test naming of throttling groups

Throttling groups are named using the 'group' parameter of the
block_set_io_throttle command and the throttling.group command-line
option. If that para

qemu-iotests: Test naming of throttling groups

Throttling groups are named using the 'group' parameter of the
block_set_io_throttle command and the throttling.group command-line
option. If that parameter is unspecified the groups get the name of
the block device.

This patch adds a new test to check the naming of throttling groups.

Signed-off-by: Alberto Garcia <berto@igalia.com>
Message-id: d87d02823a6b91609509d8bb18e2f5dbd9a6102c.1467986342.git.berto@igalia.com
Reviewed-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>

show more ...

Merge remote-tracking branch 'remotes/kevin/tags/for-upstream' into staging

Block layer patches

# gpg: Signature made Mon 22 Feb 2016 15:59:25 GMT using RSA key ID C88F2FD6
# gpg: Good signature fr

Merge remote-tracking branch 'remotes/kevin/tags/for-upstream' into staging

Block layer patches

# gpg: Signature made Mon 22 Feb 2016 15:59:25 GMT using RSA key ID C88F2FD6
# gpg: Good signature from "Kevin Wolf <kwolf@redhat.com>"

* remotes/kevin/tags/for-upstream: (34 commits)
qemu-iotests: 140: make description slightly more verbose
qemu-iotests: 140: don't use IDE device
qemu-iotests: 067: ignore QMP events
blockdev: unset inappropriate flags when changing medium
MAINTAINERS: Add myself as maintainer of the throttling code
docs: Document the throttling infrastructure
qapi: Correct the name of the iops_rd parameter
qemu-iotests: Extend iotest 093 to test bursts
throttle: Test throttle_compute_wait() during bursts
throttle: Check that burst_level leaks correctly
qapi: Add burst length fields to BlockDeviceInfo
qapi: Add burst length parameters to block_set_io_throttle
throttle: Add command-line settings to define the burst periods
throttle: Add support for burst periods
throttle: Use throttle_config_init() to initialize ThrottleConfig
throttle: Merge all functions that check the configuration into one
throttle: Set always an average value when setting a maximum value
throttle: Make throttle_is_valid() set errp
throttle: Make throttle_max_is_missing_limit() set errp
throttle: Make throttle_conflicting() set errp
...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

qemu-iotests: Extend iotest 093 to test bursts

This patch adds a new test that checks that the burst settings
('iops_max', 'iops_max_length', etc.) of the throttling code work as
expected.

Signed-o

qemu-iotests: Extend iotest 093 to test bursts

This patch adds a new test that checks that the burst settings
('iops_max', 'iops_max_length', etc.) of the throttling code work as
expected.

Signed-off-by: Alberto Garcia <berto@igalia.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>

show more ...

Merge remote-tracking branch 'remotes/stefanha/tags/block-pull-request' into staging

Pull request

v2:
* Fix C11 typedef redefinitions in ahci and libqos malloc [Peter]
* Fix lx -> PRIx64 format s

Merge remote-tracking branch 'remotes/stefanha/tags/block-pull-request' into staging

Pull request

v2:
* Fix C11 typedef redefinitions in ahci and libqos malloc [Peter]
* Fix lx -> PRIx64 format specifiers in ahci [Peter]

# gpg: Signature made Mon Feb 16 15:45:53 2015 GMT using RSA key ID 81AB73C8
# gpg: Good signature from "Stefan Hajnoczi <stefanha@redhat.com>"
# gpg: aka "Stefan Hajnoczi <stefanha@gmail.com>"

* remotes/stefanha/tags/block-pull-request: (65 commits)
block: Keep bdrv_check*_request()'s return value
block: Remove "growable" from BDS
block: Clamp BlockBackend requests
qemu-io: Use BlockBackend
qemu-io: Remove "growable" option
qemu-io: Use blk_new_open() in openfile()
qemu-nbd: Use blk_new_open() in main()
qemu-img: Use BlockBackend as far as possible
qemu-img: Use blk_new_open() in img_rebase()
qemu-img: Use blk_new_open() in img_open()
block/xen: Use blk_new_open() in blk_connect()
blockdev: Use blk_new_open() in blockdev_init()
iotests: Add test for driver=qcow2, format=qcow2
block: Add Error parameter to bdrv_find_protocol()
block: Add blk_new_open()
block: Lift some BDS functions to the BlockBackend
iotests: Add test for qemu-img convert to NBD
qemu-img: Fix qemu-img convert -n
qemu-iotests: Add 093 for IO throttling
qemu-iotests: Allow caller to disable underscore convertion for qmp
...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

qemu-iotests: Add 093 for IO throttling

This case utilizes qemu-io command "aio_{read,write} -q" to verify the
effectiveness of IO throttling options.

It's implemented by driving the vm timer from

qemu-iotests: Add 093 for IO throttling

This case utilizes qemu-io command "aio_{read,write} -q" to verify the
effectiveness of IO throttling options.

It's implemented by driving the vm timer from qtest protocol, so the
throttling timers are signaled with determinied time duration. Then we
verify the completed IO requests are within 10% error of bps and iops
limits.

"null" protocol is used as the disk backend so that no actual disk IO is
performed on host, this will make the blockstats much more
deterministic. Both "null-aio" and "null-co" are covered, which is also
a simple cross validation test for the driver code.

Signed-off-by: Fam Zheng <famz@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-id: 1422586186-9925-6-git-send-email-famz@redhat.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>

show more ...