block: Allow BDRV_REQ_FUA through blk_pwrite()

We have several block drivers that understand BDRV_REQ_FUA,
and emulate it in the block layer for the rest by a full flush.
But without a way to actual

block: Allow BDRV_REQ_FUA through blk_pwrite()

We have several block drivers that understand BDRV_REQ_FUA,
and emulate it in the block layer for the rest by a full flush.
But without a way to actually request BDRV_REQ_FUA during a
pass-through blk_pwrite(), FUA-aware block drivers like NBD are
forced to repeat the emulation logic of a full flush regardless
of whether the backend they are writing to could do it more
efficiently.

This patch just wires up a flags argument; followup patches
will actually make use of it in the NBD driver and in qemu-io.

Signed-off-by: Eric Blake <eblake@redhat.com>
Acked-by: Denis V. Lunev <den@openvz.org>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>

show more ...

Merge remote-tracking branch 'remotes/bonzini/tags/for-upstream' into staging

* Log filtering from Alex and Peter
* Chardev fix from Marc-André
* config.status tweak from David
* Header file tweaks

Merge remote-tracking branch 'remotes/bonzini/tags/for-upstream' into staging

* Log filtering from Alex and Peter
* Chardev fix from Marc-André
* config.status tweak from David
* Header file tweaks from Markus, myself and Veronia (Outreachy candidate)
* get_ticks_per_sec() removal from Rutuja (Outreachy candidate)
* Coverity fix from myself
* PKE implementation from myself, based on rth's XSAVE support

# gpg: Signature made Thu 24 Mar 2016 20:15:11 GMT using RSA key ID 78C7AE83
# gpg: Good signature from "Paolo Bonzini <bonzini@gnu.org>"
# gpg: aka "Paolo Bonzini <pbonzini@redhat.com>"

* remotes/bonzini/tags/for-upstream: (28 commits)
target-i386: implement PKE for TCG
config.status: Pass extra parameters
char: translate from QIOChannel error to errno
exec: fix error handling in file_ram_alloc
cputlb: modernise the debug support
qemu-log: support simple pid substitution for logs
target-arm: dfilter support for in_asm
qemu-log: dfilter-ise exec, out_asm, op and opt_op
qemu-log: new option -dfilter to limit output
qemu-log: Improve the "exec" TB execution logging
qemu-log: Avoid function call for disabled qemu_log_mask logging
qemu-log: correct help text for -d cpu
tcg: pass down TranslationBlock to tcg_code_gen
util: move declarations out of qemu-common.h
Replaced get_tick_per_sec() by NANOSECONDS_PER_SECOND
hw: explicitly include qemu-common.h and cpu.h
include/crypto: Include qapi-types.h or qemu/bswap.h instead of qemu-common.h
isa: Move DMA_transfer_handler from qemu-common.h to hw/isa/isa.h
Move ParallelIOArg from qemu-common.h to sysemu/char.h
Move QEMU_ALIGN_*() from qemu-common.h to qemu/osdep.h
...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

Conflicts:
scripts/clean-includes

show more ...

include/qemu/iov.h: Don't include qemu-common.h

qemu-common.h should only be included by .c files. Its file comment
explains why: "No header file should depend on qemu-common.h, as this
would easil

include/qemu/iov.h: Don't include qemu-common.h

qemu-common.h should only be included by .c files. Its file comment
explains why: "No header file should depend on qemu-common.h, as this
would easily lead to circular header dependencies."

qemu/iov.h includes qemu-common.h for QEMUIOVector stuff. Move all
that to qemu/iov.h and drop the ill-advised include. Include
qemu/iov.h where the QEMUIOVector stuff is now missing.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>

show more ...

Use scripts/clean-includes to drop redundant qemu/typedefs.h

Re-run scripts/clean-includes to apply the previous commit's
corrections and updates. Besides redundant qemu/typedefs.h, this only
finds

Use scripts/clean-includes to drop redundant qemu/typedefs.h

Re-run scripts/clean-includes to apply the previous commit's
corrections and updates. Besides redundant qemu/typedefs.h, this only
finds a redundant config-host.h include in ui/egl-helpers.c. No idea
how that escaped the previous runs.

Some manual whitespace trimming around dropped includes squashed in.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>

show more ...

Merge remote-tracking branch 'remotes/kevin/tags/for-upstream' into staging

Block layer patches

# gpg: Signature made Thu 17 Mar 2016 15:49:29 GMT using RSA key ID C88F2FD6
# gpg: Good signature fr

Merge remote-tracking branch 'remotes/kevin/tags/for-upstream' into staging

Block layer patches

# gpg: Signature made Thu 17 Mar 2016 15:49:29 GMT using RSA key ID C88F2FD6
# gpg: Good signature from "Kevin Wolf <kwolf@redhat.com>"

* remotes/kevin/tags/for-upstream: (29 commits)
iotests: Test QUORUM_REPORT_BAD in fifo mode
quorum: Emit QUORUM_REPORT_BAD for reads in fifo mode
block: Use blk_co_pwritev() in blk_co_write_zeroes()
block: Use blk_aio_prwv() for aio_read/write/write_zeroes
block: Use blk_prw() in blk_pread()/blk_pwrite()
block: Use blk_co_pwritev() in blk_write_zeroes()
block: Pull up blk_read_unthrottled() implementation
block: Use blk_co_pwritev() for blk_write()
block: Use blk_co_preadv() for blk_read()
block: Use BdrvChild in BlockBackend
block: Remove bdrv_states list
block: Use bdrv_next() instead of bdrv_states
block: Rewrite bdrv_next()
block: Add blk_next_root_bs()
block: Add bdrv_next_monitor_owned()
block: Move some bdrv_*_all() functions to BB
blockdev: Remove blk_hide_on_behalf_of_hmp_drive_del()
blockdev: Split monitor reference from BB creation
blockdev: Separate BB name management
blockdev: Add list of all BlockBackends
...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

block: Add blk_next_root_bs()

This function iterates over all BDSs attached to a BB. We are going to
need it when rewriting bdrv_next() so it no longer uses bdrv_states.

Signed-off-by: Max Reitz <m

block: Add blk_next_root_bs()

This function iterates over all BDSs attached to a BB. We are going to
need it when rewriting bdrv_next() so it no longer uses bdrv_states.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>

show more ...

blockdev: Remove blk_hide_on_behalf_of_hmp_drive_del()

We can basically inline it in hmp_drive_del(); monitor_remove_blk() is
called already, so we just need to call bdrv_make_anon(), too.

Signed-o

blockdev: Remove blk_hide_on_behalf_of_hmp_drive_del()

We can basically inline it in hmp_drive_del(); monitor_remove_blk() is
called already, so we just need to call bdrv_make_anon(), too.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>

show more ...

blockdev: Split monitor reference from BB creation

Before this patch, blk_new() automatically assigned a name to the new
BlockBackend and considered it referenced by the monitor. This patch
removes

blockdev: Split monitor reference from BB creation

Before this patch, blk_new() automatically assigned a name to the new
BlockBackend and considered it referenced by the monitor. This patch
removes the implicit monitor_add_blk() call from blk_new() (and
consequently the monitor_remove_blk() call from blk_delete(), too) and
thus blk_new() (and related functions) no longer take a BB name
argument.

In fact, there is only a single point where blk_new()/blk_new_open() is
called and the new BB is monitor-owned, and that is in blockdev_init().
Besides thus relieving us from having to invent names for all of the BBs
we use in qemu-img, this fixes a bug where qemu cannot create a new
image if there already is a monitor-owned BB named "image".

If a BB and its BDS tree are created in a single operation, as of this
patch the BDS tree will be created before the BB is given a name
(whereas it was the other way around before). This results in minor
change to the output of iotest 087, whose reference output is amended
accordingly.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>

show more ...

blockdev: Separate BB name management

Introduce separate functions (monitor_add_blk() and
monitor_remove_blk()) which set or unset a BB name. Since the name is
equivalent to the monitor's reference

blockdev: Separate BB name management

Introduce separate functions (monitor_add_blk() and
monitor_remove_blk()) which set or unset a BB name. Since the name is
equivalent to the monitor's reference to a BB, adding a name the same as
declaring the BB to be monitor-owned and removing it revokes this
status, hence the function names.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>

show more ...

block: Add blk_commit_all()

Later, we will remove bdrv_commit_all() and move its contents here, and
in order to replace bdrv_commit_all() calls by calls to blk_commit_all()
before doing so, we need

block: Add blk_commit_all()

Later, we will remove bdrv_commit_all() and move its contents here, and
in order to replace bdrv_commit_all() calls by calls to blk_commit_all()
before doing so, we need to add it as an alias now.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>

show more ...

Merge remote-tracking branch 'remotes/kevin/tags/for-upstream' into staging

Block layer patches

# gpg: Signature made Mon 14 Mar 2016 16:36:52 GMT using RSA key ID C88F2FD6
# gpg: Good signature fr

Merge remote-tracking branch 'remotes/kevin/tags/for-upstream' into staging

Block layer patches

# gpg: Signature made Mon 14 Mar 2016 16:36:52 GMT using RSA key ID C88F2FD6
# gpg: Good signature from "Kevin Wolf <kwolf@redhat.com>"

* remotes/kevin/tags/for-upstream: (40 commits)
iotests: Add test for QMP event rates
monitor: Use QEMU_CLOCK_VIRTUAL for the event queue in qtest mode
monitor: Separate QUORUM_REPORT_BAD events according to the node name
quorum: Fix crash in quorum_aio_cb()
iotests: Correct 081's reference output
block: Remove unused typedef of BlockDriverDirtyHandler
block: Move block dirty bitmap code to separate files
typedefs: Add BdrvDirtyBitmap
block: Include hbitmap.h in block.h
backup: Use Bitmap to replace "s->bitmap"
vpc: Use BB functions in .bdrv_create()
vmdk: Use BB functions in .bdrv_create()
vhdx: Use BB functions in .bdrv_create()
vdi: Use BB functions in .bdrv_create()
sheepdog: Use BB functions in .bdrv_create()
qed: Use BB functions in .bdrv_create()
qcow2: Use BB functions in .bdrv_create()
qcow: Use BB functions in .bdrv_create()
parallels: Use BB functions in .bdrv_create()
block: Introduce blk_set_allow_write_beyond_eof()
...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

block: Introduce blk_set_allow_write_beyond_eof()

We check that the guest can't write beyond the end of its disk, but for
other internal users it can make sense to allow growing a file.

Signed-off-

block: Introduce blk_set_allow_write_beyond_eof()

We check that the guest can't write beyond the end of its disk, but for
other internal users it can make sense to allow growing a file.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>

show more ...

include: Clean up includes

Clean up includes so that osdep.h is included first and headers
which it implies are not included manually.

This commit was created with scripts/clean-includes.

NB: If t

include: Clean up includes

Clean up includes so that osdep.h is included first and headers
which it implies are not included manually.

This commit was created with scripts/clean-includes.

NB: If this commit breaks compilation for your out-of-tree
patchseries or fork, then you need to make sure you add
#include "qemu/osdep.h" to any new .c files that you have.

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Eric Blake <eblake@redhat.com>

show more ...

Merge remote-tracking branch 'remotes/maxreitz/tags/pull-block-for-peter-2016-02-02' into staging

Block patches

# gpg: Signature made Tue 02 Feb 2016 17:23:44 GMT using RSA key ID E838ACAD
# gpg: G

Merge remote-tracking branch 'remotes/maxreitz/tags/pull-block-for-peter-2016-02-02' into staging

Block patches

# gpg: Signature made Tue 02 Feb 2016 17:23:44 GMT using RSA key ID E838ACAD
# gpg: Good signature from "Max Reitz <mreitz@redhat.com>"

* remotes/maxreitz/tags/pull-block-for-peter-2016-02-02: (50 commits)
block: qemu-iotests - add test for snapshot, commit, snapshot bug
block: set device_list.tqe_prev to NULL on BDS removal
iotests: Add "qemu-img map" test for VMDK extents
qemu-img: Make MapEntry a QAPI struct
qemu-img: In "map", use the returned "file" from bdrv_get_block_status
block: Use returned *file in bdrv_co_get_block_status
vmdk: Return extent's file in bdrv_get_block_status
vmdk: Fix calculation of block status's offset
vpc: Assign bs->file->bs to file in vpc_co_get_block_status
vdi: Assign bs->file->bs to file in vdi_co_get_block_status
sheepdog: Assign bs to file in sd_co_get_block_status
qed: Assign bs->file->bs to file in bdrv_qed_co_get_block_status
parallels: Assign bs->file->bs to file in parallels_co_get_block_status
iscsi: Assign bs to file in iscsi_co_get_block_status
raw: Assign bs to file in raw_co_get_block_status
qcow2: Assign bs->file->bs to file in qcow2_co_get_block_status
qcow: Assign bs->file->bs to file in qcow_co_get_block_status
block: Add "file" output parameter to block status query functions
block: acquire in bdrv_query_image_info
iotests: Add test for block jobs and BDS ejection
...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

block: Add blk_remove_all_bs()

When bdrv_close_all() is called, instead of force-closing all root
BlockDriverStates, it is better to just drop the reference from all
BlockBackends and let them be cl

block: Add blk_remove_all_bs()

When bdrv_close_all() is called, instead of force-closing all root
BlockDriverStates, it is better to just drop the reference from all
BlockBackends and let them be closed automatically. This prevents BDS
from getting closed that are still referenced by other BDS, which may
result in loss of cached data.

This patch adds a function for doing that, but does not yet incorporate
it in bdrv_close_all().

Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>

show more ...

block: Remove BDS close notifier

It is unused now, so we can remove it.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Fam Zheng <famz@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.c

block: Remove BDS close notifier

It is unused now, so we can remove it.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Fam Zheng <famz@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>

show more ...

block: Add BB-BDS remove/insert notifiers

bdrv_close() no longer signifies ejection of a medium, this is now done
by removing the BDS from the BB. Therefore, we want to have a notifier
for that in t

block: Add BB-BDS remove/insert notifiers

bdrv_close() no longer signifies ejection of a medium, this is now done
by removing the BDS from the BB. Therefore, we want to have a notifier
for that in the BB instead of a close notifier in the BDS. The former is
added now, the latter is removed later.

Symmetrically, another notifier list is added that is invoked whenever a
BDS is inserted. We will need that for virtio-blk and virtio-scsi, which
can then remove their op blockers on BDS ejection and set them up on
insertion.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Fam Zheng <famz@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>

show more ...

Merge remote-tracking branch 'remotes/bonzini/tags/for-upstream' into staging

* qemu-char logfile facility
* NBD coroutine based negotiation
* bugfixes

# gpg: Signature made Fri 15 Jan 2016 17:58:2

Merge remote-tracking branch 'remotes/bonzini/tags/for-upstream' into staging

* qemu-char logfile facility
* NBD coroutine based negotiation
* bugfixes

# gpg: Signature made Fri 15 Jan 2016 17:58:28 GMT using RSA key ID 78C7AE83
# gpg: Good signature from "Paolo Bonzini <bonzini@gnu.org>"
# gpg: aka "Paolo Bonzini <pbonzini@redhat.com>"

* remotes/bonzini/tags/for-upstream:
qemu-char: do not leak QemuMutex when freeing a character device
qemu-char: add logfile facility to all chardev backends
nbd-server: do not exit on failed memory allocation
nbd-server: do not check request length except for reads and writes
nbd-server: Coroutine based negotiation
nbd: Split nbd.c
nbd: Always call "close_fn" in nbd_client_new
SCSI device: fix to incomplete QOMify
iscsi: send readcapacity10 when readcapacity16 failed
qemu-char: delete send_all/recv_all helper methods
vmw_pvscsi: x-disable-pcie, x-old-pci-configuration back-compat props are 2.5 specific
scsi: initialise info object with appropriate size
i386: avoid null pointer dereference
target-i386: do not duplicate page protection checks
scsi: revert change to scsi_req_cancel_async and add assertions

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

nbd-server: do not exit on failed memory allocation

The amount of memory allocated in nbd_co_receive_request is driven by the
NBD client (possibly a virtual machine). Parallel I/O can cause the
ser

nbd-server: do not exit on failed memory allocation

The amount of memory allocated in nbd_co_receive_request is driven by the
NBD client (possibly a virtual machine). Parallel I/O can cause the
server to allocate a large amount of memory; check for failures and
return ENOMEM in that case.

Cc: qemu-block@nongnu.org
Reviewed-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>

show more ...

Merge remote-tracking branch 'remotes/stefanha/tags/block-pull-request' into staging

# gpg: Signature made Tue 22 Dec 2015 08:52:55 GMT using RSA key ID 81AB73C8
# gpg: Good signature from "Stefan H

Merge remote-tracking branch 'remotes/stefanha/tags/block-pull-request' into staging

# gpg: Signature made Tue 22 Dec 2015 08:52:55 GMT using RSA key ID 81AB73C8
# gpg: Good signature from "Stefan Hajnoczi <stefanha@redhat.com>"
# gpg: aka "Stefan Hajnoczi <stefanha@gmail.com>"

* remotes/stefanha/tags/block-pull-request:
sdhci: add optional quirk property to disable card insertion/removal interrupts
sdhci: don't raise a command index error for an unexpected response
sd: sdhci: Delete over-zealous power check
scripts/gdb: Fix a python exception in mtree.py
parallels: add format spec
block/mirror: replace IOV_MAX with blk_get_max_iov()
block: replace IOV_MAX with BlockLimits.max_iov
block-backend: add blk_get_max_iov()
block: add BlockLimits.max_iov field
virtio-blk: trivial code optimization

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

block-backend: add blk_get_max_iov()

Add a function to query BlockLimits.max_iov.

Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>

Merge remote-tracking branch 'remotes/jnsnow/tags/ide-pull-request' into staging

# gpg: Signature made Tue 17 Nov 2015 20:06:58 GMT using RSA key ID AAFC390E
# gpg: Good signature from "John Snow (J

Merge remote-tracking branch 'remotes/jnsnow/tags/ide-pull-request' into staging

# gpg: Signature made Tue 17 Nov 2015 20:06:58 GMT using RSA key ID AAFC390E
# gpg: Good signature from "John Snow (John Huston) <jsnow@redhat.com>"

* remotes/jnsnow/tags/ide-pull-request:
ide: enable buffered requests for PIO read requests
ide: enable buffered requests for ATAPI devices
ide: orphan all buffered requests on DMA cancel
ide: add support for IDEBufferedRequest
block: add blk_abort_aio_request
ide/atapi: make PIO read requests async

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

block: add blk_abort_aio_request

Signed-off-by: Peter Lieven <pl@kamp.de>
Reviewed-by: Fam Zheng <famz@redhat.com>
Message-id: 1447345846-15624-3-git-send-email-pl@kamp.de
Signed-off-by: John Snow <

block: add blk_abort_aio_request

Signed-off-by: Peter Lieven <pl@kamp.de>
Reviewed-by: Fam Zheng <famz@redhat.com>
Message-id: 1447345846-15624-3-git-send-email-pl@kamp.de
Signed-off-by: John Snow <jsnow@redhat.com>

show more ...

Merge remote-tracking branch 'remotes/kevin/tags/for-upstream' into staging

Block layer patches

# gpg: Signature made Wed 11 Nov 2015 16:03:19 GMT using RSA key ID C88F2FD6
# gpg: Good signature fr

Merge remote-tracking branch 'remotes/kevin/tags/for-upstream' into staging

Block layer patches

# gpg: Signature made Wed 11 Nov 2015 16:03:19 GMT using RSA key ID C88F2FD6
# gpg: Good signature from "Kevin Wolf <kwolf@redhat.com>"

* remotes/kevin/tags/for-upstream: (41 commits)
iotests: Check for quorum support in test 139
qcow2: Fix qcow2_get_cluster_offset() for zero clusters
iotests: Add tests for the x-blockdev-del command
block: Add 'x-blockdev-del' QMP command
block: Add blk_get_refcnt()
mirror: block all operations on the target image during the job
qemu-iotests: fix -valgrind option for check
qemu-iotests: fix cleanup of background processes
qemu-io: Correct error messages
qemu-io: Check for trailing chars
qemu-io: fix cvtnum lval types
block: test 'blockdev-snapshot' using a file BDS as the overlay
block: Remove inner quotation marks in iotest 085
block: Disallow snapshots if the overlay doesn't support backing files
throttle: Use bs->throttle_state instead of bs->io_limits_enabled
throttle: Check for pending requests in throttle_group_unregister_bs()
qemu-img: add check for zero-length job len
qcow2: avoid misaligned 64bit bswap
qemu-iotests: Test the reopening of overlay_bs in 'block-commit'
commit: reopen overlay_bs before base
...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

Merge remote-tracking branch 'mreitz/tags/pull-block-for-kevin-2015-11-11' into queue-block

Block patches from 2015-10-26 until 2015-11-11.

# gpg: Signature made Wed Nov 11 17:00:50 2015 CET using

Merge remote-tracking branch 'mreitz/tags/pull-block-for-kevin-2015-11-11' into queue-block

Block patches from 2015-10-26 until 2015-11-11.

# gpg: Signature made Wed Nov 11 17:00:50 2015 CET using RSA key ID E838ACAD
# gpg: Good signature from "Max Reitz <mreitz@redhat.com>"

* mreitz/tags/pull-block-for-kevin-2015-11-11:
iotests: Check for quorum support in test 139
qcow2: Fix qcow2_get_cluster_offset() for zero clusters
iotests: Add tests for the x-blockdev-del command
block: Add 'x-blockdev-del' QMP command
block: Add blk_get_refcnt()
mirror: block all operations on the target image during the job
qemu-iotests: fix -valgrind option for check
qemu-iotests: fix cleanup of background processes

Signed-off-by: Kevin Wolf <kwolf@redhat.com>

show more ...