Merge remote-tracking branch 'remotes/kevin/tags/for-upstream' into staging

Block patches

# gpg: Signature made Fri 21 Feb 2014 21:42:24 GMT using RSA key ID C88F2FD6
# gpg: Good signature from "Ke

Merge remote-tracking branch 'remotes/kevin/tags/for-upstream' into staging

Block patches

# gpg: Signature made Fri 21 Feb 2014 21:42:24 GMT using RSA key ID C88F2FD6
# gpg: Good signature from "Kevin Wolf <kwolf@redhat.com>"

* remotes/kevin/tags/for-upstream: (54 commits)
iotests: Mixed quorum child device specifications
quorum: Simplify quorum_open()
quorum: Add unit test.
quorum: Add quorum_open() and quorum_close().
quorum: Implement recursive .bdrv_recurse_is_first_non_filter in quorum.
quorum: Add quorum_co_flush().
quorum: Add quorum_invalidate_cache().
quorum: Add quorum_getlength().
quorum: Add quorum mechanism.
quorum: Add quorum_aio_readv.
blkverify: Extract qemu_iovec_clone() and qemu_iovec_compare() from blkverify.
quorum: Add quorum_aio_writev and its dependencies.
quorum: Create BDRVQuorumState and BlkDriver and do init.
quorum: Create quorum.c, add QuorumChildRequest and QuorumAIOCB.
check-qdict: Test termination of qdict_array_split()
check-qdict: Adjust test for qdict_array_split()
qdict: Extract non-QDicts in qdict_array_split()
qemu-config: Sections must consist of keys
qemu-iotests: Check qemu-img command line parsing
qemu-img: Allow -o help with incomplete argument list
...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

vhdx: correctly propagate errors

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Reviewed-by: Fam Zheng <famz@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>

block: Make bdrv_file_open() static

Add the bdrv_open() option BDRV_O_PROTOCOL which results in passing the
call to bdrv_file_open(). Additionally, make bdrv_file_open() static and
therefore bdrv_op

block: Make bdrv_file_open() static

Add the bdrv_open() option BDRV_O_PROTOCOL which results in passing the
call to bdrv_file_open(). Additionally, make bdrv_file_open() static and
therefore bdrv_open() the only way to call it.

Consequently, all existing calls to bdrv_file_open() have to be adjusted
to use bdrv_open() with the BDRV_O_PROTOCOL flag instead.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Benoit Canet <benoit@irqsave.net>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>

show more ...

Merge remote-tracking branch 'remotes/stefanha/tags/block-pull-request' into staging

Block pull request

# gpg: Signature made Fri 31 Jan 2014 21:16:43 GMT using RSA key ID 81AB73C8
# gpg: Good sign

Merge remote-tracking branch 'remotes/stefanha/tags/block-pull-request' into staging

Block pull request

# gpg: Signature made Fri 31 Jan 2014 21:16:43 GMT using RSA key ID 81AB73C8
# gpg: Good signature from "Stefan Hajnoczi <stefanha@redhat.com>"
# gpg: aka "Stefan Hajnoczi <stefanha@gmail.com>"
# gpg: WARNING: This key is not certified with a trusted signature!
# gpg: There is no indication that the signature belongs to the owner.
# Primary key fingerprint: 8695 A8BF D3F9 7CDA AC35 775A 9CA4 ABB3 81AB 73C8

* remotes/stefanha/tags/block-pull-request:
qemu-iotests: only run 071 on qcow2
dataplane: Comment fix
block/vhdx: Error checking fixes
qemu-iotests: Drop assert_no_active_commit in case 040
block/vmdk: add basic .bdrv_check support
block: remove qcow2 .bdrv_make_empty implementation
block: remove QED .bdrv_make_empty implementation
Describe flaws in qcow/qcow2 encryption in the docs

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

block/vhdx: Error checking fixes

Errors are inadvertently ignored in a few places. Has always been
broken. Spotted by Coverity.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: J

block/vhdx: Error checking fixes

Errors are inadvertently ignored in a few places. Has always been
broken. Spotted by Coverity.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Jeff Cody <jcody@redhat.com>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>

show more ...

Merge remote-tracking branch 'kwolf/tags/for-anthony' into staging

Block patches

# gpg: Signature made Fri 24 Jan 2014 08:40:53 AM PST using RSA key ID C88F2FD6
# gpg: Can't check signature: public

Merge remote-tracking branch 'kwolf/tags/for-anthony' into staging

Block patches

# gpg: Signature made Fri 24 Jan 2014 08:40:53 AM PST using RSA key ID C88F2FD6
# gpg: Can't check signature: public key not found

* kwolf/tags/for-anthony: (93 commits)
block: Switch bdrv_io_limits_intercept() to byte granularity
qemu-iotests: Test pwritev RMW logic
qemu-io: New command 'sleep'
blkdebug: Make required alignment configurable
iscsi: Set bs->request_alignment
block: Make bdrv_pwrite() a bdrv_prwv_co() wrapper
block: Make bdrv_pread() a bdrv_prwv_co() wrapper
block: Change coroutine wrapper to byte granularity
block: Assert serialisation assumptions in pwritev
block: Align requests in bdrv_co_do_pwritev()
block: Allow wait_serialising_requests() at any point
block: Make overlap range for serialisation dynamic
block: Generalise and optimise COR serialisation
block: Make zero-after-EOF work with larger alignment
block: Allow waiting for overlapping requests between begin/end
block: Switch BdrvTrackedRequest to byte granularity
block: Introduce bdrv_co_do_pwritev()
block: write: Handle COR dependency after I/O throttling
block: Introduce bdrv_aligned_pwritev()
block: Introduce bdrv_co_do_preadv()
...

Message-id: 1390584136-24703-1-git-send-email-kwolf@redhat.com
Signed-off-by: Anthony Liguori <aliguori@amazon.com>

show more ...

block: Allow reference for bdrv_file_open()

Allow specifying a reference to an existing block device (by name) for
bdrv_file_open() instead of a filename and/or options.

Signed-off-by: Max Reitz <m

block: Allow reference for bdrv_file_open()

Allow specifying a reference to an existing block device (by name) for
bdrv_file_open() instead of a filename and/or options.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>

show more ...

Merge remote-tracking branch 'stefanha/block' into staging

* stefanha/block:
commit: Remove unused check
qemu-iotests: Update test cases for commit active
commit: Support commit active layer

Merge remote-tracking branch 'stefanha/block' into staging

* stefanha/block:
commit: Remove unused check
qemu-iotests: Update test cases for commit active
commit: Support commit active layer
block: Add commit_active_start()
mirror: Move base to MirrorBlockJob
mirror: Don't close target
qemu-iotests: drop duplicate virtio-blk initialization failure
vmdk: Allow vmdk_create to work with protocol
vmdk: Check VMFS extent line field number
docs: updated qemu-img man page and qemu-doc to reflect VHDX support.
block: vhdx - improve error message, and .bdrv_check implementation
block/iscsi: Fix compilation for libiscsi 1.4.0 (API change)
qapi-schema: fix QEMU 1.8 references
dataplane: replace hostmem with memory_region_find
dataplane: change vring API to use VirtQueueElement
vring: factor common code for error exits
vring: create a common function to parse descriptors
sheepdog: fix dynamic grow for running qcow2 format

Message-id: 1387554416-5837-1-git-send-email-stefanha@redhat.com
Signed-off-by: Anthony Liguori <aliguori@amazon.com>

show more ...

block: vhdx - improve error message, and .bdrv_check implementation

If there is a dirty log file to be replayed in a VHDX image, it is
replayed in .vhdx_open(). However, if the file is opened read-

block: vhdx - improve error message, and .bdrv_check implementation

If there is a dirty log file to be replayed in a VHDX image, it is
replayed in .vhdx_open(). However, if the file is opened read-only,
then a somewhat cryptic error message results.

This adds a more helpful error message for the user. If an image file
contains a log to be replayed, and is opened read-only, the user is
instructed to run 'qemu-img check -r all' on the image file.

Running qemu-img check -r all will cause the image file to be opened
r/w, which will replay the log file. If a log file replay is detected,
this is flagged, and bdrv_check will increase the corruptions_fixed
count for the image.

[Fixed typo in error message that was pointed out by Eric Blake
<eblake@redhat.com>.
--Stefan]

Signed-off-by: Jeff Cody <jcody@redhat.com>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>

show more ...

Merge remote-tracking branch 'stefanha/block' into staging

# By Paolo Bonzini (17) and others
# Via Stefan Hajnoczi
* stefanha/block: (48 commits)
qemu-iotests: filter QEMU monitor \r\n
aio: mak

Merge remote-tracking branch 'stefanha/block' into staging

# By Paolo Bonzini (17) and others
# Via Stefan Hajnoczi
* stefanha/block: (48 commits)
qemu-iotests: filter QEMU monitor \r\n
aio: make aio_poll(ctx, true) block with no fds
block: clean up bdrv_drain_all() throttling comments
qcow2: use start_of_cluster() and offset_into_cluster() everywhere
qemu-img: decrease progress update interval on convert
qemu-img: round down request length to an aligned sector
qemu-img: dynamically adjust iobuffer size during convert
block/iscsi: set bs->bl.opt_transfer_length
block: add opt_transfer_length to BlockLimits
block/iscsi: set bdi->cluster_size
qemu-img: fix usage instruction for qemu-img convert
qemu-img: add support for skipping zeroes in input during convert
qemu-nbd: add doc for option -f
qemu-iotests: add test for snapshot in qemu-img convert
qemu-img: add -l for snapshot in convert
qemu-iotests: add 058 internal snapshot export with qemu-nbd case
qemu-nbd: support internal snapshot export
snapshot: distinguish id and name in load_tmp
qemu-iotests: Split qcow2 only cases in 048
qemu-iotests: Clean up spaces in usage output
...

Message-id: 1386347807-27359-1-git-send-email-stefanha@redhat.com
Signed-off-by: Anthony Liguori <aliguori@amazon.com>

show more ...

block drivers: add discard/write_zeroes properties to bdrv_get_info implementation

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Reviewed-by: Peter Lieven <pl@kamp.de>
Signed-off-by: Stefan Haj

block drivers: add discard/write_zeroes properties to bdrv_get_info implementation

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Reviewed-by: Peter Lieven <pl@kamp.de>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>

show more ...

vpc, vhdx: add get_info

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Reviewed-by: Peter Lieven <pl@kamp.de>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>

Merge remote-tracking branch 'stefanha/block' into staging

# By Jeff Cody (26) and others
# Via Stefan Hajnoczi
* stefanha/block: (37 commits)
block: Round up total_sectors
block: vhdx qemu-iote

Merge remote-tracking branch 'stefanha/block' into staging

# By Jeff Cody (26) and others
# Via Stefan Hajnoczi
* stefanha/block: (37 commits)
block: Round up total_sectors
block: vhdx qemu-iotest - log replay of data sector
block: qemu-iotests for vhdx, add write test support
block: vhdx - update _make_test_img() to filter out vhdx options
block: vhdx - add .bdrv_create() support
block: vhdx - fix comment typos in header, fix incorrect struct fields
block: vhdx - break out code operations to functions
block: vhdx - move more endian translations to vhdx-endian.c
block: vhdx - remove BAT file offset bit shifting
block: vhdx write support
block: vhdx - add log write support
block: vhdx - add region overlap detection for image files
block: vhdx - log parsing, replay, and flush support
block: vhdx code movement - move vhdx_close() above vhdx_open()
block: vhdx - update log guid in header, and first write tracker
block: vhdx - break endian translation functions out
block: vhdx - log support struct and defines
block: vhdx code movement - VHDXMetadataEntries and BDRVVHDXState to header.
block: vhdx - add header update capability.
block: vhdx - minor comments and typo correction.
...

Message-id: 1383905551-16411-1-git-send-email-stefanha@redhat.com
Signed-off-by: Anthony Liguori <aliguori@amazon.com>

show more ...

block: vhdx - add .bdrv_create() support

This adds support for VHDX image creation, for images of type "Fixed"
and "Dynamic". "Differencing" types (i.e., VHDX images with backing
files) are current

block: vhdx - add .bdrv_create() support

This adds support for VHDX image creation, for images of type "Fixed"
and "Dynamic". "Differencing" types (i.e., VHDX images with backing
files) are currently not supported.

Options for image creation include:
* log size:
The size of the journaling log for VHDX. Minimum is 1MB,
and it must be a multiple of 1MB. Invalid log sizes will be
silently fixed by rounding up to the nearest MB.

Default is 1MB.

* block size:
This is the size of a payload block. The range is 1MB to 256MB,
inclusive, and must be a multiple of 1MB as well. Invalid sizes
and multiples will be silently fixed. If '0' is passed, then
a sane size is chosen (depending on virtual image size).

Default is 0 (Auto-select).

* subformat:
- "dynamic"
An image without data pre-allocated.
- "fixed"
An image with data pre-allocated.

Default is "dynamic"

When creating the image file, the lettered sections are created:

-----------------------------------------------------------------.
| (A) | (B) | (C) | (D) | (E)
| File ID | Header1 | Header 2 | Region Tbl 1 | Region Tbl 2
| | | | |
.-----------------------------------------------------------------.
0 64KB 128KB 192KB 256KB 320KB

.---- ~ ----------- ~ ------------ ~ ---------------- ~ -----------.
| (F) | (G) | (H) |
| Journal Log | BAT / Bitmap | Metadata | .... data ......
| | | |
.---- ~ ----------- ~ ------------ ~ ---------------- ~ -----------.
1MB (var.) (var.) (var.)

Signed-off-by: Jeff Cody <jcody@redhat.com>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>

show more ...

block: vhdx - break out code operations to functions

This is preperation for vhdx_create(). The ability to write headers,
and calculate the number of BAT entries will be needed within the
create()

block: vhdx - break out code operations to functions

This is preperation for vhdx_create(). The ability to write headers,
and calculate the number of BAT entries will be needed within the
create() functions, so move this relevant code into helper functions.

Signed-off-by: Jeff Cody <jcody@redhat.com>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>

show more ...

block: vhdx - move more endian translations to vhdx-endian.c

In preparation for vhdx_create(), move more endian translation
functions out to vhdx-endian.c.

Signed-off-by: Jeff Cody <jcody@redhat.co

block: vhdx - move more endian translations to vhdx-endian.c

In preparation for vhdx_create(), move more endian translation
functions out to vhdx-endian.c.

Signed-off-by: Jeff Cody <jcody@redhat.com>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>

show more ...

block: vhdx - remove BAT file offset bit shifting

Bit shifting can be fun, but in this case it was unnecessary. The
upper 44 bits of the 64-bit BAT entry is specifies the File Offset,
so we shifted

block: vhdx - remove BAT file offset bit shifting

Bit shifting can be fun, but in this case it was unnecessary. The
upper 44 bits of the 64-bit BAT entry is specifies the File Offset,
so we shifted the bits to get access to the value.

However, per the spec the value is in MB. So we dutifully shifted back
to the left by 20 bits, to convert to a true uint64_t file offset.

This replaces those steps with just a bit mask, to get rid of the lower
20 bits instead.

Signed-off-by: Jeff Cody <jcody@redhat.com>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>

show more ...

block: vhdx write support

This adds support for writing to VHDX image files, using coroutines.
Writes into the BAT table goes through the VHDX log. Currently, BAT
table writes occur when expanding

block: vhdx write support

This adds support for writing to VHDX image files, using coroutines.
Writes into the BAT table goes through the VHDX log. Currently, BAT
table writes occur when expanding a dynamic VHDX file, and allocating a
new BAT entry.

Signed-off-by: Jeff Cody <jcody@redhat.com>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>

show more ...

block: vhdx - add region overlap detection for image files

Regions in the image file cannot overlap - the log, region tables,
and metdata must all be unique and non-overlapping.

This adds region ch

block: vhdx - add region overlap detection for image files

Regions in the image file cannot overlap - the log, region tables,
and metdata must all be unique and non-overlapping.

This adds region checking by means of a QLIST; there can be a variable
number of regions and metadata (there may be metadata or region tables
that we do not recognize / know about, but are not required).

This adds the capability to register a region for later checking, and
to check against registered regions for any overlap.

Also, if neither the BAT or Metadata region tables are found, return
error.

Signed-off-by: Jeff Cody <jcody@redhat.com>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>

show more ...

block: vhdx - log parsing, replay, and flush support

This adds support for VHDX v0 logs, as specified in Microsoft's
VHDX Specification Format v1.00:
https://www.microsoft.com/en-us/download/details

block: vhdx - log parsing, replay, and flush support

This adds support for VHDX v0 logs, as specified in Microsoft's
VHDX Specification Format v1.00:
https://www.microsoft.com/en-us/download/details.aspx?id=34750

The following support is added:

* Log parsing, and validation - validate that an existing log
is correct.

* Log search - search through an existing log, to find any valid
sequence of entries.

* Log replay and flush - replay an existing log, and flush/clear
the log when complete.

The VHDX log is a circular buffer, with elements (sectors) of 4KB.

A log entry is a variably-length number of sectors, that is
comprised of a header and 'descriptors', that describe each sector.

A log may contain multiple entries, know as a log sequence. In a log
sequence, each log entry immediately follows the previous entry, with an
incrementing sequence number. There can only ever be one active and
valid sequence in the log.

Each log entry must match the file log GUID in order to be valid (along
with other criteria). Once we have flushed all valid log entries, we
marked the file log GUID to be zero, which indicates a buffer with no
valid entries.

Signed-off-by: Jeff Cody <jcody@redhat.com>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>

show more ...

block: vhdx code movement - move vhdx_close() above vhdx_open()

Signed-off-by: Jeff Cody <jcody@redhat.com>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>

block: vhdx - update log guid in header, and first write tracker

Allow tracking of first file write in the VHDX image, as well as
the ability to update the GUID in the header. This is in preparatio

block: vhdx - update log guid in header, and first write tracker

Allow tracking of first file write in the VHDX image, as well as
the ability to update the GUID in the header. This is in preparation
for log support.

Signed-off-by: Jeff Cody <jcody@redhat.com>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>

show more ...

block: vhdx - break endian translation functions out

This moves the endian translation functions out from the vhdx.c source,
into a separate source file. In addition to the previously defined
endian

block: vhdx - break endian translation functions out

This moves the endian translation functions out from the vhdx.c source,
into a separate source file. In addition to the previously defined
endian functions, new endian translation functions for log support are
added as well.

Signed-off-by: Jeff Cody <jcody@redhat.com>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>

show more ...

block: vhdx code movement - VHDXMetadataEntries and BDRVVHDXState to header.

In preparation for VHDX log support, move these structures to the
header.

Signed-off-by: Jeff Cody <jcody@redhat.com>
Si

block: vhdx code movement - VHDXMetadataEntries and BDRVVHDXState to header.

In preparation for VHDX log support, move these structures to the
header.

Signed-off-by: Jeff Cody <jcody@redhat.com>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>

show more ...

block: vhdx - add header update capability.

This adds the ability to update the headers in a VHDX image, including
generating a new MS-compatible GUID.

As VHDX depends on uuid.h, VHDX is now a conf

block: vhdx - add header update capability.

This adds the ability to update the headers in a VHDX image, including
generating a new MS-compatible GUID.

As VHDX depends on uuid.h, VHDX is now a configurable build option. If
VHDX support is enabled, that will also enable uuid as well. The
default is to have VHDX enabled.

To enable/disable VHDX: --enable-vhdx, --disable-vhdx

Signed-off-by: Jeff Cody <jcody@redhat.com>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>

show more ...