Merge remote-tracking branch 'remotes/ericb/tags/pull-nbd-2019-01-21' into staging

nbd patches for 2019-01-21

Add 'qemu-nbd --list' for probing a remote NBD server's advertisements.

- Eric Blake:

Merge remote-tracking branch 'remotes/ericb/tags/pull-nbd-2019-01-21' into staging

nbd patches for 2019-01-21

Add 'qemu-nbd --list' for probing a remote NBD server's advertisements.

- Eric Blake: 0/21 nbd: add qemu-nbd --list

# gpg: Signature made Mon 21 Jan 2019 22:44:27 GMT
# gpg: using RSA key A7A16B4A2527436A
# gpg: Good signature from "Eric Blake <eblake@redhat.com>"
# gpg: aka "Eric Blake (Free Software Programmer) <ebb9@byu.net>"
# gpg: aka "[jpeg image of size 6874]"
# Primary key fingerprint: 71C2 CC22 B1C4 6029 27D2 F3AA A7A1 6B4A 2527 436A

* remotes/ericb/tags/pull-nbd-2019-01-21: (21 commits)
iotests: Enhance 223, 233 to cover 'qemu-nbd --list'
nbd/client: Work around 3.0 bug for listing meta contexts
qemu-nbd: Add --list option
nbd/client: Add meta contexts to nbd_receive_export_list()
nbd/client: Add nbd_receive_export_list()
nbd/client: Refactor nbd_opt_go() to support NBD_OPT_INFO
nbd/client: Pull out oldstyle size determination
nbd/client: Split handshake into two functions
nbd/client: Refactor return of nbd_receive_negotiate()
nbd/client: Split out nbd_receive_one_meta_context()
nbd/client: Split out nbd_send_meta_query()
nbd/client: Change signature of nbd_negotiate_simple_meta_context()
nbd/client: Move export name into NBDExportInfo
nbd/client: Refactor nbd_receive_list()
qemu-nbd: Avoid strtol open-coding
nbd/server: Favor [u]int64_t over off_t
nbd/server: Hoist length check to qmp_nbd_server_add
qemu-nbd: Sanity check partition bounds
qemu-nbd: Enhance man page
maint: Allow for EXAMPLES in texi2pod
...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

qemu-nbd: Add --list option

We want to be able to detect whether a given qemu NBD server is
exposing the right export(s) and dirty bitmaps, at least for
regression testing. We could use 'nbd-client

qemu-nbd: Add --list option

We want to be able to detect whether a given qemu NBD server is
exposing the right export(s) and dirty bitmaps, at least for
regression testing. We could use 'nbd-client -l' from the upstream
NBD project to list exports, but it's annoying to rely on
out-of-tree binaries; furthermore, nbd-client doesn't necessarily
know about all of the qemu NBD extensions. Thus, it is time to add
a new mode to qemu-nbd that merely sniffs all possible information
from the server during handshake phase, then disconnects and dumps
the information.

This patch actually implements --list/-L, while reusing other
options such as --tls-creds for now designating how to connect
as the client (rather than their non-list usage of how to operate
as the server).

I debated about adding this functionality to something akin to
'qemu-img info' - but that tool does not readily lend itself
to connecting to an arbitrary NBD server without also tying to
a specific export (I may, however, still add ImageInfoSpecificNBD
for reporting the bitmaps available when connecting to a single
export). And, while it may feel a bit odd that normally
qemu-nbd is a server but 'qemu-nbd -L' is a client, we are not
really making the qemu-nbd binary that much larger, because
'qemu-nbd -c' has to operate as both server and client
simultaneously across two threads when feeding the kernel module
for /dev/nbdN access.

Sample output:
$ qemu-nbd -L
exports available: 1
export: ''
size: 65536
flags: 0x4ed ( flush fua trim zeroes df cache )
min block: 512
opt block: 4096
max block: 33554432
available meta contexts: 1
base:allocation

Note that the output only lists sizes if the server sent
NBD_FLAG_HAS_FLAGS, because a newstyle server does not give
the size otherwise. It has the side effect that for really
old servers that did not send any flags, the size is not
output even though it was available. However, I'm not too
concerned about that - oldstyle servers are (rightfully)
getting less common to encounter (qemu 3.0 was the last
version where we even serve it), and most existing servers
that still even offer oldstyle negotiation (such as nbdkit)
still send flags (since that was added to the NBD protocol
in 2007 to permit read-only connections).

Not done here, but maybe worth future experiments: capture
the meat of NBDExportInfo into a QAPI struct, and use the
generated QAPI pretty-printers instead of hand-rolling our
output loop. It would also permit us to add a JSON output
mode for machine parsing.

Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Richard W.M. Jones <rjones@redhat.com>
Message-Id: <20190117193658.16413-20-eblake@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>

show more ...

nbd/client: Move export name into NBDExportInfo

Refactor the 'name' parameter of nbd_receive_negotiate() from
being a separate parameter into being part of the in-out 'info'.
This also spills over t

nbd/client: Move export name into NBDExportInfo

Refactor the 'name' parameter of nbd_receive_negotiate() from
being a separate parameter into being part of the in-out 'info'.
This also spills over to a simplification of nbd_opt_go().

The main driver for this refactoring is that an upcoming patch
would like to add support to qemu-nbd to list information about
all exports available on a server, where the name(s) will be
provided by the server instead of the client. But another benefit
is that we can now allow the client to explicitly specify the
empty export name "" even when connecting to an oldstyle server
(even if qemu is no longer such a server after commit 7f7dfe2a).

Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Richard W.M. Jones <rjones@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20190117193658.16413-10-eblake@redhat.com>

show more ...

qemu-nbd: Avoid strtol open-coding

Our copy-and-pasted open-coding of strtol handling forgot to
handle overflow conditions. Use qemu_strto*() instead.

In the case of --partition, since we insist o

qemu-nbd: Avoid strtol open-coding

Our copy-and-pasted open-coding of strtol handling forgot to
handle overflow conditions. Use qemu_strto*() instead.

In the case of --partition, since we insist on a user-supplied
partition to be non-zero, we can use 0 rather than -1 for our
initial value to distinguish when a partition is not being
served, for slightly more optimal code.

The error messages for out-of-bounds values are less specific,
but should not be a terrible loss in quality.

Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Richard W.M. Jones <rjones@redhat.com>
Message-Id: <20190117193658.16413-8-eblake@redhat.com>

show more ...

nbd/server: Favor [u]int64_t over off_t

Although our compile-time environment is set up so that we always
support long files with 64-bit off_t, we have no guarantee whether
off_t is the same type as

nbd/server: Favor [u]int64_t over off_t

Although our compile-time environment is set up so that we always
support long files with 64-bit off_t, we have no guarantee whether
off_t is the same type as int64_t. This requires casts when
printing values, and prevents us from directly using qemu_strtoi64()
(which will be done in the next patch). Let's just flip to uint64_t
where possible, and stick to int64_t for detecting failure of
blk_getlength(); we also keep the assertions added in the previous
patch that the resulting values fit in 63 bits. The overflow check
in nbd_co_receive_request() was already sane (request->from is
validated to fit in 63 bits, and request->len is 32 bits, so the
addition can't overflow 64 bits), but rewrite it in a form easier
to recognize as a typical overflow check.

Rename the variable 'description' to keep line lengths reasonable.

Suggested-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <20190117193658.16413-7-eblake@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>

show more ...

qemu-nbd: Sanity check partition bounds

When the user requests a partition, we were using data read
from the disk as disk offsets without a bounds check. We got
lucky that even when computed offsets

qemu-nbd: Sanity check partition bounds

When the user requests a partition, we were using data read
from the disk as disk offsets without a bounds check. We got
lucky that even when computed offsets are out-of-bounds,
blk_pread() will gracefully catch the error later (so I don't
think a malicious image can crash or exploit qemu-nbd, and am
not treating this as a security flaw), but it's better to
flag the problem up front than to risk permanent EIO death of
the block device down the road. The new bounds check adds
an assertion that will never fail, but rather exists to help
the compiler see that adding two positive 41-bit values
(given MBR constraints) can't overflow 64-bit off_t.

Using off_t to represent a partition length is a bit of a
misnomer; a later patch will update to saner types, but it
is left separate in case the bounds check needs to be
backported in isolation.

Also, note that the partition code blindly overwrites any
non-zero offset passed in by the user; so for now, make the
-o/-P combo an error for less confusion. In the future, we
may let -o and -P work together (selecting a subset of a
partition); so it is okay that an explicit '-o 0' behaves
no differently from omitting -o.

This can be tested with nbdkit:
$ echo hi > file
$ nbdkit -fv --filter=truncate partitioning file truncate=64k

Pre-patch:
$ qemu-nbd -p 10810 -P 1 -f raw nbd://localhost:10809 &
$ qemu-io -f raw nbd://localhost:10810
qemu-io> r -v 0 1
Disconnect client, due to: Failed to send reply: reading from file failed: Input/output error
Connection closed
read failed: Input/output error
qemu-io> q
[1]+ Done qemu-nbd -p 10810 -P 1 -f raw nbd://localhost:10809

Post-patch:
$ qemu-nbd -p 10810 -P 1 -f raw nbd://localhost:10809
qemu-nbd: Discovered partition 1 at offset 1048576 size 512, but size exceeds file length 65536

Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Richard W.M. Jones <rjones@redhat.com>
Message-Id: <20190117193658.16413-5-eblake@redhat.com>

show more ...

Merge remote-tracking branch 'remotes/ericb/tags/pull-nbd-2019-01-14' into staging

nbd patches for 2019-01-14

Promote bitmap/NBD interfaces to stable for use in incremental
backups. Add 'qemu-nbd -

Merge remote-tracking branch 'remotes/ericb/tags/pull-nbd-2019-01-14' into staging

nbd patches for 2019-01-14

Promote bitmap/NBD interfaces to stable for use in incremental
backups. Add 'qemu-nbd --bitmap'.

- John Snow: 0/11 bitmaps: remove x- prefix from QMP api
- Philippe Mathieu-Daudé: qemu-nbd: Rename 'exp' variable clashing with math::exp() symbol
- Eric Blake: 0/8 Promote x-nbd-server-add-bitmap to stable

# gpg: Signature made Mon 14 Jan 2019 16:13:45 GMT
# gpg: using RSA key A7A16B4A2527436A
# gpg: Good signature from "Eric Blake <eblake@redhat.com>"
# gpg: aka "Eric Blake (Free Software Programmer) <ebb9@byu.net>"
# gpg: aka "[jpeg image of size 6874]"
# Primary key fingerprint: 71C2 CC22 B1C4 6029 27D2 F3AA A7A1 6B4A 2527 436A

* remotes/ericb/tags/pull-nbd-2019-01-14:
qemu-nbd: Add --bitmap=NAME option
nbd: Merge nbd_export_bitmap into nbd_export_new
nbd: Remove x-nbd-server-add-bitmap
nbd: Allow bitmap export during QMP nbd-server-add
nbd: Merge nbd_export_set_name into nbd_export_new
nbd: Only require disabled bitmap for read-only exports
nbd: Forbid nbd-server-stop when server is not running
nbd: Add some error case testing to iotests 223
qemu-nbd: Rename 'exp' variable clashing with math::exp() symbol
iotests: add iotest 236 for testing bitmap merge
iotests: implement pretty-print for log and qmp_log
iotests: change qmp_log filters to expect QMP objects only
iotests: remove default filters from qmp_log
iotests: add qmp recursive sorting function
iotests: add filter_generated_node_ids
iotests.py: don't abort if IMGKEYSECRET is undefined
block: remove 'x' prefix from experimental bitmap APIs
blockdev: n-ary bitmap merge
block/dirty-bitmap: remove assertion from restore
blockdev: abort transactions in reverse order

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

qemu-nbd: Add --bitmap=NAME option

Having to fire up qemu, then use QMP commands for nbd-server-start
and nbd-server-add, just to expose a persistent dirty bitmap, is
rather tedious. Make it possib

qemu-nbd: Add --bitmap=NAME option

Having to fire up qemu, then use QMP commands for nbd-server-start
and nbd-server-add, just to expose a persistent dirty bitmap, is
rather tedious. Make it possible to expose a dirty bitmap using
just qemu-nbd (of course, for now this only works when qemu-nbd is
visiting a BDS formatted as qcow2).

Of course, any good feature also needs unit testing, so expand
iotest 223 to cover it.

Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <20190111194720.15671-9-eblake@redhat.com>

show more ...

nbd: Merge nbd_export_bitmap into nbd_export_new

We only have one caller that wants to export a bitmap name,
which it does right after creation of the export. But there is
still a brief window of ti

nbd: Merge nbd_export_bitmap into nbd_export_new

We only have one caller that wants to export a bitmap name,
which it does right after creation of the export. But there is
still a brief window of time where an NBD client could see the
export but not the dirty bitmap, which a robust client would
have to interpret as meaning the entire image should be treated
as dirty. Better is to eliminate the window entirely, by
inlining nbd_export_bitmap() into nbd_export_new(), and refusing
to create the bitmap in the first place if the requested bitmap
can't be located.

We also no longer need logic for setting a different bitmap
name compared to the bitmap being exported.

Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20190111194720.15671-8-eblake@redhat.com>

show more ...

nbd: Merge nbd_export_set_name into nbd_export_new

The existing NBD code had a weird split where nbd_export_new()
created an export but did not add it to the list of exported
names until a later nbd

nbd: Merge nbd_export_set_name into nbd_export_new

The existing NBD code had a weird split where nbd_export_new()
created an export but did not add it to the list of exported
names until a later nbd_export_set_name() came along and grabbed
a second reference on the object; later, the first call to
nbd_export_close() drops the second reference while removing
the export from the list. This is in part because the QAPI
NbdServerRemoveNode enum documents the possibility of adding a
mode where we could do a soft disconnect: preventing new clients,
but waiting for existing clients to gracefully quit, based on
the mode used when calling nbd_export_close().

But in spite of all that, note that we never change the name of
an NBD export while it is exposed, which means it is easier to
just inline the process of setting the name as part of creating
the export.

Inline the contents of nbd_export_set_name() and
nbd_export_set_description() into the two points in an export
lifecycle where they matter, then adjust both callers to pass
the name up front. Note that for creation, all callers pass a
non-NULL name, (passing NULL at creation was for old style
servers, but we removed support for that in commit 7f7dfe2a),
so we can add an assert and do things unconditionally; but for
cleanup, because of the dual nature of nbd_export_close(), we
still have to be careful to avoid use-after-free. Along the
way, add a comment reminding ourselves of the potential of
adding a middle mode disconnect.

Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20190111194720.15671-5-eblake@redhat.com>

show more ...

qemu-nbd: Rename 'exp' variable clashing with math::exp() symbol

The use of a variable named 'exp' prevents includes to import <math.h>.

Rename it to avoid:

qemu-nbd.c:64:19: error: ‘exp’ redecl

qemu-nbd: Rename 'exp' variable clashing with math::exp() symbol

The use of a variable named 'exp' prevents includes to import <math.h>.

Rename it to avoid:

qemu-nbd.c:64:19: error: ‘exp’ redeclared as different kind of symbol
static NBDExport *exp;
^~~
In file included from /usr/include/features.h:428,
from /usr/include/bits/libc-header-start.h:33,
from /usr/include/stdint.h:26,
from /usr/lib/gcc/x86_64-redhat-linux/8/include/stdint.h:9,
from /source/qemu/include/qemu/osdep.h:80,
from /source/qemu/qemu-nbd.c:19:
/usr/include/bits/mathcalls.h:95:1: note: previous declaration of ‘exp’ was here
__MATHCALL_VEC (exp,, (_Mdouble_ __x));
^~~~~~~~~~~~~~

Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20190111163519.11457-1-philmd@redhat.com>
Signed-off-by: Eric Blake <eblake@redhat.com>

show more ...

Merge remote-tracking branch 'remotes/ericb/tags/pull-nbd-2019-01-05' into staging

nbd patches for 2019-01-05

Error and trace improvements in NBD code, such as less noise for
common disconnect scen

Merge remote-tracking branch 'remotes/ericb/tags/pull-nbd-2019-01-05' into staging

nbd patches for 2019-01-05

Error and trace improvements in NBD code, such as less noise for
common disconnect scenarios.

- Vladimir Sementsov-Ogievskiy: 0/3 nbd-client: drop extra error noise
- Eric Blake: portions of 0/22 nbd: add qemu-nbd --list

# gpg: Signature made Sat 05 Jan 2019 13:58:54 GMT
# gpg: using RSA key A7A16B4A2527436A
# gpg: Good signature from "Eric Blake <eblake@redhat.com>"
# gpg: aka "Eric Blake (Free Software Programmer) <ebb9@byu.net>"
# gpg: aka "[jpeg image of size 6874]"
# Primary key fingerprint: 71C2 CC22 B1C4 6029 27D2 F3AA A7A1 6B4A 2527 436A

* remotes/ericb/tags/pull-nbd-2019-01-05:
nbd/client: Drop pointless buf variable
qemu-nbd: Fail earlier for -c/-d on non-linux
nbd/client: More consistent error messages
nbd: Document timeline of various features
qemu-nbd: Use program name in error messages
block/nbd-client: use traces instead of noisy error_report_err
nbd/client: Trace all server option error messages
nbd: publish _lookup functions

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

qemu-nbd: Fail earlier for -c/-d on non-linux

Connecting to a /dev/nbdN device is a Linux-specific action.
We were already masking -c and -d from 'qemu-nbd --help' on
non-linux. However, while -d f

qemu-nbd: Fail earlier for -c/-d on non-linux

Connecting to a /dev/nbdN device is a Linux-specific action.
We were already masking -c and -d from 'qemu-nbd --help' on
non-linux. However, while -d fails with a sensible error
message, it took hunting through a couple of files to prove
that. What's more, the code for -c doesn't fail until after
it has created a pthread and tried to open a device - possibly
even printing an error message with %m on a non-Linux platform
in spite of the comment that %m is glibc-specific. Make the
failure happen sooner, then get rid of stubs that are no
longer needed because of the early exits.

While at it: tweak the blank newlines in --help output to be
consistent, whether or not built on Linux.

Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <20181215135324.152629-7-eblake@redhat.com>
Reviewed-by: Richard W.M. Jones <rjones@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>

show more ...

qemu-nbd: Use program name in error messages

This changes output from:

$ qemu-nbd nosuch
Failed to blk_new_open 'nosuch': Could not open 'nosuch': No such file or directory

to something more consi

qemu-nbd: Use program name in error messages

This changes output from:

$ qemu-nbd nosuch
Failed to blk_new_open 'nosuch': Could not open 'nosuch': No such file or directory

to something more consistent with qemu-img and qemu:

$ qemu-nbd nosuch
qemu-nbd: Failed to blk_new_open 'nosuch': Could not open 'nosuch': No such file or directory

Update the lone affected test to match. (Hmm - is it sad that we don't
do much testing of expected failures?)

Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Richard W.M. Jones <rjones@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20181215135324.152629-2-eblake@redhat.com>

show more ...

Merge remote-tracking branch 'remotes/armbru/tags/pull-error-2018-10-22' into staging

Error reporting patches for 2018-10-22

# gpg: Signature made Mon 22 Oct 2018 13:20:23 BST
# gpg:

Merge remote-tracking branch 'remotes/armbru/tags/pull-error-2018-10-22' into staging

Error reporting patches for 2018-10-22

# gpg: Signature made Mon 22 Oct 2018 13:20:23 BST
# gpg: using RSA key 3870B400EB918653
# gpg: Good signature from "Markus Armbruster <armbru@redhat.com>"
# gpg: aka "Markus Armbruster <armbru@pond.sub.org>"
# Primary key fingerprint: 354B C8B3 D7EB 2A6B 6867 4E5F 3870 B400 EB91 8653

* remotes/armbru/tags/pull-error-2018-10-22: (40 commits)
error: Drop bogus "use error_setg() instead" admonitions
vpc: Fail open on bad header checksum
block: Clean up bdrv_img_create()'s error reporting
vl: Simplify call of parse_name()
vl: Fix exit status for -drive format=help
blockdev: Convert drive_new() to Error
vl: Assert drive_new() does not fail in default_drive()
fsdev: Clean up error reporting in qemu_fsdev_add()
spice: Clean up error reporting in add_channel()
tpm: Clean up error reporting in tpm_init_tpmdev()
numa: Clean up error reporting in parse_numa()
vnc: Clean up error reporting in vnc_init_func()
ui: Convert vnc_display_init(), init_keyboard_layout() to Error
ui/keymaps: Fix handling of erroneous include files
vl: Clean up error reporting in device_init_func()
vl: Clean up error reporting in parse_fw_cfg()
vl: Clean up error reporting in mon_init_func()
vl: Clean up error reporting in machine_set_property()
vl: Clean up error reporting in chardev_init_func()
qom: Clean up error reporting in user_creatable_add_opts_foreach()
...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

qom: Clean up error reporting in user_creatable_add_opts_foreach()

Calling error_report() in a function that takes an Error ** argument
is suspicious. user_creatable_add_opts_foreach() does that, a

qom: Clean up error reporting in user_creatable_add_opts_foreach()

Calling error_report() in a function that takes an Error ** argument
is suspicious. user_creatable_add_opts_foreach() does that, and then
fails without setting an error. Its caller main(), via
qemu_opts_foreach(), is fine with it, but clean it up anyway.

Cc: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20181017082702.5581-20-armbru@redhat.com>

show more ...

Use error_fatal to simplify obvious fatal errors (again)

Add a slight improvement of the Coccinelle semantic patch from commit
007b06578ab, and use it to clean up. It leaves dead Error * variables

Use error_fatal to simplify obvious fatal errors (again)

Add a slight improvement of the Coccinelle semantic patch from commit
007b06578ab, and use it to clean up. It leaves dead Error * variables
behind, cleaned up manually.

Cc: David Gibson <david@gibson.dropbear.id.au>
Cc: Alexander Graf <agraf@suse.de>
Cc: Eric Blake <eblake@redhat.com>
Cc: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Acked-by: David Gibson <david@gibson.dropbear.id.au>
Message-Id: <20181017082702.5581-3-armbru@redhat.com>

show more ...

Merge remote-tracking branch 'remotes/ericb/tags/pull-nbd-2018-10-03-v2' into staging

nbd patches for 2018-10-03

Fix bugs in NBD_CMD_CACHE, drop support for oldstyle NBD server,
minor build and doc

Merge remote-tracking branch 'remotes/ericb/tags/pull-nbd-2018-10-03-v2' into staging

nbd patches for 2018-10-03

Fix bugs in NBD_CMD_CACHE, drop support for oldstyle NBD server,
minor build and doc fixes

- Denis V. Lunev: nbd: fix NBD_CMD_CACHE negitiation... [retitled]
- Vladimir Sementsov-Ogievskiy: 0/2 server: drop old-style negotiation
- Eric Blake: qemu-nbd: Document --tls-creds
- Vladimir Sementsov-Ogievskiy: nbd/server: fix NBD_CMD_CACHE
- Peter Maydell: nbd: Don't take address of fields in packed structs

# gpg: Signature made Thu 04 Oct 2018 15:19:32 BST
# gpg: using RSA key A7A16B4A2527436A
# gpg: Good signature from "Eric Blake <eblake@redhat.com>"
# gpg: aka "Eric Blake (Free Software Programmer) <ebb9@byu.net>"
# gpg: aka "[jpeg image of size 6874]"
# Primary key fingerprint: 71C2 CC22 B1C4 6029 27D2 F3AA A7A1 6B4A 2527 436A

* remotes/ericb/tags/pull-nbd-2018-10-03-v2:
nbd: fix NBD_FLAG_SEND_CACHE value
nbd/server: drop old-style negotiation
qemu-nbd: drop old-style negotiation
qemu-nbd: Document --tls-creds
nbd/server: fix NBD_CMD_CACHE
nbd: Don't take address of fields in packed structs

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

nbd/server: drop old-style negotiation

After the previous commit, nbd_client_new's first parameter is always
NULL. Let's drop it with all corresponding old-style negotiation code
path which is unrea

nbd/server: drop old-style negotiation

After the previous commit, nbd_client_new's first parameter is always
NULL. Let's drop it with all corresponding old-style negotiation code
path which is unreachable now.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20181003170228.95973-3-vsementsov@virtuozzo.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
[eblake: re-wrap short line]
Signed-off-by: Eric Blake <eblake@redhat.com>

show more ...

qemu-nbd: drop old-style negotiation

Use new-style negotiation always, with default "" (empty) export name
if it is not specified with '-x' option.

qemu as client can manage either style since 2.6.

qemu-nbd: drop old-style negotiation

Use new-style negotiation always, with default "" (empty) export name
if it is not specified with '-x' option.

qemu as client can manage either style since 2.6.0, commit 69b49502d8

For comparison:

nbd 3.10 dropped oldstyle long ago (Mar 2015):
https://github.com/NetworkBlockDevice/nbd/commit/36940193

nbdkit 1.3 switched its default to newstyle (Jan 2018):
https://github.com/libguestfs/nbdkit/commit/b2a8aecc
https://github.com/libguestfs/nbdkit/commit/8158e773

Furthermore, if a client that only speaks oldstyle still needs to
communicate to qemu, nbdkit remains available to perform the
translation between the two protocols.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20181003170228.95973-2-vsementsov@virtuozzo.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
[eblake: enhance commit message]
Signed-off-by: Eric Blake <eblake@redhat.com>

show more ...

qemu-nbd: Document --tls-creds

Commit 145614a1 introduced --tls-creds and documented it in
qemu-nbd.texi, but forgot to document it in 'qemu-nbd --help'.

Signed-off-by: Eric Blake <eblake@redhat.co

qemu-nbd: Document --tls-creds

Commit 145614a1 introduced --tls-creds and documented it in
qemu-nbd.texi, but forgot to document it in 'qemu-nbd --help'.

Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <20181003180426.602765-1-eblake@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>

show more ...

Merge remote-tracking branch 'remotes/kevin/tags/for-upstream' into staging

Block layer patches:

- Generic background jobs
- qemu-iotests fixes for NFS and the 'migration' group
- sheepdog: Minor c

Merge remote-tracking branch 'remotes/kevin/tags/for-upstream' into staging

Block layer patches:

- Generic background jobs
- qemu-iotests fixes for NFS and the 'migration' group
- sheepdog: Minor code simplification

# gpg: Signature made Wed 23 May 2018 13:33:49 BST
# gpg: using RSA key 7F09B272C88F2FD6
# gpg: Good signature from "Kevin Wolf <kwolf@redhat.com>"
# Primary key fingerprint: DC3D EB15 9A9A F95D 3D74 56FE 7F09 B272 C88F 2FD6

* remotes/kevin/tags/for-upstream: (46 commits)
qemu-iotests: Test job-* with block jobs
iotests: Move qmp_to_opts() to VM
blockjob: Remove BlockJob.driver
job: Add query-jobs QMP command
job: Add lifecycle QMP commands
job: Add JOB_STATUS_CHANGE QMP event
job: Introduce qapi/job.json
job: Move progress fields to Job
job: Add job_transition_to_ready()
job: Add job_is_ready()
job: Add job_dismiss()
job: Add job_yield()
block: Cancel job in bdrv_close_all() callers
job: Move completion and cancellation to Job
job: Move transactions to Job
job: Switch transactions to JobTxn
job: Move job_finish_sync() to Job
job: Move .complete callback to Job
job: Add job_drain()
job: Convert block_job_cancel_async() to Job
...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

block: Cancel job in bdrv_close_all() callers

Now that we cancel all jobs and not only block jobs on shutdown, doing
that in bdrv_close_all() isn't really appropriate any more. Move the
job_cancel_s

block: Cancel job in bdrv_close_all() callers

Now that we cancel all jobs and not only block jobs on shutdown, doing
that in bdrv_close_all() isn't really appropriate any more. Move the
job_cancel_sync_all() call to the callers, and only assert that there
are no job running in bdrv_close_all().

Signed-off-by: Kevin Wolf <kwolf@redhat.com>

show more ...

Merge remote-tracking branch 'remotes/bonzini/tags/for-upstream' into staging

* Record-replay lockstep execution, log dumper and fixes (Alex, Pavel)
* SCSI fix to pass maximum transfer size (Daniel

Merge remote-tracking branch 'remotes/bonzini/tags/for-upstream' into staging

* Record-replay lockstep execution, log dumper and fixes (Alex, Pavel)
* SCSI fix to pass maximum transfer size (Daniel Barboza)
* chardev fixes and improved iothread support (Daniel Berrangé, Peter)
* checkpatch tweak (Eric)
* make help tweak (Marc-André)
* make more PCI NICs available with -net or -nic (myself)
* change default q35 NIC to e1000e (myself)
* SCSI support for NDOB bit (myself)
* membarrier system call support (myself)
* SuperIO refactoring (Philippe)
* miscellaneous cleanups and fixes (Thomas)

# gpg: Signature made Mon 12 Mar 2018 16:10:52 GMT
# gpg: using RSA key BFFBD25F78C7AE83
# gpg: Good signature from "Paolo Bonzini <bonzini@gnu.org>"
# gpg: aka "Paolo Bonzini <pbonzini@redhat.com>"
# Primary key fingerprint: 46F5 9FBD 57D6 12E7 BFD4 E2F7 7E15 100C CD36 69B1
# Subkey fingerprint: F133 3857 4B66 2389 866C 7682 BFFB D25F 78C7 AE83

* remotes/bonzini/tags/for-upstream: (69 commits)
tcg: fix cpu_io_recompile
replay: update documentation
replay: save vmstate of the asynchronous events
replay: don't process async events when warping the clock
scripts/replay-dump.py: replay log dumper
replay: avoid recursive call of checkpoints
replay: check return values of fwrite
replay: push replay_mutex_lock up the call tree
replay: don't destroy mutex at exit
replay: make locking visible outside replay code
replay/replay-internal.c: track holding of replay_lock
replay/replay.c: bump REPLAY_VERSION again
replay: save prior value of the host clock
replay: added replay log format description
replay: fix save/load vm for non-empty queue
replay: fixed replay_enable_events
replay: fix processing async events
cpu-exec: fix exception_index handling
hw/i386/pc: Factor out the superio code
hw/alpha/dp264: Use the TYPE_SMC37C669_SUPERIO
...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

# Conflicts:
# default-configs/i386-softmmu.mak
# default-configs/x86_64-softmmu.mak

show more ...

Polish the version strings containing the package version

Since commit 67a1de0d195a there is no space anymore between the
version number and the parentheses when running configure with
--with-pkgver

Polish the version strings containing the package version

Since commit 67a1de0d195a there is no space anymore between the
version number and the parentheses when running configure with
--with-pkgversion=foo :

$ qemu-system-s390x --version
QEMU emulator version 2.11.50(foo)

But the space is included when building without that option
when building from a git checkout:

$ qemu-system-s390x --version
QEMU emulator version 2.11.50 (v2.11.0-1494-gbec9c64-dirty)

The same confusion exists with the "query-version" QMP command.
Let's fix this by introducing a proper QEMU_FULL_VERSION definition
that includes the space and parentheses, while the QEMU_PKGVERSION
should just cleanly contain the package version string itself.
Note that this also changes the behavior of the "query-version" QMP
command (the space and parentheses are not included there anymore),
but that's supposed to be OK since the strings there are not meant
to be parsed by other tools.

Fixes: 67a1de0d195a6185c39b436159c9ffc7720bf979
Buglink: https://bugs.launchpad.net/qemu/+bug/1673373
Signed-off-by: Thomas Huth <thuth@redhat.com>
Message-Id: <1518692807-25859-1-git-send-email-thuth@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>

show more ...