Merge remote-tracking branch 'remotes/ericb/tags/pull-nbd-2021-09-27-v2' into staging

nbd patches for 2021-09-27

- Vladimir Sementsov-Ogievskiy: Rework coroutines of qemu NBD client
to improve re

Merge remote-tracking branch 'remotes/ericb/tags/pull-nbd-2021-09-27-v2' into staging

nbd patches for 2021-09-27

- Vladimir Sementsov-Ogievskiy: Rework coroutines of qemu NBD client
to improve reconnect support
- Eric Blake: Relax server in regards to NBD_OPT_LIST_META_CONTEXT
- Vladimir Sementsov-Ogievskiy: Plumb up 64-bit bulk-zeroing support
in block layer, in preparation for future NBD spec extensions
- Nir Soffer: Default to writeback cache in qemu-nbd

# gpg: Signature made Wed 29 Sep 2021 22:07:58 BST
# gpg: using RSA key 71C2CC22B1C4602927D2F3AAA7A16B4A2527436A
# gpg: Good signature from "Eric Blake <eblake@redhat.com>" [full]
# gpg: aka "Eric Blake (Free Software Programmer) <ebb9@byu.net>" [full]
# gpg: aka "[jpeg image of size 6874]" [full]
# Primary key fingerprint: 71C2 CC22 B1C4 6029 27D2 F3AA A7A1 6B4A 2527 436A

* remotes/ericb/tags/pull-nbd-2021-09-27-v2:
block/nbd: check that received handle is valid
block/nbd: drop connection_co
block/nbd: refactor nbd_recv_coroutines_wake_all()
block/nbd: move nbd_recv_coroutines_wake_all() up
block/nbd: nbd_channel_error() shutdown channel unconditionally
nbd/client-connection: nbd_co_establish_connection(): fix non set errp
nbd/server: Allow LIST_META_CONTEXT without STRUCTURED_REPLY
block/io: allow 64bit discard requests
block: use int64_t instead of int in driver discard handlers
block: make BlockLimits::max_pdiscard 64bit
block/io: allow 64bit write-zeroes requests
block: use int64_t instead of int in driver write_zeroes handlers
block: make BlockLimits::max_pwrite_zeroes 64bit
block: use int64_t instead of uint64_t in copy_range driver handlers
block: use int64_t instead of uint64_t in driver write handlers
block: use int64_t instead of uint64_t in driver read handlers
qcow2: check request on vmstate save/load path
block/io: bring request check to bdrv_co_(read,write)v_vmstate
qemu-nbd: Change default cache mode to writeback

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

qemu-nbd: Change default cache mode to writeback

Both qemu and qemu-img use writeback cache mode by default, which is
already documented in qemu(1). qemu-nbd uses writethrough cache mode by
default,

qemu-nbd: Change default cache mode to writeback

Both qemu and qemu-img use writeback cache mode by default, which is
already documented in qemu(1). qemu-nbd uses writethrough cache mode by
default, and the default cache mode is not documented.

According to the qemu-nbd(8):

--cache=CACHE
The cache mode to be used with the file. See the
documentation of the emulator's -drive cache=... option for
allowed values.

qemu(1) says:

The default mode is cache=writeback.

So users have no reason to assume that qemu-nbd is using writethough
cache mode. The only hint is the painfully slow writing when using the
defaults.

Looking in git history, it seems that qemu used writethrough in the past
to support broken guests that did not flush data properly, or could not
flush due to limitations in qemu. But qemu-nbd clients can use
NBD_CMD_FLUSH to flush data, so using writethrough does not help anyone.

Change the default cache mode to writback, and document the default and
available values properly in the online help and manual.

With this change converting image via qemu-nbd is 3.5 times faster.

$ qemu-img create dst.img 50g
$ qemu-nbd -t -f raw -k /tmp/nbd.sock dst.img

Before this change:

$ hyperfine -r3 "./qemu-img convert -p -f raw -O raw -T none -W fedora34.img nbd+unix:///?socket=/tmp/nbd.sock"
Benchmark #1: ./qemu-img convert -p -f raw -O raw -T none -W fedora34.img nbd+unix:///?socket=/tmp/nbd.sock
Time (mean ± σ): 83.639 s ± 5.970 s [User: 2.733 s, System: 6.112 s]
Range (min … max): 76.749 s … 87.245 s 3 runs

After this change:

$ hyperfine -r3 "./qemu-img convert -p -f raw -O raw -T none -W fedora34.img nbd+unix:///?socket=/tmp/nbd.sock"
Benchmark #1: ./qemu-img convert -p -f raw -O raw -T none -W fedora34.img nbd+unix:///?socket=/tmp/nbd.sock
Time (mean ± σ): 23.522 s ± 0.433 s [User: 2.083 s, System: 5.475 s]
Range (min … max): 23.234 s … 24.019 s 3 runs

Users can avoid the issue by using --cache=writeback[1] but the defaults
should give good performance for the common use case.

[1] https://bugzilla.redhat.com/1990656

Signed-off-by: Nir Soffer <nsoffer@redhat.com>
Message-Id: <20210813205519.50518-1-nsoffer@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
CC: qemu-stable@nongnu.org
Signed-off-by: Eric Blake <eblake@redhat.com>

show more ...

Merge remote-tracking branch 'remotes/armbru/tags/pull-error-2021-08-26' into staging

Error reporting patches for 2021-08-26

# gpg: Signature made Thu 26 Aug 2021 16:17:05 BST
# gpg:

Merge remote-tracking branch 'remotes/armbru/tags/pull-error-2021-08-26' into staging

Error reporting patches for 2021-08-26

# gpg: Signature made Thu 26 Aug 2021 16:17:05 BST
# gpg: using RSA key 354BC8B3D7EB2A6B68674E5F3870B400EB918653
# gpg: issuer "armbru@redhat.com"
# gpg: Good signature from "Markus Armbruster <armbru@redhat.com>" [full]
# gpg: aka "Markus Armbruster <armbru@pond.sub.org>" [full]
# Primary key fingerprint: 354B C8B3 D7EB 2A6B 6867 4E5F 3870 B400 EB91 8653

* remotes/armbru/tags/pull-error-2021-08-26:
vl: Clean up -smp error handling
Remove superfluous ERRP_GUARD()
vhost: Clean up how VhostOpts method vhost_backend_init() fails
vhost: Clean up how VhostOpts method vhost_get_config() fails
microvm: Drop dead error handling in microvm_machine_state_init()
migration: Handle migration_incoming_setup() errors consistently
migration: Unify failure check for migrate_add_blocker()
whpx nvmm: Drop useless migrate_del_blocker()
vfio: Avoid error_propagate() after migrate_add_blocker()
i386: Never free migration blocker objects instead of sometimes
vhost-scsi: Plug memory leak on migrate_add_blocker() failure
multi-process: Fix pci_proxy_dev_realize() error handling
spapr: Explain purpose of ->fwnmi_migration_blocker more clearly
spapr: Plug memory leak when we can't add a migration blocker
error: Use error_fatal to simplify obvious fatal errors (again)

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

error: Use error_fatal to simplify obvious fatal errors (again)

We did this with scripts/coccinelle/use-error_fatal.cocci before, in
commit 50beeb68094 and 007b06578ab. This commit cleans up rarer

error: Use error_fatal to simplify obvious fatal errors (again)

We did this with scripts/coccinelle/use-error_fatal.cocci before, in
commit 50beeb68094 and 007b06578ab. This commit cleans up rarer
variations that don't seem worth matching with Coccinelle.

Cc: Thomas Huth <thuth@redhat.com>
Cc: Cornelia Huck <cornelia.huck@de.ibm.com>
Cc: Peter Xu <peterx@redhat.com>
Cc: Juan Quintela <quintela@redhat.com>
Cc: Stefan Hajnoczi <stefanha@redhat.com>
Cc: Paolo Bonzini <pbonzini@redhat.com>
Cc: Marc-André Lureau <marcandre.lureau@redhat.com>
Cc: Gerd Hoffmann <kraxel@redhat.com>
Cc: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20210720125408.387910-2-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Peter Xu <peterx@redhat.com>
Acked-by: Michael S. Tsirkin <mst@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>

show more ...

Merge remote-tracking branch 'remotes/berrange-gitlab/tags/tls-deps-pull-request' into staging

Hide build time dependancy on gnutls fom non-crypto code

# gpg: Signature made Wed 30 Jun 2021 12:18:4

Merge remote-tracking branch 'remotes/berrange-gitlab/tags/tls-deps-pull-request' into staging

Hide build time dependancy on gnutls fom non-crypto code

# gpg: Signature made Wed 30 Jun 2021 12:18:47 BST
# gpg: using RSA key DAF3A6FDB26B62912D0E8E3FBE86EBB415104FDF
# gpg: Good signature from "Daniel P. Berrange <dan@berrange.com>" [full]
# gpg: aka "Daniel P. Berrange <berrange@redhat.com>" [full]
# Primary key fingerprint: DAF3 A6FD B26B 6291 2D0E 8E3F BE86 EBB4 1510 4FDF

* remotes/berrange-gitlab/tags/tls-deps-pull-request:
crypto: Make QCryptoTLSCreds* structures private
ui/vnc: Use qcrypto_tls_creds_check_endpoint()
migration/tls: Use qcrypto_tls_creds_check_endpoint()
chardev/socket: Use qcrypto_tls_creds_check_endpoint()
qemu-nbd: Use qcrypto_tls_creds_check_endpoint()
block/nbd: Use qcrypto_tls_creds_check_endpoint()
crypto/tlscreds: Introduce qcrypto_tls_creds_check_endpoint() helper

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

qemu-nbd: Use qcrypto_tls_creds_check_endpoint()

Avoid accessing QCryptoTLSCreds internals by using
the qcrypto_tls_creds_check_endpoint() helper.

Tested-by: Akihiko Odaki <akihiko.odaki@gmail.com>

qemu-nbd: Use qcrypto_tls_creds_check_endpoint()

Avoid accessing QCryptoTLSCreds internals by using
the qcrypto_tls_creds_check_endpoint() helper.

Tested-by: Akihiko Odaki <akihiko.odaki@gmail.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>

show more ...

Merge remote-tracking branch 'remotes/kevin/tags/for-upstream' into staging

Block layer patches and object-add QAPIfication

- QAPIfy object-add and --object
- stream: Fail gracefully if permission

Merge remote-tracking branch 'remotes/kevin/tags/for-upstream' into staging

Block layer patches and object-add QAPIfication

- QAPIfy object-add and --object
- stream: Fail gracefully if permission is denied
- storage-daemon: Fix crash on quit when job is still running
- curl: Fix use after free
- char: Deprecate backend aliases, fix QMP query-chardev-backends
- Fix image creation option defaults that exist in both the format and
the protocol layer (e.g. 'cluster_size' in qcow2 and rbd; the qcow2
default was incorrectly applied to the rbd layer)

# gpg: Signature made Fri 19 Mar 2021 09:18:22 GMT
# gpg: using RSA key DC3DEB159A9AF95D3D7456FE7F09B272C88F2FD6
# gpg: issuer "kwolf@redhat.com"
# gpg: Good signature from "Kevin Wolf <kwolf@redhat.com>" [full]
# Primary key fingerprint: DC3D EB15 9A9A F95D 3D74 56FE 7F09 B272 C88F 2FD6

* remotes/kevin/tags/for-upstream: (42 commits)
vl: allow passing JSON to -object
qom: move user_creatable_add_opts logic to vl.c and QAPIfy it
tests: convert check-qom-proplist to keyval
qom: Support JSON in HMP object_add and tools --object
char: Simplify chardev_name_foreach()
char: Deprecate backend aliases 'tty' and 'parport'
char: Skip CLI aliases in query-chardev-backends
qom: Add user_creatable_parse_str()
hmp: QAPIfy object_add
qemu-img: Use user_creatable_process_cmdline() for --object
qom: Add user_creatable_add_from_str()
qemu-nbd: Use user_creatable_process_cmdline() for --object
qemu-io: Use user_creatable_process_cmdline() for --object
qom: Factor out user_creatable_process_cmdline()
qom: Remove user_creatable_add_dict()
qemu-storage-daemon: Implement --object with qmp_object_add()
qom: Make "object" QemuOptsList optional
qapi/qom: QAPIfy object-add
qapi/qom: Add ObjectOptions for x-remote-object
qapi/qom: Add ObjectOptions for input-*
...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

qemu-nbd: Use user_creatable_process_cmdline() for --object

This switches qemu-nbd from a QemuOpts-based parser for --object to
user_creatable_process_cmdline() which uses a keyval parser and enforc

qemu-nbd: Use user_creatable_process_cmdline() for --object

This switches qemu-nbd from a QemuOpts-based parser for --object to
user_creatable_process_cmdline() which uses a keyval parser and enforces
the QAPI schema.

Apart from being a cleanup, this makes non-scalar properties accessible.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Acked-by: Paolo Bonzini <pbonzini@redhat.com>
Acked-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>

show more ...

Merge remote-tracking branch 'remotes/ericb/tags/pull-nbd-2021-02-12' into staging

nbd patches for 2021-02-12

- let qemu-nbd handle larger backlog of connecting clients
- fix a few NBD-related iote

Merge remote-tracking branch 'remotes/ericb/tags/pull-nbd-2021-02-12' into staging

nbd patches for 2021-02-12

- let qemu-nbd handle larger backlog of connecting clients
- fix a few NBD-related iotest failures
- add block cancellation hook for faster response to NBD failures

# gpg: Signature made Fri 12 Feb 2021 19:57:56 GMT
# gpg: using RSA key 71C2CC22B1C4602927D2F3AAA7A16B4A2527436A
# gpg: Good signature from "Eric Blake <eblake@redhat.com>" [full]
# gpg: aka "Eric Blake (Free Software Programmer) <ebb9@byu.net>" [full]
# gpg: aka "[jpeg image of size 6874]" [full]
# Primary key fingerprint: 71C2 CC22 B1C4 6029 27D2 F3AA A7A1 6B4A 2527 436A

* remotes/ericb/tags/pull-nbd-2021-02-12:
iotests/264: add backup-cancel test-case
block/backup: implement .cancel job handler
iotests/264: add mirror-cancel test-case
iotests.py: qemu_nbd_popen: remove pid file after use
iotests/264: move to python unittest
block/mirror: implement .cancel job handler
job: add .cancel handler for the driver
block/raw-format: implement .bdrv_cancel_in_flight handler
block/nbd: implement .bdrv_cancel_in_flight
block: add new BlockDriver handler: bdrv_cancel_in_flight
io: error_prepend() in qio_channel_readv_full_all() causes segfault
iotests/210: Fix reference output
qemu-nbd: Permit --shared=0 for unlimited clients
qemu-nbd: Use SOMAXCONN for socket listen() backlog

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

qemu-nbd: Permit --shared=0 for unlimited clients

This gives us better feature parity with QMP nbd-server-start, where
max-connections defaults to 0 for unlimited.

Signed-off-by: Eric Blake <eblake

qemu-nbd: Permit --shared=0 for unlimited clients

This gives us better feature parity with QMP nbd-server-start, where
max-connections defaults to 0 for unlimited.

Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <20210209152759.209074-3-eblake@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>

show more ...

qemu-nbd: Use SOMAXCONN for socket listen() backlog

Our default of a backlog of 1 connection is rather puny; it gets in
the way when we are explicitly allowing multiple clients (such as
qemu-nbd -e

qemu-nbd: Use SOMAXCONN for socket listen() backlog

Our default of a backlog of 1 connection is rather puny; it gets in
the way when we are explicitly allowing multiple clients (such as
qemu-nbd -e N [--shared], or nbd-server-start with its default
"max-connections":0 for unlimited), but is even a problem when we
stick to qemu-nbd's default of only 1 active client but use -t
[--persistent] where a second client can start using the server once
the first finishes. While the effects are less noticeable on TCP
sockets (since the client can poll() to learn when the server is ready
again), it is definitely observable on Unix sockets, where on Linux, a
client will fail with EAGAIN and no recourse but to sleep an arbitrary
amount of time before retrying if the server backlog is already full.

Since QMP nbd-server-start is always persistent, it now always
requests a backlog of SOMAXCONN; meanwhile, qemu-nbd will request
SOMAXCONN if persistent, otherwise its backlog should be based on the
expected number of clients.

See https://bugzilla.redhat.com/1925045 for a demonstration of where
our low backlog prevents libnbd from connecting as many parallel
clients as it wants.

Reported-by: Richard W.M. Jones <rjones@redhat.com>
Signed-off-by: Eric Blake <eblake@redhat.com>
CC: qemu-stable@nongnu.org
Message-Id: <20210209152759.209074-2-eblake@redhat.com>
Tested-by: Richard W.M. Jones <rjones@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Eric Blake <eblake@redhat.com>

show more ...

Merge remote-tracking branch 'remotes/kevin/tags/for-upstream' into staging

Block layer patches:

- Fix double processing of nodes in bdrv_set_aio_context()
- Fix potential hang in block export shut

Merge remote-tracking branch 'remotes/kevin/tags/for-upstream' into staging

Block layer patches:

- Fix double processing of nodes in bdrv_set_aio_context()
- Fix potential hang in block export shutdown
- block/nvme: Minor tracing improvements
- iotests: Some more fixups for the 'check' rewrite
- MAINTAINERS: Add Vladimir as co-maintainer for Block Jobs

# gpg: Signature made Tue 02 Feb 2021 16:26:02 GMT
# gpg: using RSA key DC3DEB159A9AF95D3D7456FE7F09B272C88F2FD6
# gpg: issuer "kwolf@redhat.com"
# gpg: Good signature from "Kevin Wolf <kwolf@redhat.com>" [full]
# Primary key fingerprint: DC3D EB15 9A9A F95D 3D74 56FE 7F09 B272 C88F 2FD6

* remotes/kevin/tags/for-upstream:
block: Fix VM size column width in bdrv_snapshot_dump()
block/nvme: Trace NVMe spec version supported by the controller
block/nvme: Properly display doorbell stride length in trace event
iotests: Fix -makecheck output
iotests: check: return 1 on failure
iotests: Revert emulator selection to old behaviour
iotests/297: pylint: ignore too many statements
block: move blk_exp_close_all() to qemu_cleanup()
block: Avoid processing BDS twice in bdrv_set_aio_context_ignore()
MAINTAINERS: Add Vladimir as co-maintainer for Block Jobs

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

block: move blk_exp_close_all() to qemu_cleanup()

Move blk_exp_close_all() from bdrv_close() to qemu_cleanup(), before
bdrv_drain_all_begin().

Export drivers may have coroutines yielding at some po

block: move blk_exp_close_all() to qemu_cleanup()

Move blk_exp_close_all() from bdrv_close() to qemu_cleanup(), before
bdrv_drain_all_begin().

Export drivers may have coroutines yielding at some point in the block
layer, so we need to shut them down before draining the block layer,
as otherwise they may get stuck blk_wait_while_drained().

RHBZ: https://bugzilla.redhat.com/show_bug.cgi?id=1900505
Signed-off-by: Sergio Lopez <slp@redhat.com>
Message-Id: <20210201125032.44713-3-slp@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>

show more ...

Merge remote-tracking branch 'remotes/ericb/tags/pull-nbd-2021-01-20' into staging

nbd patches for 2021-01-20

- minor resource leak fixes in qemu-nbd
- ensure proper aio context when nbd server use

Merge remote-tracking branch 'remotes/ericb/tags/pull-nbd-2021-01-20' into staging

nbd patches for 2021-01-20

- minor resource leak fixes in qemu-nbd
- ensure proper aio context when nbd server uses iothreads
- iotest refactorings in preparation for rewriting ./check to be more
flexible, and preparing for more nbd server reconnect features

# gpg: Signature made Thu 21 Jan 2021 02:28:19 GMT
# gpg: using RSA key 71C2CC22B1C4602927D2F3AAA7A16B4A2527436A
# gpg: Good signature from "Eric Blake <eblake@redhat.com>" [full]
# gpg: aka "Eric Blake (Free Software Programmer) <ebb9@byu.net>" [full]
# gpg: aka "[jpeg image of size 6874]" [full]
# Primary key fingerprint: 71C2 CC22 B1C4 6029 27D2 F3AA A7A1 6B4A 2527 436A

* remotes/ericb/tags/pull-nbd-2021-01-20:
iotests.py: qemu_io(): reuse qemu_tool_pipe_and_status()
iotests.py: fix qemu_tool_pipe_and_status()
iotests/264: fix style
iotests: define group in each iotest
iotests/294: add shebang line
iotests: make tests executable
iotests: fix some whitespaces in test output files
iotests/303: use dot slash for qcow2.py running
iotests/277: use dot slash for nbd-fault-injector.py running
nbd/server: Quiesce coroutines on context switch
block: Honor blk_set_aio_context() context requirements
qemu-nbd: Fix a memleak in nbd_client_thread()
qemu-nbd: Fix a memleak in qemu_nbd_client_list()

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

qemu-nbd: Fix a memleak in nbd_client_thread()

When the qio_channel_socket_connect_sync() fails
we should goto 'out_socket' label to free the 'sioc' instead of
goto 'out' label.
In addition, there's

qemu-nbd: Fix a memleak in nbd_client_thread()

When the qio_channel_socket_connect_sync() fails
we should goto 'out_socket' label to free the 'sioc' instead of
goto 'out' label.
In addition, there's a lot of redundant code in the successful branch
and the error branch, optimize it.

Reported-by: Euler Robot <euler.robot@huawei.com>
Signed-off-by: Alex Chen <alex.chen@huawei.com>
Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20201208134944.27962-1-alex.chen@huawei.com>

show more ...

qemu-nbd: Fix a memleak in qemu_nbd_client_list()

When the qio_channel_socket_connect_sync() fails
we should goto 'out' label to free the 'sioc' instead of return.

Reported-by: Euler Robot <euler.r

qemu-nbd: Fix a memleak in qemu_nbd_client_list()

When the qio_channel_socket_connect_sync() fails
we should goto 'out' label to free the 'sioc' instead of return.

Reported-by: Euler Robot <euler.robot@huawei.com>
Signed-off-by: Alex Chen <alex.chen@huawei.com>
Message-Id: <20201130123651.17543-1-alex.chen@huawei.com>
Reviewed-by: Alberto Garcia <berto@igalia.com>
Signed-off-by: Eric Blake <eblake@redhat.com>

show more ...

Merge remote-tracking branch 'remotes/stefanha-gitlab/tags/tracing-pull-request' into staging

Tracing pull request

# gpg: Signature made Wed 11 Nov 2020 15:56:18 GMT
# gpg: using RSA

Merge remote-tracking branch 'remotes/stefanha-gitlab/tags/tracing-pull-request' into staging

Tracing pull request

# gpg: Signature made Wed 11 Nov 2020 15:56:18 GMT
# gpg: using RSA key 8695A8BFD3F97CDAAC35775A9CA4ABB381AB73C8
# gpg: Good signature from "Stefan Hajnoczi <stefanha@redhat.com>" [full]
# gpg: aka "Stefan Hajnoczi <stefanha@gmail.com>" [full]
# Primary key fingerprint: 8695 A8BF D3F9 7CDA AC35 775A 9CA4 ABB3 81AB 73C8

* remotes/stefanha-gitlab/tags/tracing-pull-request:
scripts/tracetool: silence SystemTap dtrace(1) long long warnings
trace: remove argument from trace_init_file

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

trace: remove argument from trace_init_file

It is not needed, all the callers are just saving what was
retrieved from -trace and trace_init_file can retrieve it
on its own.

Signed-off-by: Paolo Bon

trace: remove argument from trace_init_file

It is not needed, all the callers are just saving what was
retrieved from -trace and trace_init_file can retrieve it
on its own.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-id: 20201102115841.4017692-1-pbonzini@redhat.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>

show more ...

Merge remote-tracking branch 'remotes/ericb/tags/pull-nbd-2020-10-27-v2' into staging

nbd patches for 2020-10-27

- Tweak the new block-export-add QMP command
- Allow multiple -B options for qemu-nb

Merge remote-tracking branch 'remotes/ericb/tags/pull-nbd-2020-10-27-v2' into staging

nbd patches for 2020-10-27

- Tweak the new block-export-add QMP command
- Allow multiple -B options for qemu-nbd
- Add qemu:allocation-depth metadata context as qemu-nbd -A
- Improve iotest use of NBD

# gpg: Signature made Fri 30 Oct 2020 20:22:42 GMT
# gpg: using RSA key 71C2CC22B1C4602927D2F3AAA7A16B4A2527436A
# gpg: Good signature from "Eric Blake <eblake@redhat.com>" [full]
# gpg: aka "Eric Blake (Free Software Programmer) <ebb9@byu.net>" [full]
# gpg: aka "[jpeg image of size 6874]" [full]
# Primary key fingerprint: 71C2 CC22 B1C4 6029 27D2 F3AA A7A1 6B4A 2527 436A

* remotes/ericb/tags/pull-nbd-2020-10-27-v2:
nbd: Add 'qemu-nbd -A' to expose allocation depth
nbd: Add new qemu:allocation-depth metadata context
block: Return depth level during bdrv_is_allocated_above
nbd: Allow export of multiple bitmaps for one device
nbd: Refactor counting of metadata contexts
nbd: Simplify qemu bitmap context name
nbd: Update qapi to support exporting multiple bitmaps
nbd: Utilize QAPI_CLONE for type conversion
qapi: Add QAPI_LIST_PREPEND() macro
block: Simplify QAPI_LIST_ADD
iotests/291: Stop NBD server
iotests/291: Filter irrelevant parts of img-info

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

nbd: Add 'qemu-nbd -A' to expose allocation depth

Allow the server to expose an additional metacontext to be requested
by savvy clients. qemu-nbd adds a new option -A to expose the
qemu:allocation-

nbd: Add 'qemu-nbd -A' to expose allocation depth

Allow the server to expose an additional metacontext to be requested
by savvy clients. qemu-nbd adds a new option -A to expose the
qemu:allocation-depth metacontext through NBD_CMD_BLOCK_STATUS; this
can also be set via QMP when using block-export-add.

qemu as client is hacked into viewing the key aspects of this new
context by abusing the already-experimental x-dirty-bitmap option to
collapse all depths greater than 2, which results in a tri-state value
visible in the output of 'qemu-img map --output=json' (yes, that means
x-dirty-bitmap is now a bit of a misnomer, but I didn't feel like
renaming it as it would introduce a needless break of back-compat,
even though we make no compat guarantees with x- members):

unallocated (depth 0) => "zero":false, "data":true
local (depth 1) => "zero":false, "data":false
backing (depth 2+) => "zero":true, "data":true

libnbd as client is probably a nicer way to get at the information
without having to decipher such hacks in qemu as client. ;)

Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <20201027050556.269064-11-eblake@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>

show more ...

nbd: Update qapi to support exporting multiple bitmaps

Since 'block-export-add' is new to 5.2, we can still tweak the
interface; there, allowing 'bitmaps':['str'] is nicer than
'bitmap':'str'. This

nbd: Update qapi to support exporting multiple bitmaps

Since 'block-export-add' is new to 5.2, we can still tweak the
interface; there, allowing 'bitmaps':['str'] is nicer than
'bitmap':'str'. This wires up the qapi and qemu-nbd changes to permit
passing multiple bitmaps as distinct metadata contexts that the NBD
client may request, but the actual support for more than one will
require a further patch to the server.

Note that there are no changes made to the existing deprecated
'nbd-server-add' command; this required splitting the QAPI type
BlockExportOptionsNbd, which fortunately does not affect QMP
introspection.

Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <20201027050556.269064-5-eblake@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Peter Krempa <pkrempa@redhat.com>

show more ...

Merge remote-tracking branch 'remotes/stefanha-gitlab/tags/block-pull-request' into staging

Pull request

v3:
* Fix linker error on FreeBSD and other !CONFIG_LINUX host OSes [Peter]

# gpg: Signatu

Merge remote-tracking branch 'remotes/stefanha-gitlab/tags/block-pull-request' into staging

Pull request

v3:
* Fix linker error on FreeBSD and other !CONFIG_LINUX host OSes [Peter]

# gpg: Signature made Fri 23 Oct 2020 16:21:19 BST
# gpg: using RSA key 8695A8BFD3F97CDAAC35775A9CA4ABB381AB73C8
# gpg: Good signature from "Stefan Hajnoczi <stefanha@redhat.com>" [full]
# gpg: aka "Stefan Hajnoczi <stefanha@gmail.com>" [full]
# Primary key fingerprint: 8695 A8BF D3F9 7CDA AC35 775A 9CA4 ABB3 81AB 73C8

* remotes/stefanha-gitlab/tags/block-pull-request: (28 commits)
iotests: add commit top->base cases to 274
block/io: fix bdrv_is_allocated_above
block/io: bdrv_common_block_status_above: support bs == base
block/io: bdrv_common_block_status_above: support include_base
block/io: fix bdrv_co_block_status_above
block/export: add vhost-user-blk multi-queue support
block/export: add iothread and fixed-iothread options
block: move block exports to libblockdev
qemu-storage-daemon: avoid compiling blockdev_ss twice
util/vhost-user-server: use static library in meson.build
util/vhost-user-server: move header to include/
block/export: convert vhost-user-blk server to block export API
block/export: report flush errors
util/vhost-user-server: rework vu_client_trip() coroutine lifecycle
util/vhost-user-server: check EOF when reading payload
util/vhost-user-server: fix memory leak in vu_message_read()
util/vhost-user-server: drop unused DevicePanicNotifier
block/export: consolidate request structs into VuBlockReq
util/vhost-user-server: drop unnecessary watch deletion
util/vhost-user-server: drop unnecessary QOM cast
...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

block: move block exports to libblockdev

Block exports are used by softmmu, qemu-storage-daemon, and qemu-nbd.
They are not used by other programs and are not otherwise needed in
libblock.

Undo the

block: move block exports to libblockdev

Block exports are used by softmmu, qemu-storage-daemon, and qemu-nbd.
They are not used by other programs and are not otherwise needed in
libblock.

Undo the recent move of blockdev-nbd.c from blockdev_ss into block_ss.
Since bdrv_close_all() (libblock) calls blk_exp_close_all()
(libblockdev) a stub function is required..

Make qemu-nbd.c use signal handling utility functions instead of
duplicating the code. This helps because os-posix.c is in libblockdev
and it depends on a qemu_system_killed() symbol that qemu-nbd.c lacks.
Once we use the signal handling utility functions we also end up
providing the necessary symbol.

Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id: 20200929125516.186715-4-stefanha@redhat.com
[Fixed s/ndb/nbd/ typo in commit description as suggested by Eric Blake
--Stefan]
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>

show more ...

Merge remote-tracking branch 'remotes/ericb/tags/pull-nbd-2020-10-08-v3' into staging

nbd patches for 2020-10-08

- silence compilation warnings
- more fixes to prevent reconnect hangs
- improve 'qe

Merge remote-tracking branch 'remotes/ericb/tags/pull-nbd-2020-10-08-v3' into staging

nbd patches for 2020-10-08

- silence compilation warnings
- more fixes to prevent reconnect hangs
- improve 'qemu-nbd' termination behavior
- cleaner NBD protocol compliance on string handling

# gpg: Signature made Fri 09 Oct 2020 21:05:30 BST
# gpg: using RSA key 71C2CC22B1C4602927D2F3AAA7A16B4A2527436A
# gpg: Good signature from "Eric Blake <eblake@redhat.com>" [full]
# gpg: aka "Eric Blake (Free Software Programmer) <ebb9@byu.net>" [full]
# gpg: aka "[jpeg image of size 6874]" [full]
# Primary key fingerprint: 71C2 CC22 B1C4 6029 27D2 F3AA A7A1 6B4A 2527 436A

* remotes/ericb/tags/pull-nbd-2020-10-08-v3:
nbd: Simplify meta-context parsing
nbd/server: Reject embedded NUL in NBD strings
qemu-nbd: Honor SIGINT and SIGHUP
block/nbd: nbd_co_reconnect_loop(): don't connect if drained
block/nbd: fix reconnect-delay
block/nbd: correctly use qio_channel_detach_aio_context when needed
block/nbd: fix drain dead-lock because of nbd reconnect-delay
nbd: silence maybe-uninitialized warnings

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

qemu-nbd: Honor SIGINT and SIGHUP

Honoring just SIGTERM on Linux is too weak; we also want to handle
other common signals, and do so even on BSD. Why? Because at least
'qemu-nbd -B bitmap' needs a

qemu-nbd: Honor SIGINT and SIGHUP

Honoring just SIGTERM on Linux is too weak; we also want to handle
other common signals, and do so even on BSD. Why? Because at least
'qemu-nbd -B bitmap' needs a chance to clean up the in-use bit on
bitmaps when the server is shut down via a signal.

See also: http://bugzilla.redhat.com/1883608

Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <20200930121105.667049-2-eblake@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
[eblake: apply comment tweak suggested by Vladimir; fix ifdef around
termsig_handler]
Signed-off-by: Eric Blake <eblake@redhat.com>

show more ...