Merge tag 'pull-nbd-2023-10-05' of https://repo.or.cz/qemu/ericb into staging

NBD patches for 2023-10-05

- various: mailmap cleanups
- Eric Blake: enable use of NBD 64-bit extended headers

# -----

Merge tag 'pull-nbd-2023-10-05' of https://repo.or.cz/qemu/ericb into staging

NBD patches for 2023-10-05

- various: mailmap cleanups
- Eric Blake: enable use of NBD 64-bit extended headers

# -----BEGIN PGP SIGNATURE-----
#
# iQEzBAABCAAdFiEEccLMIrHEYCkn0vOqp6FrSiUnQ2oFAmUfA94ACgkQp6FrSiUn
# Q2o2hAf/Q4q6RoEG9WoOIa6WB1nk9tZN6GRfW4jS+09hJTFVbYUhJOcvQwwZjPT6
# 6oIwde8w7uE+AxBOA3XPbgTOBnTnpt2RH2AYVctNYB2vonuSrx3/KE3XSi4nrJRo
# +XiPbOsDzQu+vYsI7XJ+5e13BE4iZVmzbyke0U6hO88uR6tQstDJV1Mhem4jIbCG
# uLzkVDs3yY5sUFoUe77lFGWgKqfh87eJXV18T76df97ZZ5O2/w9G8MknG/CTSsR0
# fmzzC1Q4a2UEFtX8M3etRQ/b5WbFYhM+XwFevm2YBpod89ejGEA0ohhpn+GpZDU9
# SU66lXL/5jM9N7RkiSFwvupaot9hvw==
# =UieX
# -----END PGP SIGNATURE-----
# gpg: Signature made Thu 05 Oct 2023 14:43:42 EDT
# gpg: using RSA key 71C2CC22B1C4602927D2F3AAA7A16B4A2527436A
# gpg: Good signature from "Eric Blake <eblake@redhat.com>" [full]
# gpg: aka "Eric Blake (Free Software Programmer) <ebb9@byu.net>" [full]
# gpg: aka "[jpeg image of size 6874]" [full]
# Primary key fingerprint: 71C2 CC22 B1C4 6029 27D2 F3AA A7A1 6B4A 2527 436A

* tag 'pull-nbd-2023-10-05' of https://repo.or.cz/qemu/ericb:
nbd/server: Add FLAG_PAYLOAD support to CMD_BLOCK_STATUS
nbd/server: Prepare for per-request filtering of BLOCK_STATUS
nbd/server: Refactor list of negotiated meta contexts
nbd/client: Request extended headers during negotiation
nbd/client: Accept 64-bit block status chunks
nbd/client: Initial support for extended headers
nbd/client: Plumb errp through nbd_receive_replies
nbd/server: Enable initial support for extended headers
nbd/server: Support 64-bit block status
nbd/server: Prepare to send extended header replies
nbd/server: Prepare to receive extended header requests
nbd/server: Support a request payload
mailmap: Fix BALATON Zoltan author email
maint: Tweak comments in mailmap regarding SPF
mailmap: Fix Andrey Drobyshev author email

Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>

show more ...

nbd/client: Accept 64-bit block status chunks

Once extended mode is enabled, we need to accept 64-bit status replies
(even for replies that don't exceed a 32-bit length). It is easier to
normalize

nbd/client: Accept 64-bit block status chunks

Once extended mode is enabled, we need to accept 64-bit status replies
(even for replies that don't exceed a 32-bit length). It is easier to
normalize narrow replies into wide format so that the rest of our code
only has to handle one width. Although a server is non-compliant if
it sends a 64-bit reply in compact mode, or a 32-bit reply in extended
mode, it is still easy enough to tolerate these mismatches.

In normal execution, we are only requesting "base:allocation" which
never exceeds 32 bits for flag values. But during testing with
x-dirty-bitmap, we can force qemu to connect to some other context
that might have 64-bit status bit; however, we ignore those upper bits
(other than mapping qemu:allocation-depth into something that
'qemu-img map --output=json' can expose), and since that only affects
testing, we really don't bother with checking whether more than the
two least-significant bits are set.

Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
Message-ID: <20230925192229.3186470-22-eblake@redhat.com>

show more ...

Merge tag 'pull-nbd-2023-09-25' of https://repo.or.cz/qemu/ericb into staging

NBD patches through 2023-09-25

- Denis V. Lunev: iotest improvements
- Eric Blake: further work towards 64-bit NBD exte

Merge tag 'pull-nbd-2023-09-25' of https://repo.or.cz/qemu/ericb into staging

NBD patches through 2023-09-25

- Denis V. Lunev: iotest improvements
- Eric Blake: further work towards 64-bit NBD extensions

# -----BEGIN PGP SIGNATURE-----
#
# iQEzBAABCAAdFiEEccLMIrHEYCkn0vOqp6FrSiUnQ2oFAmUR2MUACgkQp6FrSiUn
# Q2q6jAf+PT65XzMAhgKvu1vIeMSQqyCocNB2MCOzNp+46uB9bNbPPLQSH2EX+t6p
# kQfHyHUl4YMi0EqgCfodiewlaUKeMxP3cPWMGYaYZ16uNMOIYL1boreDAcM25rb5
# P3TV3DAWTWSclUxrkTC2DxAIBPgsPsGG/2daqOMDEdinxlIywCMJDEIHc9gwwd/t
# 7laz9V1cOW9NbQXrM7eTofJKPKIeqZ+w0kvqrf9HBvZl9CqwHADi7xoz9xP+fN+f
# 713ED/hwt0FIlixtIm2/8vu7nn09cu6m9NaKsMOomsYg9Z6wU3ctivViG5NLq3MD
# OOUu51dV8gRRAXAFU5vKb0d93D27zQ==
# =Ik02
# -----END PGP SIGNATURE-----
# gpg: Signature made Mon 25 Sep 2023 15:00:21 EDT
# gpg: using RSA key 71C2CC22B1C4602927D2F3AAA7A16B4A2527436A
# gpg: Good signature from "Eric Blake <eblake@redhat.com>" [full]
# gpg: aka "Eric Blake (Free Software Programmer) <ebb9@byu.net>" [full]
# gpg: aka "[jpeg image of size 6874]" [full]
# Primary key fingerprint: 71C2 CC22 B1C4 6029 27D2 F3AA A7A1 6B4A 2527 436A

* tag 'pull-nbd-2023-09-25' of https://repo.or.cz/qemu/ericb:
nbd/server: Refactor handling of command sanity checks
nbd: Prepare for 64-bit request effect lengths
nbd: Add types for extended headers
nbd/client: Pass mode through to nbd_send_request
nbd: Replace bool structured_reply with mode enum
iotests: improve 'not run' message for nbd-multiconn test
iotests: use TEST_IMG_FILE instead of TEST_IMG in _require_large_file

Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>

show more ...

nbd: Prepare for 64-bit request effect lengths

Widen the length field of NBDRequest to 64-bits, although we can
assert that all current uses are still under 32 bits: either because
of NBD_MAX_BUFFER

nbd: Prepare for 64-bit request effect lengths

Widen the length field of NBDRequest to 64-bits, although we can
assert that all current uses are still under 32 bits: either because
of NBD_MAX_BUFFER_SIZE which is even smaller (and where size_t can
still be appropriate, even on 32-bit platforms), or because nothing
ever puts us into NBD_MODE_EXTENDED yet (and while future patches will
allow larger transactions, the lengths in play here are still capped
at 32-bit). There are no semantic changes, other than a typo fix in a
couple of error messages.

Signed-off-by: Eric Blake <eblake@redhat.com>
Message-ID: <20230829175826.377251-23-eblake@redhat.com>
[eblake: fix assertion bug in nbd_co_send_simple_reply]
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>

show more ...

Merge tag 'block-pull-request' of https://gitlab.com/stefanha/qemu into staging

Pull request

- Stefano Garzarella's blkio block driver 'fd' parameter
- My thread-local blk_io_plug() series

# -----

Merge tag 'block-pull-request' of https://gitlab.com/stefanha/qemu into staging

Pull request

- Stefano Garzarella's blkio block driver 'fd' parameter
- My thread-local blk_io_plug() series

# -----BEGIN PGP SIGNATURE-----
#
# iQEzBAABCAAdFiEEhpWov9P5fNqsNXdanKSrs4Grc8gFAmR4uHoACgkQnKSrs4Gr
# c8hFBAgAo+SFrOteYgdELM9s0EWb0AU39MTOyNXW7i5mPZNXrn5J7pfRD/5wvI6l
# wl5GNMQ+M5HVYO7CumKWr4M1IpKV5Jin6FN/2h15fWkeg17lBOmNHUF+LctLYQbq
# HwtNA4hdw1+SEv8kQLBgiqSJMqWcn80X09emgPMCIwET9zxokRYwVjQJx2alM5bd
# SqgitDp5qlHyj5HQPX2orT9KrXYWQdGr8i50bn0S67r1wdqTRMu93wrWdEUUncId
# 7otlUaq8cARbRMJzIwDmy/cF24Ynr0wCJb4aHW+trRtf+PNgx1Ki+YOiz+LFyjq7
# t6KOMeignzhz9Uzq8EVG4XW8SHpGkw==
# =Ms48
# -----END PGP SIGNATURE-----
# gpg: Signature made Thu 01 Jun 2023 08:25:46 AM PDT
# gpg: using RSA key 8695A8BFD3F97CDAAC35775A9CA4ABB381AB73C8
# gpg: Good signature from "Stefan Hajnoczi <stefanha@redhat.com>" [full]
# gpg: aka "Stefan Hajnoczi <stefanha@gmail.com>" [full]

* tag 'block-pull-request' of https://gitlab.com/stefanha/qemu:
qapi: add '@fdset' feature for BlockdevOptionsVirtioBlkVhostVdpa
block/blkio: use qemu_open() to support fd passing for virtio-blk
block: remove bdrv_co_io_plug() API
block/linux-aio: convert to blk_io_plug_call() API
block/io_uring: convert to blk_io_plug_call() API
block/blkio: convert to blk_io_plug_call() API
block/nvme: convert to blk_io_plug_call() API
block: add blk_io_plug_call() API

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>

show more ...

block/io_uring: convert to blk_io_plug_call() API

Stop using the .bdrv_co_io_plug() API because it is not multi-queue
block layer friendly. Use the new blk_io_plug_call() API to batch I/O
submission

block/io_uring: convert to blk_io_plug_call() API

Stop using the .bdrv_co_io_plug() API because it is not multi-queue
block layer friendly. Use the new blk_io_plug_call() API to batch I/O
submission instead.

Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Stefano Garzarella <sgarzare@redhat.com>
Acked-by: Kevin Wolf <kwolf@redhat.com>
Message-id: 20230530180959.1108766-5-stefanha@redhat.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>

show more ...

block/nvme: convert to blk_io_plug_call() API

Stop using the .bdrv_co_io_plug() API because it is not multi-queue
block layer friendly. Use the new blk_io_plug_call() API to batch I/O
submission ins

block/nvme: convert to blk_io_plug_call() API

Stop using the .bdrv_co_io_plug() API because it is not multi-queue
block layer friendly. Use the new blk_io_plug_call() API to batch I/O
submission instead.

Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Stefano Garzarella <sgarzare@redhat.com>
Acked-by: Kevin Wolf <kwolf@redhat.com>
Message-id: 20230530180959.1108766-3-stefanha@redhat.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>

show more ...

Merge tag 'block-pull-request' of https://gitlab.com/stefanha/qemu into staging

Pull request

This pull request contain's Sam Li's zoned storage support in the QEMU block
layer and virtio-blk emulat

Merge tag 'block-pull-request' of https://gitlab.com/stefanha/qemu into staging

Pull request

This pull request contain's Sam Li's zoned storage support in the QEMU block
layer and virtio-blk emulation.

v2:
- Sam fixed the CI failures. CI passes for me now. [Richard]

# -----BEGIN PGP SIGNATURE-----
#
# iQEzBAABCAAdFiEEhpWov9P5fNqsNXdanKSrs4Grc8gFAmRiWCgACgkQnKSrs4Gr
# c8h/7gf+MMm2cGEaf376t8HMwTc6wbXVfbmAlZrge2EXPZfFvEaxj7HClcEraOgV
# yJsGWeU6mOw4r68ICJ/4KhrY1cdv+VZym/LsMLMcFUTXFHnyX4pyU3am31FPOI4K
# +wrDYJOJhc4DkAESWGgEWiMKpuO/uUEgBmHdW+qPFCl77Yl/eP6H5uNP6nGFn55p
# QpS/l8iha7PDkc81EsrjA+e/YI0ubfNSP7+zZElhQ98354CQ0MCfmZ6h9bT+o2bu
# R7SBUj80e+2X0a1b9s/2Jz/x8l4TEsl8kr48/Q1usq3GVVkbjEgqsk6wTN13Q/4g
# CeIR7E61ZeYzmpb4tLFRIqK2Jw+NEQ==
# =Q8xW
# -----END PGP SIGNATURE-----
# gpg: Signature made Mon 15 May 2023 09:04:56 AM PDT
# gpg: using RSA key 8695A8BFD3F97CDAAC35775A9CA4ABB381AB73C8
# gpg: Good signature from "Stefan Hajnoczi <stefanha@redhat.com>" [full]
# gpg: aka "Stefan Hajnoczi <stefanha@gmail.com>" [full]

* tag 'block-pull-request' of https://gitlab.com/stefanha/qemu:
docs/zoned-storage:add zoned emulation use case
virtio-blk: add some trace events for zoned emulation
block: add accounting for zone append operation
virtio-blk: add zoned storage emulation for zoned devices
block: add some trace events for zone append
qemu-iotests: test zone append operation
block: introduce zone append write for zoned devices
file-posix: add tracking of the zone write pointers
docs/zoned-storage: add zoned device documentation
block: add some trace events for new block layer APIs
iotests: test new zone operations
block: add zoned BlockDriver check to block layer
block/raw-format: add zone operations to pass through requests
block/block-backend: add block layer APIs resembling Linux ZonedBlockDevice ioctls
block/file-posix: introduce helper functions for sysfs attributes
block/block-common: add zoned device structs

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>

show more ...

block: add some trace events for zone append

Signed-off-by: Sam Li <faithilikerun@gmail.com>
Reviewed-by: Dmitry Fomichev <dmitry.fomichev@wdc.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>

block: add some trace events for zone append

Signed-off-by: Sam Li <faithilikerun@gmail.com>
Reviewed-by: Dmitry Fomichev <dmitry.fomichev@wdc.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-id: 20230508051510.177850-5-faithilikerun@gmail.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>

show more ...

block: add some trace events for new block layer APIs

Signed-off-by: Sam Li <faithilikerun@gmail.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Dmitry Fomichev <dmitry.fomichev

block: add some trace events for new block layer APIs

Signed-off-by: Sam Li <faithilikerun@gmail.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Dmitry Fomichev <dmitry.fomichev@wdc.com>
Acked-by: Kevin Wolf <kwolf@redhat.com>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-id: 20230508045533.175575-8-faithilikerun@gmail.com
Message-id: 20230324090605.28361-8-faithilikerun@gmail.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>

show more ...

Merge tag 'pull-block-2022-06-14-v2' of https://gitlab.com/vsementsov/qemu into staging

Block jobs & NBD patches

v2: - add arguments to QEMUMachine constructor in test, to make it work
on arm

Merge tag 'pull-block-2022-06-14-v2' of https://gitlab.com/vsementsov/qemu into staging

Block jobs & NBD patches

v2: - add arguments to QEMUMachine constructor in test, to make it work
on arm in gitlab pipeline
- use bdrv_inc_in_flight() / bdrv_dec_in_flight() instead of direct
manipulation with bs->in_flight

- add new options for copy-before-write filter
- new trace points for NBD
- prefer unsigned type for some 'in_flight' fields

# -----BEGIN PGP SIGNATURE-----
#
# iQIzBAABCgAdFiEEi5wmzbL9FHyIDoahVh8kwfGfefsFAmK8BqkACgkQVh8kwfGf
# efuiMw/+P9FFLfGFSjVP+LYeT0Ah6kN1ypCMQzIk3Qq/J6qgMZhtRqpQoZOfIFQL
# U9fGmEtQZ7gvEMD/nJToL6uOYlnQfPxDcA6GrRwWWE3rcFiPK4J0q2LlnPLn4QaU
# W/qag5l/QnZfLlj/iV6neWOEvqdnvY1L8IS+T8xV6N0iBYlwgMC/6FGshQwehzcV
# T5F1qPGB0vjFDjf92LFPEsvsFFHjHIVPwOyJMvF64QtSk57utikq/la9PI/yA9AH
# Ll4mNQuZKx6rSI5wE6b21jc8iOUvaoHdPSEDQZfNILSdgGdiKvFwE51y+baGnIAD
# TpjxfG59q0jyGxMjQVxMRSFaxAC4+Mqi82diSPv4xbiUdsE4byJH0oENn4z7+wAv
# EvjuU9yx4FfHHltoUNwfn2pv00o/ELdZIoBNmW36rPxSGZMvfVfRtuBL7XWNUFbW
# Fr4BwsjC4KfIxb16QTBGhXVv6grxdlwoU9N23npdi0YpW1ftZzXGDa85+gINQ807
# eK/gP/OtYPwIql0bgmLiuaRNzC9psmQOO6vQbdvd/e4BEVWkxiez37+e+zFMStmL
# OAL8rS6jckUoxVZjCYFEWg97XOobLUIhQxt9Fwh2omMDGKTwv861ghUAivxSWs93
# IRNxfwqNPxnpDDXjnK1ayZgU08IL98AUYVKcPN1y8JvEhB4Hr1k=
# =ndKk
# -----END PGP SIGNATURE-----
# gpg: Signature made Wed 29 Jun 2022 01:30:41 PM +0530
# gpg: using RSA key 8B9C26CDB2FD147C880E86A1561F24C1F19F79FB
# gpg: Good signature from "Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>" [unknown]
# gpg: aka "Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>" [unknown]
# gpg: WARNING: This key is not certified with a trusted signature!
# gpg: There is no indication that the signature belongs to the owner.
# Primary key fingerprint: 8B9C 26CD B2FD 147C 880E 86A1 561F 24C1 F19F 79FB

* tag 'pull-block-2022-06-14-v2' of https://gitlab.com/vsementsov/qemu:
block: use 'unsigned' for in_flight field on driver state
nbd: trace long NBD operations
iotests: copy-before-write: add cases for cbw-timeout option
block/copy-before-write: implement cbw-timeout option
block/block-copy: block_copy(): add timeout_ns parameter
util: add qemu-co-timeout
iotests: add copy-before-write: on-cbw-error tests
block/copy-before-write: add on-cbw-error open parameter
block/copy-before-write: refactor option parsing

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>

show more ...

nbd: trace long NBD operations

At the moment there are 2 sources of lengthy operations if configured:
* open connection, which could retry inside and
* reconnect of already opened connection
These o

nbd: trace long NBD operations

At the moment there are 2 sources of lengthy operations if configured:
* open connection, which could retry inside and
* reconnect of already opened connection
These operations could be quite lengthy and cumbersome to catch thus
it would be quite natural to add trace points for them.

This patch is based on the original downstream work made by Vladimir.

Signed-off-by: Denis V. Lunev <den@openvz.org>
CC: Eric Blake <eblake@redhat.com>
CC: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
CC: Kevin Wolf <kwolf@redhat.com>
CC: Hanna Reitz <hreitz@redhat.com>
CC: Paolo Bonzini <pbonzini@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>

show more ...

Merge remote-tracking branch 'remotes/kwolf/tags/for-upstream' into staging

Block layer patches

- Fail gracefully when blockdev-snapshot creates loops
- ide: Fix IDENTIFY DEVICE for disks > 128 GiB

Merge remote-tracking branch 'remotes/kwolf/tags/for-upstream' into staging

Block layer patches

- Fail gracefully when blockdev-snapshot creates loops
- ide: Fix IDENTIFY DEVICE for disks > 128 GiB
- file-posix: Fix return value translation for AIO discards
- file-posix: add 'aio-max-batch' option
- rbd: implement bdrv_co_block_status
- Code cleanups and build fixes

# gpg: Signature made Tue 02 Nov 2021 12:04:02 PM EDT
# gpg: using RSA key DC3DEB159A9AF95D3D7456FE7F09B272C88F2FD6
# gpg: issuer "kwolf@redhat.com"
# gpg: Good signature from "Kevin Wolf <kwolf@redhat.com>" [full]

* remotes/kwolf/tags/for-upstream:
block/nvme: Extract nvme_free_queue() from nvme_free_queue_pair()
block/nvme: Display CQ/SQ pointer in nvme_free_queue_pair()
block/nvme: Automatically free qemu_memalign() with QEMU_AUTO_VFREE
block-backend: Silence clang -m32 compiler warning
linux-aio: add `dev_max_batch` parameter to laio_io_unplug()
linux-aio: add `dev_max_batch` parameter to laio_co_submit()
file-posix: add `aio-max-batch` option
block/export/fuse.c: fix musl build
ide: Cap LBA28 capacity announcement to 2^28-1
block/rbd: implement bdrv_co_block_status
block: Fail gracefully when blockdev-snapshot creates loops
block/file-posix: Fix return value translation for AIO discards

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>

show more ...

block/nvme: Display CQ/SQ pointer in nvme_free_queue_pair()

For debugging purpose it is helpful to know the CQ/SQ pointers.
We already have a trace event in nvme_free_queue_pair(), extend
it to repo

block/nvme: Display CQ/SQ pointer in nvme_free_queue_pair()

For debugging purpose it is helpful to know the CQ/SQ pointers.
We already have a trace event in nvme_free_queue_pair(), extend
it to report these pointer addresses.

Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20211006164931.172349-3-philmd@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>

show more ...

Merge remote-tracking branch 'remotes/ericb/tags/pull-nbd-2021-10-15' into staging

nbd patches for 2021-10-15

- Vladimir Sementsov-Ogievskiy: Consistent use of 64-bit parameters in
block operatio

Merge remote-tracking branch 'remotes/ericb/tags/pull-nbd-2021-10-15' into staging

nbd patches for 2021-10-15

- Vladimir Sementsov-Ogievskiy: Consistent use of 64-bit parameters in
block operations
- Hanna Reitz: Silence 32-bit compiler warning

# gpg: Signature made Fri 15 Oct 2021 02:08:10 PM PDT
# gpg: using RSA key 71C2CC22B1C4602927D2F3AAA7A16B4A2527436A
# gpg: Good signature from "Eric Blake <eblake@redhat.com>" [full]
# gpg: aka "Eric Blake (Free Software Programmer) <ebb9@byu.net>" [full]
# gpg: aka "[jpeg image of size 6874]" [full]

* remotes/ericb/tags/pull-nbd-2021-10-15:
block-backend: update blk_co_pwrite() and blk_co_pread() wrappers
block-backend: fix blk_co_flush prototype to mention coroutine_fn
block-backend: drop INT_MAX restriction from blk_check_byte_request()
block-backend: blk_pread, blk_pwrite: rename count parameter to bytes
block-backend: convert blk_aio_ functions to int64_t bytes paramter
block-backend: convert blk_co_copy_range to int64_t bytes
block-backend: convert blk_foo wrappers to use int64_t bytes parameter
block-backend: drop blk_prw, use block-coroutine-wrapper
block-coroutine-wrapper.py: support BlockBackend first argument
block-backend: rename _do_ helper functions to _co_do_
block-backend: convert blk_co_pdiscard to int64_t bytes
block-backend: convert blk_co_pwritev_part to int64_t bytes
block-backend: make blk_co_preadv() 64bit
block-backend: blk_check_byte_request(): int64_t bytes
qcow2: Silence clang -m32 compiler warning

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>

show more ...

block-backend: convert blk_co_pwritev_part to int64_t bytes

We convert blk_do_pwritev_part() and some wrappers:
blk_co_pwritev_part(), blk_co_pwritev(), blk_co_pwrite_zeroes().

All functions are co

block-backend: convert blk_co_pwritev_part to int64_t bytes

We convert blk_do_pwritev_part() and some wrappers:
blk_co_pwritev_part(), blk_co_pwritev(), blk_co_pwrite_zeroes().

All functions are converted so that the parameter type becomes wider, so
all callers should be OK with it.

Look at blk_do_pwritev_part() body:
bytes is passed to:

- trace_blk_co_pwritev (we update it here)
- blk_check_byte_request, throttle_group_co_io_limits_intercept,
bdrv_co_pwritev_part - all already have int64_t argument.

Note that requests exceeding INT_MAX are still restricted by
blk_check_byte_request().

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20211006131718.214235-4-vsementsov@virtuozzo.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
[eblake: grammar tweaks]
Signed-off-by: Eric Blake <eblake@redhat.com>

show more ...

block-backend: make blk_co_preadv() 64bit

For both updated functions, the type of bytes becomes wider, so all callers
should be OK with it.

blk_co_preadv() only passes its arguments to blk_do_pread

block-backend: make blk_co_preadv() 64bit

For both updated functions, the type of bytes becomes wider, so all callers
should be OK with it.

blk_co_preadv() only passes its arguments to blk_do_preadv().

blk_do_preadv() passes bytes to:

- trace_blk_co_preadv, which is updated too
- blk_check_byte_request, throttle_group_co_io_limits_intercept,
bdrv_co_preadv, which are already int64_t.

Note that requests exceeding INT_MAX are still restricted by
blk_check_byte_request().

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20211006131718.214235-3-vsementsov@virtuozzo.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
[eblake: grammar tweaks]
Signed-off-by: Eric Blake <eblake@redhat.com>

show more ...

Merge remote-tracking branch 'remotes/ericb/tags/pull-nbd-2021-09-27-v2' into staging

nbd patches for 2021-09-27

- Vladimir Sementsov-Ogievskiy: Rework coroutines of qemu NBD client
to improve re

Merge remote-tracking branch 'remotes/ericb/tags/pull-nbd-2021-09-27-v2' into staging

nbd patches for 2021-09-27

- Vladimir Sementsov-Ogievskiy: Rework coroutines of qemu NBD client
to improve reconnect support
- Eric Blake: Relax server in regards to NBD_OPT_LIST_META_CONTEXT
- Vladimir Sementsov-Ogievskiy: Plumb up 64-bit bulk-zeroing support
in block layer, in preparation for future NBD spec extensions
- Nir Soffer: Default to writeback cache in qemu-nbd

# gpg: Signature made Wed 29 Sep 2021 22:07:58 BST
# gpg: using RSA key 71C2CC22B1C4602927D2F3AAA7A16B4A2527436A
# gpg: Good signature from "Eric Blake <eblake@redhat.com>" [full]
# gpg: aka "Eric Blake (Free Software Programmer) <ebb9@byu.net>" [full]
# gpg: aka "[jpeg image of size 6874]" [full]
# Primary key fingerprint: 71C2 CC22 B1C4 6029 27D2 F3AA A7A1 6B4A 2527 436A

* remotes/ericb/tags/pull-nbd-2021-09-27-v2:
block/nbd: check that received handle is valid
block/nbd: drop connection_co
block/nbd: refactor nbd_recv_coroutines_wake_all()
block/nbd: move nbd_recv_coroutines_wake_all() up
block/nbd: nbd_channel_error() shutdown channel unconditionally
nbd/client-connection: nbd_co_establish_connection(): fix non set errp
nbd/server: Allow LIST_META_CONTEXT without STRUCTURED_REPLY
block/io: allow 64bit discard requests
block: use int64_t instead of int in driver discard handlers
block: make BlockLimits::max_pdiscard 64bit
block/io: allow 64bit write-zeroes requests
block: use int64_t instead of int in driver write_zeroes handlers
block: make BlockLimits::max_pwrite_zeroes 64bit
block: use int64_t instead of uint64_t in copy_range driver handlers
block: use int64_t instead of uint64_t in driver write handlers
block: use int64_t instead of uint64_t in driver read handlers
qcow2: check request on vmstate save/load path
block/io: bring request check to bdrv_co_(read,write)v_vmstate
qemu-nbd: Change default cache mode to writeback

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

block: use int64_t instead of int in driver discard handlers

We are generally moving to int64_t for both offset and bytes parameters
on all io paths.

Main motivation is realization of 64-bit write_

block: use int64_t instead of int in driver discard handlers

We are generally moving to int64_t for both offset and bytes parameters
on all io paths.

Main motivation is realization of 64-bit write_zeroes operation for
fast zeroing large disk chunks, up to the whole disk.

We chose signed type, to be consistent with off_t (which is signed) and
with possibility for signed return type (where negative value means
error).

So, convert driver discard handlers bytes parameter to int64_t.

The only caller of all updated function is bdrv_co_pdiscard in
block/io.c. It is already prepared to work with 64bit requests, but
pass at most max(bs->bl.max_pdiscard, INT_MAX) to the driver.

Let's look at all updated functions:

blkdebug: all calculations are still OK, thanks to
bdrv_check_qiov_request().
both rule_check and bdrv_co_pdiscard are 64bit

blklogwrites: pass to blk_loc_writes_co_log which is 64bit

blkreplay, copy-on-read, filter-compress: pass to bdrv_co_pdiscard, OK

copy-before-write: pass to bdrv_co_pdiscard which is 64bit and to
cbw_do_copy_before_write which is 64bit

file-posix: one handler calls raw_account_discard() is 64bit and both
handlers calls raw_do_pdiscard(). Update raw_do_pdiscard, which pass
to RawPosixAIOData::aio_nbytes, which is 64bit (and calls
raw_account_discard())

gluster: somehow, third argument of glfs_discard_async is size_t.
Let's set max_pdiscard accordingly.

iscsi: iscsi_allocmap_set_invalid is 64bit,
!is_byte_request_lun_aligned is 64bit.
list.num is uint32_t. Let's clarify max_pdiscard and
pdiscard_alignment.

mirror_top: pass to bdrv_mirror_top_do_write() which is
64bit

nbd: protocol limitation. max_pdiscard is alredy set strict enough,
keep it as is for now.

nvme: buf.nlb is uint32_t and we do shift. So, add corresponding limits
to nvme_refresh_limits().

preallocate: pass to bdrv_co_pdiscard() which is 64bit.

rbd: pass to qemu_rbd_start_co() which is 64bit.

qcow2: calculations are still OK, thanks to bdrv_check_qiov_request(),
qcow2_cluster_discard() is 64bit.

raw-format: raw_adjust_offset() is 64bit, bdrv_co_pdiscard too.

throttle: pass to bdrv_co_pdiscard() which is 64bit and to
throttle_group_co_io_limits_intercept() which is 64bit as well.

test-block-iothread: bytes argument is unused

Great! Now all drivers are prepared to handle 64bit discard requests,
or else have explicit max_pdiscard limits.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20210903102807.27127-11-vsementsov@virtuozzo.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Eric Blake <eblake@redhat.com>

show more ...

block: use int64_t instead of int in driver write_zeroes handlers

We are generally moving to int64_t for both offset and bytes parameters
on all io paths.

Main motivation is realization of 64-bit w

block: use int64_t instead of int in driver write_zeroes handlers

We are generally moving to int64_t for both offset and bytes parameters
on all io paths.

Main motivation is realization of 64-bit write_zeroes operation for
fast zeroing large disk chunks, up to the whole disk.

We chose signed type, to be consistent with off_t (which is signed) and
with possibility for signed return type (where negative value means
error).

So, convert driver write_zeroes handlers bytes parameter to int64_t.

The only caller of all updated function is bdrv_co_do_pwrite_zeroes().

bdrv_co_do_pwrite_zeroes() itself is of course OK with widening of
callee parameter type. Also, bdrv_co_do_pwrite_zeroes()'s
max_write_zeroes is limited to INT_MAX. So, updated functions all are
safe, they will not get "bytes" larger than before.

Still, let's look through all updated functions, and add assertions to
the ones which are actually unprepared to values larger than INT_MAX.
For these drivers also set explicit max_pwrite_zeroes limit.

Let's go:

blkdebug: calculations can't overflow, thanks to
bdrv_check_qiov_request() in generic layer. rule_check() and
bdrv_co_pwrite_zeroes() both have 64bit argument.

blklogwrites: pass to blk_log_writes_co_log() with 64bit argument.

blkreplay, copy-on-read, filter-compress: pass to
bdrv_co_pwrite_zeroes() which is OK

copy-before-write: Calls cbw_do_copy_before_write() and
bdrv_co_pwrite_zeroes, both have 64bit argument.

file-posix: both handler calls raw_do_pwrite_zeroes, which is updated.
In raw_do_pwrite_zeroes() calculations are OK due to
bdrv_check_qiov_request(), bytes go to RawPosixAIOData::aio_nbytes
which is uint64_t.
Check also where that uint64_t gets handed:
handle_aiocb_write_zeroes_block() passes a uint64_t[2] to
ioctl(BLKZEROOUT), handle_aiocb_write_zeroes() calls do_fallocate()
which takes off_t (and we compile to always have 64-bit off_t), as
does handle_aiocb_write_zeroes_unmap. All look safe.

gluster: bytes go to GlusterAIOCB::size which is int64_t and to
glfs_zerofill_async works with off_t.

iscsi: Aha, here we deal with iscsi_writesame16_task() that has
uint32_t num_blocks argument and iscsi_writesame16_task() has
uint16_t argument. Make comments, add assertions and clarify
max_pwrite_zeroes calculation.
iscsi_allocmap_() functions already has int64_t argument
is_byte_request_lun_aligned is simple to update, do it.

mirror_top: pass to bdrv_mirror_top_do_write which has uint64_t
argument

nbd: Aha, here we have protocol limitation, and NBDRequest::len is
uint32_t. max_pwrite_zeroes is cleanly set to 32bit value, so we are
OK for now.

nvme: Again, protocol limitation. And no inherent limit for
write-zeroes at all. But from code that calculates cdw12 it's obvious
that we do have limit and alignment. Let's clarify it. Also,
obviously the code is not prepared to handle bytes=0. Let's handle
this case too.
trace events already 64bit

preallocate: pass to handle_write() and bdrv_co_pwrite_zeroes(), both
64bit.

rbd: pass to qemu_rbd_start_co() which is 64bit.

qcow2: offset + bytes and alignment still works good (thanks to
bdrv_check_qiov_request()), so tail calculation is OK
qcow2_subcluster_zeroize() has 64bit argument, should be OK
trace events updated

qed: qed_co_request wants int nb_sectors. Also in code we have size_t
used for request length which may be 32bit. So, let's just keep
INT_MAX as a limit (aligning it down to pwrite_zeroes_alignment) and
don't care.

raw-format: Is OK. raw_adjust_offset and bdrv_co_pwrite_zeroes are both
64bit.

throttle: Both throttle_group_co_io_limits_intercept() and
bdrv_co_pwrite_zeroes() are 64bit.

vmdk: pass to vmdk_pwritev which is 64bit

quorum: pass to quorum_co_pwritev() which is 64bit

Hooray!

At this point all block drivers are prepared to support 64bit
write-zero requests, or have explicitly set max_pwrite_zeroes.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20210903102807.27127-8-vsementsov@virtuozzo.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
[eblake: use <= rather than < in assertions relying on max_pwrite_zeroes]
Signed-off-by: Eric Blake <eblake@redhat.com>

show more ...

block: use int64_t instead of uint64_t in driver write handlers

We are generally moving to int64_t for both offset and bytes parameters
on all io paths.

Main motivation is realization of 64-bit wri

block: use int64_t instead of uint64_t in driver write handlers

We are generally moving to int64_t for both offset and bytes parameters
on all io paths.

Main motivation is realization of 64-bit write_zeroes operation for
fast zeroing large disk chunks, up to the whole disk.

We chose signed type, to be consistent with off_t (which is signed) and
with possibility for signed return type (where negative value means
error).

So, convert driver write handlers parameters which are already 64bit to
signed type.

While being here, convert also flags parameter to be BdrvRequestFlags.

Now let's consider all callers. Simple

git grep '\->bdrv_\(aio\|co\)_pwritev\(_part\)\?'

shows that's there three callers of driver function:

bdrv_driver_pwritev() and bdrv_driver_pwritev_compressed() in
block/io.c, both pass int64_t, checked by bdrv_check_qiov_request() to
be non-negative.

qcow2_save_vmstate() does bdrv_check_qiov_request().

Still, the functions may be called directly, not only by drv->...
Let's check:

git grep '\.bdrv_\(aio\|co\)_pwritev\(_part\)\?\s*=' | \
awk '{print $4}' | sed 's/,//' | sed 's/&//' | sort | uniq | \
while read func; do git grep "$func(" | \
grep -v "$func(BlockDriverState"; done

shows several callers:

qcow2:
qcow2_co_truncate() write at most up to @offset, which is checked in
generic qcow2_co_truncate() by bdrv_check_request().
qcow2_co_pwritev_compressed_task() pass the request (or part of the
request) that already went through normal write path, so it should
be OK

qcow:
qcow_co_pwritev_compressed() pass int64_t, it's updated by this patch

quorum:
quorum_co_pwrite_zeroes() pass int64_t and int - OK

throttle:
throttle_co_pwritev_compressed() pass int64_t, it's updated by this
patch

vmdk:
vmdk_co_pwritev_compressed() pass int64_t, it's updated by this
patch

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20210903102807.27127-5-vsementsov@virtuozzo.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Eric Blake <eblake@redhat.com>

show more ...

Merge remote-tracking branch 'remotes/stefanha-gitlab/tags/block-pull-request' into staging

Pull request

Userspace NVMe driver patches.

# gpg: Signature made Tue 07 Sep 2021 09:13:57 BST
# gpg:

Merge remote-tracking branch 'remotes/stefanha-gitlab/tags/block-pull-request' into staging

Pull request

Userspace NVMe driver patches.

# gpg: Signature made Tue 07 Sep 2021 09:13:57 BST
# gpg: using RSA key 8695A8BFD3F97CDAAC35775A9CA4ABB381AB73C8
# gpg: Good signature from "Stefan Hajnoczi <stefanha@redhat.com>" [full]
# gpg: aka "Stefan Hajnoczi <stefanha@gmail.com>" [full]
# Primary key fingerprint: 8695 A8BF D3F9 7CDA AC35 775A 9CA4 ABB3 81AB 73C8

* remotes/stefanha-gitlab/tags/block-pull-request:
block/nvme: Only report VFIO error on failed retry
util/vfio-helpers: Let qemu_vfio_do_mapping() propagate Error
util/vfio-helpers: Simplify qemu_vfio_dma_map() returning directly
util/vfio-helpers: Use error_setg in qemu_vfio_find_[fixed/temp]_iova
util/vfio-helpers: Extract qemu_vfio_water_mark_reached()
util/vfio-helpers: Pass Error handle to qemu_vfio_dma_map()
block/nvme: Have nvme_create_queue_pair() report errors consistently
util/vfio-helpers: Remove unreachable code in qemu_vfio_dma_map()
util/vfio-helpers: Replace qemu_mutex_lock() calls with QEMU_LOCK_GUARD
util/vfio-helpers: Let qemu_vfio_verify_mappings() use error_report()
block/nvme: Use safer trace format string

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

block/nvme: Use safer trace format string

Fix when building with -Wshorten-64-to-32:

warning: implicit conversion loses integer precision: 'unsigned long' to 'int' [-Wshorten-64-to-32]

Reviewed-

block/nvme: Use safer trace format string

Fix when building with -Wshorten-64-to-32:

warning: implicit conversion loses integer precision: 'unsigned long' to 'int' [-Wshorten-64-to-32]

Reviewed-by: Klaus Jensen <k.jensen@samsung.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-id: 20210902070025.197072-2-philmd@redhat.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>

show more ...

Merge remote-tracking branch 'remotes/berrange-gitlab/tags/misc-fixes-pull-request' into staging

Merge misc patches

# gpg: Signature made Mon 14 Jun 2021 15:14:48 BST
# gpg: using RS

Merge remote-tracking branch 'remotes/berrange-gitlab/tags/misc-fixes-pull-request' into staging

Merge misc patches

# gpg: Signature made Mon 14 Jun 2021 15:14:48 BST
# gpg: using RSA key DAF3A6FDB26B62912D0E8E3FBE86EBB415104FDF
# gpg: Good signature from "Daniel P. Berrange <dan@berrange.com>" [full]
# gpg: aka "Daniel P. Berrange <berrange@redhat.com>" [full]
# Primary key fingerprint: DAF3 A6FD B26B 6291 2D0E 8E3F BE86 EBB4 1510 4FDF

* remotes/berrange-gitlab/tags/misc-fixes-pull-request:
usb/dev-mtp: use GDateTime for formatting timestamp for objects
block: use GDateTime for formatting timestamp when dumping snapshot info
migration: use GDateTime for formatting timestamp in snapshot names
block: remove duplicate trace.h include
block: add trace point when fdatasync fails
block: preserve errno from fdatasync failures
softmmu: add trace point when bdrv_flush_all fails
migration: add trace point when vm_stop_force_state fails
sasl: remove comment about obsolete kerberos versions
docs: recommend SCRAM-SHA-256 SASL mech instead of SHA-1 variant
docs: document usage of the authorization framework
docs: document how to pass secret data to QEMU
docs: add table of contents to QAPI references

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

block: add trace point when fdatasync fails

A flush failure is a critical failure scenario for some operations.
For example, it will prevent migration from completing, as it will
make vm_stop() repo

block: add trace point when fdatasync fails

A flush failure is a critical failure scenario for some operations.
For example, it will prevent migration from completing, as it will
make vm_stop() report an error. Thus it is important to have a
trace point present for debugging.

Reviewed-by: Connor Kuehl <ckuehl@redhat.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>

show more ...