iotests: fix expected output from gnutls

Error reporting from gnutls was improved by:

commit 57941c9c86357a6a642f9ee3279d881df4043b6d
Author: Daniel P. Berrangé <berrange@redhat.com>
Date:

iotests: fix expected output from gnutls

Error reporting from gnutls was improved by:

commit 57941c9c86357a6a642f9ee3279d881df4043b6d
Author: Daniel P. Berrangé <berrange@redhat.com>
Date: Fri Mar 15 14:07:58 2024 +0000

crypto: push error reporting into TLS session I/O APIs

This has the effect of changing the output from one of the NBD
tests.

Reported-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
(cherry picked from commit 48b8583698d96d6290726400789fcd51c55691b1)
Signed-off-by: Michael Tokarev <mjt@tls.msk.ru>

show more ...

Merge tag 'crypto-fixes-pull-request' of https://gitlab.com/berrange/qemu into staging

Various crypto fixes

* Support sha384 with glib crypto backend
* Improve error reporting for unsupported cip

Merge tag 'crypto-fixes-pull-request' of https://gitlab.com/berrange/qemu into staging

Various crypto fixes

* Support sha384 with glib crypto backend
* Improve error reporting for unsupported cipher modes
* Avoid memory leak when bad cipher mode is given
* Run pbkdf tests on macOS
* Runtime check for pbkdf hash impls with gnutls & gcrypt
* Avoid hangs counter pbkdf iterations on some Linux kernels
by using a throwaway thread for benchmarking performance
* Fix iotests expected output from gnutls errors

# -----BEGIN PGP SIGNATURE-----
#
# iQIzBAABCAAdFiEE2vOm/bJrYpEtDo4/vobrtBUQT98FAmbfAy4ACgkQvobrtBUQ
# T99ZLhAAomQ7GeUNYM2/Fh9VptSAy9nddp7WwMd2egTZ+wPMnvalxXiMupf0WNzb
# 3CJtXojeNCCV8PtBuTmlCmLg+HxFA+zQgkizS9WqUOMies54woGrwQCUIlyez1Au
# Y2+9GbRRrkJA860R1aK2EJI5C2ofJf+CJd/nWSxsTzZSPu6Iu3V9ZahSdAq8o96R
# AMh//6MOuX5pauyTYLXL3jpxEJEM4YjQ/+AF8D5FYustcN1Icjv2KPL1PArwUOTe
# m1NZtcWLxZpmis5vXO8davMbB16bda6YUxuBQ++pFlF3ars7U2JldZ4DIqECAKkI
# sxtw6Wq/IjdVwyJLj/+c7CX3/T1p4IuJ8ch6sfVnQz1KUf2NxPtBwSXqQneLUa2G
# b46swxL695nCBYkbcfgWYfL5BaU1b0W8Xkk4sRoTNN5tDcYOuE9nMTMu71pHifmz
# 1itkxvLdLkwH7mxzTAxVV+vdQk3KiXlmt42/hOJMgAC3WRp2JJsEv64Jpq9huooA
# a+7fM5c2r3b77q7hjIwp8X6HmNehCt2KQiGvn0DvMmqb22r/RT9VzE89iNhPNSUx
# rCj7b2+19Xrfe1wxwl07GJ7yUXX4XIcphH66iO9nu1RQDBATqNSiJ/dHCfP9iiEP
# 7PHf4krOSzA+wL67FP+u8x0sVhpPmbPRvU8VW4+D+Av91TK5wo0=
# =wLAZ
# -----END PGP SIGNATURE-----
# gpg: Signature made Mon 09 Sep 2024 15:16:14 BST
# gpg: using RSA key DAF3A6FDB26B62912D0E8E3FBE86EBB415104FDF
# gpg: Good signature from "Daniel P. Berrange <dan@berrange.com>" [full]
# gpg: aka "Daniel P. Berrange <berrange@redhat.com>" [full]
# Primary key fingerprint: DAF3 A6FD B26B 6291 2D0E 8E3F BE86 EBB4 1510 4FDF

* tag 'crypto-fixes-pull-request' of https://gitlab.com/berrange/qemu:
crypto: Introduce x509 utils
crypto: Support SHA384 hash when using glib
crypto: Define macros for hash algorithm digest lengths
crypto: use consistent error reporting pattern for unsupported cipher modes
crypto: avoid leak of ctx when bad cipher mode is given
tests/unit: build pbkdf test on macOS
tests/unit: always build the pbkdf crypto unit test
crypto: check gnutls & gcrypt support the requested pbkdf hash
crypto: run qcrypto_pbkdf2_count_iters in a new thread
iotests: fix expected output from gnutls

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

iotests: fix expected output from gnutls

Error reporting from gnutls was improved by:

commit 57941c9c86357a6a642f9ee3279d881df4043b6d
Author: Daniel P. Berrangé <berrange@redhat.com>
Date:

iotests: fix expected output from gnutls

Error reporting from gnutls was improved by:

commit 57941c9c86357a6a642f9ee3279d881df4043b6d
Author: Daniel P. Berrangé <berrange@redhat.com>
Date: Fri Mar 15 14:07:58 2024 +0000

crypto: push error reporting into TLS session I/O APIs

This has the effect of changing the output from one of the NBD
tests.

Reported-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>

show more ...

Merge tag 'pull-nbd-2023-10-05' of https://repo.or.cz/qemu/ericb into staging

NBD patches for 2023-10-05

- various: mailmap cleanups
- Eric Blake: enable use of NBD 64-bit extended headers

# -----

Merge tag 'pull-nbd-2023-10-05' of https://repo.or.cz/qemu/ericb into staging

NBD patches for 2023-10-05

- various: mailmap cleanups
- Eric Blake: enable use of NBD 64-bit extended headers

# -----BEGIN PGP SIGNATURE-----
#
# iQEzBAABCAAdFiEEccLMIrHEYCkn0vOqp6FrSiUnQ2oFAmUfA94ACgkQp6FrSiUn
# Q2o2hAf/Q4q6RoEG9WoOIa6WB1nk9tZN6GRfW4jS+09hJTFVbYUhJOcvQwwZjPT6
# 6oIwde8w7uE+AxBOA3XPbgTOBnTnpt2RH2AYVctNYB2vonuSrx3/KE3XSi4nrJRo
# +XiPbOsDzQu+vYsI7XJ+5e13BE4iZVmzbyke0U6hO88uR6tQstDJV1Mhem4jIbCG
# uLzkVDs3yY5sUFoUe77lFGWgKqfh87eJXV18T76df97ZZ5O2/w9G8MknG/CTSsR0
# fmzzC1Q4a2UEFtX8M3etRQ/b5WbFYhM+XwFevm2YBpod89ejGEA0ohhpn+GpZDU9
# SU66lXL/5jM9N7RkiSFwvupaot9hvw==
# =UieX
# -----END PGP SIGNATURE-----
# gpg: Signature made Thu 05 Oct 2023 14:43:42 EDT
# gpg: using RSA key 71C2CC22B1C4602927D2F3AAA7A16B4A2527436A
# gpg: Good signature from "Eric Blake <eblake@redhat.com>" [full]
# gpg: aka "Eric Blake (Free Software Programmer) <ebb9@byu.net>" [full]
# gpg: aka "[jpeg image of size 6874]" [full]
# Primary key fingerprint: 71C2 CC22 B1C4 6029 27D2 F3AA A7A1 6B4A 2527 436A

* tag 'pull-nbd-2023-10-05' of https://repo.or.cz/qemu/ericb:
nbd/server: Add FLAG_PAYLOAD support to CMD_BLOCK_STATUS
nbd/server: Prepare for per-request filtering of BLOCK_STATUS
nbd/server: Refactor list of negotiated meta contexts
nbd/client: Request extended headers during negotiation
nbd/client: Accept 64-bit block status chunks
nbd/client: Initial support for extended headers
nbd/client: Plumb errp through nbd_receive_replies
nbd/server: Enable initial support for extended headers
nbd/server: Support 64-bit block status
nbd/server: Prepare to send extended header replies
nbd/server: Prepare to receive extended header requests
nbd/server: Support a request payload
mailmap: Fix BALATON Zoltan author email
maint: Tweak comments in mailmap regarding SPF
mailmap: Fix Andrey Drobyshev author email

Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>

show more ...

nbd/client: Request extended headers during negotiation

All the pieces are in place for a client to finally request extended
headers. Note that we must not request extended headers when qemu-nbd
is

nbd/client: Request extended headers during negotiation

All the pieces are in place for a client to finally request extended
headers. Note that we must not request extended headers when qemu-nbd
is used to connect to the kernel module (as nbd.ko does not expect
them, but expects us to do the negotiation in userspace before handing
the socket over to the kernel), but there is no harm in all other
clients requesting them.

Extended headers are not essential to the information collected during
'qemu-nbd --list', but probing for it gives us one more piece of
information in that output. Update the iotests affected by the new
line of output.

Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
Message-ID: <20230925192229.3186470-23-eblake@redhat.com>

show more ...

Merge remote-tracking branch 'remotes/ericb/tags/pull-nbd-2022-03-07' into staging

nbd patches for 2022-03-07

- Dan Berrange: Allow qemu-nbd to support TLS over Unix sockets
- Eric Blake: Minor cle

Merge remote-tracking branch 'remotes/ericb/tags/pull-nbd-2022-03-07' into staging

nbd patches for 2022-03-07

- Dan Berrange: Allow qemu-nbd to support TLS over Unix sockets
- Eric Blake: Minor cleanups related to 64-bit block operations

# gpg: Signature made Tue 08 Mar 2022 01:41:35 GMT
# gpg: using RSA key 71C2CC22B1C4602927D2F3AAA7A16B4A2527436A
# gpg: Good signature from "Eric Blake <eblake@redhat.com>" [full]
# gpg: aka "Eric Blake (Free Software Programmer) <ebb9@byu.net>" [full]
# gpg: aka "[jpeg image of size 6874]" [full]
# Primary key fingerprint: 71C2 CC22 B1C4 6029 27D2 F3AA A7A1 6B4A 2527 436A

* remotes/ericb/tags/pull-nbd-2022-03-07:
qemu-io: Allow larger write zeroes under no fallback
qemu-io: Utilize 64-bit status during map
nbd/server: Minor cleanups
tests/qemu-iotests: validate NBD TLS with UNIX sockets and PSK
tests/qemu-iotests: validate NBD TLS with UNIX sockets
tests/qemu-iotests: validate NBD TLS with hostname mismatch
tests/qemu-iotests: convert NBD TLS test to use standard filters
tests/qemu-iotests: introduce filter for qemu-nbd export list
tests/qemu-iotests: expand _filter_nbd rules
tests/qemu-iotests: add QEMU_IOTESTS_REGEN=1 to update reference file
block/nbd: don't restrict TLS usage to IP sockets
qemu-nbd: add --tls-hostname option for TLS certificate validation
block/nbd: support override of hostname for TLS certificate validation
block: pass desired TLS hostname through from block driver client
crypto: mandate a hostname when checking x509 creds on a client

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

tests/qemu-iotests: validate NBD TLS with UNIX sockets and PSK

This validates that connections to an NBD server running on a UNIX
socket can use TLS with pre-shared keys (PSK).

Reviewed-by: Eric Bl

tests/qemu-iotests: validate NBD TLS with UNIX sockets and PSK

This validates that connections to an NBD server running on a UNIX
socket can use TLS with pre-shared keys (PSK).

Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20220304193610.3293146-13-berrange@redhat.com>
[eblake: squash in rebase fix]
Tested-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Eric Blake <eblake@redhat.com>

show more ...

tests/qemu-iotests: validate NBD TLS with UNIX sockets

This validates that connections to an NBD server running on a UNIX
socket can use TLS, and require a TLS hostname override to pass
certificate

tests/qemu-iotests: validate NBD TLS with UNIX sockets

This validates that connections to an NBD server running on a UNIX
socket can use TLS, and require a TLS hostname override to pass
certificate validation.

Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20220304193610.3293146-12-berrange@redhat.com>
[eblake: squash in rebase fix]
Tested-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Eric Blake <eblake@redhat.com>

show more ...

tests/qemu-iotests: validate NBD TLS with hostname mismatch

This validates that connections to an NBD server where the certificate
hostname does not match will fail. It further validates that using

tests/qemu-iotests: validate NBD TLS with hostname mismatch

This validates that connections to an NBD server where the certificate
hostname does not match will fail. It further validates that using the
new 'tls-hostname' override option can solve the failure.

Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20220304193610.3293146-11-berrange@redhat.com>
Signed-off-by: Eric Blake <eblake@redhat.com>

show more ...

tests/qemu-iotests: convert NBD TLS test to use standard filters

Using standard filters is more future proof than rolling our own.

Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Daniel

tests/qemu-iotests: convert NBD TLS test to use standard filters

Using standard filters is more future proof than rolling our own.

Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20220304193610.3293146-10-berrange@redhat.com>
Signed-off-by: Eric Blake <eblake@redhat.com>

show more ...

Merge remote-tracking branch 'remotes/hreitz/tags/pull-block-2021-08-09' into staging

Block patches for 6.1-rc3:
- Build fix for FUSE block exports
- iotest 233 fix

# gpg: Signature made Mon 09 Aug

Merge remote-tracking branch 'remotes/hreitz/tags/pull-block-2021-08-09' into staging

Block patches for 6.1-rc3:
- Build fix for FUSE block exports
- iotest 233 fix

# gpg: Signature made Mon 09 Aug 2021 17:59:29 BST
# gpg: using RSA key 91BEB60A30DB3E8857D11829F407DB0061D5CF40
# gpg: issuer "mreitz@redhat.com"
# gpg: Good signature from "Max Reitz <mreitz@redhat.com>" [full]
# Primary key fingerprint: 91BE B60A 30DB 3E88 57D1 1829 F407 DB00 61D5 CF40

* remotes/hreitz/tags/pull-block-2021-08-09:
tests: filter out TLS distinguished name in certificate checks
block/export/fuse.c: fix musl build

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

tests: filter out TLS distinguished name in certificate checks

The version of GNUTLS in Fedora 34 has changed the order in which encodes
fields when generating new TLS certificates. This in turn cha

tests: filter out TLS distinguished name in certificate checks

The version of GNUTLS in Fedora 34 has changed the order in which encodes
fields when generating new TLS certificates. This in turn changes the
order seen when querying the distinguished name. This ultimately breaks
the expected output in the NBD TLS iotests. We don't need to be
comparing the exact distinguished name text for the purpose of the test
though, so it is fine to filter it out.

Reported-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20210804180330.3469683-1-berrange@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Tested-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Hanna Reitz <hreitz@redhat.com>

show more ...

Merge remote-tracking branch 'remotes/ericb/tags/pull-nbd-2019-09-24-v2' into staging

nbd patches for 2019-09-24

- Improved error message for plaintext client of encrypted server
- Fix various asse

Merge remote-tracking branch 'remotes/ericb/tags/pull-nbd-2019-09-24-v2' into staging

nbd patches for 2019-09-24

- Improved error message for plaintext client of encrypted server
- Fix various assertions when -object iothread is in use
- Silence a Coverity error for use-after-free on error path

# gpg: Signature made Wed 25 Sep 2019 14:35:52 BST
# gpg: using RSA key 71C2CC22B1C4602927D2F3AAA7A16B4A2527436A
# gpg: Good signature from "Eric Blake <eblake@redhat.com>" [full]
# gpg: aka "Eric Blake (Free Software Programmer) <ebb9@byu.net>" [full]
# gpg: aka "[jpeg image of size 6874]" [full]
# Primary key fingerprint: 71C2 CC22 B1C4 6029 27D2 F3AA A7A1 6B4A 2527 436A

* remotes/ericb/tags/pull-nbd-2019-09-24-v2:
util/qemu-sockets: fix keep_alive handling in inet_connect_saddr
tests: Use iothreads during iotest 223
nbd: Grab aio context lock in more places
nbd/server: attach client channel to the export's AioContext
nbd/client: Add hint when TLS is missing

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

nbd/client: Add hint when TLS is missing

I received an off-list report of failure to connect to an NBD server
expecting an x509 certificate, when the client was attempting something
similar to this

nbd/client: Add hint when TLS is missing

I received an off-list report of failure to connect to an NBD server
expecting an x509 certificate, when the client was attempting something
similar to this command line:

$ ./x86_64-softmmu/qemu-system-x86_64 -name 'blah' -machine q35 -nodefaults \
-object tls-creds-x509,id=tls0,endpoint=client,dir=$path_to_certs \
-device virtio-scsi-pci,id=virtio_scsi_pci0,bus=pcie.0,addr=0x6 \
-drive id=drive_image1,if=none,snapshot=off,aio=threads,cache=none,format=raw,file=nbd:localhost:9000,werror=stop,rerror=stop,tls-creds=tls0 \
-device scsi-hd,id=image1,drive=drive_image1,bootindex=0
qemu-system-x86_64: -drive id=drive_image1,if=none,snapshot=off,aio=threads,cache=none,format=raw,file=nbd:localhost:9000,werror=stop,rerror=stop,tls-creds=tls0: TLS negotiation required before option 7 (go)
server reported: Option 0x7 not permitted before TLS

The problem? As specified, -drive is trying to pass tls-creds to the
raw format driver instead of the nbd protocol driver, but before we
get to the point where we can detect that raw doesn't know what to do
with tls-creds, the nbd driver has already failed because the server
complained. The fix to the broken command line? Pass
'...,file.tls-creds=tls0' to ensure the tls-creds option is handed to
nbd, not raw. But since the error message was rather cryptic, I'm
trying to improve the error message.

With this patch, the error message adds a line:

qemu-system-x86_64: -drive id=drive_image1,if=none,snapshot=off,aio=threads,cache=none,format=raw,file=nbd:localhost:9000,werror=stop,rerror=stop,tls-creds=tls0: TLS negotiation required before option 7 (go)
Did you forget a valid tls-creds?
server reported: Option 0x7 not permitted before TLS

And with luck, someone grepping for that error message will find this
commit message and figure out their command line mistake. Sadly, the
only mention of file.tls-creds in our docs relates to an --image-opts
use of PSK encryption with qemu-img as the client, rather than x509
certificate encryption with qemu-kvm as the client.

CC: Tingting Mao <timao@redhat.com>
CC: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <20190907172055.26870-1-eblake@redhat.com>
[eblake: squash in iotest 233 fix]
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>

show more ...

Merge remote-tracking branch 'remotes/ericb/tags/pull-nbd-2019-09-05-v2' into staging

nbd patches for 2019-09-05

- Advertise NBD_FLAG_CAN_MULTI_CONN on readonly images
- Tolerate larger set of serv

Merge remote-tracking branch 'remotes/ericb/tags/pull-nbd-2019-09-05-v2' into staging

nbd patches for 2019-09-05

- Advertise NBD_FLAG_CAN_MULTI_CONN on readonly images
- Tolerate larger set of server error responses during handshake
- More precision on handling fallocate() failures due to alignment
- Better documentation of NBD connection URIs
- Implement new extension NBD_CMD_FLAG_FAST_ZERO to benefit qemu-img convert

# gpg: Signature made Thu 05 Sep 2019 22:08:17 BST
# gpg: using RSA key 71C2CC22B1C4602927D2F3AAA7A16B4A2527436A
# gpg: Good signature from "Eric Blake <eblake@redhat.com>" [full]
# gpg: aka "Eric Blake (Free Software Programmer) <ebb9@byu.net>" [full]
# gpg: aka "[jpeg image of size 6874]" [full]
# Primary key fingerprint: 71C2 CC22 B1C4 6029 27D2 F3AA A7A1 6B4A 2527 436A

* remotes/ericb/tags/pull-nbd-2019-09-05-v2:
nbd: Implement server use of NBD FAST_ZERO
nbd: Implement client use of NBD FAST_ZERO
nbd: Prepare for NBD_CMD_FLAG_FAST_ZERO
nbd: Improve per-export flag handling in server
docs: Update preferred NBD device syntax
block: workaround for unaligned byte range in fallocate()
nbd: Tolerate more errors to structured reply request
nbd: Use g_autofree in a few places
nbd: Advertise multi-conn for shared read-only connections

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

nbd: Implement server use of NBD FAST_ZERO

The server side is fairly straightforward: we can always advertise
support for detection of fast zero, and implement it by mapping the
request to the block

nbd: Implement server use of NBD FAST_ZERO

The server side is fairly straightforward: we can always advertise
support for detection of fast zero, and implement it by mapping the
request to the block layer BDRV_REQ_NO_FALLBACK.

Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <20190823143726.27062-5-eblake@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
[eblake: update iotests 223, 233]

show more ...

nbd: Tolerate more errors to structured reply request

A server may have a reason to reject a request for structured replies,
beyond just not recognizing them as a valid request; similarly, it may
ha

nbd: Tolerate more errors to structured reply request

A server may have a reason to reject a request for structured replies,
beyond just not recognizing them as a valid request; similarly, it may
have a reason for rejecting a request for a meta context. It doesn't
hurt us to continue talking to such a server; otherwise 'qemu-nbd
--list' of such a server fails to display all available details about
the export.

Encountered when temporarily tweaking nbdkit to reply with
NBD_REP_ERR_POLICY. Present since structured reply support was first
added (commit d795299b reused starttls handling, but starttls is
different in that we can't fall back to other behavior on any error).

Note that for an unencrypted client trying to connect to a server that
requires encryption, this defers the point of failure to when we
finally execute a strict command (such as NBD_OPT_GO or NBD_OPT_LIST),
now that the intermediate NBD_OPT_STRUCTURED_REPLY does not diagnose
NBD_REP_ERR_TLS_REQD as fatal; but as the protocol eventually gets us
to a command where we can't continue onwards, the changed error
message doesn't cause any security concerns.

Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <20190824172813.29720-3-eblake@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
[eblake: fix iotest 233]

show more ...

Merge remote-tracking branch 'remotes/ericb/tags/pull-nbd-2019-05-07' into staging

nbd patches for 2019-05-07

- iotest improvements

# gpg: Signature made Tue 07 May 2019 16:00:18 BST
# gpg:

Merge remote-tracking branch 'remotes/ericb/tags/pull-nbd-2019-05-07' into staging

nbd patches for 2019-05-07

- iotest improvements

# gpg: Signature made Tue 07 May 2019 16:00:18 BST
# gpg: using RSA key A7A16B4A2527436A
# gpg: Good signature from "Eric Blake <eblake@redhat.com>" [full]
# gpg: aka "Eric Blake (Free Software Programmer) <ebb9@byu.net>" [full]
# gpg: aka "[jpeg image of size 6874]" [full]
# Primary key fingerprint: 71C2 CC22 B1C4 6029 27D2 F3AA A7A1 6B4A 2527 436A

* remotes/ericb/tags/pull-nbd-2019-05-07:
iotests: Make 182 do without device_add
iotests: Tweak 221 sizing for different hole granularities
tests/qemu-iotests: Fix more reference output files due to recent qemu-io change
iotests: Fix 233 for ports other than 10809
qemu-nbd: Look up flag names in array

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

iotests: Fix 233 for ports other than 10809

233 generally filters the port, but in two cases does not. If some
other concurrently running application has already taken port 10809,
this will result

iotests: Fix 233 for ports other than 10809

233 generally filters the port, but in two cases does not. If some
other concurrently running application has already taken port 10809,
this will result in an output mismatch. Fix this by applying the
filter in these two cases, too.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20190506160529.6955-1-mreitz@redhat.com>
Signed-off-by: Eric Blake <eblake@redhat.com>

show more ...

Merge remote-tracking branch 'remotes/kevin/tags/for-upstream' into staging

Block layer patches:

- iotests: Fix output of qemu-io related tests
- Don't ignore bdrv_set_aio_context() for nodes with

Merge remote-tracking branch 'remotes/kevin/tags/for-upstream' into staging

Block layer patches:

- iotests: Fix output of qemu-io related tests
- Don't ignore bdrv_set_aio_context() for nodes with bs->drv = NUL
- vmdk: Set vmdk parent backing_format to vmdk
- qcow2: Preallocation fixes (especially for external data files)
- Add linear-buffer-based APIs (as wrappers around qiov-based ones)
- Various code cleanups and small corner case fixes

# gpg: Signature made Tue 30 Apr 2019 16:35:09 BST
# gpg: using RSA key 7F09B272C88F2FD6
# gpg: Good signature from "Kevin Wolf <kwolf@redhat.com>" [full]
# Primary key fingerprint: DC3D EB15 9A9A F95D 3D74 56FE 7F09 B272 C88F 2FD6

* remotes/kevin/tags/for-upstream: (27 commits)
block/qed: add missed coroutine_fn markers
iotests: Check that images are in read-only mode after block-commit
commit: Make base read-only if there is an early failure
qemu-img: use buffer-based io
block/stream: use buffer-based io
block/commit: use buffer-based io
block/backup: use buffer-based io
block/parallels: use buffer-based io
block/qed: use buffer-based io
block/qcow: use buffer-based io
block/qcow2: use buffer-based io
block: introduce byte-based io helpers
qcow2: Fix error handling in the compression code
qcow2: Fix qcow2_make_empty() with external data file
qemu-img: Make create hint at protocol options
iotests: Perform the correct test in 082
qcow2: Fix full preallocation with external data file
qcow2: Add errp to preallocate_co()
qcow2: Avoid COW during metadata preallocation
qemu-img: Saner printing of large file sizes
...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

qemu-img: Saner printing of large file sizes

Disk sizes close to INT64_MAX cause overflow, for some pretty
ridiculous output:

$ ./nbdkit -U - memory size=$((2**63 - 512)) --run 'qemu-img info $nb

qemu-img: Saner printing of large file sizes

Disk sizes close to INT64_MAX cause overflow, for some pretty
ridiculous output:

$ ./nbdkit -U - memory size=$((2**63 - 512)) --run 'qemu-img info $nbd'
image: nbd+unix://?socket=/tmp/nbdkitHSAzNz/socket
file format: raw
virtual size: -8388607T (9223372036854775296 bytes)
disk size: unavailable

But there's no reason to have two separate implementations of integer
to human-readable abbreviation, where one has overflow and stops at
'T', while the other avoids overflow and goes all the way to 'E'. With
this patch, the output now claims 8EiB instead of -8388607T, which
really is the correct rounding of largest file size supported by qemu
(we could go 511 bytes larger if we used byte-accurate sizing instead
of rounding up to the next sector boundary, but that wouldn't change
the human-readable result).

Quite a few iotests need updates to expected output to match.

Reported-by: Richard W.M. Jones <rjones@redhat.com>
Signed-off-by: Eric Blake <eblake@redhat.com>
Tested-by: Richard W.M. Jones <rjones@redhat.com>
Reviewed-by: Alberto Garcia <berto@igalia.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Tested-by: Max Reitz <mreitz@redhat.com>

show more ...

Merge remote-tracking branch 'remotes/ericb/tags/pull-nbd-2019-04-01' into staging

nbd patches for 2019-04-01

- Better behavior of qemu-img map on NBD images
- Fixes for NBD protocol alignment corn

Merge remote-tracking branch 'remotes/ericb/tags/pull-nbd-2019-04-01' into staging

nbd patches for 2019-04-01

- Better behavior of qemu-img map on NBD images
- Fixes for NBD protocol alignment corner cases:
- the server has fewer places where it sends reads or block status
not aligned to its advertised block size
- the client has more cases where it can work around server
non-compliance present in qemu 3.1
- the client now avoids non-compliant requests when interoperating
with nbdkit or other servers not advertising block size

# gpg: Signature made Mon 01 Apr 2019 15:06:54 BST
# gpg: using RSA key A7A16B4A2527436A
# gpg: Good signature from "Eric Blake <eblake@redhat.com>" [full]
# gpg: aka "Eric Blake (Free Software Programmer) <ebb9@byu.net>" [full]
# gpg: aka "[jpeg image of size 6874]" [full]
# Primary key fingerprint: 71C2 CC22 B1C4 6029 27D2 F3AA A7A1 6B4A 2527 436A

* remotes/ericb/tags/pull-nbd-2019-04-01:
nbd/client: Trace server noncompliance on structured reads
nbd/server: Advertise actual minimum block size
block: Add bdrv_get_request_alignment()
nbd/client: Support qemu-img convert from unaligned size
nbd/client: Reject inaccessible tail of inconsistent server
nbd/client: Report offsets in bdrv_block_status
nbd/client: Lower min_block for block-status, unaligned size
iotests: Add 241 to test NBD on unaligned images
nbd-client: Work around server BLOCK_STATUS misalignment at EOF
qemu-img: Gracefully shutdown when map can't finish
nbd: Permit simple error to NBD_CMD_BLOCK_STATUS
nbd: Don't lose server's error to NBD_CMD_BLOCK_STATUS
nbd: Tolerate some server non-compliance in NBD_CMD_BLOCK_STATUS
qemu-img: Report bdrv_block_status failures

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

nbd/server: Advertise actual minimum block size

Both NBD_CMD_BLOCK_STATUS and structured NBD_CMD_READ will split their
reply according to bdrv_block_status() boundaries. If the block device
has a re

nbd/server: Advertise actual minimum block size

Both NBD_CMD_BLOCK_STATUS and structured NBD_CMD_READ will split their
reply according to bdrv_block_status() boundaries. If the block device
has a request_alignment smaller than 512, but we advertise a block
alignment of 512 to the client, then this can result in the server
reply violating client expectations by reporting a smaller region of
the export than what the client is permitted to address (although this
is less of an issue for qemu 4.0 clients, given recent client patches
to overlook our non-compliance at EOF). Since it's always better to
be strict in what we send, it is worth advertising the actual minimum
block limit rather than blindly rounding it up to 512.

Note that this patch is not foolproof - it is still possible to
provoke non-compliant server behavior using:

$ qemu-nbd --image-opts driver=blkdebug,align=512,image.driver=file,image.filename=/path/to/non-aligned-file

That is arguably a bug in the blkdebug driver (it should never pass
back block status smaller than its alignment, even if it has to make
multiple bdrv_get_status calls and determine the
least-common-denominator status among the group to return). It may
also be possible to observe issues with a backing layer with smaller
alignment than the active layer, although so far I have been unable to
write a reliable iotest for that scenario (but again, an issue like
that could be argued to be a bug in the block layer, or something
where we need a flag to bdrv_block_status() to state whether the
result must be aligned to the current layer's limits or can be
subdivided for accuracy when chasing backing files).

Anyways, as blkdebug is not normally used, and as this patch makes our
server more interoperable with qemu 3.1 clients, it is worth applying
now, even while we still work on a larger patch series for the 4.1
timeframe to have byte-accurate file lengths.

Note that the iotests output changes - for 223 and 233, we can see the
server's better granularity advertisement; and for 241, the three test
cases have the following effects:
- natural alignment: the server's smaller alignment is now advertised,
and the hole reported at EOF is now the right result; we've gotten rid
of the server's non-compliance
- forced server alignment: the server still advertises 512 bytes, but
still sends a mid-sector hole. This is still a server compliance bug,
which needs to be fixed in the block layer in a later patch; output
does not change because the client is already being tolerant of the
non-compliance
- forced client alignment: the server's smaller alignment means that
the client now sees the server's status change mid-sector without any
protocol violations, but the fact that the map shows an unaligned
mid-sector hole is evidence of the block layer problems with aligned
block status, to be fixed in a later patch

Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <20190329042750.14704-7-eblake@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
[eblake: rebase to enhanced iotest 241 coverage]

show more ...

Merge remote-tracking branch 'remotes/ericb/tags/pull-nbd-2019-03-08' into staging

nbd patches for 2019-03-08

- support TLS client authorization in NBD servers
- iotest 223 race fix

# gpg: Signatu

Merge remote-tracking branch 'remotes/ericb/tags/pull-nbd-2019-03-08' into staging

nbd patches for 2019-03-08

- support TLS client authorization in NBD servers
- iotest 223 race fix

# gpg: Signature made Fri 08 Mar 2019 17:37:59 GMT
# gpg: using RSA key A7A16B4A2527436A
# gpg: Good signature from "Eric Blake <eblake@redhat.com>" [full]
# gpg: aka "Eric Blake (Free Software Programmer) <ebb9@byu.net>" [full]
# gpg: aka "[jpeg image of size 6874]" [full]
# Primary key fingerprint: 71C2 CC22 B1C4 6029 27D2 F3AA A7A1 6B4A 2527 436A

* remotes/ericb/tags/pull-nbd-2019-03-08:
iotests: Wait for qemu to end in 223
nbd: fix outdated qapi docs syntax for tls-creds
nbd: allow authorization with nbd-server-start QMP command
qemu-nbd: add support for authorization of TLS clients

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

qemu-nbd: add support for authorization of TLS clients

Currently any client which can complete the TLS handshake is able to use
the NBD server. The server admin can turn on the 'verify-peer' option

qemu-nbd: add support for authorization of TLS clients

Currently any client which can complete the TLS handshake is able to use
the NBD server. The server admin can turn on the 'verify-peer' option
for the x509 creds to require the client to provide a x509 certificate.
This means the client will have to acquire a certificate from the CA
before they are permitted to use the NBD server. This is still a fairly
low bar to cross.

This adds a '--tls-authz OBJECT-ID' option to the qemu-nbd command which
takes the ID of a previously added 'QAuthZ' object instance. This will
be used to validate the client's x509 distinguished name. Clients
failing the authorization check will not be permitted to use the NBD
server.

For example to setup authorization that only allows connection from a client
whose x509 certificate distinguished name is

CN=laptop.example.com,O=Example Org,L=London,ST=London,C=GB

escape the commas in the name and use:

qemu-nbd --object tls-creds-x509,id=tls0,dir=/home/berrange/qemutls,\
endpoint=server,verify-peer=yes \
--object 'authz-simple,id=auth0,identity=CN=laptop.example.com,,\
O=Example Org,,L=London,,ST=London,,C=GB' \
--tls-creds tls0 \
--tls-authz authz0 \
....other qemu-nbd args...

NB: a real shell command line would not have leading whitespace after
the line continuation, it is just included here for clarity.

Reviewed-by: Juan Quintela <quintela@redhat.com>
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
Message-Id: <20190227162035.18543-2-berrange@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
[eblake: split long line in --help text, tweak 233 to show that whitespace
after ,, in identity= portion is actually okay]
Signed-off-by: Eric Blake <eblake@redhat.com>

show more ...