Merge tag 'for-upstream' of https://repo.or.cz/qemu/kevin into staging

Block layer patches

- scsi-block: Fix error handling with r/werror=stop
- Depend on newer clang for TSA, make WITH_GRAPH_RDLOC

Merge tag 'for-upstream' of https://repo.or.cz/qemu/kevin into staging

Block layer patches

- scsi-block: Fix error handling with r/werror=stop
- Depend on newer clang for TSA, make WITH_GRAPH_RDLOCK_GUARD() fully
checked, fix block-copy to add missing lock
- vvfat: Fix write bugs for large files and add iotests
- Clean up blockdev-snapshot-internal-sync doc
- Fix iotests 024 for qed

# -----BEGIN PGP SIGNATURE-----
#
# iQJFBAABCAAvFiEE3D3rFZqa+V09dFb+fwmycsiPL9YFAmayag4RHGt3b2xmQHJl
# ZGhhdC5jb20ACgkQfwmycsiPL9Y0yhAArDpKYNsOmJerL/abIetchJ84suuR2MHZ
# iziAsTXk1iiSNYrAfXyiGhMsovvApluW1bojB80XLjaWFeN76zCRq0bnYVhv/xeX
# bQddC4JyWkcYGmdASiFpvQ7+p37jBh+OebmxsF557s4uM6b0/QN1xnOyyjBpyJbB
# aBTNgUYaTXmD6RD8h9SscnroNqhckuv6+zm0SX2Z4wRTF2uEmVWdL2yz2I3P8G7W
# dhVfgOCYQmW0cSfTueBQJClaUoHyJeibd4TzHR12hFAKIYobXMGfcE3AhfpBvO3t
# 0SEQ5MUx3zasGVENSJA6UnzVnpHl8HRtdDIFhSWb6yZJJ6RPPGynj7UVvFOK1SXM
# iXzj1kcYzFO/AFO3JxkSr6IHZdzZr4e5wtuFbw8Je6Ai0P5prc53jBDovtbAT0Wt
# +dAP7cnntYLDcAIsJqGUdr2FJfSOh9gApH/I3kF3scDwLRpb6OlWJ60T5b98VcR4
# +J67AXuGN7OXtYEU6GupZpWTQ/nZQ63egrCfJlqL67QduuF1YvcgOo2+TdAwDYkf
# 8nU7AEUgzWox8EcTkof/BXYYabOjn0D6/1+aLc7J7vGGlnKVyQMK9Kn5MRBzkyb3
# iWOtuv8aoNfnxtuMnpwe/Uf2hhOGi8IldnoP2+Yb9urWnFQ3Jbbmnv8Ga7mDQmRs
# ue4gDS51MCc=
# =ouBM
# -----END PGP SIGNATURE-----
# gpg: Signature made Wed 07 Aug 2024 04:23:10 AM AEST
# gpg: using RSA key DC3DEB159A9AF95D3D7456FE7F09B272C88F2FD6
# gpg: issuer "kwolf@redhat.com"
# gpg: Good signature from "Kevin Wolf <kwolf@redhat.com>" [full]

* tag 'for-upstream' of https://repo.or.cz/qemu/kevin:
iotests/024: exclude 'backing file format' field from the output
iotests: Add `vvfat` tests
vvfat: Fix reading files with non-continuous clusters
vvfat: Fix wrong checks for cluster mappings invariant
vvfat: Fix usage of `info.file.offset`
vvfat: Fix bug in writing to middle of file
scsi-disk: Always report RESERVATION_CONFLICT to guest
scsi-disk: Add warning comments that host_status errors take a shortcut
scsi-block: Don't skip callback for sgio error status/driver_status
scsi-disk: Use positive return value for status in dma_readv/writev
block/graph-lock: Make WITH_GRAPH_RDLOCK_GUARD() fully checked
block-copy: Fix missing graph lock
qapi-block-core: Clean up blockdev-snapshot-internal-sync doc

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>

show more ...

block/graph-lock: Make WITH_GRAPH_RDLOCK_GUARD() fully checked

Upstream clang 18 (and backports to clang 17 in Fedora and RHEL)
implemented support for __attribute__((cleanup())) in its Thread Safet

block/graph-lock: Make WITH_GRAPH_RDLOCK_GUARD() fully checked

Upstream clang 18 (and backports to clang 17 in Fedora and RHEL)
implemented support for __attribute__((cleanup())) in its Thread Safety
Analysis, so we can now actually have a proper implementation of
WITH_GRAPH_RDLOCK_GUARD() that understands when we acquire and when we
release the lock.

-Wthread-safety is now only enabled if the compiler is new enough to
understand this pattern. In theory, we could have used some #ifdefs to
keep the existing basic checks on old compilers, but as long as someone
runs a newer compiler (and our CI does), we will catch locking problems,
so it's probably not worth keeping multiple implementations for this.

The implementation can't use g_autoptr any more because the glib macros
define wrapper functions that don't have the right TSA attributes, so
the compiler would complain about them. Just use the cleanup attribute
directly instead.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Message-ID: <20240627181245.281403-3-kwolf@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Manos Pitsidianakis <manos.pitsidianakis@linaro.org>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>

show more ...

Merge tag 'for-upstream' of https://repo.or.cz/qemu/kevin into staging

Block layer patches

- virtio-blk: Multiqueue support (configurable iothread per queue)
- Made NBD export and hw/scsi thread-sa

Merge tag 'for-upstream' of https://repo.or.cz/qemu/kevin into staging

Block layer patches

- virtio-blk: Multiqueue support (configurable iothread per queue)
- Made NBD export and hw/scsi thread-safe without AioContext lock
- Fix crash when loading snapshot on inactive node

# -----BEGIN PGP SIGNATURE-----
#
# iQJFBAABCAAvFiEE3D3rFZqa+V09dFb+fwmycsiPL9YFAmWEw/8RHGt3b2xmQHJl
# ZGhhdC5jb20ACgkQfwmycsiPL9bX0Q/9G+Qx8mQGmbxJzvtW7/1eaeJ5CPCYT8w3
# 033S5hCil43mvX2aQKTFrh1Nz4aYlqMDyURvNu7nigyADY+kBpzzJ1MFr6WQrzYv
# QEk4jf/FOllfKn8+/A0z2NJDhtpVgqKKHBsFZl8FBUcxd79daTaoPPM3BNNsOHQD
# o7Z7hR/iEdG9dkAh/fpwctsgMO/CoN0BRRyN2OByj03zeu1TlDJ6lX0hxlcJl9Jw
# vLo81rWTCqKRu+SbjBsb0HfYE2hP54A4hvxn4I9vYGYDz8ElucluYyeqUEK+mdrX
# /DQBdb+Osl1FD6MuIaFR+Rgp9Mu5h6ZOdvUyCY0zuByti851hV8qjW9BtrTfqaMh
# LMOKoL6c5B8XJYWVGAGrJexIw1hHq5WKdXN9zp4FZA4tOyHUMRjHuR1+zScU6gnU
# WRSIQR46w75A13clWyJs9Hf/q5Fp/1KT4nfuZ/hmiXvxdsYY5x1w/W3s9tRNjYKL
# d6FVk17cFc6Ksb7lWvDCgg61BNZtGm4Clmw0kJ6V1reiQz7AvDLmduLUQbmrVt7G
# gWAY4b2L9YXJpEx5en0kE50KLAUw/E9ozbOq6ZT9nFUKeNAPC8PS5lK7vYVwebCk
# VA0t8pFzKhdB1bJaG5fMSRPBuqkvhsaDEEDABlSro8dyyjoQBaEdk5P9Kxe66hBc
# xhTmDPdv/JM=
# =E3Zh
# -----END PGP SIGNATURE-----
# gpg: Signature made Thu 21 Dec 2023 18:02:23 EST
# gpg: using RSA key DC3DEB159A9AF95D3D7456FE7F09B272C88F2FD6
# gpg: issuer "kwolf@redhat.com"
# gpg: Good signature from "Kevin Wolf <kwolf@redhat.com>" [full]
# Primary key fingerprint: DC3D EB15 9A9A F95D 3D74 56FE 7F09 B272 C88F 2FD6

* tag 'for-upstream' of https://repo.or.cz/qemu/kevin: (33 commits)
virtio-blk: add iothread-vq-mapping parameter
qdev: add IOThreadVirtQueueMappingList property type
qdev-properties: alias all object class properties
string-output-visitor: show structs as "<omitted>"
block-coroutine-wrapper: use qemu_get_current_aio_context()
block: remove outdated AioContext locking comments
job: remove outdated AioContext locking comments
scsi: remove outdated AioContext lock comment
docs: remove AioContext lock from IOThread docs
aio: remove aio_context_acquire()/aio_context_release() API
aio-wait: draw equivalence between AIO_WAIT_WHILE() and AIO_WAIT_WHILE_UNLOCKED()
scsi: remove AioContext locking
block: remove bdrv_co_lock()
block: remove AioContext locking
graph-lock: remove AioContext locking
aio: make aio_context_acquire()/aio_context_release() a no-op
tests: remove aio_context_acquire() tests
scsi: assert that callbacks run in the correct AioContext
virtio-scsi: replace AioContext lock with tmf_bh_lock
dma-helpers: don't lock AioContext in dma_blk_cb()
...

Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>

show more ...

graph-lock: remove AioContext locking

Stop acquiring/releasing the AioContext lock in
bdrv_graph_wrlock()/bdrv_graph_unlock() since the lock no longer has any
effect.

The distinction between bdrv_g

graph-lock: remove AioContext locking

Stop acquiring/releasing the AioContext lock in
bdrv_graph_wrlock()/bdrv_graph_unlock() since the lock no longer has any
effect.

The distinction between bdrv_graph_wrunlock() and
bdrv_graph_wrunlock_ctx() becomes meaningless and they can be collapsed
into one function.

Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Message-ID: <20231205182011.1976568-6-stefanha@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>

show more ...

Merge tag 'for-upstream' of https://repo.or.cz/qemu/kevin into staging

Block layer patches

- Fix graph lock related deadlocks with the stream job
- ahci: Fix legacy software reset
- ide/via: Fix sw

Merge tag 'for-upstream' of https://repo.or.cz/qemu/kevin into staging

Block layer patches

- Fix graph lock related deadlocks with the stream job
- ahci: Fix legacy software reset
- ide/via: Fix switch between compatibility and native mode

# -----BEGIN PGP SIGNATURE-----
#
# iQJFBAABCAAvFiEE3D3rFZqa+V09dFb+fwmycsiPL9YFAmVcmYoRHGt3b2xmQHJl
# ZGhhdC5jb20ACgkQfwmycsiPL9YDzw/7BD6wZpyCsDbFu9Jbt0L894tYQls7otnR
# yeAIaZVqSkDcMK8VBD/xAjV8UgX194oKPi42CDgS73avd0cSHLIM5cNgGkwCrMWS
# ry5uuOP6EWVMPPR/129cpH8uGvkl+qwCQf5gB13/8NvMbeN2mHOTC6WW+VA20vb0
# V0DJXhYszVzXa3L1a/m6f4Jwj54tTeZ56JcBblL3wi/soklb45gsnPJaHeGb3rzK
# yjPkw+kpVXTVbpacobGmzmjlD3Yqk69NexP2kyU1w2lqPnemYPH+9sa+7RxMspkj
# InQvqq6TFtMOrC/65/527p2ENRUOxn7Xwsa1+Hnar2i3BoyGugWE8GPxJDBxAWW4
# INJtpxIpiA7Scd26VBCNVstVe5EuyxkP97T85cgNUMgeE58y3i51i6eHd4GUIR7v
# PNc5TsSbnVV8sQ7RsXka4hRyjndIPRB0CBePydDoBz6zaGmcVU6ep0Oppah9gVu9
# CU0dBz2jV0r1dFhU1eZkCbd1ufdR93R/iD3gBD4vj1xSL3l+9OE/FKdrVE66uElL
# iAsHp3cimkPuWAx/jZaeAC7BDI0XS6s1TimddqJx90f2mZjkq8cmVp+HoVNP0jRQ
# VP6AIQy6is+P4QtDSekgXVJE8K95ngBzsr+ittR8jF4q67QzHVjLmJ9ZBXyrowlz
# gtZTy2WPxbM=
# =8dXj
# -----END PGP SIGNATURE-----
# gpg: Signature made Tue 21 Nov 2023 06:50:34 EST
# gpg: using RSA key DC3DEB159A9AF95D3D7456FE7F09B272C88F2FD6
# gpg: issuer "kwolf@redhat.com"
# gpg: Good signature from "Kevin Wolf <kwolf@redhat.com>" [full]
# Primary key fingerprint: DC3D EB15 9A9A F95D 3D74 56FE 7F09 B272 C88F 2FD6

* tag 'for-upstream' of https://repo.or.cz/qemu/kevin:
hw/ide/via: implement legacy/native mode switching
ide/via: don't attempt to set default BAR addresses
ide/pci: introduce pci_ide_update_mode() function
ide/ioport: move ide_portio_list[] and ide_portio_list2[] definitions to IDE core
iotests: Test two stream jobs in a single iothread
stream: Fix AioContext locking during bdrv_graph_wrlock()
block: Fix deadlocks in bdrv_graph_wrunlock()
block: Fix bdrv_graph_wrlock() call in blk_remove_bs()
hw/ide/ahci: fix legacy software reset

Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>

show more ...

block: Fix deadlocks in bdrv_graph_wrunlock()

bdrv_graph_wrunlock() calls aio_poll(), which may run callbacks that
have a nested event loop. Nested event loops can depend on other
iothreads making p

block: Fix deadlocks in bdrv_graph_wrunlock()

bdrv_graph_wrunlock() calls aio_poll(), which may run callbacks that
have a nested event loop. Nested event loops can depend on other
iothreads making progress, so in order to allow them to make progress it
must not hold the AioContext lock of another thread while calling
aio_poll().

This introduces a @bs parameter to bdrv_graph_wrunlock() whose
AioContext is temporarily dropped (which matches bdrv_graph_wrlock()),
and a bdrv_graph_wrunlock_ctx() that can be used if the BlockDriverState
doesn't necessarily exist any more when unlocking.

This also requires a change to bdrv_schedule_unref(), which was relying
on the incorrectly taken lock. It needs to take the lock itself now.
While this is a separate bug, it can't be fixed a separate patch because
otherwise the intermediate state would either deadlock or try to release
a lock that we don't even hold.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Message-ID: <20231115172012.112727-3-kwolf@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
[kwolf: Fixed up bdrv_schedule_unref()]
Signed-off-by: Kevin Wolf <kwolf@redhat.com>

show more ...

Merge tag 'for-upstream' of https://repo.or.cz/qemu/kevin into staging

Block layer patches

- Clean up coroutine versions of bdrv_{is_allocated,block_status}*
- Graph locking part 5 (protect childre

Merge tag 'for-upstream' of https://repo.or.cz/qemu/kevin into staging

Block layer patches

- Clean up coroutine versions of bdrv_{is_allocated,block_status}*
- Graph locking part 5 (protect children/parent links)

# -----BEGIN PGP SIGNATURE-----
#
# iQJFBAABCAAvFiEE3D3rFZqa+V09dFb+fwmycsiPL9YFAmUoHL8RHGt3b2xmQHJl
# ZGhhdC5jb20ACgkQfwmycsiPL9b4uRAAjryVAaA5jXZ3mdGB80nhGtARZlIaIVO/
# tlXk065q2Cj+98f+fBPCPWvmEz28vJwBhJUsFwpHzLZrxecBpwZp0MPAkFBNkouq
# +AiO9xyTAqccEp/dnIys4Bun9Rp0Jq9lk9y29zzEmQuK5uCB56lpx2cDn/JkzSQt
# ZFtnxxTwi3MDTNvXATub8Ia/1suui0zvESS7J/NBxQNI3cFaQszp1vMwlRIoPiWo
# 15YZFPZZQ2pvu6/1nL1Vl9OLbPAVcEGJpjHZv0XhudYOwRiDvjYnwfPL7BuwYEsU
# Dos4mZZd/KMU695s7OzlVYi1q4ATKUTUxyyylVhXZrFBXSE5ntnfoHTKHEruTyPb
# G31h5mribSTWjdvY5HewHbSSPjByAWsSQg9yzcHybhORiqGQCpcGQ8zuW7oNKMPV
# JicWdoRVY4U4hR0nRdDxz9zdpQ8QYok/ginBxFaOzrCfClUB7ZOBxwRMclIghuRH
# FV+ZJk0ylVOz2tbfNxUa3KhUgTPd8jgCHFI7xak5EBRtTJiJjE03Xag1Fdxy5/D5
# tRsBBW4sOFygAhjN/xyeaRv9L8rAv3x/akriFjPUbOMLkPcJpe/DTWsP8+5LaZF8
# GkQvjsg5UvmfcJ3LFtecXxfYH4UWhDmyAjF+BswiRqafDDi2CCUmdwDnzEPbwuWO
# x1y7cgxe9SE=
# =4d/s
# -----END PGP SIGNATURE-----
# gpg: Signature made Thu 12 Oct 2023 12:20:15 EDT
# gpg: using RSA key DC3DEB159A9AF95D3D7456FE7F09B272C88F2FD6
# gpg: issuer "kwolf@redhat.com"
# gpg: Good signature from "Kevin Wolf <kwolf@redhat.com>" [full]
# Primary key fingerprint: DC3D EB15 9A9A F95D 3D74 56FE 7F09 B272 C88F 2FD6

* tag 'for-upstream' of https://repo.or.cz/qemu/kevin: (26 commits)
block: Add assertion for bdrv_graph_wrlock()
block: Protect bs->children with graph_lock
block: Protect bs->parents with graph_lock
block: Mark bdrv_get_specific_info() and callers GRAPH_RDLOCK
block: Mark bdrv_apply_auto_read_only() and callers GRAPH_RDLOCK
block: Mark bdrv_op_is_blocked() and callers GRAPH_RDLOCK
qcow2: Mark check_constraints_on_bitmap() GRAPH_RDLOCK
qcow2: Mark qcow2_inactivate() and callers GRAPH_RDLOCK
qcow2: Mark qcow2_signal_corruption() and callers GRAPH_RDLOCK
block: Mark bdrv_amend_options() and callers GRAPH_RDLOCK
block: Mark bdrv_get_parent_name() and callers GRAPH_RDLOCK
block: Mark bdrv_primary_child() and callers GRAPH_RDLOCK
block: Mark bdrv_refresh_filename() and callers GRAPH_RDLOCK
block: Mark bdrv_get_xdbg_block_graph() and callers GRAPH_RDLOCK
block: Take graph rdlock in parts of reopen
block: Mark bdrv_snapshot_fallback() and callers GRAPH_RDLOCK
block: Mark bdrv_parent_cb_resize() and callers GRAPH_RDLOCK
block: Mark drain related functions GRAPH_RDLOCK
block: Mark bdrv_first_blk() and bdrv_is_root_node() GRAPH_RDLOCK
block: Take graph rdlock in bdrv_inactivate_all()
...

Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>

show more ...

block: Add assertion for bdrv_graph_wrlock()

bdrv_graph_wrlock() can't run in a coroutine (because it polls) and
requires holding the BQL. We already have GLOBAL_STATE_CODE() to assert
the latter. A

block: Add assertion for bdrv_graph_wrlock()

bdrv_graph_wrlock() can't run in a coroutine (because it polls) and
requires holding the BQL. We already have GLOBAL_STATE_CODE() to assert
the latter. Assert the former as well and add a no_coroutine_fn marker.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Message-ID: <20230929145157.45443-23-kwolf@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>

show more ...

Merge tag 'for-upstream' of https://repo.or.cz/qemu/kevin into staging

Block layer patches

- Re-enable the graph lock
- More fixes to coroutine_fn marking

# -----BEGIN PGP SIGNATURE-----
#
# iQJFB

Merge tag 'for-upstream' of https://repo.or.cz/qemu/kevin into staging

Block layer patches

- Re-enable the graph lock
- More fixes to coroutine_fn marking

# -----BEGIN PGP SIGNATURE-----
#
# iQJFBAABCAAvFiEE3D3rFZqa+V09dFb+fwmycsiPL9YFAmScQCQRHGt3b2xmQHJl
# ZGhhdC5jb20ACgkQfwmycsiPL9bNSA//WIzPT45rFhl2U9QgyOJu26ho6ahsgwgI
# Z3QM5kCDB1dAN9USRPxhGboLGo8CyY7eeSwSrR7RtwBGYrWrAoJfGp5gK/7d9s5Q
# o0AGgRPnJGhFkBhRRMytsDsewM6Kk4IRmk4HMK3cOH3rsSM8RHs6KmDSBKesllu0
# QVGf3qW4u8LHyZyGM5OlPVUbtuDuK6/52FGhpXBp+x4oyNegOhjwO4mGOvTG+xIk
# Q5zwWZaPfjxaEDkvW8iahB6/D7Tpt64BmMf1Ydhxcd5eKEp932CiBI36aAlNKoRD
# Al5wztRx1GEh12ekN39jIi7Ypp3JX26keJcieKU0q656pT551UFRYjU0Rk08/Cca
# qv2oiQDu6bHgQ9zCQ1nMfa9+K2MyBwx0b5qfYkvs2RzgCTl8ImgBQANHfw8tz6Bq
# HUo1zsFBXCaK0boUB5iFwdf3rlx3t9UTEuDej/RaHqZjZD5xeG/smCcOlSfHaKUa
# wXfYxvm8ZfefJn1D6io1A+7M956uvIQNtmh13cU44clgFX9Y/bBNMg/5lMRsJKo8
# xxjvqCAyxo/pPfUsVWx4pc8AXbfVa85gyoSiaLEYZnqP54sJ2lFccqykCsTy58Lo
# VDcoPnoSc+LNqBOvtzxXgQbEWFCXU6fe0+TZgVYUvExWFIAOImeDWg2GD1JVrwsX
# e9QrPhL3DXg=
# =ZQcP
# -----END PGP SIGNATURE-----
# gpg: Signature made Wed 28 Jun 2023 04:13:56 PM CEST
# gpg: using RSA key DC3DEB159A9AF95D3D7456FE7F09B272C88F2FD6
# gpg: issuer "kwolf@redhat.com"
# gpg: Good signature from "Kevin Wolf <kwolf@redhat.com>" [full]

* tag 'for-upstream' of https://repo.or.cz/qemu/kevin: (23 commits)
block: use bdrv_co_debug_event in coroutine context
block: use bdrv_co_getlength in coroutine context
qcow2: mark more functions as coroutine_fns and GRAPH_RDLOCK
vhdx: mark more functions as coroutine_fns and GRAPH_RDLOCK
vmdk: mark more functions as coroutine_fns and GRAPH_RDLOCK
dmg: mark more functions as coroutine_fns and GRAPH_RDLOCK
cloop: mark more functions as coroutine_fns and GRAPH_RDLOCK
block: mark another function as coroutine_fns and GRAPH_UNLOCKED
bochs: mark more functions as coroutine_fns and GRAPH_RDLOCK
vpc: mark more functions as coroutine_fns and GRAPH_RDLOCK
qed: mark more functions as coroutine_fns and GRAPH_RDLOCK
file-posix: remove incorrect coroutine_fn calls
Revert "graph-lock: Disable locking for now"
graph-lock: Unlock the AioContext while polling
blockjob: Fix AioContext locking in block_job_add_bdrv()
block: Fix AioContext locking in bdrv_open_backing_file()
block: Fix AioContext locking in bdrv_open_inherit()
block: Fix AioContext locking in bdrv_reopen_parse_file_or_backing()
block: Fix AioContext locking in bdrv_attach_child_common()
block: Fix AioContext locking in bdrv_open_child()
...

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>

show more ...

graph-lock: Unlock the AioContext while polling

If the caller keeps the AioContext lock for a block node in an iothread,
polling in bdrv_graph_wrlock() deadlocks if the condition isn't
fulfilled imm

graph-lock: Unlock the AioContext while polling

If the caller keeps the AioContext lock for a block node in an iothread,
polling in bdrv_graph_wrlock() deadlocks if the condition isn't
fulfilled immediately.

Now that all callers make sure to actually have the AioContext locked
when they call bdrv_replace_child_noperm() like they should, we can
change bdrv_graph_wrlock() to take a BlockDriverState whose AioContext
lock the caller holds (NULL if it doesn't) and unlock it temporarily
while polling.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Message-ID: <20230605085711.21261-11-kwolf@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>

show more ...

Merge tag 'for-upstream' of https://repo.or.cz/qemu/kevin into staging

Block layer patches

- Graph locking, part 3 (more block drivers)
- Compile out assert_bdrv_graph_readable() by default
- Add c

Merge tag 'for-upstream' of https://repo.or.cz/qemu/kevin into staging

Block layer patches

- Graph locking, part 3 (more block drivers)
- Compile out assert_bdrv_graph_readable() by default
- Add configure options for vmdk, vhdx and vpc
- Fix use after free in blockdev_mark_auto_del()
- migration: Attempt disk reactivation in more failure scenarios
- Coroutine correctness fixes

# -----BEGIN PGP SIGNATURE-----
#
# iQJFBAABCAAvFiEE3D3rFZqa+V09dFb+fwmycsiPL9YFAmRbi6ERHGt3b2xmQHJl
# ZGhhdC5jb20ACgkQfwmycsiPL9Y66A//ZRk/0M6EZUJPAKG6m/XLTDNrOCNBZ1Tu
# kBGvxXsVQZMt4gGpBad4l2INN6IQKTIdIf+lK71EpxMPmFG6xK32btn38yywCAfQ
# lr1p5nR0Y/zSlT+XzP4yKy/CtQl6U0rkysmjCIk35bZc7uLy6eo4oFR4vmhRRt2M
# UGltB50/Nicx12YFufVjodbhv+apxTGwS2XHatmwqtjKeYReSz8mJHslEy6DvC8m
# ziNThD6YBy7hMktAhNaqUqtZD0OSWz66VMObco/4i2++sOAMZIspXQkjv3AjH74e
# lmgMhNc/xgJKPwFBPsj6F7dOKxwhdKD9jzZlx3yaBtAU18hpWX54QWuA3/CFlySc
# 5QbbqIstFTC8lqoRWThQrcHHRKbDBJCP4ImRXUIKhuPaxEzXA9zb3+f3QPTIjLSA
# KO7nxuSmO+tC7hQ1K9kAjRZHWlxxAk4clk+7UrK4UrWgGxfCUKgFg4Tyx7RrpwA6
# j4L5vwAY60LW74tikWe9xJx2QbdRoWBTTZhUyirbO7rLX1e8mS1nUWmtIsFSQxAq
# Z7nX7ygN0WEF+8qIsk3jTGaEeJoCM7+7B+X2RpSy0sftFjFYmybIiUgLMO7e+ozK
# rvUPnwlHAbGCVIJOKrUDj3cGt6k3/xnrTajUc7pCB3KKqG4pe+IlZuHyKIUMActb
# dBLaBnj0M2o=
# =hw9E
# -----END PGP SIGNATURE-----
# gpg: Signature made Wed 10 May 2023 01:18:41 PM BST
# gpg: using RSA key DC3DEB159A9AF95D3D7456FE7F09B272C88F2FD6
# gpg: issuer "kwolf@redhat.com"
# gpg: Good signature from "Kevin Wolf <kwolf@redhat.com>" [full]

* tag 'for-upstream' of https://repo.or.cz/qemu/kevin: (28 commits)
block: compile out assert_bdrv_graph_readable() by default
block: Mark bdrv_refresh_limits() and callers GRAPH_RDLOCK
block: Mark bdrv_recurse_can_replace() and callers GRAPH_RDLOCK
block: Mark bdrv_query_block_graph_info() and callers GRAPH_RDLOCK
block: Mark bdrv_query_bds_stats() and callers GRAPH_RDLOCK
block: Mark BlockDriver callbacks for amend job GRAPH_RDLOCK
block: Mark bdrv_co_debug_event() GRAPH_RDLOCK
block: Mark bdrv_co_get_info() and callers GRAPH_RDLOCK
block: Mark bdrv_co_get_allocated_file_size() and callers GRAPH_RDLOCK
mirror: Require GRAPH_RDLOCK for accessing a node's parent list
vhdx: Require GRAPH_RDLOCK for accessing a node's parent list
nbd: Mark nbd_co_do_establish_connection() and callers GRAPH_RDLOCK
nbd: Remove nbd_co_flush() wrapper function
block: .bdrv_open is non-coroutine and unlocked
graph-lock: Fix GRAPH_RDLOCK_GUARD*() to be reader lock
graph-lock: Add GRAPH_UNLOCKED(_PTR)
test-bdrv-drain: Don't modify the graph in coroutines
iotests: Test resizing image attached to an iothread
block: Don't call no_coroutine_fns in qmp_block_resize()
block: bdrv/blk_co_unref() for calls in coroutine context
...

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>

show more ...

graph-lock: Fix GRAPH_RDLOCK_GUARD*() to be reader lock

GRAPH_RDLOCK_GUARD() and GRAPH_RDLOCK_GUARD_MAINLOOP() only take a
reader lock for the graph, so the correct annotation for them to use is
TSA

graph-lock: Fix GRAPH_RDLOCK_GUARD*() to be reader lock

GRAPH_RDLOCK_GUARD() and GRAPH_RDLOCK_GUARD_MAINLOOP() only take a
reader lock for the graph, so the correct annotation for them to use is
TSA_ASSERT_SHARED rather than TSA_ASSERT.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20230504115750.54437-8-kwolf@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>

show more ...

graph-lock: Add GRAPH_UNLOCKED(_PTR)

For some functions, it is part of their interface to be called without
holding the graph lock. Add a new macro to document this.

The macro expands to TSA_EXCLUD

graph-lock: Add GRAPH_UNLOCKED(_PTR)

For some functions, it is part of their interface to be called without
holding the graph lock. Add a new macro to document this.

The macro expands to TSA_EXCLUDES(), which is a relatively weak check
because it passes in cases where the compiler just doesn't know if the
lock is held. Function pointers can't be checked at all. Therefore, its
primary purpose is documentation.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-Id: <20230504115750.54437-7-kwolf@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>

show more ...

block: add missing coroutine_fn annotations

After the recent introduction of many new coroutine callbacks,
a couple calls from non-coroutine_fn to coroutine_fn have sneaked
in; fix them.

Signed-off

block: add missing coroutine_fn annotations

After the recent introduction of many new coroutine callbacks,
a couple calls from non-coroutine_fn to coroutine_fn have sneaked
in; fix them.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20230406101752.242125-1-pbonzini@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>

show more ...

Merge tag 'pull-include-2023-02-06-v2' of https://repo.or.cz/qemu/armbru into staging

Header cleanup patches for 2023-02-06

# -----BEGIN PGP SIGNATURE-----
#
# iQJGBAABCAAwFiEENUvIs9frKmtoZ05fOHC0A

Merge tag 'pull-include-2023-02-06-v2' of https://repo.or.cz/qemu/armbru into staging

Header cleanup patches for 2023-02-06

# -----BEGIN PGP SIGNATURE-----
#
# iQJGBAABCAAwFiEENUvIs9frKmtoZ05fOHC0AOuRhlMFAmPjQRUSHGFybWJydUBy
# ZWRoYXQuY29tAAoJEDhwtADrkYZTWHwQAI5D2HTRt3peFSPg/tFuYqSGtOobzM5x
# xdZxjWCtGV+dZ4TsyQ7yJkQ2i6aPjs0LzmTnTIsmf+p1OJSthvp4fQUzfXQauiJy
# OnNA76v7WjiXB8u8tcXtEkkHNmccp8n4KMjk33TfK/HQVx7lZ2EFurlCkvBr5wki
# FuDVad6R43ChmvBWdCUOi6G1IklAihm8AN4lBJu3iC7U8bjW4FmLLMitcu5OyKgt
# v9V4XFAe4eYUIPZ6uH5Lpr5m/qtrRXLe9KOdRUR2vDVL18Cf6Zl7mrUNtv7iV5TI
# hBOA12ZP5XXf81FXl7e8y3Xi2KXvb/el0wQ7SvtE7XB3Pdbfa5WSGKOc3VxYLmBA
# 4xUnEbzAogrrruovdk1bmh2LnVzCH66m72xaLCHBvjOU3M6V2B0eeaZ35FxNuFcB
# toHPAjeuzWkDAscVYcYKuPwnkNOMNqHxEdihrgy6mYLr6nauIYr7Lqgs3SqGqpct
# /HGy683+J+AqnHFTk1MTRftDxqk/Nku6ntAxLXndkpm3uDvu+iV5d3BfK3A9t7d1
# A2Y983DU6SiVwpMIv2eDL8sXYxuwIs56ZmPYIcSbqzpCXtdFqwWOTeFET/4vD+8t
# V3YKJ27jmWQ9bxbLHGPPYSKheuCVBIGsqxouE/Pbj5nXRKm/TeGp+20a4dWdE08r
# 2WTLAQbVQGD4
# =5TPW
# -----END PGP SIGNATURE-----
# gpg: Signature made Wed 08 Feb 2023 06:28:37 GMT
# gpg: using RSA key 354BC8B3D7EB2A6B68674E5F3870B400EB918653
# gpg: issuer "armbru@redhat.com"
# gpg: Good signature from "Markus Armbruster <armbru@redhat.com>" [full]
# gpg: aka "Markus Armbruster <armbru@pond.sub.org>" [full]
# Primary key fingerprint: 354B C8B3 D7EB 2A6B 6867 4E5F 3870 B400 EB91 8653

* tag 'pull-include-2023-02-06-v2' of https://repo.or.cz/qemu/armbru:
Drop duplicate #include
Don't include headers already included by qemu/osdep.h
Fix non-first inclusions of qemu/osdep.h
accel: Clean up includes
block: Clean up includes
riscv: Clean up includes
target/hexagon: Clean up includes
net: Clean up includes
migration: Clean up includes
qga: Clean up includes
hw/tricore: Clean up includes
hw/input: Clean up includes
hw/cxl: Clean up includes
crypto: Clean up includes
bsd-user: Clean up includes
scripts/clean-includes: Improve --git commit message
scripts/clean-includes: Skip symbolic links
scripts/clean-includes: Don't claim duplicate headers found when not
scripts/clean-includes: Fully skip / ignore files

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

block: Clean up includes

This commit was created with scripts/clean-includes.

All .c should include qemu/osdep.h first. The script performs three
related cleanups:

* Ensure .c files include qemu/

block: Clean up includes

This commit was created with scripts/clean-includes.

All .c should include qemu/osdep.h first. The script performs three
related cleanups:

* Ensure .c files include qemu/osdep.h first.
* Including it in a .h is redundant, since the .c already includes
it. Drop such inclusions.
* Likewise, including headers qemu/osdep.h includes is redundant.
Drop these, too.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20230202133830.2152150-16-armbru@redhat.com>

show more ...

Merge tag 'pull-include-2023-01-20' of https://repo.or.cz/qemu/armbru into staging

Header cleanup patches for 2023-01-20

# -----BEGIN PGP SIGNATURE-----
#
# iQJGBAABCAAwFiEENUvIs9frKmtoZ05fOHC0AOuR

Merge tag 'pull-include-2023-01-20' of https://repo.or.cz/qemu/armbru into staging

Header cleanup patches for 2023-01-20

# -----BEGIN PGP SIGNATURE-----
#
# iQJGBAABCAAwFiEENUvIs9frKmtoZ05fOHC0AOuRhlMFAmPKN6YSHGFybWJydUBy
# ZWRoYXQuY29tAAoJEDhwtADrkYZTPeoQAIKl/BF6PFRNq0/k3vPqMe6nltjgkpa/
# p7E5qRlo31RCeUB+f0iW26mySnNTgYkE28yy57HxUML/9Lp1bbxyDgRNiJ406a4L
# kFVF04kOIFez1+mfvWN92DZqcl/EAAqNL6XqSFyO38kYwcsFsi+BZ7DLZbL9Ea8v
# wVywB96mN6KyrLWCJ2D0OqIVuPHSHol+5zt9e6+ShBgN0FfElLbv0F4KH3VJ1olA
# psKl6w6V9+c2zV1kT/H+S763m6mQdwtVo/UuOJoElI+Qib/UBxDOrhdYf4Zg7hKf
# ByUuhJUASm8y9yD/42mFs90B6eUNzLSBC8v1PgRqSqDHtllveP4RysklBlyIMlOs
# DKtqEuRuIJ/qDXliIFHY6tBnUkeITSd7BCxkQYfaGyaSOcviDSlE3AyaaBC0sY4F
# P/lTTiRg5ksvhDYtJnW3mSfmT2PY7aBtyE3D1Z84v9hek6D0reMQTE97yL/j4m7P
# wJP8aM3Z8GILCVxFIh02wmqWZhZUCGsIDS/vxVm+u060n66qtDIQFBoazsFJrCME
# eWI+qDNDr6xhLegeYajGDM9pdpQc3x0siiuHso4wMSI9NZxwP+tkCVhTpqmrRcs4
# GSH/4IlUXqEZdUQDL38DfA22C1TV8BzyMhGLTUERWWYki1sr99yv0pdFyk5r3nLB
# SURwr58rB2zo
# =dOfq
# -----END PGP SIGNATURE-----
# gpg: Signature made Fri 20 Jan 2023 06:41:42 GMT
# gpg: using RSA key 354BC8B3D7EB2A6B68674E5F3870B400EB918653
# gpg: issuer "armbru@redhat.com"
# gpg: Good signature from "Markus Armbruster <armbru@redhat.com>" [full]
# gpg: aka "Markus Armbruster <armbru@pond.sub.org>" [full]
# Primary key fingerprint: 354B C8B3 D7EB 2A6B 6867 4E5F 3870 B400 EB91 8653

* tag 'pull-include-2023-01-20' of https://repo.or.cz/qemu/armbru:
include/hw/ppc include/hw/pci-host: Drop extra typedefs
include/hw/ppc: Don't include hw/pci-host/pnv_phb.h from pnv.h
include/hw/ppc: Supply a few missing includes
include/hw/ppc: Split pnv_chip.h off pnv.h
include/hw/block: Include hw/block/block.h where needed
hw/sparc64/niagara: Use blk_name() instead of open-coding it
include/block: Untangle inclusion loops
coroutine: Use Coroutine typedef name instead of structure tag
coroutine: Split qemu/coroutine-core.h off qemu/coroutine.h
coroutine: Clean up superfluous inclusion of qemu/lockable.h
coroutine: Move coroutine_fn to qemu/osdep.h, trim includes
coroutine: Clean up superfluous inclusion of qemu/coroutine.h

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

coroutine: Move coroutine_fn to qemu/osdep.h, trim includes

block/block-hmp-cmds.h and qemu/co-shared-resource.h use coroutine_fn
without including qemu/coroutine.h. They compile only if it's alrea

coroutine: Move coroutine_fn to qemu/osdep.h, trim includes

block/block-hmp-cmds.h and qemu/co-shared-resource.h use coroutine_fn
without including qemu/coroutine.h. They compile only if it's already
included from elsewhere.

I could fix that, but pulling in qemu/coroutine.h and everything it
includes just for a macro that expands into nothing feels silly.
Instead, move the macro to qemu/osdep.h.

Inclusions of qemu/coroutine.h just for coroutine_fn become
superfluous. Drop them.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-Id: <20221221131435.3851212-3-armbru@redhat.com>

show more ...

Merge tag 'for-upstream' of https://repo.or.cz/qemu/kevin into staging

Block layer patches

- Code cleanups around block graph modification
- Simplify drain
- coroutine_fn correctness fixes, includi

Merge tag 'for-upstream' of https://repo.or.cz/qemu/kevin into staging

Block layer patches

- Code cleanups around block graph modification
- Simplify drain
- coroutine_fn correctness fixes, including splitting generated
coroutine wrappers into co_wrapper (to be called only from
non-coroutine context) and co_wrapper_mixed (both coroutine and
non-coroutine context)
- Introduce a block graph rwlock

# gpg: Signature made Thu 15 Dec 2022 15:08:34 GMT
# gpg: using RSA key DC3DEB159A9AF95D3D7456FE7F09B272C88F2FD6
# gpg: issuer "kwolf@redhat.com"
# gpg: Good signature from "Kevin Wolf <kwolf@redhat.com>" [full]
# Primary key fingerprint: DC3D EB15 9A9A F95D 3D74 56FE 7F09 B272 C88F 2FD6

* tag 'for-upstream' of https://repo.or.cz/qemu/kevin: (50 commits)
block: GRAPH_RDLOCK for functions only called by co_wrappers
block: use co_wrapper_mixed_bdrv_rdlock in functions taking the rdlock
block-coroutine-wrapper.py: introduce annotations that take the graph rdlock
Mark assert_bdrv_graph_readable/writable() GRAPH_RD/WRLOCK
graph-lock: TSA annotations for lock/unlock functions
block: assert that graph read and writes are performed correctly
block: remove unnecessary assert_bdrv_graph_writable()
block: wrlock in bdrv_replace_child_noperm
block: Fix locking in external_snapshot_prepare()
test-bdrv-drain: Fix incorrrect drain assumptions
clang-tsa: Add macros for shared locks
clang-tsa: Add TSA_ASSERT() macro
Import clang-tsa.h
async: Register/unregister aiocontext in graph lock list
graph-lock: Implement guard macros
graph-lock: Introduce a lock to protect block graph operations
block: Factor out bdrv_drain_all_begin_nopoll()
block/dirty-bitmap: convert coroutine-only functions to co_wrapper
block: convert bdrv_create to co_wrapper
block-coroutine-wrapper.py: support also basic return types
...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

Mark assert_bdrv_graph_readable/writable() GRAPH_RD/WRLOCK

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Message-Id: <20221207131838.239125-16-kwolf@redhat.com>
Reviewed-by: Emanuele Giuseppe Esposit

Mark assert_bdrv_graph_readable/writable() GRAPH_RD/WRLOCK

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Message-Id: <20221207131838.239125-16-kwolf@redhat.com>
Reviewed-by: Emanuele Giuseppe Esposito <eesposit@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>

show more ...

graph-lock: TSA annotations for lock/unlock functions

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Message-Id: <20221207131838.239125-15-kwolf@redhat.com>
Reviewed-by: Emanuele Giuseppe Esposito <ee

graph-lock: TSA annotations for lock/unlock functions

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Message-Id: <20221207131838.239125-15-kwolf@redhat.com>
Reviewed-by: Emanuele Giuseppe Esposito <eesposit@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>

show more ...

block: assert that graph read and writes are performed correctly

Remove the old assert_bdrv_graph_writable, and replace it with
the new version using graph-lock API.

See the function documentation

block: assert that graph read and writes are performed correctly

Remove the old assert_bdrv_graph_writable, and replace it with
the new version using graph-lock API.

See the function documentation for more information.

Signed-off-by: Emanuele Giuseppe Esposito <eesposit@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Message-Id: <20221207131838.239125-14-kwolf@redhat.com>
Reviewed-by: Emanuele Giuseppe Esposito <eesposit@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>

show more ...

graph-lock: Implement guard macros

Similar to the implementation in lockable.h, implement macros to
automatically take and release the rdlock.

Create the empty GraphLockable and GraphLockableMainlo

graph-lock: Implement guard macros

Similar to the implementation in lockable.h, implement macros to
automatically take and release the rdlock.

Create the empty GraphLockable and GraphLockableMainloop structs only to
use it as a type for G_DEFINE_AUTOPTR_CLEANUP_FUNC.

Signed-off-by: Emanuele Giuseppe Esposito <eesposit@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Message-Id: <20221207131838.239125-4-kwolf@redhat.com>
Reviewed-by: Emanuele Giuseppe Esposito <eesposit@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>

show more ...

graph-lock: Introduce a lock to protect block graph operations

Block layer graph operations are always run under BQL in the main loop.
This is proved by the assertion qemu_in_main_thread() and its w

graph-lock: Introduce a lock to protect block graph operations

Block layer graph operations are always run under BQL in the main loop.
This is proved by the assertion qemu_in_main_thread() and its wrapper
macro GLOBAL_STATE_CODE.

However, there are also concurrent coroutines running in other iothreads
that always try to traverse the graph. Currently this is protected
(among various other things) by the AioContext lock, but once this is
removed, we need to make sure that reads do not happen while modifying
the graph.

We distinguish between writer (main loop, under BQL) that modifies the
graph, and readers (all other coroutines running in various AioContext),
that go through the graph edges, reading ->parents and->children.

The writer (main loop) has "exclusive" access, so it first waits for any
current read to finish, and then prevents incoming ones from entering
while it has the exclusive access.

The readers (coroutines in multiple AioContext) are free to access the
graph as long the writer is not modifying the graph. In case it is, they
go in a CoQueue and sleep until the writer is done.

If a coroutine changes AioContext, the counter in the original and new
AioContext are left intact, since the writer does not care where the
reader is, but only if there is one.

As a result, some AioContexts might have a negative reader count, to
balance the positive count of the AioContext that took the lock. This
also means that when an AioContext is deleted it may have a nonzero
reader count. In that case we transfer the count to a global shared
counter so that the writer is always aware of all readers.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Emanuele Giuseppe Esposito <eesposit@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Message-Id: <20221207131838.239125-3-kwolf@redhat.com>
Reviewed-by: Emanuele Giuseppe Esposito <eesposit@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>

show more ...