aio: remove aio_disable_external() API

All callers now pass is_external=false to aio_set_fd_handler() and
aio_set_event_notifier(). The aio_disable_external() API that
temporarily disables fd handle

aio: remove aio_disable_external() API

All callers now pass is_external=false to aio_set_fd_handler() and
aio_set_event_notifier(). The aio_disable_external() API that
temporarily disables fd handlers that were registered is_external=true
is therefore dead code.

Remove aio_disable_external(), aio_enable_external(), and the
is_external arguments to aio_set_fd_handler() and
aio_set_event_notifier().

The entire test-fdmon-epoll test is removed because its sole purpose was
testing aio_disable_external().

Parts of this patch were generated using the following coccinelle
(https://coccinelle.lip6.fr/) semantic patch:

@@
expression ctx, fd, is_external, io_read, io_write, io_poll, io_poll_ready, opaque;
@@
- aio_set_fd_handler(ctx, fd, is_external, io_read, io_write, io_poll, io_poll_ready, opaque)
+ aio_set_fd_handler(ctx, fd, io_read, io_write, io_poll, io_poll_ready, opaque)

@@
expression ctx, notifier, is_external, io_read, io_poll, io_poll_ready;
@@
- aio_set_event_notifier(ctx, notifier, is_external, io_read, io_poll, io_poll_ready)
+ aio_set_event_notifier(ctx, notifier, io_read, io_poll, io_poll_ready)

Reviewed-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-Id: <20230516190238.8401-21-stefanha@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>

show more ...

aio-posix: do not nest poll handlers

QEMU's event loop supports nesting, which means that event handler
functions may themselves call aio_poll(). The condition that triggered a
handler must be reset

aio-posix: do not nest poll handlers

QEMU's event loop supports nesting, which means that event handler
functions may themselves call aio_poll(). The condition that triggered a
handler must be reset before the nested aio_poll() call, otherwise the
same handler will be called and immediately re-enter aio_poll. This
leads to an infinite loop and stack exhaustion.

Poll handlers are especially prone to this issue, because they typically
reset their condition by finishing the processing of pending work.
Unfortunately it is during the processing of pending work that nested
aio_poll() calls typically occur and the condition has not yet been
reset.

Disable a poll handler during ->io_poll_ready() so that a nested
aio_poll() call cannot invoke ->io_poll_ready() again. As a result, the
disabled poll handler and its associated fd handler do not run during
the nested aio_poll(). Calling aio_set_fd_handler() from inside nested
aio_poll() could cause it to run again. If the fd handler is pending
inside nested aio_poll(), then it will also run again.

In theory fd handlers can be affected by the same issue, but they are
more likely to reset the condition before calling nested aio_poll().

This is a special case and it's somewhat complex, but I don't see a way
around it as long as nested aio_poll() is supported.

Cc: qemu-stable@nongnu.org
Buglink: https://bugzilla.redhat.com/show_bug.cgi?id=2186181
Fixes: c38270692593 ("block: Mark bdrv_co_io_(un)plug() and callers GRAPH_RDLOCK")
Cc: Kevin Wolf <kwolf@redhat.com>
Cc: Emanuele Giuseppe Esposito <eesposit@redhat.com>
Cc: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-Id: <20230502184134.534703-2-stefanha@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>

show more ...

aio: make aio_set_fd_poll() static to aio-posix.c

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Stefan Berger <stefanb@linux.ibm.com>
Message-Id: <20230221124802.410355

aio: make aio_set_fd_poll() static to aio-posix.c

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Stefan Berger <stefanb@linux.ibm.com>
Message-Id: <20230221124802.4103554-9-marcandre.lureau@redhat.com>

show more ...

util/aio: Defer disabling poll mode as long as possible

When we measure FIO read performance (cache=writethrough, bs=4k,
iodepth=64) in VMs, ~80K/s notifications (e.g., EPT_MISCONFIG) are observed
f

util/aio: Defer disabling poll mode as long as possible

When we measure FIO read performance (cache=writethrough, bs=4k,
iodepth=64) in VMs, ~80K/s notifications (e.g., EPT_MISCONFIG) are observed
from guest to qemu.

It turns out those frequent notificatons are caused by interference from
worker threads. Worker threads queue bottom halves after completing IO
requests. Pending bottom halves may lead to either aio_compute_timeout()
zeros timeout and pass it to try_poll_mode() or run_poll_handlers() returns
no progress after noticing pending aio_notify() events. Both cause
run_poll_handlers() to call poll_set_started(false) to disable poll mode.
However, for both cases, as timeout is already zeroed, the event loop
(i.e., aio_poll()) just processes bottom halves and then starts the next
event loop iteration. So, disabling poll mode has no value but leads to
unnecessary notifications from guest.

To minimize unnecessary notifications from guest, defer disabling poll
mode to when the event loop is about to be blocked.

With this patch applied, FIO seq-read performance (bs=4k, iodepth=64,
cache=writethrough) in VMs increases from 330K/s to 413K/s IOPS.

Suggested-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Chao Gao <chao.gao@intel.com>
Message-id: 20220710120849.63086-1-chao.gao@intel.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>

show more ...

util/event-loop-base: Introduce options to set the thread pool size

The thread pool regulates itself: when idle, it kills threads until
empty, when in demand, it creates new threads until full. This

util/event-loop-base: Introduce options to set the thread pool size

The thread pool regulates itself: when idle, it kills threads until
empty, when in demand, it creates new threads until full. This behaviour
doesn't play well with latency sensitive workloads where the price of
creating a new thread is too high. For example, when paired with qemu's
'-mlock', or using safety features like SafeStack, creating a new thread
has been measured take multiple milliseconds.

In order to mitigate this let's introduce a new 'EventLoopBase'
property to set the thread pool size. The threads will be created during
the pool's initialization or upon updating the property's value, remain
available during its lifetime regardless of demand, and destroyed upon
freeing it. A properly characterized workload will then be able to
configure the pool to avoid any latency spikes.

Signed-off-by: Nicolas Saenz Julienne <nsaenzju@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
Message-id: 20220425075723.20019-4-nsaenzju@redhat.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>

show more ...

aio-posix: fix spurious ->poll_ready() callbacks in main loop

When ->poll() succeeds the AioHandler is placed on the ready list with
revents set to the magic value 0. This magic value causes
aio_dis

aio-posix: fix spurious ->poll_ready() callbacks in main loop

When ->poll() succeeds the AioHandler is placed on the ready list with
revents set to the magic value 0. This magic value causes
aio_dispatch_handler() to invoke ->poll_ready() instead of ->io_read()
for G_IO_IN or ->io_write() for G_IO_OUT.

This magic value 0 hack works for the IOThread where AioHandlers are
placed on ->ready_list and processed by aio_dispatch_ready_handlers().
It does not work for the main loop where all AioHandlers are processed
by aio_dispatch_handlers(), even those that are not ready and have a
revents value of 0.

As a result the main loop invokes ->poll_ready() on AioHandlers that are
not ready. These spurious ->poll_ready() calls waste CPU cycles and
could lead to crashes if the code assumes ->poll() must have succeeded
before ->poll_ready() is called (a reasonable asumption but I haven't
seen it in practice).

Stop using revents to track whether ->poll_ready() will be called on an
AioHandler. Introduce a separate AioHandler->poll_ready field instead.
This eliminates spurious ->poll_ready() calls in the main loop.

Fixes: 826cc32423db2a99d184dbf4f507c737d7e7a4ae ("aio-posix: split poll check from ready handler")
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Reported-by: Jason Wang <jasowang@redhat.com>
Tested-by: Jason Wang <jasowang@redhat.com>
Message-id: 20220223155703.136833-1-stefanha@redhat.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>

show more ...

aio-posix: split poll check from ready handler

Adaptive polling measures the execution time of the polling check plus
handlers called when a polled event becomes ready. Handlers can take a
significa

aio-posix: split poll check from ready handler

Adaptive polling measures the execution time of the polling check plus
handlers called when a polled event becomes ready. Handlers can take a
significant amount of time, making it look like polling was running for
a long time when in fact the event handler was running for a long time.

For example, on Linux the io_submit(2) syscall invoked when a virtio-blk
device's virtqueue becomes ready can take 10s of microseconds. This
can exceed the default polling interval (32 microseconds) and cause
adaptive polling to stop polling.

By excluding the handler's execution time from the polling check we make
the adaptive polling calculation more accurate. As a result, the event
loop now stays in polling mode where previously it would have fallen
back to file descriptor monitoring.

The following data was collected with virtio-blk num-queues=2
event_idx=off using an IOThread. Before:

168k IOPS, IOThread syscalls:

9837.115 ( 0.020 ms): IO iothread1/620155 io_submit(ctx_id: 140512552468480, nr: 16, iocbpp: 0x7fcb9f937db0) = 16
9837.158 ( 0.002 ms): IO iothread1/620155 write(fd: 103, buf: 0x556a2ef71b88, count: 8) = 8
9837.161 ( 0.001 ms): IO iothread1/620155 write(fd: 104, buf: 0x556a2ef71b88, count: 8) = 8
9837.163 ( 0.001 ms): IO iothread1/620155 ppoll(ufds: 0x7fcb90002800, nfds: 4, tsp: 0x7fcb9f1342d0, sigsetsize: 8) = 3
9837.164 ( 0.001 ms): IO iothread1/620155 read(fd: 107, buf: 0x7fcb9f939cc0, count: 512) = 8
9837.174 ( 0.001 ms): IO iothread1/620155 read(fd: 105, buf: 0x7fcb9f939cc0, count: 512) = 8
9837.176 ( 0.001 ms): IO iothread1/620155 read(fd: 106, buf: 0x7fcb9f939cc0, count: 512) = 8
9837.209 ( 0.035 ms): IO iothread1/620155 io_submit(ctx_id: 140512552468480, nr: 32, iocbpp: 0x7fca7d0cebe0) = 32

174k IOPS (+3.6%), IOThread syscalls:

9809.566 ( 0.036 ms): IO iothread1/623061 io_submit(ctx_id: 140539805028352, nr: 32, iocbpp: 0x7fd0cdd62be0) = 32
9809.625 ( 0.001 ms): IO iothread1/623061 write(fd: 103, buf: 0x5647cfba5f58, count: 8) = 8
9809.627 ( 0.002 ms): IO iothread1/623061 write(fd: 104, buf: 0x5647cfba5f58, count: 8) = 8
9809.663 ( 0.036 ms): IO iothread1/623061 io_submit(ctx_id: 140539805028352, nr: 32, iocbpp: 0x7fd0d0388b50) = 32

Notice that ppoll(2) and eventfd read(2) syscalls are eliminated because
the IOThread stays in polling mode instead of falling back to file
descriptor monitoring.

As usual, polling is not implemented on Windows so this patch ignores
the new io_poll_read() callback in aio-win32.c.

Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Stefano Garzarella <sgarzare@redhat.com>
Message-id: 20211207132336.36627-2-stefanha@redhat.com

[Fixed up aio_set_event_notifier() calls in
tests/unit/test-fdmon-epoll.c added after this series was queued.
--Stefan]

Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>

show more ...

Merge remote-tracking branch 'remotes/quic/tags/pull-hex-20210725' into staging

The Hexagon target was silently failing the SIGSEGV test because
the signal handler was not called.

Merge remote-tracking branch 'remotes/quic/tags/pull-hex-20210725' into staging

The Hexagon target was silently failing the SIGSEGV test because
the signal handler was not called.

Patch 1/2 fixes the Hexagon target
Patch 2/2 drops include qemu.h from target/hexagon/op_helper.c

**** Changes in v2 ****
Drop changes to linux-test.c due to intermittent failures on riscv

# gpg: Signature made Sun 25 Jul 2021 22:39:38 BST
# gpg: using RSA key 7B0244FB12DE4422
# gpg: Good signature from "Taylor Simpson (Rock on) <tsimpson@quicinc.com>" [undefined]
# gpg: WARNING: This key is not certified with a trusted signature!
# gpg: There is no indication that the signature belongs to the owner.
# Primary key fingerprint: 3635 C788 CE62 B91F D4C5 9AB4 7B02 44FB 12DE 4422

* remotes/quic/tags/pull-hex-20210725:
target/hexagon: Drop include of qemu.h
Hexagon (target/hexagon) remove put_user_*/get_user_*

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

Merge remote-tracking branch 'remotes/bonzini-gitlab/tags/for-upstream' into staging

Bugfixes.

# gpg: Signature made Thu 22 Jul 2021 14:11:27 BST
# gpg: using RSA

Merge remote-tracking branch 'remotes/bonzini-gitlab/tags/for-upstream' into staging

Bugfixes.

# gpg: Signature made Thu 22 Jul 2021 14:11:27 BST
# gpg: using RSA key F13338574B662389866C7682BFFBD25F78C7AE83
# gpg: issuer "pbonzini@redhat.com"
# gpg: Good signature from "Paolo Bonzini <bonzini@gnu.org>" [full]
# gpg: aka "Paolo Bonzini <pbonzini@redhat.com>" [full]
# Primary key fingerprint: 46F5 9FBD 57D6 12E7 BFD4 E2F7 7E15 100C CD36 69B1
# Subkey fingerprint: F133 3857 4B66 2389 866C 7682 BFFB D25F 78C7 AE83

* remotes/bonzini-gitlab/tags/for-upstream:
configure: Let --without-default-features disable vhost-kernel and vhost-vdpa
configure: Fix the default setting of the "xen" feature
configure: Allow vnc to get disabled with --without-default-features
configure: Fix --without-default-features propagation to meson
meson: fix dependencies for modinfo
configure: Drop obsolete check for the alloc_size attribute
target/i386: Added consistency checks for EFER
target/i386: Added consistency checks for CR4
target/i386: Added V_INTR_PRIO check to virtual interrupts
qemu-config: restore "machine" in qmp_query_command_line_options()
usb: fix usb-host dependency check
chardev-spice: add missing module_obj directive
vl: Parse legacy default_machine_opts
qemu-config: fix memory leak on ferror()
qemu-config: never call the callback after an error, fix leak

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

Merge remote-tracking branch 'remotes/stefanha-gitlab/tags/block-pull-request' into staging

Pull request

Stefano's performance regression fix for commit 2558cb8dd4 ("linux-aio:

Merge remote-tracking branch 'remotes/stefanha-gitlab/tags/block-pull-request' into staging

Pull request

Stefano's performance regression fix for commit 2558cb8dd4 ("linux-aio:
increasing MAX_EVENTS to a larger hardcoded value").

# gpg: Signature made Wed 21 Jul 2021 14:12:47 BST
# gpg: using RSA key 8695A8BFD3F97CDAAC35775A9CA4ABB381AB73C8
# gpg: Good signature from "Stefan Hajnoczi <stefanha@redhat.com>" [full]
# gpg: aka "Stefan Hajnoczi <stefanha@gmail.com>" [full]
# Primary key fingerprint: 8695 A8BF D3F9 7CDA AC35 775A 9CA4 ABB3 81AB 73C8

* remotes/stefanha-gitlab/tags/block-pull-request:
linux-aio: limit the batch size using `aio-max-batch` parameter
iothread: add aio-max-batch parameter
iothread: generalize iothread_set_param/iothread_get_param

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

iothread: add aio-max-batch parameter

The `aio-max-batch` parameter will be propagated to AIO engines
and it will be used to control the maximum number of queued requests.

When

iothread: add aio-max-batch parameter

The `aio-max-batch` parameter will be propagated to AIO engines
and it will be used to control the maximum number of queued requests.

When there are in queue a number of requests equal to `aio-max-batch`,
the engine invokes the system call to forward the requests to the kernel.

This parameter allows us to control the maximum batch size to reduce
the latency that requests might accumulate while queued in the AIO
engine queue.

If `aio-max-batch` is equal to 0 (default value), the AIO engine will
use its default maximum batch size value.

Signed-off-by: Stefano Garzarella <sgarzare@redhat.com>
Message-id: 20210721094211.69853-3-sgarzare@redhat.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>

show more ...

Merge remote-tracking branch 'remotes/dgibson/tags/ppc-for-5.2-20201009' into staging

ppc patch queue 2020-10-09

Here's the next set of ppc related patches for qemu-5.2. There are

Merge remote-tracking branch 'remotes/dgibson/tags/ppc-for-5.2-20201009' into staging

ppc patch queue 2020-10-09

Here's the next set of ppc related patches for qemu-5.2. There are
two main things here:

* Cleanups to error handling in spapr from Greg Kurz
* Improvements to NUMA handling for spapr from Daniel Barboza

There are also a handful of other bugfixes.

# gpg: Signature made Fri 09 Oct 2020 07:02:29 BST
# gpg: using RSA key 75F46586AE61A66CC44E87DC6C38CACA20D9B392
# gpg: Good signature from "David Gibson <david@gibson.dropbear.id.au>" [full]
# gpg: aka "David Gibson (Red Hat) <dgibson@redhat.com>" [full]
# gpg: aka "David Gibson (ozlabs.org) <dgibson@ozlabs.org>" [full]
# gpg: aka "David Gibson (kernel.org) <dwg@kernel.org>" [unknown]
# Primary key fingerprint: 75F4 6586 AE61 A66C C44E 87DC 6C38 CACA 20D9 B392

* remotes/dgibson/tags/ppc-for-5.2-20201009:
specs/ppc-spapr-numa: update with new NUMA support
spapr_numa: consider user input when defining associativity
spapr_numa: change reference-points and maxdomain settings
spapr_numa: forbid asymmetrical NUMA setups
spapr: add spapr_machine_using_legacy_numa() helper
ppc/pnv: Increase max firmware size
spapr: Add a return value to spapr_check_pagesize()
spapr: Add a return value to spapr_nvdimm_validate()
spapr: Simplify error handling in spapr_cpu_core_realize()
spapr: Add a return value to spapr_set_vcpu_id()
spapr: Simplify error handling in prop_get_fdt()
spapr: Add a return value to spapr_drc_attach()
spapr: Simplify error handling in spapr_vio_busdev_realize()
spapr: Simplify error handling in do_client_architecture_support()
spapr: Get rid of cas_check_pvr() error reporting
spapr: Simplify error handling in callers of ppc_set_compat()
ppc: Fix return value in cpu_post_load() error path
ppc: Add a return value to ppc_set_compat() and ppc_set_compat_all()
spapr: Fix error leak in spapr_realize_vcpu()
spapr: Handle HPT allocation failure in nested guest

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

Merge remote-tracking branch 'remotes/armbru/tags/pull-error-2020-10-09' into staging

Error reporting patches for 2020-10-09

# gpg: Signature made Fri 09 Oct 2020 07:45:17 BST
#

Merge remote-tracking branch 'remotes/armbru/tags/pull-error-2020-10-09' into staging

Error reporting patches for 2020-10-09

# gpg: Signature made Fri 09 Oct 2020 07:45:17 BST
# gpg: using RSA key 354BC8B3D7EB2A6B68674E5F3870B400EB918653
# gpg: issuer "armbru@redhat.com"
# gpg: Good signature from "Markus Armbruster <armbru@redhat.com>" [full]
# gpg: aka "Markus Armbruster <armbru@pond.sub.org>" [full]
# Primary key fingerprint: 354B C8B3 D7EB 2A6B 6867 4E5F 3870 B400 EB91 8653

* remotes/armbru/tags/pull-error-2020-10-09:
error: Use error_fatal to simplify obvious fatal errors (again)
error: Remove NULL checks on error_propagate() calls (again)

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

qmp: Move dispatcher to a coroutine

This moves the QMP dispatcher to a coroutine and runs all QMP command
handlers that declare 'coroutine': true in coroutine context so they
can avo

qmp: Move dispatcher to a coroutine

This moves the QMP dispatcher to a coroutine and runs all QMP command
handlers that declare 'coroutine': true in coroutine context so they
can avoid blocking the main loop while doing I/O or waiting for other
events.

For commands that are not declared safe to run in a coroutine, the
dispatcher drops out of coroutine context by calling the QMP command
handler from a bottom half.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20201005155855.256490-10-kwolf@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>

show more ...

Merge remote-tracking branch 'remotes/vivier2/tags/trivial-branch-for-5.2-pull-request' into staging

Trivial Patches Pull request 20200928

# gpg: Signature made Mon 28 Sep 2020 10:1

Merge remote-tracking branch 'remotes/vivier2/tags/trivial-branch-for-5.2-pull-request' into staging

Trivial Patches Pull request 20200928

# gpg: Signature made Mon 28 Sep 2020 10:15:00 BST
# gpg: using RSA key CD2F75DDC8E3A4DC2E4F5173F30C38BD3F2FBE3C
# gpg: issuer "laurent@vivier.eu"
# gpg: Good signature from "Laurent Vivier <lvivier@redhat.com>" [full]
# gpg: aka "Laurent Vivier <laurent@vivier.eu>" [full]
# gpg: aka "Laurent Vivier (Red Hat) <lvivier@redhat.com>" [full]
# Primary key fingerprint: CD2F 75DD C8E3 A4DC 2E4F 5173 F30C 38BD 3F2F BE3C

* remotes/vivier2/tags/trivial-branch-for-5.2-pull-request:
docs/system/deprecated: Move lm32 and unicore32 to the right section
migration/multifd: Remove superfluous semicolons
timer: Fix timer_mod_anticipate() documentation
vhost-vdpa: remove useless variable
Add *.pyc back to the .gitignore file
virtio: vdpa: omit check return of g_malloc
meson: fix static flag summary
vhost-vdpa: fix indentation in vdpa_ops

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

Merge remote-tracking branch 'remotes/stefanha/tags/block-pull-request' into staging

Pull request

This includes the atomic_ -> qatomic_ rename that touches many files and is
pro

Merge remote-tracking branch 'remotes/stefanha/tags/block-pull-request' into staging

Pull request

This includes the atomic_ -> qatomic_ rename that touches many files and is
prone to conflicts.

# gpg: Signature made Wed 23 Sep 2020 17:08:43 BST
# gpg: using RSA key 8695A8BFD3F97CDAAC35775A9CA4ABB381AB73C8
# gpg: Good signature from "Stefan Hajnoczi <stefanha@redhat.com>" [full]
# gpg: aka "Stefan Hajnoczi <stefanha@gmail.com>" [full]
# Primary key fingerprint: 8695 A8BF D3F9 7CDA AC35 775A 9CA4 ABB3 81AB 73C8

* remotes/stefanha/tags/block-pull-request:
qemu/atomic.h: rename atomic_ to qatomic_
tests: add test-fdmon-epoll
fdmon-poll: reset npfd when upgrading to fdmon-epoll
gitmodules: add qemu.org vbootrom submodule
gitmodules: switch to qemu.org meson mirror
gitmodules: switch to qemu.org qboot mirror
docs/system: clarify deprecation schedule
virtio-crypto: don't modify elem->in/out_sg
virtio-blk: undo destructive iov_discard_*() operations
util/iov: add iov_discard_undo()
virtio: add vhost-user-fs-ccw device
libvhost-user: handle endianness as mandated by the spec
MAINTAINERS: add Stefan Hajnoczi as block/nvme.c maintainer

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

qemu/atomic.h: rename atomic_ to qatomic_

clang's C11 atomic_fetch_*() functions only take a C11 atomic type
pointer argument. QEMU uses direct types (int, etc) and this causes a
com

qemu/atomic.h: rename atomic_ to qatomic_

clang's C11 atomic_fetch_*() functions only take a C11 atomic type
pointer argument. QEMU uses direct types (int, etc) and this causes a
compiler error when a QEMU code calls these functions in a source file
that also included <stdatomic.h> via a system header file:

$ CC=clang CXX=clang++ ./configure ... && make
../util/async.c:79:17: error: address argument to atomic operation must be a pointer to _Atomic type ('unsigned int *' invalid)

Avoid using atomic_*() names in QEMU's atomic.h since that namespace is
used by <stdatomic.h>. Prefix QEMU's APIs with 'q' so that atomic.h
and <stdatomic.h> can co-exist. I checked /usr/include on my machine and
searched GitHub for existing "qatomic_" users but there seem to be none.

This patch was generated using:

$ git grep -h -o '\<atomic\(64\)\?_[a-z0-9_]\+' include/qemu/atomic.h | \
sort -u >/tmp/changed_identifiers
$ for identifier in $(</tmp/changed_identifiers); do
sed -i "s%\<$identifier\>%q$identifier%g" \
$(git grep -I -l "\<$identifier\>")
done

I manually fixed line-wrap issues and misaligned rST tables.

Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Acked-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20200923105646.47864-1-stefanha@redhat.com>

show more ...

Merge remote-tracking branch 'remotes/hdeller/tags/target-hppa-v3-pull-request' into staging

artist out of bounds fixes

# gpg: Signature made Wed 26 Aug 2020 22:09:55 BST
# gpg:

Merge remote-tracking branch 'remotes/hdeller/tags/target-hppa-v3-pull-request' into staging

artist out of bounds fixes

# gpg: Signature made Wed 26 Aug 2020 22:09:55 BST
# gpg: using EDDSA key BCE9123E1AD29F07C049BBDEF712B510A23A0F5F
# gpg: Good signature from "Helge Deller <deller@gmx.de>" [unknown]
# gpg: aka "Helge Deller <deller@kernel.org>" [unknown]
# gpg: WARNING: This key is not certified with a trusted signature!
# gpg: There is no indication that the signature belongs to the owner.
# Primary key fingerprint: 4544 8228 2CD9 10DB EF3D 25F8 3E5F 3D04 A7A2 4603
# Subkey fingerprint: BCE9 123E 1AD2 9F07 C049 BBDE F712 B510 A23A 0F5F

* remotes/hdeller/tags/target-hppa-v3-pull-request:
hw/display/artist: Fix invalidation of lines near screen border
hw/display/artist: Fix invalidation of lines in artist_draw_line()
hw/display/artist: Unbreak size mismatch memory accesses
hw/display/artist: Prevent out of VRAM buffer accesses
Revert "hw/display/artist: Avoid drawing line when nothing to display"
hw/display/artist: Refactor artist_rop8() to avoid buffer over-run
hw/display/artist: Check offset in draw_line to avoid buffer over-run
hw/hppa/lasi: Don't abort on invalid IMR value
hw/display/artist.c: fix out of bounds check
hw/hppa: Implement proper SeaBIOS version check
seabios-hppa: Update to SeaBIOS hppa version 1
hw/hppa: Sync hppa_hardware.h file with SeaBIOS sources

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

Merge remote-tracking branch 'remotes/cschoenebeck/tags/pull-9p-20200812' into staging

9pfs: Fix severe performance issue of Treaddir requests.

# gpg: Signature made Wed 12 Aug 2020

Merge remote-tracking branch 'remotes/cschoenebeck/tags/pull-9p-20200812' into staging

9pfs: Fix severe performance issue of Treaddir requests.

# gpg: Signature made Wed 12 Aug 2020 11:06:21 BST
# gpg: using RSA key 96D8D110CF7AF8084F88590134C2B58765A47395
# gpg: issuer "qemu_oss@crudebyte.com"
# gpg: Good signature from "Christian Schoenebeck <qemu_oss@crudebyte.com>" [unknown]
# gpg: WARNING: This key is not certified with a trusted signature!
# gpg: There is no indication that the signature belongs to the owner.
# Primary key fingerprint: ECAB 1A45 4014 1413 BA38 4926 30DB 47C3 A012 D5F4
# Subkey fingerprint: 96D8 D110 CF7A F808 4F88 5901 34C2 B587 65A4 7395

* remotes/cschoenebeck/tags/pull-9p-20200812:
9pfs: clarify latency of v9fs_co_run_in_worker()
9pfs: differentiate readdir lock between 9P2000.u vs. 9P2000.L
9pfs: T_readdir latency optimization
9pfs: add new function v9fs_co_readdir_many()
9pfs: split out fs driver core of v9fs_co_readdir()
9pfs: make v9fs_readdir_response_size() public
tests/virtio-9p: added split readdir tests

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

Merge remote-tracking branch 'remotes/dgibson/tags/ppc-for-5.2-20200818' into staging

ppc patch queue 2020-08-18

Here's my first pull request for qemu-5.2, which has quite a few

Merge remote-tracking branch 'remotes/dgibson/tags/ppc-for-5.2-20200818' into staging

ppc patch queue 2020-08-18

Here's my first pull request for qemu-5.2, which has quite a few
accumulated things. Highlights are:

* Preliminary support for POWER10 (Power ISA 3.1) instruction emulation
* Add documentation on the (very confusing) pseries NUMA configuration
* Fix some bugs handling edge cases with XICS, XIVE and kernel_irqchip
* Fix icount for a number of POWER registers
* Many cleanups to error handling in XIVE code
* Validate size of -prom-env data

# gpg: Signature made Tue 18 Aug 2020 05:18:36 BST
# gpg: using RSA key 75F46586AE61A66CC44E87DC6C38CACA20D9B392
# gpg: Good signature from "David Gibson <david@gibson.dropbear.id.au>" [full]
# gpg: aka "David Gibson (Red Hat) <dgibson@redhat.com>" [full]
# gpg: aka "David Gibson (ozlabs.org) <dgibson@ozlabs.org>" [full]
# gpg: aka "David Gibson (kernel.org) <dwg@kernel.org>" [unknown]
# Primary key fingerprint: 75F4 6586 AE61 A66C C44E 87DC 6C38 CACA 20D9 B392

* remotes/dgibson/tags/ppc-for-5.2-20200818: (40 commits)
spapr/xive: Use xive_source_esb_len()
nvram: Exit QEMU if NVRAM cannot contain all -prom-env data
spapr/xive: Simplify error handling of kvmppc_xive_cpu_synchronize_state()
ppc/xive: Simplify error handling in xive_tctx_realize()
spapr/xive: Simplify error handling in kvmppc_xive_connect()
ppc/xive: Fix error handling in vmstate_xive_tctx_*() callbacks
spapr/xive: Fix error handling in kvmppc_xive_post_load()
spapr/kvm: Fix error handling in kvmppc_xive_pre_save()
spapr/xive: Rework error handling of kvmppc_xive_set_source_config()
spapr/xive: Rework error handling in kvmppc_xive_get_queues()
spapr/xive: Rework error handling of kvmppc_xive_[gs]et_queue_config()
spapr/xive: Rework error handling of kvmppc_xive_cpu_[gs]et_state()
spapr/xive: Rework error handling of kvmppc_xive_mmap()
spapr/xive: Rework error handling of kvmppc_xive_source_reset()
spapr/xive: Rework error handling of kvmppc_xive_cpu_connect()
spapr: Simplify error handling in spapr_phb_realize()
spapr/xive: Convert KVM device fd checks to assert()
ppc/xive: Introduce dedicated kvm_irqchip_in_kernel() wrappers
ppc/xive: Rework setup of XiveSource::esb_mmio
target/ppc: Integrate icount to purr, vtb, and tbu40
...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

Merge remote-tracking branch 'remotes/philmd-gitlab/tags/sd-next-20200821' into staging

SD/MMC patches

- Convert legacy SD host controller to the SDBus API
- Move legacy API to

Merge remote-tracking branch 'remotes/philmd-gitlab/tags/sd-next-20200821' into staging

SD/MMC patches

- Convert legacy SD host controller to the SDBus API
- Move legacy API to a separate "sdcard_legacy.h" header
- Introduce methods to access multiple bytes on SDBus data lines
- Fix 'switch function' group location
- Fix SDSC maximum card size (2GB)

CI jobs result:
https://gitlab.com/philmd/qemu/-/pipelines/180605963

# gpg: Signature made Fri 21 Aug 2020 18:27:50 BST
# gpg: using RSA key FAABE75E12917221DCFD6BB2E3E32C2CDEADC0DE
# gpg: Good signature from "Philippe Mathieu-Daudé (F4BUG) <f4bug@amsat.org>" [full]
# Primary key fingerprint: FAAB E75E 1291 7221 DCFD 6BB2 E3E3 2C2C DEAD C0DE

* remotes/philmd-gitlab/tags/sd-next-20200821: (23 commits)
hw/sd: Correct the maximum size of a Standard Capacity SD Memory Card
hw/sd: Fix incorrect populated function switch status data structure
hw/sd: Use sdbus_read_data() instead of sdbus_read_byte() when possible
hw/sd: Add sdbus_read_data() to read multiples bytes on the data line
hw/sd: Use sdbus_write_data() instead of sdbus_write_byte when possible
hw/sd: Add sdbus_write_data() to write multiples bytes on the data line
hw/sd: Rename sdbus_read_data() as sdbus_read_byte()
hw/sd: Rename sdbus_write_data() as sdbus_write_byte()
hw/sd: Rename read/write_data() as read/write_byte()
hw/sd: Move sdcard legacy API to 'hw/sd/sdcard_legacy.h'
hw/sd/sdcard: Make sd_data_ready() static
hw/sd/pl181: Replace disabled fprintf()s by trace events
hw/sd/pl181: Do not create SD card within the SD host controller
hw/sd/pl181: Expose a SDBus and connect the SDCard to it
hw/sd/pl181: Use named GPIOs
hw/sd/pl181: Add TODO to use Fifo32 API
hw/sd/pl181: Rename pl181_send_command() as pl181_do_command()
hw/sd/pl181: Replace fprintf(stderr, "*\n") with error_report()
hw/sd/milkymist: Do not create SD card within the SD host controller
hw/sd/milkymist: Create the SDBus at init()
...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

Merge remote-tracking branch 'remotes/bonzini-gitlab/tags/for-upstream' into staging

meson fixes:

* --disable-tools --enable-system build
* s390 no-TCG build
* fdmon-io_urin

Merge remote-tracking branch 'remotes/bonzini-gitlab/tags/for-upstream' into staging

meson fixes:

* --disable-tools --enable-system build
* s390 no-TCG build
* fdmon-io_uring
* 'shift' error message in version_ge()

# gpg: Signature made Fri 21 Aug 2020 22:12:29 BST
# gpg: using RSA key F13338574B662389866C7682BFFBD25F78C7AE83
# gpg: issuer "pbonzini@redhat.com"
# gpg: Good signature from "Paolo Bonzini <bonzini@gnu.org>" [full]
# gpg: aka "Paolo Bonzini <pbonzini@redhat.com>" [full]
# Primary key fingerprint: 46F5 9FBD 57D6 12E7 BFD4 E2F7 7E15 100C CD36 69B1
# Subkey fingerprint: F133 3857 4B66 2389 866C 7682 BFFB D25F 78C7 AE83

* remotes/bonzini-gitlab/tags/for-upstream:
keymaps: update
meson: Fix --disable-tools --enable-system builds
meson: convert pc-bios/keymaps/Makefile
configure: silence 'shift' error message in version_ge()
util/meson.build: fix fdmon-io_uring build
target/s390x: fix meson.build issue

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

Merge remote-tracking branch 'remotes/ericb/tags/pull-bitmaps-2020-08-21' into staging

bitmaps patches for 2020-08-21

- Andrey Shinkevich: Enhance qcow2.py for iotest inspection of

Merge remote-tracking branch 'remotes/ericb/tags/pull-bitmaps-2020-08-21' into staging

bitmaps patches for 2020-08-21

- Andrey Shinkevich: Enhance qcow2.py for iotest inspection of qcow2 images
- Max Reitz: Add block-bitmap-mapping migration parameter

# gpg: Signature made Fri 21 Aug 2020 15:05:07 BST
# gpg: using RSA key 71C2CC22B1C4602927D2F3AAA7A16B4A2527436A
# gpg: Good signature from "Eric Blake <eblake@redhat.com>" [full]
# gpg: aka "Eric Blake (Free Software Programmer) <ebb9@byu.net>" [full]
# gpg: aka "[jpeg image of size 6874]" [full]
# Primary key fingerprint: 71C2 CC22 B1C4 6029 27D2 F3AA A7A1 6B4A 2527 436A

* remotes/ericb/tags/pull-bitmaps-2020-08-21:
iotests: Test node/bitmap aliases during migration
iotests.py: Let wait_migration() return on failure
migration: Add block-bitmap-mapping parameter
iotests: dump QCOW2 header in JSON in #303
qcow2_format.py: support dumping metadata in JSON format
qcow2_format.py: collect fields to dump in JSON format
qcow2.py: Introduce '-j' key to dump in JSON format
qcow2_format.py: Dump bitmap table serialized entries
qcow2_format.py: pass cluster size to substructures
qcow2_format.py: Dump bitmap directory information
qcow2_format.py: dump bitmap flags in human readable way.
qcow2_format.py: change Qcow2BitmapExt initialization method
qcow2_format.py: make printable data an extension class member
iotests: add test for QCOW2 header dump

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

Merge remote-tracking branch 'remotes/ehabkost/tags/machine-next-pull-request' into staging

Machine queue 2020-08-19

Regular post-release changes:
* hw: add compat machines for

Merge remote-tracking branch 'remotes/ehabkost/tags/machine-next-pull-request' into staging

Machine queue 2020-08-19

Regular post-release changes:
* hw: add compat machines for 5.2 (Cornelia Huck)

Features:
* qmp: Expose MachineClass::default_ram_id
(Michal Privoznik)

Cleanups:
* qdev: Document qdev_prop_set_drive_err() return value
(Philippe Mathieu-Daudé)

# gpg: Signature made Wed 19 Aug 2020 16:21:55 BST
# gpg: using RSA key 5A322FD5ABC4D3DBACCFD1AA2807936F984DC5A6
# gpg: issuer "ehabkost@redhat.com"
# gpg: Good signature from "Eduardo Habkost <ehabkost@redhat.com>" [full]
# Primary key fingerprint: 5A32 2FD5 ABC4 D3DB ACCF D1AA 2807 936F 984D C5A6

* remotes/ehabkost/tags/machine-next-pull-request:
hw: add compat machines for 5.2
qmp: Expose MachineClass::default_ram_id
qdev: Document qdev_prop_set_drive_err() return value

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

Merge remote-tracking branch 'remotes/kraxel/tags/seabios-20200819-pull-request' into staging

seabios: update to 1.14-final

# gpg: Signature made Wed 19 Aug 2020 07:36:36 BST
#

Merge remote-tracking branch 'remotes/kraxel/tags/seabios-20200819-pull-request' into staging

seabios: update to 1.14-final

# gpg: Signature made Wed 19 Aug 2020 07:36:36 BST
# gpg: using RSA key 4CB6D8EED3E87138
# gpg: Good signature from "Gerd Hoffmann (work) <kraxel@redhat.com>" [full]
# gpg: aka "Gerd Hoffmann <gerd@kraxel.org>" [full]
# gpg: aka "Gerd Hoffmann (private) <kraxel@gmail.com>" [full]
# Primary key fingerprint: A032 8CFF B93A 17A7 9901 FE7D 4CB6 D8EE D3E8 7138

* remotes/kraxel/tags/seabios-20200819-pull-request:
seabios: update to 1.14-final

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...