Merge tag 'pull-xenfv-stable-20231106' of git://git.infradead.org/users/dwmw2/qemu into staging

Bugfixes for emulated Xen support

Selected bugfixes for mainline and stable, especially to the per-vC

Merge tag 'pull-xenfv-stable-20231106' of git://git.infradead.org/users/dwmw2/qemu into staging

Bugfixes for emulated Xen support

Selected bugfixes for mainline and stable, especially to the per-vCPU
local APIC vector delivery mode for event channel notifications, which
was broken in a number of ways.

The xen-block driver has been defaulting to the wrong protocol for x86
guest, and this fixes that — which is technically an incompatible change
but I'm fairly sure nobody relies on the broken behaviour (and in
production I *have* seen guests which rely on the correct behaviour,
which now matches the blkback driver in the Linux kernel).

A handful of other simple fixes for issues which came to light as new
features (qv) were being developed.

# -----BEGIN PGP SIGNATURE-----
#
# iQJIBAABCAAyFiEEvgfZ/VSAmrLEsP9fY3Ys2mfi81kFAmVIvv4UHGR3bXcyQGlu
# ZnJhZGVhZC5vcmcACgkQY3Ys2mfi81nFmRAAvK3VNuGDV56TJqFdtEWD+3jzSZU0
# CoL1mxggvwnlFn1SdHvbC5jl+UscknErcNbqlxMTTg9jQiiQqzFuaWujJnL0dEOY
# RJiS2scKln/1gv9NRbLE31FjPwoNz+zJI/iMvdutjT7Ll//v34jY0vd1Y5Wo53ay
# MBschuuxD1sUUTHNj5f9afrgZaetJfgBSNZraiLR5T2HEadJVJuhItdGxW1+KaPI
# zBIcflIeZmJl9b/L1a2bP3KJmRo8QzHB56X3uzwkPhYhYSU2dnCaJTLCkiNfK+Qh
# SgCBMlzsvJbIZqDA9YPOGdKK1ArfTJRmRDwAkqH0YQknQGoIkpN+7eQiiSv6PMS5
# U/93V7r6MfaftIs6YdWSnFozWeBuyKZL9H2nAXqZgL5t6uEMVR8Un/kFnGfslTFY
# 9gQ1o4IM6ECLiXhIP/sPNOprrbFb0HU7QPtEDJOxrJzBM+IfLbldRHn4p9CccqQA
# LHvJF98VhX1d0nA0iZBT3qqfKPbmUhRV9Jrm+WamqNrRXhiGdF8EidsUf8RWX+JD
# xZWJiqhTwShxdLE6TC/JgFz4cQCVHG8QiZstZUbdq59gtz9YO5PGByMgI3ds7iNQ
# lGXAPFm+1wU85W4dZOH7qyim6d9ytFm2Fm110BKM8l9B6UKEuKHpsxXMqdo65JXI
# 7uBKbVpdPKul0DY=
# =dQ7h
# -----END PGP SIGNATURE-----
# gpg: Signature made Mon 06 Nov 2023 18:25:02 HKT
# gpg: using RSA key BE07D9FD54809AB2C4B0FF5F63762CDA67E2F359
# gpg: issuer "dwmw2@infradead.org"
# gpg: Good signature from "David Woodhouse <dwmw2@infradead.org>" [unknown]
# gpg: aka "David Woodhouse <dwmw2@exim.org>" [unknown]
# gpg: aka "David Woodhouse <david@woodhou.se>" [unknown]
# gpg: aka "David Woodhouse <dwmw2@kernel.org>" [unknown]
# gpg: WARNING: The key's User ID is not certified with a trusted signature!
# gpg: There is no indication that the signature belongs to the owner.
# Primary key fingerprint: BE07 D9FD 5480 9AB2 C4B0 FF5F 6376 2CDA 67E2 F359

* tag 'pull-xenfv-stable-20231106' of git://git.infradead.org/users/dwmw2/qemu:
hw/xen: use correct default protocol for xen-block on x86
hw/xen: take iothread mutex in xen_evtchn_reset_op()
hw/xen: fix XenStore watch delivery to guest
hw/xen: don't clear map_track[] in xen_gnttab_reset()
hw/xen: select kernel mode for per-vCPU event channel upcall vector
i386/xen: fix per-vCPU upcall vector for Xen emulation
i386/xen: Don't advertise XENFEAT_supervisor_mode_kernel

Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>

show more ...

hw/xen: select kernel mode for per-vCPU event channel upcall vector

A guest which has configured the per-vCPU upcall vector may set the
HVM_PARAM_CALLBACK_IRQ param to fairly much anything other tha

hw/xen: select kernel mode for per-vCPU event channel upcall vector

A guest which has configured the per-vCPU upcall vector may set the
HVM_PARAM_CALLBACK_IRQ param to fairly much anything other than zero.

For example, Linux v6.0+ after commit b1c3497e604 ("x86/xen: Add support
for HVMOP_set_evtchn_upcall_vector") will just do this after setting the
vector:

/* Trick toolstack to think we are enlightened. */
if (!cpu)
rc = xen_set_callback_via(1);

That's explicitly setting the delivery to GSI#1, but it's supposed to be
overridden by the per-vCPU vector setting. This mostly works in Qemu
*except* for the logic to enable the in-kernel handling of event channels,
which falsely determines that the kernel cannot accelerate GSI delivery
in this case.

Add a kvm_xen_has_vcpu_callback_vector() to report whether vCPU#0 has
the vector set, and use that in xen_evtchn_set_callback_param() to
enable the kernel acceleration features even when the param *appears*
to be set to target a GSI.

Preserve the Xen behaviour that when HVM_PARAM_CALLBACK_IRQ is set to
*zero* the event channel delivery is disabled completely. (Which is
what that bizarre guest behaviour is working round in the first place.)

Cc: qemu-stable@nongnu.org
Fixes: 91cce756179 ("hw/xen: Add xen_evtchn device for event channel emulation")
Signed-off-by: David Woodhouse <dwmw@amazon.co.uk>
Reviewed-by: Paul Durrant <paul@xen.org>

show more ...

Merge tag 'for-upstream' of https://gitlab.com/bonzini/qemu into staging

* bugfixes
* show machine ACPI support in QAPI
* Core Xen emulation support for KVM/x86

# -----BEGIN PGP SIGNATURE-----
#
#

Merge tag 'for-upstream' of https://gitlab.com/bonzini/qemu into staging

* bugfixes
* show machine ACPI support in QAPI
* Core Xen emulation support for KVM/x86

# -----BEGIN PGP SIGNATURE-----
#
# iQFIBAABCAAyFiEE8TM4V0tmI4mGbHaCv/vSX3jHroMFAmQAlrYUHHBib256aW5p
# QHJlZGhhdC5jb20ACgkQv/vSX3jHroONWwf/fxDUMcZUvvatNxiVMhNfqEt/cL0F
# Durv1PmbbeVh9PP0W7XFkEXO3LCIRDyR4rtmCs7gHGdmzDOWQ+QIWgQijQ/y7ElQ
# bTVsvs0+s/6H3csP3dJTJaXSHshbQvrAZTsyk5KcAB6xdL1KqulfLUoGvXJhAmRs
# NKZN8un+nuAhFhL0VBWA9eQaP+BVHQI5ItAj8PaoBby4+Q9fNnat6j1/G4iLly8J
# dxIwCnuRHLiB3melWtadwbv6ddLJFeZNa50HUIsynqoItTzmRVr+oXz1yfq087dB
# 9uksmoqb+icGEdwqs0iYbQ/dhVnIrMDpn/n2Us28S5VdIMVvxr1JEbEkSQ==
# =0jY8
# -----END PGP SIGNATURE-----
# gpg: Signature made Thu 02 Mar 2023 12:29:42 GMT
# gpg: using RSA key F13338574B662389866C7682BFFBD25F78C7AE83
# gpg: issuer "pbonzini@redhat.com"
# gpg: Good signature from "Paolo Bonzini <bonzini@gnu.org>" [full]
# gpg: aka "Paolo Bonzini <pbonzini@redhat.com>" [full]
# Primary key fingerprint: 46F5 9FBD 57D6 12E7 BFD4 E2F7 7E15 100C CD36 69B1
# Subkey fingerprint: F133 3857 4B66 2389 866C 7682 BFFB D25F 78C7 AE83

* tag 'for-upstream' of https://gitlab.com/bonzini/qemu: (62 commits)
Makefile: qemu-bundle is a directory
qapi: Add 'acpi' field to 'query-machines' output
hw/xen: Subsume xen_be_register_common() into xen_be_init()
i386/xen: Document Xen HVM emulation
kvm/i386: Add xen-evtchn-max-pirq property
hw/xen: Support MSI mapping to PIRQ
hw/xen: Support GSI mapping to PIRQ
hw/xen: Implement emulated PIRQ hypercall support
i386/xen: Implement HYPERVISOR_physdev_op
hw/xen: Automatically add xen-platform PCI device for emulated Xen guests
hw/xen: Add basic ring handling to xenstore
hw/xen: Add xen_xenstore device for xenstore emulation
hw/xen: Add backend implementation of interdomain event channel support
i386/xen: handle HVMOP_get_param
i386/xen: Reserve Xen special pages for console, xenstore rings
i386/xen: handle PV timer hypercalls
hw/xen: Implement GNTTABOP_query_size
i386/xen: Implement HYPERVISOR_grant_table_op and GNTTABOP_[gs]et_verson
hw/xen: Support mapping grant frames
hw/xen: Add xen_gnttab device for grant table emulation
...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

Merge branch 'xenfv-kvm-15' of git://git.infradead.org/users/dwmw2/qemu into HEAD

This adds support for emulating Xen under Linux/KVM, based on kernel
patches which have been present since Linux v5.

Merge branch 'xenfv-kvm-15' of git://git.infradead.org/users/dwmw2/qemu into HEAD

This adds support for emulating Xen under Linux/KVM, based on kernel
patches which have been present since Linux v5.12. As with the kernel
support, it's derived from work started by João Martins of Oracle in
2018.

This series just adds the basic platform support — CPUID, hypercalls,
event channels, a stub of XenStore.

A full single-tenant internal implementation of XenStore, and patches
to make QEMU's Xen PV drivers work with this Xen emulation, are waiting
in the wings to be submitted in a follow-on patch series.

As noted in the documentation, it's enabled by setting the xen-version
property on the KVM accelerator, e.g.:

qemu-system-x86_64 -serial mon:stdio -M q35 -display none -m 1G -smp 2 \
-accel kvm,xen-version=0x4000e,kernel-irqchip=split \
-kernel vmlinuz-6.0.7-301.fc37.x86_64 \
-append "console=ttyS0 root=/dev/sda1" \
-drive file=/var/lib/libvirt/images/fedora28.qcow2,if=none,id=disk \
-device ahci,id=ahci -device ide-hd,drive=disk,bus=ahci.0

Even before this was merged, we've already been using it to find and fix
bugs in the Linux kernel Xen guest support:

https://lore.kernel.org/all/4bffa69a949bfdc92c4a18e5a1c3cbb3b94a0d32.camel@infradead.org/
https://lore.kernel.org/all/871qnunycr.ffs@tglx/

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>

show more ...

kvm/i386: Add xen-evtchn-max-pirq property

The default number of PIRQs is set to 256 to avoid issues with 32-bit MSI
devices. Allow it to be increased if the user desires.

Signed-off-by: David Wood

kvm/i386: Add xen-evtchn-max-pirq property

The default number of PIRQs is set to 256 to avoid issues with 32-bit MSI
devices. Allow it to be increased if the user desires.

Signed-off-by: David Woodhouse <dwmw@amazon.co.uk>
Reviewed-by: Paul Durrant <paul@xen.org>

show more ...

i386/xen: Reserve Xen special pages for console, xenstore rings

Xen has eight frames at 0xfeff8000 for this; we only really need two for
now and KVM puts the identity map at 0xfeffc000, so limit our

i386/xen: Reserve Xen special pages for console, xenstore rings

Xen has eight frames at 0xfeff8000 for this; we only really need two for
now and KVM puts the identity map at 0xfeffc000, so limit ourselves to
four.

Signed-off-by: David Woodhouse <dwmw@amazon.co.uk>
Reviewed-by: Paul Durrant <paul@xen.org>

show more ...

kvm/i386: Add xen-gnttab-max-frames property

Signed-off-by: David Woodhouse <dwmw@amazon.co.uk>
Reviewed-by: Paul Durrant <paul@xen.org>

hw/xen: Support HVM_PARAM_CALLBACK_TYPE_GSI callback

The GSI callback (and later PCI_INTX) is a level triggered interrupt. It
is asserted when an event channel is delivered to vCPU0, and is supposed

hw/xen: Support HVM_PARAM_CALLBACK_TYPE_GSI callback

The GSI callback (and later PCI_INTX) is a level triggered interrupt. It
is asserted when an event channel is delivered to vCPU0, and is supposed
to be cleared when the vcpu_info->evtchn_upcall_pending field for vCPU0
is cleared again.

Thankfully, Xen does *not* assert the GSI if the guest sets its own
evtchn_upcall_pending field; we only need to assert the GSI when we
have delivered an event for ourselves. So that's the easy part, kind of.

There's a slight complexity in that we need to hold the BQL before we
can call qemu_set_irq(), and we definitely can't do that while holding
our own port_lock (because we'll need to take that from the qemu-side
functions that the PV backend drivers will call). So if we end up
wanting to set the IRQ in a context where we *don't* already hold the
BQL, defer to a BH.

However, we *do* need to poll for the evtchn_upcall_pending flag being
cleared. In an ideal world we would poll that when the EOI happens on
the PIC/IOAPIC. That's how it works in the kernel with the VFIO eventfd
pairs — one is used to trigger the interrupt, and the other works in the
other direction to 'resample' on EOI, and trigger the first eventfd
again if the line is still active.

However, QEMU doesn't seem to do that. Even VFIO level interrupts seem
to be supported by temporarily unmapping the device's BARs from the
guest when an interrupt happens, then trapping *all* MMIO to the device
and sending the 'resample' event on *every* MMIO access until the IRQ
is cleared! Maybe in future we'll plumb the 'resample' concept through
QEMU's irq framework but for now we'll do what Xen itself does: just
check the flag on every vmexit if the upcall GSI is known to be
asserted.

Signed-off-by: David Woodhouse <dwmw@amazon.co.uk>
Reviewed-by: Paul Durrant <paul@xen.org>

show more ...

hw/xen: Implement EVTCHNOP_bind_virq

Add the array of virq ports to each vCPU so that we can deliver timers,
debug ports, etc. Global virqs are allocated against vCPU 0 initially,
but can be migrate

hw/xen: Implement EVTCHNOP_bind_virq

Add the array of virq ports to each vCPU so that we can deliver timers,
debug ports, etc. Global virqs are allocated against vCPU 0 initially,
but can be migrated to other vCPUs (when we implement that).

The kernel needs to know about VIRQ_TIMER in order to accelerate timers,
so tell it via KVM_XEN_VCPU_ATTR_TYPE_TIMER. Also save/restore the value
of the singleshot timer across migration, as the kernel will handle the
hypercalls automatically now.

Signed-off-by: David Woodhouse <dwmw@amazon.co.uk>
Reviewed-by: Paul Durrant <paul@xen.org>

show more ...

i386/xen: Add support for Xen event channel delivery to vCPU

The kvm_xen_inject_vcpu_callback_vector() function will either deliver
the per-vCPU local APIC vector (as an MSI), or just kick the vCPU

i386/xen: Add support for Xen event channel delivery to vCPU

The kvm_xen_inject_vcpu_callback_vector() function will either deliver
the per-vCPU local APIC vector (as an MSI), or just kick the vCPU out
of the kernel to trigger KVM's automatic delivery of the global vector.
Support for asserting the GSI/PCI_INTX callbacks will come later.

Also add kvm_xen_get_vcpu_info_hva() which returns the vcpu_info of
a given vCPU.

Signed-off-by: David Woodhouse <dwmw@amazon.co.uk>
Reviewed-by: Paul Durrant <paul@xen.org>

show more ...

hw/xen: Add xen_overlay device for emulating shared xenheap pages

For the shared info page and for grant tables, Xen shares its own pages
from the "Xen heap" to the guest. The guest requests that a

hw/xen: Add xen_overlay device for emulating shared xenheap pages

For the shared info page and for grant tables, Xen shares its own pages
from the "Xen heap" to the guest. The guest requests that a given page
from a certain address space (XENMAPSPACE_shared_info, etc.) be mapped
to a given GPA using the XENMEM_add_to_physmap hypercall.

To support that in qemu when *emulating* Xen, create a memory region
(migratable) and allow it to be mapped as an overlay when requested.

Xen theoretically allows the same page to be mapped multiple times
into the guest, but that's hard to track and reinstate over migration,
so we automatically *unmap* any previous mapping when creating a new
one. This approach has been used in production with.... a non-trivial
number of guests expecting true Xen, without any problems yet being
noticed.

This adds just the shared info page for now. The grant tables will be
a larger region, and will need to be overlaid one page at a time. I
think that means I need to create separate aliases for each page of
the overall grant_frames region, so that they can be mapped individually.

Signed-off-by: David Woodhouse <dwmw@amazon.co.uk>
Reviewed-by: Paul Durrant <paul@xen.org>

show more ...

i386/xen: implement HYPERVISOR_sched_op, SCHEDOP_shutdown

It allows to shutdown itself via hypercall with any of the 3 reasons:
1) self-reboot
2) shutdown
3) crash

Implementing SCHEDOP_shutdo

i386/xen: implement HYPERVISOR_sched_op, SCHEDOP_shutdown

It allows to shutdown itself via hypercall with any of the 3 reasons:
1) self-reboot
2) shutdown
3) crash

Implementing SCHEDOP_shutdown sub op let us handle crashes gracefully rather
than leading to triple faults if it remains unimplemented.

In addition, the SHUTDOWN_soft_reset reason is used for kexec, to reset
Xen shared pages and other enlightenments and leave a clean slate for the
new kernel without the hypervisor helpfully writing information at
unexpected addresses.

Signed-off-by: Joao Martins <joao.m.martins@oracle.com>
[dwmw2: Ditch sched_op_compat which was never available for HVM guests,
Add SCHEDOP_soft_reset]
Signed-off-by: David Woodhouse <dwmw@amazon.co.uk>
Reviewed-by: Paul Durrant <paul@xen.org>

show more ...

i386/kvm: Add xen-version KVM accelerator property and init KVM Xen support

This just initializes the basic Xen support in KVM for now. Only permitted
on TYPE_PC_MACHINE because that's where the sys

i386/kvm: Add xen-version KVM accelerator property and init KVM Xen support

This just initializes the basic Xen support in KVM for now. Only permitted
on TYPE_PC_MACHINE because that's where the sysbus devices for Xen heap
overlay, event channel, grant tables and other stuff will exist. There's
no point having the basic hypercall support if nothing else works.

Provide sysemu/kvm_xen.h and a kvm_xen_get_caps() which will be used
later by support devices.

Signed-off-by: David Woodhouse <dwmw@amazon.co.uk>
Reviewed-by: Paul Durrant <paul@xen.org>

show more ...