Merge tag 'for-upstream' of https://gitlab.com/bonzini/qemu into staging

* kvm: enable dirty ring for arm64
* target/i386: new features
* target/i386: AVX fixes
* configure: create a python venv unc

Merge tag 'for-upstream' of https://gitlab.com/bonzini/qemu into staging

* kvm: enable dirty ring for arm64
* target/i386: new features
* target/i386: AVX fixes
* configure: create a python venv unconditionally
* meson: bump to 0.63.0 and move tests from configure
* meson: Pass -j option to sphinx
* drop support for Python 3.6
* fix check-python-tox
* fix "make clean" in the source directory

# -----BEGIN PGP SIGNATURE-----
#
# iQFIBAABCAAyFiEE8TM4V0tmI4mGbHaCv/vSX3jHroMFAmRmDYQUHHBib256aW5p
# QHJlZGhhdC5jb20ACgkQv/vSX3jHroOXSwf/WKmYPe09yHfxfVSFsSz83QpB3e+f
# KJx6FdyMMt26ZQJpcqorobrDV23R8FyxngXPkwoxqobAEtXB/AH0/S/u8RUZ46Qt
# IrF8FXr4ZdyLW7CW6nmIejmlul0iRmFD7D98E6dZ3QXfype3Ifra7gG74spZ1B44
# ZNvaomJKUK8Ga8rbChs9KtgrxlOC5q8IfTWF5ZExmZszPC9NRnZmU5Oncnuwek9T
# Ic6zDPoAeF3jDtovZhxg1HAB9e/ENZX/V9NjO92yZa8u/TITQ88l4tJctf7uiLxO
# 2oGY12ln8i//pbjyUe4iM+bNh5+reAChEI8iv7WxEsj9s2HBUJ68f3tpbQ==
# =Zg00
# -----END PGP SIGNATURE-----
# gpg: Signature made Thu 18 May 2023 04:35:32 AM PDT
# gpg: using RSA key F13338574B662389866C7682BFFBD25F78C7AE83
# gpg: issuer "pbonzini@redhat.com"
# gpg: Good signature from "Paolo Bonzini <bonzini@gnu.org>" [undefined]
# gpg: aka "Paolo Bonzini <pbonzini@redhat.com>" [undefined]
# gpg: WARNING: This key is not certified with a trusted signature!
# gpg: There is no indication that the signature belongs to the owner.
# Primary key fingerprint: 46F5 9FBD 57D6 12E7 BFD4 E2F7 7E15 100C CD36 69B1
# Subkey fingerprint: F133 3857 4B66 2389 866C 7682 BFFB D25F 78C7 AE83

* tag 'for-upstream' of https://gitlab.com/bonzini/qemu: (68 commits)
docs/devel: update build system docs
configure: remove unnecessary check
configure: reorder option parsing code
configure: remove unnecessary mkdir
configure: do not rerun the tests with -Werror
configure: remove compiler sanity check
build: move --disable-debug-info to meson
build: move compiler version check to meson
build: move remaining compiler flag tests to meson
build: move warning flag selection to meson
build: move stack protector flag selection to meson
build: move coroutine backend selection to meson
build: move SafeStack tests to meson
build: move sanitizer tests to meson
meson: prepare move of QEMU_CFLAGS to meson
configure, meson: move --enable-modules to Meson
configure: remove pkg-config functions
build: move glib detection and workarounds to meson
meson: drop unnecessary declare_dependency()
meson: add more version numbers to the summary
...

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>

show more ...

configure: remove unnecessary check

All calls to probe_target_compiler are conditioned on
some "have_target" invocation, or inside a loop on target_list.
Therefore there is no issue with building un

configure: remove unnecessary check

All calls to probe_target_compiler are conditioned on
some "have_target" invocation, or inside a loop on target_list.
Therefore there is no issue with building unnecessary
firmware images and tests.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>

show more ...

configure: reorder option parsing code

Move some variable assignments around for clarity and to remove
one of three loops on the command line arguments.

Signed-off-by: Paolo Bonzini <pbonzini@redha

configure: reorder option parsing code

Move some variable assignments around for clarity and to remove
one of three loops on the command line arguments.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>

show more ...

configure: remove unnecessary mkdir

It is taken care of by the symlink shell function.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>

configure: do not rerun the tests with -Werror

Tests run in configure are pretty trivial at this point, so
do not bother with the extra complication of running tests
both with and without -Werror.

configure: do not rerun the tests with -Werror

Tests run in configure are pretty trivial at this point, so
do not bother with the extra complication of running tests
both with and without -Werror.

Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>

show more ...

configure: remove compiler sanity check

The comment is not correct anymore, in that the usability test for
the compiler and linker are done after probing $cpu, and Meson will
redo them anyway.

Sign

configure: remove compiler sanity check

The comment is not correct anymore, in that the usability test for
the compiler and linker are done after probing $cpu, and Meson will
redo them anyway.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>

show more ...

build: move --disable-debug-info to meson

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>

build: move compiler version check to meson

Use the slighly nicer .version_compare() function for GCC; for Clang that is
not possible due to the mess that Apple does with version numbers.

Signed-of

build: move compiler version check to meson

Use the slighly nicer .version_compare() function for GCC; for Clang that is
not possible due to the mess that Apple does with version numbers.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>

show more ...

build: move remaining compiler flag tests to meson

Remove the only remaining uses of QEMU_CFLAGS. Now that no
feature tests are done in configure, it is possible to remove
CONFIGURE_CFLAGS and CONF

build: move remaining compiler flag tests to meson

Remove the only remaining uses of QEMU_CFLAGS. Now that no
feature tests are done in configure, it is possible to remove
CONFIGURE_CFLAGS and CONFIGURE_LDFLAGS as well.

Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>

show more ...

build: move warning flag selection to meson

Meson already knows to test with the positive form of the flag, which
simplifies the test. Warnings are now tested explicitly for the C++
compiler, inste

build: move warning flag selection to meson

Meson already knows to test with the positive form of the flag, which
simplifies the test. Warnings are now tested explicitly for the C++
compiler, instead of hardcoding those that are only available for
the C language.

At this point all compiler flags in QEMU_CFLAGS are global and only
depend on the OS. No feature tests are performed in configure.

Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>

show more ...

build: move stack protector flag selection to meson

Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>

build: move coroutine backend selection to meson

To simplify the code, rename coroutine-win32.c to match the option
passed to configure.

Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>

build: move coroutine backend selection to meson

To simplify the code, rename coroutine-win32.c to match the option
passed to configure.

Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>

show more ...

build: move SafeStack tests to meson

This disables the old behavior of detecting SafeStack from environment
CFLAGS. SafeStack is now enabled purely based on the configure arguments.

Reviewed-by: P

build: move SafeStack tests to meson

This disables the old behavior of detecting SafeStack from environment
CFLAGS. SafeStack is now enabled purely based on the configure arguments.

Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>

show more ...

build: move sanitizer tests to meson

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>

configure, meson: move --enable-modules to Meson

Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>

configure: remove pkg-config functions

All uses of pkg-config have been moved to Meson.

Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.co

configure: remove pkg-config functions

All uses of pkg-config have been moved to Meson.

Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>

show more ...

build: move glib detection and workarounds to meson

QEMU adds the path to glib.h to all compilation commands. This is simpler
due to the pervasive use of static_library, and was grandfathered in fr

build: move glib detection and workarounds to meson

QEMU adds the path to glib.h to all compilation commands. This is simpler
due to the pervasive use of static_library, and was grandfathered in from
the previous Make-based build system. Until Meson 0.63 the only way to
do this was to detect glib in configure and use add_project_arguments,
but now it is possible to use add_project_dependencies instead.

gmodule is detected in a separate variable, with export enabled for
modules and disabled for plugin.

Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>

show more ...

meson: use prefer_static option

The option is new in Meson 0.63 and removes the need to pass "static:
true" to all dependency and find_library invocation. Actually cleaning
up the invocations is le

meson: use prefer_static option

The option is new in Meson 0.63 and removes the need to pass "static:
true" to all dependency and find_library invocation. Actually cleaning
up the invocations is left for a separate patch.

Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>

show more ...

meson: require 0.63.0

This version allows cleanups in modinfo collection, but they only
work with Ninja 1.9.x and 1.8.x is still supported. It also supports the
equivalent of QEMU's --static option

meson: require 0.63.0

This version allows cleanups in modinfo collection, but they only
work with Ninja 1.9.x and 1.8.x is still supported. It also supports the
equivalent of QEMU's --static option to configure.

The wheel file is bumped to 0.63.3, the last release in the 0.63 branch.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>

show more ...

configure: Add courtesy hint to Python version failure message

If we begin requiring Python 3.7+, a few platforms are going to need to
install an additional Python interpreter package.

As a courtes

configure: Add courtesy hint to Python version failure message

If we begin requiring Python 3.7+, a few platforms are going to need to
install an additional Python interpreter package.

As a courtesy to the user, suggest the optional package they might need
to install. This will hopefully minimize any downtime caused by the
change in Python dependency.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-Id: <20230221012456.2607692-3-jsnow@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20230511035435.734312-25-jsnow@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>

show more ...

Python: Drop support for Python 3.6

Python 3.6 was EOL 2021-12-31. Newer versions of upstream libraries have
begun dropping support for this version and it is becoming more
cumbersome to support. Av

Python: Drop support for Python 3.6

Python 3.6 was EOL 2021-12-31. Newer versions of upstream libraries have
begun dropping support for this version and it is becoming more
cumbersome to support. Avocado-framework and qemu.qmp each have their
own reasons for wanting to drop Python 3.6, but won't until QEMU does.

Versions of Python available in our supported build platforms as of today,
with optional versions available in parentheses:

openSUSE Leap 15.4: 3.6.15 (3.9.10, 3.10.2)
CentOS Stream 8: 3.6.8 (3.8.13, 3.9.16)
CentOS Stream 9: 3.9.13
Fedora 36: 3.10
Fedora 37: 3.11
Debian 11: 3.9.2
Alpine 3.14, 3.15: 3.9.16
Alpine 3.16, 3.17: 3.10.10
Ubuntu 20.04 LTS: 3.8.10
Ubuntu 22.04 LTS: 3.10.4
NetBSD 9.3: 3.9.13*
FreeBSD 12.4: 3.9.16
FreeBSD 13.1: 3.9.16
OpenBSD 7.2: 3.9.16

Note: Our VM tests install 3.9 explicitly for FreeBSD and 3.10 for
NetBSD; the default for "python" or "python3" in FreeBSD is
3.9.16. NetBSD does not appear to have a default meta-package, but
offers several options, the lowest of which is 3.7.15. "python39"
appears to be a pre-requisite to one of the other packages we request in
tests/vm/netbsd. pip, ensurepip and other Python essentials are
currently only available for Python 3.10 for NetBSD.

CentOS and OpenSUSE support parallel installation of multiple Python
interpreters, and binaries in /usr/bin will always use Python 3.6. However,
the newly introduced support for virtual environments ensures that all build
steps that execute QEMU Python code use a single interpreter.

Since it is safe to under our supported platform policy, bump our
minimum supported version of Python to 3.7.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20230511035435.734312-24-jsnow@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>

show more ...

configure: add --enable-pypi and --disable-pypi

In the event that there's no vendored source present and no sufficient
version of $package can be found, we will attempt to connect to PyPI to
install

configure: add --enable-pypi and --disable-pypi

In the event that there's no vendored source present and no sufficient
version of $package can be found, we will attempt to connect to PyPI to
install the package if '--disable-pypi' was not passed.

This means that PyPI access is "enabled by default", but there are some
subtleties that make this action occur much less frequently than you
might imagine:

(1) While --enable-pypi is the default, vendored source will always be
preferred when found, making PyPI a fallback. This should ensure
that configure-time venv building "just works" for almost everyone
in almost every circumstance.

(2) Because meson source is, at time of writing, vendored directly into
qemu.git, PyPI will never be used for sourcing meson.

(3) Because Sphinx is an optional dependency, if docs are set to "auto",
PyPI will not be used to obtain Sphinx source as a fallback and
instead docs will be disabled. If PyPI sourcing of sphinx is
desired, --enable-docs should be passed to force the lookup. I chose
this as the default behavior to avoid adding new internet lookups to
a "default" invocation of configure.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20230511035435.734312-23-jsnow@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>

show more ...

configure: bootstrap sphinx with mkvenv

When docs are explicitly requested, require Sphinx>=1.6.0. When docs are
explicitly disabled, don't bother to check for Sphinx at all. If docs
are set to "aut

configure: bootstrap sphinx with mkvenv

When docs are explicitly requested, require Sphinx>=1.6.0. When docs are
explicitly disabled, don't bother to check for Sphinx at all. If docs
are set to "auto", attempt to locate Sphinx, but continue onward if it
wasn't located.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20230511035435.734312-22-jsnow@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>

show more ...

configure: move --enable-docs and --disable-docs back to configure

Move this option back from meson into configure for the purposes of
using the configuration value to bootstrap Sphinx in different

configure: move --enable-docs and --disable-docs back to configure

Move this option back from meson into configure for the purposes of
using the configuration value to bootstrap Sphinx in different ways
based on this value.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20230511035435.734312-21-jsnow@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>

show more ...

configure: use 'mkvenv ensure meson' to bootstrap meson

This commit changes how we detect and install meson. It notably removes
'--meson='.

Currently, configure creates a lightweight Python virtual

configure: use 'mkvenv ensure meson' to bootstrap meson

This commit changes how we detect and install meson. It notably removes
'--meson='.

Currently, configure creates a lightweight Python virtual environment
unconditionally using the user's configured $python that inherits system
packages. Temporarily, we forced the use of meson source present via git
submodule or in the release tarball.

With this patch, we restore the ability to use a system-provided meson:

If Meson is installed in the build venv and meets our minimum version
requirements, we will use that Meson. This includes a system provided
meson, which would be visible via system-site packages inside the venv.

In the event that Meson is installed but *not for the chosen Python
interpreter*, not found, or of insufficient version, we will attempt to
install Meson from vendored source into the newly created Python virtual
environment. This vendored installation replaces both the git submodule
and tarball source mechanisms for sourcing meson.

As a result of this patch, the Python interpreter we use for both our
own build scripts *and* Meson extensions are always known to be the
exact same Python. As a further benefit, there will also be a symlink
available in the build directory that points to the correct, configured
python and can be used by e.g. manual tests to invoke the correct,
configured Python unambiguously.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20230511035435.734312-18-jsnow@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>

show more ...