scripts: adjust url to Coverity tools

The URL to the Coverity tools download has changed; the old one points
to an obsolete version that is not supported anymore. Adjust to point
to the correct and

scripts: adjust url to Coverity tools

The URL to the Coverity tools download has changed; the old one points
to an obsolete version that is not supported anymore. Adjust to point
to the correct and supported tools.

Suggested-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>

show more ...

coverity: physmem: use simple assertions instead of modelling

Unfortunately Coverity doesn't follow the logic aroung "len" and "l"
variables in stacks finishing with flatview_{read,write}_continue()

coverity: physmem: use simple assertions instead of modelling

Unfortunately Coverity doesn't follow the logic aroung "len" and "l"
variables in stacks finishing with flatview_{read,write}_continue() and
generate a lot of OVERRUN false-positives. When small buffer (2 or 4
bytes) is passed to mem read/write path, Coverity assumes the worst
case of sz=8 in stn_he_p()/ldn_he_p() (defined in
include/qemu/bswap.h), and reports buffer overrun.

To silence these false-positives we have model functions, which hide
real logic from Coverity.

However, it turned out that these new two assertions are enough to
quiet Coverity.

Assertions are better than hiding the logic, so let's drop the
modelling and move to assertions for memory r/w call stacks.

After patch, the sequence

cov-make-library --output-file /tmp/master.xmldb \
scripts/coverity-scan/model.c
cov-build --dir ~/covtmp/master make -j9
cov-analyze --user-model-file /tmp/master.xmldb \
--dir ~/covtmp/master --all --strip-path "$(pwd)
cov-format-errors --dir ~/covtmp/master \
--html-output ~/covtmp/master_html_report

Generate for me the same big set of CIDs excepept for 6 disappeared (so
it becomes even better).

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
Acked-by: David Hildenbrand <david@redhat.com>
Message-ID: <20231005140326.332830-1-vsementsov@yandex-team.ru>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>

show more ...

block: Fix deadlocks in bdrv_graph_wrunlock()

bdrv_graph_wrunlock() calls aio_poll(), which may run callbacks that
have a nested event loop. Nested event loops can depend on other
iothreads making p

block: Fix deadlocks in bdrv_graph_wrunlock()

bdrv_graph_wrunlock() calls aio_poll(), which may run callbacks that
have a nested event loop. Nested event loops can depend on other
iothreads making progress, so in order to allow them to make progress it
must not hold the AioContext lock of another thread while calling
aio_poll().

This introduces a @bs parameter to bdrv_graph_wrunlock() whose
AioContext is temporarily dropped (which matches bdrv_graph_wrlock()),
and a bdrv_graph_wrunlock_ctx() that can be used if the BlockDriverState
doesn't necessarily exist any more when unlocking.

This also requires a change to bdrv_schedule_unref(), which was relying
on the incorrectly taken lock. It needs to take the lock itself now.
While this is a separate bug, it can't be fixed a separate patch because
otherwise the intermediate state would either deadlock or try to release
a lock that we don't even hold.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Message-ID: <20231115172012.112727-3-kwolf@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
[kwolf: Fixed up bdrv_schedule_unref()]
Signed-off-by: Kevin Wolf <kwolf@redhat.com>

show more ...

qapi: Fix QAPISchemaEntity.__repr__()

I messed it up on merge. It's a debugging aid, so no impact on build.

Fixes: e307a8174bb8 (qapi: provide a friendly string representation of QAPI classes)
Sig

qapi: Fix QAPISchemaEntity.__repr__()

I messed it up on merge. It's a debugging aid, so no impact on build.

Fixes: e307a8174bb8 (qapi: provide a friendly string representation of QAPI classes)
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-ID: <20231024104841.1569250-1-armbru@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>

show more ...

tracetool: avoid invalid escape in Python string

This is an error in Python 3.12; fix it by using a raw string literal.

Cc: <qemu-stable@nongnu.org>
Signed-off-by: Marc-André Lureau <marcandre.lur

tracetool: avoid invalid escape in Python string

This is an error in Python 3.12; fix it by using a raw string literal.

Cc: <qemu-stable@nongnu.org>
Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-ID: <20231108105649.60453-1-marcandre.lureau@redhat.com>

show more ...

gdbstub: Add num_regs member to GDBFeature

Currently the number of registers exposed to GDB is written as magic
numbers in code. Derive the number of registers GDB actually see from
XML files to rep

gdbstub: Add num_regs member to GDBFeature

Currently the number of registers exposed to GDB is written as magic
numbers in code. Derive the number of registers GDB actually see from
XML files to replace the magic numbers in code later.

Signed-off-by: Akihiko Odaki <akihiko.odaki@daynix.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20231025093128.33116-2-akihiko.odaki@daynix.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20231106185112.2755262-9-alex.bennee@linaro.org>

show more ...

scripts/cpu-x86-uarch-abi.py: Fix parameter error of cmd

When run this script, there's the error:

python3 scripts/cpu-x86-uarch-abi.py /tmp/qmp
Traceback (most recent call last):
File "/path-to-q

scripts/cpu-x86-uarch-abi.py: Fix parameter error of cmd

When run this script, there's the error:

python3 scripts/cpu-x86-uarch-abi.py /tmp/qmp
Traceback (most recent call last):
File "/path-to-qemu/qemu/scripts/cpu-x86-uarch-abi.py", line 96, in <module>
cpu = shell.cmd("query-cpu-model-expansion",
TypeError: QEMUMonitorProtocol.cmd() takes 2 positional arguments but 3 were given

Commit 7f521b023bc28 ("scripts/cpu-x86-uarch-abi.py: use .command()
instead of .cmd()") converts the the original .cmd() to .command()
(which was later renamed to "cmd" to replace the original one).

But the new .cmd() only accepts typing.Mapping as the parameter instead
of typing.Dict (see _qmp.execute()).

Change the paremeters of "query-cpu-model-expansion" to typing.Mapping
format to fix this error.

Fixes: 7f521b023bc28 ("scripts/cpu-x86-uarch-abi.py: use .command() instead of .cmd()")

Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
Signed-off-by: Zhao Liu <zhao1.liu@intel.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>

show more ...

build-sys: add a "pixman" feature

For now, pixman is mandatory, but we set config_host.h and Kconfig.
Once compilation is fixed, "pixman" will become actually optional.

Signed-off-by: Marc-André Lu

build-sys: add a "pixman" feature

For now, pixman is mandatory, but we set config_host.h and Kconfig.
Once compilation is fixed, "pixman" will become actually optional.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>

show more ...

Add Hyper-V Dynamic Memory Protocol driver (hv-balloon) base

This driver is like virtio-balloon on steroids: it allows both changing the
guest memory allocation via ballooning and (in the next patch

Add Hyper-V Dynamic Memory Protocol driver (hv-balloon) base

This driver is like virtio-balloon on steroids: it allows both changing the
guest memory allocation via ballooning and (in the next patch) inserting
pieces of extra RAM into it on demand from a provided memory backend.

The actual resizing is done via ballooning interface (for example, via
the "balloon" HMP command).
This includes resizing the guest past its boot size - that is, hot-adding
additional memory in granularity limited only by the guest alignment
requirements, as provided by the next patch.

In contrast with ACPI DIMM hotplug where one can only request to unplug a
whole DIMM stick this driver allows removing memory from guest in single
page (4k) units via ballooning.

After a VM reboot the guest is back to its original (boot) size.

In the future, the guest boot memory size might be changed on reboot
instead, taking into account the effective size that VM had before that
reboot (much like Hyper-V does).

For performance reasons, the guest-released memory is tracked in a few
range trees, as a series of (start, count) ranges.
Each time a new page range is inserted into such tree its neighbors are
checked as candidates for possible merging with it.

Besides performance reasons, the Dynamic Memory protocol itself uses page
ranges as the data structure in its messages, so relevant pages need to be
merged into such ranges anyway.

One has to be careful when tracking the guest-released pages, since the
guest can maliciously report returning pages outside its current address
space, which later clash with the address range of newly added memory.
Similarly, the guest can report freeing the same page twice.

The above design results in much better ballooning performance than when
using virtio-balloon with the same guest: 230 GB / minute with this driver
versus 70 GB / minute with virtio-balloon.

During a ballooning operation most of time is spent waiting for the guest
to come up with newly freed page ranges, processing the received ranges on
the host side (in QEMU and KVM) is nearly instantaneous.

The unballoon operation is also pretty much instantaneous:
thanks to the merging of the ballooned out page ranges 200 GB of memory can
be returned to the guest in about 1 second.
With virtio-balloon this operation takes about 2.5 minutes.

These tests were done against a Windows Server 2019 guest running on a
Xeon E5-2699, after dirtying the whole memory inside guest before each
balloon operation.

Using a range tree instead of a bitmap to track the removed memory also
means that the solution scales well with the guest size: even a 1 TB range
takes just a few bytes of such metadata.

Since the required GTree operations aren't present in every Glib version
a check for them was added to the meson build script, together with new
"--enable-hv-balloon" and "--disable-hv-balloon" configure arguments.
If these GTree operations are missing in the system's Glib version this
driver will be skipped during QEMU build.

An optional "status-report=on" device parameter requests memory status
events from the guest (typically sent every second), which allow the host
to learn both the guest memory available and the guest memory in use
counts.

Following commits will add support for their external emission as
"HV_BALLOON_STATUS_REPORT" QMP events.

The driver is named hv-balloon since the Linux kernel client driver for
the Dynamic Memory Protocol is named as such and to follow the naming
pattern established by the virtio-balloon driver.
The whole protocol runs over Hyper-V VMBus.

The driver was tested against Windows Server 2012 R2, Windows Server 2016
and Windows Server 2019 guests and obeys the guest alignment requirements
reported to the host via DM_CAPABILITIES_REPORT message.

Acked-by: David Hildenbrand <david@redhat.com>
Signed-off-by: Maciej S. Szmigiero <maciej.szmigiero@oracle.com>

show more ...

qapi: provide a friendly string representation of QAPI classes

If printing a QAPI schema object for debugging we get the classname and
a hex value for the instance:

<qapi.schema.QAPISchemaEnumTyp

qapi: provide a friendly string representation of QAPI classes

If printing a QAPI schema object for debugging we get the classname and
a hex value for the instance:

<qapi.schema.QAPISchemaEnumType object at 0x7f0ab4c2dad0>
<qapi.schema.QAPISchemaObjectType object at 0x7f0ab4c2dd90>
<qapi.schema.QAPISchemaArrayType object at 0x7f0ab4c2df90>

With this change we instead get the classname and the human friendly
name of the QAPI type instance:

<QAPISchemaEnumType:CpuS390State at 0x7f0ab4c2dad0>
<QAPISchemaObjectType:CpuInfoS390 at 0x7f0ab4c2dd90>
<QAPISchemaArrayType:CpuInfoFastList at 0x7f0ab4c2df90>

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-ID: <20231018120500.2028642-1-berrange@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
[Conditional swapped to avoid negation]
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
[Tweaked to mollify pylint]
Signed-off-by: Markus Armbruster <armbru@redhat.com>

show more ...

qapi: re-establish linting baseline

Some very minor housekeeping to make the linters happy once more.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-ID: <20231004230532.3002201-4-jsnow@redhat.

qapi: re-establish linting baseline

Some very minor housekeeping to make the linters happy once more.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-ID: <20231004230532.3002201-4-jsnow@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Markus Armbruster <armbru@redhat.com>

show more ...

scripts/update-linux-headers: Add iommufd.h

Update the script to import iommufd.h

Signed-off-by: Eric Auger <eric.auger@redhat.com>
Signed-off-by: Yi Liu <yi.l.liu@intel.com>
Signed-off-by: Zhenzho

scripts/update-linux-headers: Add iommufd.h

Update the script to import iommufd.h

Signed-off-by: Eric Auger <eric.auger@redhat.com>
Signed-off-by: Yi Liu <yi.l.liu@intel.com>
Signed-off-by: Zhenzhong Duan <zhenzhong.duan@intel.com>
Reviewed-by: Cédric Le Goater <clg@redhat.com>
Signed-off-by: Cédric Le Goater <clg@redhat.com>

show more ...

meson-buildoptions: document the data at the top

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>

configure, meson: use command line options to configure qemu-ga

Preserve the functionality of the environment variables, but
allow using the command line instead.

Signed-off-by: Paolo Bonzini <pbon

configure, meson: use command line options to configure qemu-ga

Preserve the functionality of the environment variables, but
allow using the command line instead.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>

show more ...

configure: move environment-specific defaults to config-meson.cross

Store the -Werror and SMBD defaults in the machine file, which still allows
them to be overridden on the command line and enables

configure: move environment-specific defaults to config-meson.cross

Store the -Werror and SMBD defaults in the machine file, which still allows
them to be overridden on the command line and enables automatic parsing
of the related options.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>

show more ...

configure: move target-specific defaults to an external machine file

Enable Windows-specific defaults with a machine file, so that related
options can be automatically parsed and included in the hel

configure: move target-specific defaults to an external machine file

Enable Windows-specific defaults with a machine file, so that related
options can be automatically parsed and included in the help message.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>

show more ...

configure: clean up plugin option handling

Keep together all the conditions that lead to disabling plugins, and
remove now-dead code.

Since the option was not in SKIP_OPTIONS, it was present twice

configure: clean up plugin option handling

Keep together all the conditions that lead to disabling plugins, and
remove now-dead code.

Since the option was not in SKIP_OPTIONS, it was present twice in
the help message, both from configure and from meson-buildoptions.sh.
Remove the duplication and take the occasion to document the option as
autodetected, which it is.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>

show more ...

meson, cutils: allow non-relocatable installs

Say QEMU is configured with bindir = "/usr/bin" and a firmware path
that starts with "/usr/share/qemu". Ever since QEMU 5.2, QEMU's
install has been re

meson, cutils: allow non-relocatable installs

Say QEMU is configured with bindir = "/usr/bin" and a firmware path
that starts with "/usr/share/qemu". Ever since QEMU 5.2, QEMU's
install has been relocatable: if you move qemu-system-x86_64 from
/usr/bin to /home/username/bin, it will start looking for firmware in
/home/username/share/qemu. Previously, you would get a non-relocatable
install where the moved QEMU will keep looking for firmware in
/usr/share/qemu.

Windows almost always wants relocatable installs, and in fact that
is why QEMU 5.2 introduced relocatability in the first place.
However, newfangled distribution mechanisms such as AppImage
(https://docs.appimage.org/reference/best-practices.html), and
possibly NixOS, also dislike using at runtime the absolute paths
that were established at build time.

On POSIX systems you almost never care; if you do, your usecase
dictates which one is desirable, so there's no single answer.
Obviously relocatability works fine most of the time, because not many
people have complained about QEMU's switch to relocatable install,
and that's why until now there was no way to disable relocatability.

But a non-relocatable, non-modular binary can help if you want to do
experiments with old firmware and new QEMU or vice versa (because you
can just upgrade/downgrade the firmware package, and use rpm2cpio or
similar to extract the QEMU binaries outside /usr), so allow both.
This patch allows one to build a non-relocatable install using a new
option to configure. Why? Because it's not too hard, and because
it helps the user double check the relocatability of their install.

Note that the same code that handles relocation also lets you run QEMU
from the build tree and pick e.g. firmware files from the source tree
transparently. Therefore that part remains active with this patch,
even if you configure with --disable-relocatable.

Suggested-by: Michael Tokarev <mjt@tls.msk.ru>
Reviewed-by: Emmanouil Pitsidianakis <manos.pitsidianakis@linaro.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>

show more ...

tracetool: avoid invalid escape in Python string

This is an error in Python 3.12; fix it by using a raw string literal.

Cc: qemu-stable@nongnu.org
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>

scripts/get_maintainer.pl: don't print parentheses

When called from git-send-email, some results contain unclosed
parentheses from the subsystem title, for example:

(cc-cmd) Adding cc: qemu-ppc

scripts/get_maintainer.pl: don't print parentheses

When called from git-send-email, some results contain unclosed
parentheses from the subsystem title, for example:

(cc-cmd) Adding cc: qemu-ppc@nongnu.org (open list:PowerNV (Non-Virt...) from: 'scripts/get_maintainer.pl --nogit-fallback'
(cc-cmd) Adding cc: qemu-devel@nongnu.org (open list:All patches CC here) from: 'scripts/get_maintainer.pl --nogit-fallback'
Unmatched () '(open list:PowerNV (Non-Virt...)' '' at /usr/lib/git-core/git-send-email line 642.
error: unable to extract a valid address from: qemu-ppc@nongnu.org (open list:PowerNV (Non-Virt...)
What to do with this address? ([q]uit|[d]rop|[e]dit): d

This commit removes all parentheses from results.

Signed-off-by: Emmanouil Pitsidianakis <manos.pitsidianakis@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-ID: <20231013091628.669415-1-manos.pitsidianakis@linaro.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>

show more ...

scripts: Mark feature_to_c.py as non-executable to fix a build issue

Meson tries to run scripts via the shebang line if they files are
marked as executable. If "python3" is not in the $PATH, or if i

scripts: Mark feature_to_c.py as non-executable to fix a build issue

Meson tries to run scripts via the shebang line if they files are
marked as executable. If "python3" is not in the $PATH, or if it
is a version that is too old, then the script execution fails.
We should make sure to run scripts via the python3 interpreter
that is used for Meson itself. For this, the files need to be marked
as non-executable, then meson will use the python3 binary that has
been used to run itself.

Fixes: 956af7daad ("gdbstub: Introduce GDBFeature structure")
Signed-off-by: Thomas Huth <thuth@redhat.com>
Message-ID: <20231016094917.19044-1-thuth@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>

show more ...

migration: Fix analyze-migration read operation signedness

The migration code uses unsigned values for 16, 32 and 64-bit
operations. Fix the script to do the same.

This was causing an issue when pa

migration: Fix analyze-migration read operation signedness

The migration code uses unsigned values for 16, 32 and 64-bit
operations. Fix the script to do the same.

This was causing an issue when parsing the migration stream generated
on the ppc64 target because one of instance_ids was larger than the
32bit signed maximum:

Traceback (most recent call last):
File "/home/fabiano/kvm/qemu/build/scripts/analyze-migration.py", line 658, in <module>
dump.read(dump_memory = args.memory)
File "/home/fabiano/kvm/qemu/build/scripts/analyze-migration.py", line 592, in read
classdesc = self.section_classes[section_key]
KeyError: ('spapr_iommu', -2147483648)

Signed-off-by: Fabiano Rosas <farosas@suse.de>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>
Message-ID: <20231009184326.15777-6-farosas@suse.de>

show more ...

migration: Fix analyze-migration.py when ignore-shared is used

The script is currently broken when the x-ignore-shared capability is
used:

Traceback (most recent call last):
File "./scripts/analy

migration: Fix analyze-migration.py when ignore-shared is used

The script is currently broken when the x-ignore-shared capability is
used:

Traceback (most recent call last):
File "./scripts/analyze-migration.py", line 656, in <module>
dump.read(dump_memory = args.memory)
File "./scripts/analyze-migration.py", line 593, in read
section.read()
File "./scripts/analyze-migration.py", line 163, in read
self.name = self.file.readstr(len = namelen)
File "./scripts/analyze-migration.py", line 53, in readstr
return self.readvar(len).decode('utf-8')
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x82 in position 55: invalid start byte

We're currently adding data to the middle of the ram section depending
on the presence of the capability. As a consequence, any code loading
the ram section needs to know about capabilities so it can interpret
the stream.

Skip the byte that's added when x-ignore-shared is used to fix the
script.

Signed-off-by: Fabiano Rosas <farosas@suse.de>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>
Message-ID: <20231009184326.15777-5-farosas@suse.de>

show more ...

migration: Add capability parsing to analyze-migration.py

The script is broken when the configuration/capabilities section is
present. Add support for parsing the capabilities so we can fix it in
th

migration: Add capability parsing to analyze-migration.py

The script is broken when the configuration/capabilities section is
present. Add support for parsing the capabilities so we can fix it in
the next patch.

Signed-off-by: Fabiano Rosas <farosas@suse.de>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>
Message-ID: <20231009184326.15777-4-farosas@suse.de>

show more ...

migration: Fix analyze-migration.py 'configuration' parsing

The 'configuration' state subsections are currently not being parsed
and the script fails when analyzing an aarch64 stream:

Traceback (mo

migration: Fix analyze-migration.py 'configuration' parsing

The 'configuration' state subsections are currently not being parsed
and the script fails when analyzing an aarch64 stream:

Traceback (most recent call last):
File "./scripts/analyze-migration.py", line 625, in <module>
dump.read(dump_memory = args.memory)
File "./scripts/analyze-migration.py", line 571, in read
raise Exception("Unknown section type: %d" % section_type)
Exception: Unknown section type: 5

Signed-off-by: Fabiano Rosas <farosas@suse.de>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>
Message-ID: <20231009184326.15777-3-farosas@suse.de>

show more ...