python: use vm.cmd() instead of vm.qmp() where appropriate

In many cases we just want an effect of qmp command and want to raise
on failure. Use vm.cmd() method which does exactly this.

The commit

python: use vm.cmd() instead of vm.qmp() where appropriate

In many cases we just want an effect of qmp command and want to raise
on failure. Use vm.cmd() method which does exactly this.

The commit is generated by command

git grep -l '\.qmp(' | xargs ./scripts/python_qmp_updater.py

And then, fix self.assertRaises to expect ExecuteError exception in
tests/qemu-iotests/124

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id: 20231006154125.1068348-16-vsementsov@yandex-team.ru
Signed-off-by: John Snow <jsnow@redhat.com>

show more ...

iotests/281: Let NBD connection yield in iothread

Put an NBD block device into an I/O thread, and then read data from it,
hoping that the NBD connection will yield during that read. When it
does, t

iotests/281: Let NBD connection yield in iothread

Put an NBD block device into an I/O thread, and then read data from it,
hoping that the NBD connection will yield during that read. When it
does, the coroutine must be reentered in the block device's I/O thread,
which will only happen if the NBD block driver attaches the connection's
QIOChannel to the new AioContext. It did not do that after 4ddb5d2fde
("block/nbd: drop connection_co") and prior to "block/nbd: Move s->ioc
on AioContext change", which would cause an assertion failure.

To improve our chances of yielding, the NBD server is throttled to
reading 64 kB/s, and the NBD client reads 128 kB, so it should yield at
some point.

Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Signed-off-by: Hanna Reitz <hreitz@redhat.com>
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>

show more ...

iotests/281: Test lingering timers

Prior to "block/nbd: Delete reconnect delay timer when done" and
"block/nbd: Delete open timer when done", both of those timers would
remain scheduled even after s

iotests/281: Test lingering timers

Prior to "block/nbd: Delete reconnect delay timer when done" and
"block/nbd: Delete open timer when done", both of those timers would
remain scheduled even after successfully (re-)connecting to the server,
and they would not even be deleted when the BDS is deleted.

This test constructs exactly this situation:
(1) Configure an @open-timeout, so the open timer is armed, and
(2) Configure a @reconnect-delay and trigger a reconnect situation
(which succeeds immediately), so the reconnect delay timer is armed.
Then we immediately delete the BDS, and sleep for longer than the
@open-timeout and @reconnect-delay. Prior to said patches, this caused
one (or both) of the timer CBs to access already-freed data.

Accessing freed data may or may not crash, so this test can produce
false successes, but I do not know how to show the problem in a better
or more reliable way. If you run this test on "block/nbd: Assert there
are no timers when closed" and without the fix patches mentioned above,
you should reliably see an assertion failure.
(But all other tests that use the reconnect delay timer (264 and 277)
will fail in that configuration, too; as will nbd-reconnect-on-open,
which uses the open timer.)

Remove this test from the quick group because of the two second sleep
this patch introduces.

(I decided to put this test case into 281, because the main bug this
series addresses is in the interaction of the NBD block driver and I/O
threads, which is precisely the scope of 281. The test case for that
other bug will also be put into the test class added here.

Also, excuse the test class's name, I couldn't come up with anything
better. The "yield" part will make sense two patches from now.)

Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Signed-off-by: Hanna Reitz <hreitz@redhat.com>
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>

show more ...

iotests: specify some unsupported_imgopts for python iotests

We are going to support IMGOPTS for python iotests. Still some iotests
will not work with common IMGOPTS used with bash iotests like
spec

iotests: specify some unsupported_imgopts for python iotests

We are going to support IMGOPTS for python iotests. Still some iotests
will not work with common IMGOPTS used with bash iotests like
specifying refcount_bits and compat qcow2 options. So we
should define corresponding unsupported_imgopts for now.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20211223160144.1097696-4-vsementsov@virtuozzo.com>
Signed-off-by: Hanna Reitz <hreitz@redhat.com>

show more ...

Merge remote-tracking branch 'remotes/vivier2/tags/linux-user-for-6.0-pull-request' into staging

linux-user pull request 20210119-v3

Remove obsolete F_SHLCK and F_EXLCK translation

Merge remote-tracking branch 'remotes/vivier2/tags/linux-user-for-6.0-pull-request' into staging

linux-user pull request 20210119-v3

Remove obsolete F_SHLCK and F_EXLCK translation
Update sockopt
Add F_ADD_SEALS and F_GET_SEALS

# gpg: Signature made Thu 21 Jan 2021 12:33:04 GMT
# gpg: using RSA key CD2F75DDC8E3A4DC2E4F5173F30C38BD3F2FBE3C
# gpg: issuer "laurent@vivier.eu"
# gpg: Good signature from "Laurent Vivier <lvivier@redhat.com>" [full]
# gpg: aka "Laurent Vivier <laurent@vivier.eu>" [full]
# gpg: aka "Laurent Vivier (Red Hat) <lvivier@redhat.com>" [full]
# Primary key fingerprint: CD2F 75DD C8E3 A4DC 2E4F 5173 F30C 38BD 3F2F BE3C

* remotes/vivier2/tags/linux-user-for-6.0-pull-request:
linux-user: Remove obsolete F_SHLCK and F_EXLCK translation
linux-user: Add IPv6 options to do_print_sockopt()
linux-user: add missing IPv6 get/setsockopt option
linux-user: add missing UDP get/setsockopt option
linux-user: Support F_ADD_SEALS and F_GET_SEALS fcntls

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

Merge remote-tracking branch 'remotes/cohuck-gitlab/tags/s390x-20210121' into staging

s390x updates:
- headers update to Linux 5.11-rc2
- fix tcg emulation for some instructions that

Merge remote-tracking branch 'remotes/cohuck-gitlab/tags/s390x-20210121' into staging

s390x updates:
- headers update to Linux 5.11-rc2
- fix tcg emulation for some instructions that are generated by
clang Linux kernel builds
- vfio-ccw: wire up the device unplug notification mechanism
- fix a gcc 11 warning

# gpg: Signature made Thu 21 Jan 2021 12:08:12 GMT
# gpg: using RSA key C3D0D66DC3624FF6A8C018CEDECF6B93C6F02FAF
# gpg: issuer "cohuck@redhat.com"
# gpg: Good signature from "Cornelia Huck <conny@cornelia-huck.de>" [unknown]
# gpg: aka "Cornelia Huck <huckc@linux.vnet.ibm.com>" [full]
# gpg: aka "Cornelia Huck <cornelia.huck@de.ibm.com>" [full]
# gpg: aka "Cornelia Huck <cohuck@kernel.org>" [unknown]
# gpg: aka "Cornelia Huck <cohuck@redhat.com>" [unknown]
# Primary key fingerprint: C3D0 D66D C362 4FF6 A8C0 18CE DECF 6B93 C6F0 2FAF

* remotes/cohuck-gitlab/tags/s390x-20210121:
s390x: Use strpadcpy for copying vm name
vfio-ccw: Connect the device request notifier
Update linux headers to 5.11-rc2
update-linux-headers: Include const.h
s390x/tcg: Ignore register content if b1/b2 is zero when handling EXECUTE
tests/tcg/s390x: Fix EXRL tests
s390x/tcg: Don't ignore content in r0 when not specified via "b" or "x"
s390x/tcg: Fix RISBHG
s390x/tcg: Fix ALGSI

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

Merge remote-tracking branch 'remotes/ericb/tags/pull-nbd-2021-01-20' into staging

nbd patches for 2021-01-20

- minor resource leak fixes in qemu-nbd
- ensure proper aio context

Merge remote-tracking branch 'remotes/ericb/tags/pull-nbd-2021-01-20' into staging

nbd patches for 2021-01-20

- minor resource leak fixes in qemu-nbd
- ensure proper aio context when nbd server uses iothreads
- iotest refactorings in preparation for rewriting ./check to be more
flexible, and preparing for more nbd server reconnect features

# gpg: Signature made Thu 21 Jan 2021 02:28:19 GMT
# gpg: using RSA key 71C2CC22B1C4602927D2F3AAA7A16B4A2527436A
# gpg: Good signature from "Eric Blake <eblake@redhat.com>" [full]
# gpg: aka "Eric Blake (Free Software Programmer) <ebb9@byu.net>" [full]
# gpg: aka "[jpeg image of size 6874]" [full]
# Primary key fingerprint: 71C2 CC22 B1C4 6029 27D2 F3AA A7A1 6B4A 2527 436A

* remotes/ericb/tags/pull-nbd-2021-01-20:
iotests.py: qemu_io(): reuse qemu_tool_pipe_and_status()
iotests.py: fix qemu_tool_pipe_and_status()
iotests/264: fix style
iotests: define group in each iotest
iotests/294: add shebang line
iotests: make tests executable
iotests: fix some whitespaces in test output files
iotests/303: use dot slash for qcow2.py running
iotests/277: use dot slash for nbd-fault-injector.py running
nbd/server: Quiesce coroutines on context switch
block: Honor blk_set_aio_context() context requirements
qemu-nbd: Fix a memleak in nbd_client_thread()
qemu-nbd: Fix a memleak in qemu_nbd_client_list()

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

iotests: define group in each iotest

We are going to drop group file. Define group in tests as a preparatory
step.

The patch is generated by

cd tests/qemu-iotests

iotests: define group in each iotest

We are going to drop group file. Define group in tests as a preparatory
step.

The patch is generated by

cd tests/qemu-iotests

grep '^[0-9]\{3\} ' group | while read line; do
file=$(awk '{print $1}' <<< "$line");
groups=$(sed -e 's/^... //' <<< "$line");
awk "NR==2{print \"# group: $groups\"}1" $file > tmp;
cat tmp > $file;
done

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20210116134424.82867-7-vsementsov@virtuozzo.com>
Signed-off-by: Eric Blake <eblake@redhat.com>

show more ...

Merge remote-tracking branch 'remotes/bonzini/tags/for-upstream' into staging

* various small fixes and cleanups
* fixes for the ucode revision patch from the previous pull request

Merge remote-tracking branch 'remotes/bonzini/tags/for-upstream' into staging

* various small fixes and cleanups
* fixes for the ucode revision patch from the previous pull request

# gpg: Signature made Wed 12 Feb 2020 15:30:24 GMT
# gpg: using RSA key BFFBD25F78C7AE83
# gpg: Good signature from "Paolo Bonzini <bonzini@gnu.org>" [full]
# gpg: aka "Paolo Bonzini <pbonzini@redhat.com>" [full]
# Primary key fingerprint: 46F5 9FBD 57D6 12E7 BFD4 E2F7 7E15 100C CD36 69B1
# Subkey fingerprint: F133 3857 4B66 2389 866C 7682 BFFB D25F 78C7 AE83

* remotes/bonzini/tags/for-upstream:
target/i386: enable monitor and ucode revision with -cpu max
target/i386: check for availability of MSR_IA32_UCODE_REV as an emulated MSR
target/i386: fix TCG UCODE_REV access
build: move TARGET_GPROF to config-host.mak
exec: do not define use_icount for user-mode emulation
minikconf: accept alnum identifiers
Remove support for CLOCK_MONOTONIC not being defined
seqlock: fix seqlock_write_unlock_impl function
vl: Don't mismatch g_strsplit()/g_free()

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

Merge remote-tracking branch 'remotes/stefanha/tags/block-pull-request' into staging

Pull request

# gpg: Signature made Mon 10 Feb 2020 09:23:42 GMT
# gpg: using

Merge remote-tracking branch 'remotes/stefanha/tags/block-pull-request' into staging

Pull request

# gpg: Signature made Mon 10 Feb 2020 09:23:42 GMT
# gpg: using RSA key 8695A8BFD3F97CDAAC35775A9CA4ABB381AB73C8
# gpg: Good signature from "Stefan Hajnoczi <stefanha@redhat.com>" [full]
# gpg: aka "Stefan Hajnoczi <stefanha@gmail.com>" [full]
# Primary key fingerprint: 8695 A8BF D3F9 7CDA AC35 775A 9CA4 ABB3 81AB 73C8

* remotes/stefanha/tags/block-pull-request:
hw/core: Allow setting 'virtio-blk-device.scsi' property on OSX host
block: fix crash on zero-length unaligned write and read

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

Merge remote-tracking branch 'remotes/gkurz/tags/9p-next-2020-02-08' into staging

9p patches:
- some more protocol sanity checks
- qtest for readdir
- Christian Schoenebeck now o

Merge remote-tracking branch 'remotes/gkurz/tags/9p-next-2020-02-08' into staging

9p patches:
- some more protocol sanity checks
- qtest for readdir
- Christian Schoenebeck now official reviewer

# gpg: Signature made Sat 08 Feb 2020 08:38:10 GMT
# gpg: using RSA key B4828BAF943140CEF2A3491071D4D5E5822F73D6
# gpg: Good signature from "Greg Kurz <groug@kaod.org>" [full]
# gpg: aka "Gregory Kurz <gregory.kurz@free.fr>" [full]
# gpg: aka "[jpeg image of size 3330]" [full]
# Primary key fingerprint: B482 8BAF 9431 40CE F2A3 4910 71D4 D5E5 822F 73D6

* remotes/gkurz/tags/9p-next-2020-02-08:
MAINTAINERS: 9pfs: Add myself as reviewer
tests/virtio-9p: added readdir test
hw/9pfs/9p-synth: added directory for readdir test
9pfs: validate count sent by client with T_readdir
9pfs: require msize >= 4096
tests/virtio-9p: add terminating null in v9fs_string_read()

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

Merge remote-tracking branch 'remotes/philmd-gitlab/tags/python-next-20200207' into staging

- Python 3 cleanups:
. Remove text about Python 2 in qemu-deprecated (Thomas)
. Remove

Merge remote-tracking branch 'remotes/philmd-gitlab/tags/python-next-20200207' into staging

- Python 3 cleanups:
. Remove text about Python 2 in qemu-deprecated (Thomas)
. Remove shebang header (Paolo, Philippe)
. scripts/checkpatch.pl now allows Python 3 interpreter (Philippe)
. Explicit usage of Python 3 interpreter in scripts (Philippe)
. Fix Python scripts permissions (Paolo, Philippe)
. Drop 'from __future__ import print_function' (Paolo)
. Specify minimum python requirements in ReadTheDocs configuration (Alex)
- Test UNIX/EXEC transports with migration (Oksana)
- Added extract_from_rpm helper, improved extract_from_deb (Liam)
- Allow to use other serial consoles than default one (Philippe)
- Various improvements in QEMUMonitorProtocol (Wainer)
- Fix kvm_available() on ppc64le (Wainer)

# gpg: Signature made Fri 07 Feb 2020 15:01:56 GMT
# gpg: using RSA key FAABE75E12917221DCFD6BB2E3E32C2CDEADC0DE
# gpg: Good signature from "Philippe Mathieu-Daudé (F4BUG) <f4bug@amsat.org>" [full]
# Primary key fingerprint: FAAB E75E 1291 7221 DCFD 6BB2 E3E3 2C2C DEAD C0DE

* remotes/philmd-gitlab/tags/python-next-20200207: (46 commits)
.readthedocs.yml: specify some minimum python requirements
drop "from __future__ import print_function"
make all Python scripts executable
scripts/signrom: remove Python 2 support, add shebang
tests/qemu-iotests/check: Only check for Python 3 interpreter
scripts: Explicit usage of Python 3 (scripts without __main__)
tests/qemu-iotests: Explicit usage of Python3 (scripts without __main__)
tests/vm: Remove shebang header
tests/acceptance: Remove shebang header
scripts/tracetool: Remove shebang header
scripts/minikconf: Explicit usage of Python 3
scripts: Explicit usage of Python 3 (scripts with __main__)
tests: Explicit usage of Python 3
tests/qemu-iotests: Explicit usage of Python 3 (scripts with __main__)
tests/qemu-iotests/check: Allow use of python3 interpreter
scripts/checkpatch.pl: Only allow Python 3 interpreter
tests/acceptance/migration: Default to -nodefaults
tests/acceptance/migration: Add the 'migration' tag
tests/acceptance/migration: Test EXEC transport when migrating
tests/acceptance/migration: Test UNIX transport when migrating
...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

tests/qemu-iotests: Explicit usage of Python 3 (scripts with __main__)

Use the program search path to find the Python 3 interpreter.

Patch created mechanically by running:

tests/qemu-iotests: Explicit usage of Python 3 (scripts with __main__)

Use the program search path to find the Python 3 interpreter.

Patch created mechanically by running:

$ sed -i "s,^#\!/usr/bin/\(env\ \)\?python$,#\!/usr/bin/env python3," \
$(git grep -l 'if __name__.*__main__')

Reported-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Suggested-by: Daniel P. Berrangé <berrange@redhat.com>
Suggested-by: Stefan Hajnoczi <stefanha@redhat.com>
Acked-by: Stefan Hajnoczi <stefanha@redhat.com>
Acked-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20200130163232.10446-4-philmd@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>

show more ...

Merge remote-tracking branch 'remotes/jnsnow/tags/ide-pull-request' into staging

Pull request

# gpg: Signature made Tue 28 Jan 2020 01:05:19 GMT
# gpg: using RSA

Merge remote-tracking branch 'remotes/jnsnow/tags/ide-pull-request' into staging

Pull request

# gpg: Signature made Tue 28 Jan 2020 01:05:19 GMT
# gpg: using RSA key F9B7ABDBBCACDF95BE76CBD07DEF8106AAFC390E
# gpg: Good signature from "John Snow (John Huston) <jsnow@redhat.com>" [full]
# Primary key fingerprint: FAEB 9711 A12C F475 812F 18F2 88A9 064D 1835 61EB
# Subkey fingerprint: F9B7 ABDB BCAC DF95 BE76 CBD0 7DEF 8106 AAFC 390E

* remotes/jnsnow/tags/ide-pull-request:
tests/ide-test: Create a single unit-test covering more PRDT cases
ide: Fix incorrect handling of some PRDTs in ide_dma_cb()

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

Merge remote-tracking branch 'remotes/rth/tags/pull-pa-20200127' into staging

Improve LASI emulation
Add Artist graphics
Fix main memory allocation
Improve LDCW emulation wrt rea

Merge remote-tracking branch 'remotes/rth/tags/pull-pa-20200127' into staging

Improve LASI emulation
Add Artist graphics
Fix main memory allocation
Improve LDCW emulation wrt real hw

# gpg: Signature made Mon 27 Jan 2020 18:53:35 GMT
# gpg: using RSA key 7A481E78868B4DB6A85A05C064DF38E8AF7E215F
# gpg: issuer "richard.henderson@linaro.org"
# gpg: Good signature from "Richard Henderson <richard.henderson@linaro.org>" [full]
# Primary key fingerprint: 7A48 1E78 868B 4DB6 A85A 05C0 64DF 38E8 AF7E 215F

* remotes/rth/tags/pull-pa-20200127:
target/hppa: Allow, but diagnose, LDCW aligned only mod 4
hw/hppa/machine: Map the PDC memory region with higher priority
hw/hppa/machine: Restrict the total memory size to 3GB
hw/hppa/machine: Correctly check the firmware is in PDC range
hppa: Add emulation of Artist graphics
seabios-hppa: update to latest version
hppa: Switch to tulip NIC by default
hppa: add emulation of LASI PS2 controllers
ps2: accept 'Set Key Make and Break' commands
hppa: Add support for LASI chip with i82596 NIC
hw/hppa/dino.c: Improve emulation of Dino PCI chip

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

iotests: Test handling of AioContexts with some blockdev actions

Includes the following tests:

- Adding a dirty bitmap.
* RHBZ: 1782175

- Starting a drive-mirror t

iotests: Test handling of AioContexts with some blockdev actions

Includes the following tests:

- Adding a dirty bitmap.
* RHBZ: 1782175

- Starting a drive-mirror to an NBD-backed target.
* RHBZ: 1746217, 1773517

- Aborting an external snapshot transaction.
* RHBZ: 1779036

- Aborting a blockdev backup transaction.
* RHBZ: 1782111

For each one of them, a VM with a number of disks running in an
IOThread AioContext is used.

Signed-off-by: Sergio Lopez <slp@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>

show more ...