gitlab: add optional job to run flaky avocado tests

One problem with flaky tests is they often only fail under CI
conditions which makes it hard to debug. We add an optional allow_fail
job so develo

gitlab: add optional job to run flaky avocado tests

One problem with flaky tests is they often only fail under CI
conditions which makes it hard to debug. We add an optional allow_fail
job so developers can trigger the only the flaky tests in the CI
environment if they are debugging.

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Acked-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20231201093633.2551497-8-alex.bennee@linaro.org>

show more ...

docs/devel: rationalise unstable gitlab tests under FLAKY_TESTS

It doesn't make sense to have two classes of flaky tests. While it may
take the constrained environment of CI to trigger failures easi

docs/devel: rationalise unstable gitlab tests under FLAKY_TESTS

It doesn't make sense to have two classes of flaky tests. While it may
take the constrained environment of CI to trigger failures easily it
doesn't mean they don't occasionally happen on developer machines. As
CI is the gating factor to passing there is no point developers
running the tests locally anyway unless they are trying to fix things.

While we are at it update the language in the docs to discourage the
QEMU_TEST_FLAKY_TESTS becoming a permanent solution.

Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Acked-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20231201093633.2551497-3-alex.bennee@linaro.org>

show more ...

tests/avocado/machine_s390_ccw_virtio.py: Fix SyntaxWarnings from python 3.12

Python 3.12 now warns about backslashes in strings that aren't used
for escaping a special character from Python. Silenc

tests/avocado/machine_s390_ccw_virtio.py: Fix SyntaxWarnings from python 3.12

Python 3.12 now warns about backslashes in strings that aren't used
for escaping a special character from Python. Silence the warning
by using raw strings here instead.

Signed-off-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-ID: <20231113140721.46903-1-thuth@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>

show more ...

python/qemu: rename command() to cmd()

Use a shorter name. We are going to move in iotests from qmp() to
command() where possible. But command() is longer than qmp() and don't
look better. Let's ren

python/qemu: rename command() to cmd()

Use a shorter name. We are going to move in iotests from qmp() to
command() where possible. But command() is longer than qmp() and don't
look better. Let's rename.

You can simply grep for '\.command(' and for 'def command(' to check
that everything is updated (command() in tests/docker/docker.py is
unrelated).

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Cédric Le Goater <clg@kaod.org>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Message-id: 20231006154125.1068348-6-vsementsov@yandex-team.ru
[vsementsov: also update three occurrences in
tests/avocado/machine_aspeed.py and keep r-b]
Signed-off-by: John Snow <jsnow@redhat.com>

show more ...

tests/avocado/machine_s390_ccw_virtio: Skip the flaky virtio-gpu test by default

The virtio-gpu test is known to be flaky - that's why we also did
not enable the test_s390x_fedora in the gitlab CI.

tests/avocado/machine_s390_ccw_virtio: Skip the flaky virtio-gpu test by default

The virtio-gpu test is known to be flaky - that's why we also did
not enable the test_s390x_fedora in the gitlab CI. However, a flaky
test can also be annoying when testing locally, so let's rather skip
this subtest by default and start running the test_s390x_fedora test
in the gitlab CI again (since the other things that are tested here
are quite valuable).

Message-Id: <20230724084851.24251-1-thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Reviewed-by: Cédric Le Goater <clg@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>

show more ...

target/s390x: support PRNO_TRNG instruction

In order for hosts running inside of TCG to initialize the kernel's
random number generator, we should support the PRNO_TRNG instruction,
backed in the us

target/s390x: support PRNO_TRNG instruction

In order for hosts running inside of TCG to initialize the kernel's
random number generator, we should support the PRNO_TRNG instruction,
backed in the usual way with the qemu_guest_getrandom helper. This is
confirmed working on Linux 5.19.

Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
Message-Id: <20220921100729.2942008-2-Jason@zx2c4.com>
Reviewed-by: David Hildenbrand <david@redhat.com>
[thuth: turn prno-trng off in avocado test to avoid breaking it]
Signed-off-by: Thomas Huth <thuth@redhat.com>

show more ...

tests/avocado/machine_s390_ccw_virtio: Adapt test to new default resolution

QEMU's default screen resolution recently changed to 1280x800, so the
resolution in the screen shot header changed of cour

tests/avocado/machine_s390_ccw_virtio: Adapt test to new default resolution

QEMU's default screen resolution recently changed to 1280x800, so the
resolution in the screen shot header changed of course, too.

Fixes: de72c4b7cd ("edid: set default resolution to 1280x800 (WXGA)")
Reported-by: Peter Maydell <peter.maydell@linaro.org>
Message-Id: <20220221101933.307525-1-thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Acked-by: Halil Pasic <pasic@linux.ibm.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>

show more ...

tests/avocado: Rename avocado_qemu.Test -> QemuSystemTest

To run user-mode emulation tests, we introduced the
avocado_qemu.QemuUserTest which inherits from avocado_qemu.QemuBaseTest.
System-mode emu

tests/avocado: Rename avocado_qemu.Test -> QemuSystemTest

To run user-mode emulation tests, we introduced the
avocado_qemu.QemuUserTest which inherits from avocado_qemu.QemuBaseTest.
System-mode emulation tests are based on the avocado_qemu.Test class,
which also inherits avocado_qemu.QemuBaseTest. To avoid confusion,
rename it as avocado_qemu.QemuSystemTest.

Suggested-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20211105143416.148332-7-f4bug@amsat.org>

show more ...

tests/acceptance: rename tests acceptance to tests avocado

In the discussion about renaming the `tests/acceptance` [1], the
conclusion was that the folders inside `tests` are related to the
framewor

tests/acceptance: rename tests acceptance to tests avocado

In the discussion about renaming the `tests/acceptance` [1], the
conclusion was that the folders inside `tests` are related to the
framework running the tests and not directly related to the type of
the tests.

This changes the folder to `tests/avocado` and adjusts the MAKEFILE, the
CI related files and the documentation.

[1] https://lists.gnu.org/archive/html/qemu-devel/2021-05/msg06553.html

Reviewed-by: Niek Linnenbank <nieklinnenbank@gmail.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Willian Rampazzo <willianr@redhat.com>
Message-Id: <20211105155354.154864-3-willianr@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>

show more ...