tests/avocado/machine_aspeed.py: Update buildroot images to 2023.11

Compared to mainline buildroot, these images have some customization :

- Linux version is bumped to 6.6.3 and built with a custom

tests/avocado/machine_aspeed.py: Update buildroot images to 2023.11

Compared to mainline buildroot, these images have some customization :

- Linux version is bumped to 6.6.3 and built with a custom config
- U-Boot is switched to the one provided by OpenBMC for more support
- defconfigs extra tools for dev

See branch [1] for more details.

There are a few changes since last update, commit ed1f5ff84209. Images
all have a password now and I2C devices have been updated in the Linux
ast2600-evb device tree [2]. Do the necessary adjustements.

[1] https://github.com/legoater/buildroot/commits/aspeed-2023.11
[2] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=9deb10cf160e

Signed-off-by: Cédric Le Goater <clg@kaod.org>

show more ...

gitlab: add optional job to run flaky avocado tests

One problem with flaky tests is they often only fail under CI
conditions which makes it hard to debug. We add an optional allow_fail
job so develo

gitlab: add optional job to run flaky avocado tests

One problem with flaky tests is they often only fail under CI
conditions which makes it hard to debug. We add an optional allow_fail
job so developers can trigger the only the flaky tests in the CI
environment if they are debugging.

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Acked-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20231201093633.2551497-8-alex.bennee@linaro.org>

show more ...

docs/devel: rationalise unstable gitlab tests under FLAKY_TESTS

It doesn't make sense to have two classes of flaky tests. While it may
take the constrained environment of CI to trigger failures easi

docs/devel: rationalise unstable gitlab tests under FLAKY_TESTS

It doesn't make sense to have two classes of flaky tests. While it may
take the constrained environment of CI to trigger failures easily it
doesn't mean they don't occasionally happen on developer machines. As
CI is the gating factor to passing there is no point developers
running the tests locally anyway unless they are trying to fix things.

While we are at it update the language in the docs to discourage the
QEMU_TEST_FLAKY_TESTS becoming a permanent solution.

Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Acked-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20231201093633.2551497-3-alex.bennee@linaro.org>

show more ...

python/qemu: rename command() to cmd()

Use a shorter name. We are going to move in iotests from qmp() to
command() where possible. But command() is longer than qmp() and don't
look better. Let's ren

python/qemu: rename command() to cmd()

Use a shorter name. We are going to move in iotests from qmp() to
command() where possible. But command() is longer than qmp() and don't
look better. Let's rename.

You can simply grep for '\.command(' and for 'def command(' to check
that everything is updated (command() in tests/docker/docker.py is
unrelated).

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Cédric Le Goater <clg@kaod.org>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Message-id: 20231006154125.1068348-6-vsementsov@yandex-team.ru
[vsementsov: also update three occurrences in
tests/avocado/machine_aspeed.py and keep r-b]
Signed-off-by: John Snow <jsnow@redhat.com>

show more ...

python/machine: remove unused sock_dir argument

By using a socketpair for all of the sockets managed by the VM class and
its extensions, we don't need the sock_dir argument anymore, so remove
it.

W

python/machine: remove unused sock_dir argument

By using a socketpair for all of the sockets managed by the VM class and
its extensions, we don't need the sock_dir argument anymore, so remove
it.

We only added this argument so that we could specify a second, shorter
temporary directory for cases where the temp/log dirs were "too long" as
a socket name on macOS. We don't need it for this class now. In one
case, avocado testing takes over responsibility for creating an
appropriate sockdir.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-id: 20230928044943.849073-7-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>

show more ...

tests/avocado/machine_aspeed.py: Update SDK images

Switch to the latest v8.06 release which introduces interesting
changes for the AST2600 I2C and I3C models. Also take the AST2600 A2
images instead

tests/avocado/machine_aspeed.py: Update SDK images

Switch to the latest v8.06 release which introduces interesting
changes for the AST2600 I2C and I3C models. Also take the AST2600 A2
images instead of the default since QEMU tries to model The AST2600 A3
SoC.

Signed-off-by: Cédric Le Goater <clg@kaod.org>
Reviewed-by: Joel Stanley <joel@jms.id.au>
Signed-off-by: Cédric Le Goater <clg@kaod.org>

show more ...

tests/avocado/machine_aspeed.py: Update SDK images

Switch to the latest v8.06 release which introduces interesting
changes for the AST2600 I2C and I3C models. Also take the AST2600 A2
images instead

tests/avocado/machine_aspeed.py: Update SDK images

Switch to the latest v8.06 release which introduces interesting
changes for the AST2600 I2C and I3C models. Also take the AST2600 A2
images instead of the default since QEMU tries to model The AST2600 A3
SoC.

Signed-off-by: Cédric Le Goater <clg@kaod.org>

show more ...

tests/avocado/machine_aspeed: Fix the broken ast2[56]00_evb_sdk tests

test_arm_ast2500_evb_sdk and test_arm_ast2600_evb_sdk are currently
failing. The problem is that they are trying to look for the

tests/avocado/machine_aspeed: Fix the broken ast2[56]00_evb_sdk tests

test_arm_ast2500_evb_sdk and test_arm_ast2600_evb_sdk are currently
failing. The problem is that they are trying to look for the login
prompt that does not have a newline at the end - but the logic in
_console_interaction() only handles full lines. It used to work by
accident in the past since there were sometimes kernel (warning and
error) messages popping up that finally provided a newline character
in the output, but since the tests have been changed to run with the
"quiet" kernel parameter, this is not working anymore.

To make this work reliably, we must not look for the "login:" prompt,
but have to use some text ending with a newline instead. And in the
ast2600 test, switch to ssh instead of trying to log into the serial
console - this works much more reliable and also has the benefit of
excercising the network interface here a little bit, too.

Signed-off-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Cédric Le Goater <clg@kaod.org>
Message-Id: <20230421110345.1294131-3-thuth@redhat.com>
[AJB: remove stray debug log]
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20230424092249.58552-10-alex.bennee@linaro.org>

show more ...

tests/avocado/aspeed: Add TPM TIS I2C test

Add a new buildroot image based test that attaches a TPM emulator to the
I2C bus and checks for a known PCR0 value for the image that was booted.

Note tha

tests/avocado/aspeed: Add TPM TIS I2C test

Add a new buildroot image based test that attaches a TPM emulator to the
I2C bus and checks for a known PCR0 value for the image that was booted.

Note that this does not tear down swtpm process when qemu execution fails.
The swtpm process will exit when qemu exits if a connection has been
made, but if the test errors before connection then the swtpm process
will still be around.

Signed-off-by: Joel Stanley <joel@jms.id.au>
Reviewed-by: Stefan Berger <stefanb@linux.ibm.com>
Reviewed-by: Cédric Le Goater <clg@kaod.org>
Reviewed-by: Ninad Palsule <ninad@linux.ibm.com>
Message-id: 20230328120844.190914-1-joel@jms.id.au

show more ...

tests/avocado/machine_aspeed.py: Update SDK images

v8.03 : I2C and I3C changes for AST2600

Signed-off-by: Cédric Le Goater <clg@kaod.org>

tests/avocado/machine_aspeed.py: Add shutdown to the SDK tests

This is adding an extra minute of shutdown to each test :/

Signed-off-by: Cédric Le Goater <clg@kaod.org>

tests/avocado/aspeed: Add TPM TIS I2C test

Add a new buildroot image based test that attaches a TPM emulator to the
I2C bus and checks for a known PCR0 value for the image that was booted.

Note tha

tests/avocado/aspeed: Add TPM TIS I2C test

Add a new buildroot image based test that attaches a TPM emulator to the
I2C bus and checks for a known PCR0 value for the image that was booted.

Note that this does not tear down swtpm process when qemu execution fails.
The swtpm process will exit when qemu exits if a connection has been
made, but if the test errors before connection then the swtpm process
will still be around.

Signed-off-by: Joel Stanley <joel@jms.id.au>
Reviewed-by: Stefan Berger <stefanb@linux.ibm.com>
Message-Id: <20230328120844.190914-1-joel@jms.id.au>
Signed-off-by: Cédric Le Goater <clg@kaod.org>

show more ...

tests/avocado/machine_aspeed.py: Add an I2C slave test

The Aspeed 2600 I2C controller supports a slave mode which can be
tested with the I2C echo device. Test extracted from :

https://lists.nongn

tests/avocado/machine_aspeed.py: Add an I2C slave test

The Aspeed 2600 I2C controller supports a slave mode which can be
tested with the I2C echo device. Test extracted from :

https://lists.nongnu.org/archive/html/qemu-devel/2022-06/msg00183.html

Suggested-by: Klaus Jensen <k.jensen@samsung.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Cédric Le Goater <clg@kaod.org>

show more ...

tests/avocado: Test Aspeed Zephyr SDK v00.01.08 on AST1030 board

Add a very quick test that runs some commands in a Zephyr shell:

$ tests/venv/bin/avocado --show=app,console run -t os:zephyr test

tests/avocado: Test Aspeed Zephyr SDK v00.01.08 on AST1030 board

Add a very quick test that runs some commands in a Zephyr shell:

$ tests/venv/bin/avocado --show=app,console run -t os:zephyr tests/avocado
(2/2) tests/avocado/machine_aspeed.py:AST1030Machine.test_ast1030_zephyros_1_07:
console: *** Booting Zephyr OS build v00.01.07 ***
console: ast1030_evb demo
console: SOC: AST1030-A1
console: uart:~$ kernel stacks
console: 0x36910 wdt_background (real size 1024): unused 988 usage 36 / 1024 (3 %)
console: 0x36ad8 shell_uart (real size 4096): unused 3084 usage 1012 / 4096 (24 %)
console: 0x2edb8 ADC0 (real size 400): unused 260 usage 140 / 400 (35 %)
console: 0x2f0f0 ADC1 (real size 400): unused 260 usage 140 / 400 (35 %)
console: 0x3b098 sysworkq (real size 1024): unused 860 usage 164 / 1024 (16 %)
console: 0x36cc0 usbdworkq (real size 1024): unused 860 usage 164 / 1024 (16 %)
console: 0x36bd8 usbworkq (real size 1024): unused 860 usage 164 / 1024 (16 %)
console: 0x36a10 logging (real size 768): unused 548 usage 220 / 768 (28 %)
console: 0x36ef8 idle 00 (real size 320): unused 268 usage 52 / 320 (16 %)
console: 0x47800 IRQ 00 (real size 2048): unused 1504 usage 544 / 2048 (26 %)
console: uart:~$ otp info scu
console: SCU BIT reg_protect Description
console: ____________________________________________________________________
console: 0x500 0x0 0x0 Disable ARM CM4 CPU boot (TXD5)
console: 0x500 0x1 0x0 /Reserved
console: 0x500 0x2 0x0 \ "
console: 0x500 0x3 0x0 Address offset of single chip ABR mode
console: 0x500 0x4 0x0 /Reserved
console: 0x500 0x5 0x0 | "
console: 0x500 0x6 0x0 | "
console: 0x500 0x7 0x0 | "
console: 0x500 0x8 0x0 | "
console: 0x500 0x9 0x0 | "
console: 0x500 0xA 0x0 | "
console: 0x500 0xB 0x0 | "
console: 0x500 0xC 0x0 | "
console: 0x500 0xD 0x0 | "
console: 0x500 0xE 0x0 | "
console: 0x500 0xF 0x0 | "
console: 0x500 0x10 0x0 \ "
console: 0x500 0x11 0x0 Disabl3 ARM JTAG debug
console: 0x500 0x12 0x0 /Reserved
console: 0x500 0x13 0x0 | "
console: 0x500 0x14 0x0 | "
console: 0x500 0x15 0x0 | "
console: 0x500 0x16 0x0 | "
console: 0x500 0x17 0x0 | "
console: 0x500 0x18 0x0 | "
console: 0x500 0x19 0x0 | "
console: 0x500 0x1A 0x0 | "
console: 0x500 0x1B 0x0 | "
console: 0x500 0x1C 0x0 | "
console: 0x500 0x1D 0x0 | "
console: 0x500 0x1E 0x0 | "
console: 0x500 0x1F 0x0 \ "
console: 0x510 0x0 0x0 /Reserved
console: 0x510 0x1 0x0 | "
console: 0x510 0x2 0x0 | "
console: 0x510 0x3 0x0 \ "
console: 0x510 0x4 0x0 Disable debug interfaces
console: 0x510 0x5 0x0 /Reserved
console: 0x510 0x6 0x0 | "
console: 0x510 0x7 0x0 \ "
console: 0x510 0x8 0x0 Enable boot from Uart5 by Pin Strap
console: 0x510 0x9 0x0 /Reserved
console: 0x510 0xA 0x0 \ "
console: 0x510 0xB 0x0 Enable boot SPI ABR
console: 0x510 0xC 0x0 Boot SPI ABR Mode
console: 0x510 0xD 0x0 /Boot SPI flash size
console: 0x510 0xE 0x0 | "
console: 0x510 0xF 0x0 \ "
console: 0x510 0x10 0x0 /Reserved
console: 0x510 0x11 0x0 | "
console: 0x510 0x12 0x0 | "
console: 0x510 0x13 0x0 | "
console: 0x510 0x14 0x0 | "
console: 0x510 0x15 0x0 \ "
console: 0x510 0x16 0x0 Enable boot SPI auxiliary control pins
console: 0x510 0x19 0x0 /Reserved
console: 0x510 0x1A 0x0 | "
console: 0x510 0x1B 0x0 | "
console: 0x510 0x1C 0x0 | "
console: 0x510 0x1D 0x0 | "
console: 0x510 0x1E 0x0 | "
console: 0x510 0x1F 0x0 \ "
console: 0x510 0x1E 0x0 Enable dedicate GPIO strap pins
console: 0x510 0x1F 0x0 Enable Secure Boot by Pin Strap
console: uart:~$ hwinfo devid
console: Length: 8
console: ID: 0x0000018000000180
console: uart:~$ crypto aes256_cbc_vault
console: aes256_cbc vault key 1
console: Was waiting for:
console: 6b c1 be e2 2e 40 9f 96 e9 3d 7e 11 73 93 17 2a
console: ae 2d 8a 57 1e 03 ac 9c 9e b7 6f ac 45 af 8e 51
console: 30 c8 1c 46 a3 5c e4 11 e5 fb c1 19 1a 0a 52 ef
console: f6 9f 24 45 df 4f 9b 17 ad 2b 41 7b e6 6c 37 10
console: But got:
console: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
console: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
console: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
console: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
console: uart:~$ random get
console: 0x862460d
console: uart:~$ i2c scan I2C_0
console: 0 1 2 3 4 5 6 7 8 9 a b c d e f
console: 00: -- -- -- -- -- -- -- -- -- -- -- --
console: 10: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
console: 20: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
console: 30: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
console: 40: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
console: 50: 50 -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
console: 60: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
console: 70: -- -- -- -- -- -- -- --
console: 1 devices found on I2C_0
console: uart:~$ kernel uptime
console: Uptime: 9897 ms
console: uart:~$ kernel reboot warm
console: *** Booting Zephyr OS build v00.01.07 ***
PASS (1.08 s)

Ref: https://github.com/AspeedTech-BMC/zephyr/releases/download/v00.01.07/Aspeed_Zephy_SDK_User_Guide_v00.01.07.pdf

Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Reviewed-by: Peter Delevoryas <peter@pjd.dev>
Reviewed-by: Cédric Le Goater <clg@kaod.org>
Signed-off-by: Cédric Le Goater <clg@kaod.org>

show more ...

tests/avocado/machine_aspeed.py: Mask systemd services to speed up SDK boot

Reviewed-by: Joel Stanley <joel@jms.id.au>
Message-Id: <20230119123449.531826-8-clg@kaod.org>
Signed-off-by: Cédric Le Goa

tests/avocado/machine_aspeed.py: Mask systemd services to speed up SDK boot

Reviewed-by: Joel Stanley <joel@jms.id.au>
Message-Id: <20230119123449.531826-8-clg@kaod.org>
Signed-off-by: Cédric Le Goater <clg@kaod.org>

show more ...

tests/avocado/machine_aspeed.py: update buildroot tests

Use buildroot 2022.11 based images plus some customization :

- Linux version is bumped to 6.0.9 and kernel is built with a custom
confi

tests/avocado/machine_aspeed.py: update buildroot tests

Use buildroot 2022.11 based images plus some customization :

- Linux version is bumped to 6.0.9 and kernel is built with a custom
config similar to what OpenBMC provides.
- U-Boot is switched to the one provided by OpenBMC for better support.
- defconfigs includes more target tools for dev.

Reviewed-by: Joel Stanley <joel@jms.id.au>
Message-Id: <20230119123449.531826-7-clg@kaod.org>
Signed-off-by: Cédric Le Goater <clg@kaod.org>

show more ...

tests/avocado/machine_aspeed.py: Add I2C slave tests

Test extracted from :

https://lists.nongnu.org/archive/html/qemu-devel/2022-06/msg00183.html

Signed-off-by: Cédric Le Goater <clg@kaod.org>

tests/avocado/machine_aspeed.py: Update SDK images

v8.03 : I2C and I3C changes for AST2600

Signed-off-by: Cédric Le Goater <clg@kaod.org>

tests/avocado/machine_aspeed.py: Add shutdown to the SDK tests

This is adding an extra minute of shutdown to each test :/

Signed-off-by: Cédric Le Goater <clg@kaod.org>

tests/avocado/machine_aspeed.py: Mask systemd services to speed up SDK boot

Signed-off-by: Cédric Le Goater <clg@kaod.org>

tests/avocado/machine_aspeed.py: update buildroot tests

Use buildroot 2022.11 based images plus some customization :

- Linux version is bumped to 6.0.9 and kernel is built with a custom
confi

tests/avocado/machine_aspeed.py: update buildroot tests

Use buildroot 2022.11 based images plus some customization :

- Linux version is bumped to 6.0.9 and kernel is built with a custom
config similar to what OpenBMC provides.
- U-Boot is switched to the one provided by OpenBMC for better support.
- defconfigs includes more target tools for dev.

Signed-off-by: Cédric Le Goater <clg@kaod.org>

show more ...

tests/avocado/machine_aspeed.py: Reduce noise on the console for SDK tests

The Aspeed SDK images are based on OpenBMC which starts a lot of
services. The output noise on the console can break from t

tests/avocado/machine_aspeed.py: Reduce noise on the console for SDK tests

The Aspeed SDK images are based on OpenBMC which starts a lot of
services. The output noise on the console can break from time to time
the test waiting for the logging prompt.

Change the U-Boot bootargs variable to add "quiet" to the kernel
command line and reduce the output volume. This also drops the test on
the CPU id which was nice to have but not essential.

Signed-off-by: Cédric Le Goater <clg@kaod.org>
Message-Id: <20221104075347.370503-1-clg@kaod.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20221117172532.538149-3-alex.bennee@linaro.org>

show more ...

tests/avocado/machine_aspeed.py: Fix typos on buildroot

Replace 'buidroot' and 'builroot' by 'buildroot'.

Fixes: f7bc7da0724f ("test/avocado/machine_aspeed.py: Add tests using buildroot images")
Re

tests/avocado/machine_aspeed.py: Fix typos on buildroot

Replace 'buidroot' and 'builroot' by 'buildroot'.

Fixes: f7bc7da0724f ("test/avocado/machine_aspeed.py: Add tests using buildroot images")
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Joel Stanley <joel@jms.id.au>
Message-Id: <20220923084803.498337-2-clg@kaod.org>
Signed-off-by: Cédric Le Goater <clg@kaod.org>

show more ...

tests/avocado: Add missing require_netdev('user') checks

Some avocado tests fail if QEMU was built without libslirp. Add
require_netdev('user') checks where necessary:

These tests try to ping 10.0.

tests/avocado: Add missing require_netdev('user') checks

Some avocado tests fail if QEMU was built without libslirp. Add
require_netdev('user') checks where necessary:

These tests try to ping 10.0.2.2 and expect it to succeed:
boot_linux_console.py:BootLinuxConsole.test_arm_emcraft_sf2
boot_linux_console.py:BootLinuxConsole.test_arm_orangepi_sd
ppc_bamboo.py:BambooMachine.test_ppc_bamboo

These tests run a commandline that includes '-net user':
machine_aspeed.py:AST2x00Machine.test_arm_ast2500_evb_builroot
(and others that use the do_test_arm_aspeed_buidroot_start()
or do_test_arm_aspeed_sdk_start() helper functions)

These changes seem to be sufficient for 'make check-avocado'
to not fail on a --disable-slirp build.

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Message-Id: <20221001195224.2453581-1-peter.maydell@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>

show more ...

tests/avocado: split the AST2x00Machine classes

The SDK tests take a lot longer to run and hence need a longer
timeout. As they run well over the 60 second maximum for CI lets also
disable them for

tests/avocado: split the AST2x00Machine classes

The SDK tests take a lot longer to run and hence need a longer
timeout. As they run well over the 60 second maximum for CI lets also
disable them for CI as well.

I suspect they also suffer from the inability to detect the login
prompt due to no newlines being processed.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220914155950.804707-7-alex.bennee@linaro.org>

show more ...