gitlab: centralize the container tag name

We use a fixed container tag of 'latest' so that contributors' forks
don't end up with an ever growing number of containers as they work
on throwaway featur

gitlab: centralize the container tag name

We use a fixed container tag of 'latest' so that contributors' forks
don't end up with an ever growing number of containers as they work
on throwaway feature branches.

This fixed tag causes problems running CI upstream in stable staging
branches, however, because the stable staging branch will publish old
container content that clashes with that needed by primary staging
branch. This makes it impossible to reliably run CI pipelines in
parallel in upstream for different staging branches.

This introduces $QEMU_CI_CONTAINER_TAG global variable as a way to
change which tag container publishing uses. Initially it can be set
by contributors as a git push option if they want to override the
default use of 'latest' eg

git push gitlab <branch> -o ci.variable=QEMU_CONTAINER_TAG=fish

this is useful if contributors need to run pipelines for different
branches concurrently in their forks.

Reviewed-by: Michael Tokarev <mjt@tls.msk.ru>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20230608164018.2520330-2-berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>

show more ...

gitlab-ci: Remove unused Python package

Python should have been removed in this commit:
https://gitlab.com/qemu-project/qemu/-/commit/94b8b146df84ba472f461398d93fb9cdf0db8f94

Signed-off-by: Camilla

gitlab-ci: Remove unused Python package

Python should have been removed in this commit:
https://gitlab.com/qemu-project/qemu/-/commit/94b8b146df84ba472f461398d93fb9cdf0db8f94

Signed-off-by: Camilla Conte <cconte@redhat.com>
Message-Id: <20230531150824.32349-2-cconte@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>

show more ...

gitlab: switch from 'stable' to 'latest' docker container tags

The 'stable' and 'stable-dind' tags are not documented as supported
tags at:

https://hub.docker.com/_/docker

Looking at their conte

gitlab: switch from 'stable' to 'latest' docker container tags

The 'stable' and 'stable-dind' tags are not documented as supported
tags at:

https://hub.docker.com/_/docker

Looking at their content they reflect docker 19.x.x release series,
were last built in Dec 2020, and have 3 critical and 20 high rated
CVEs unfixed. This obsolete status is attested by this commit:

https://github.com/docker-library/docker/commit/606c63960a4845af7077721eb3900c706f5d0c5e

The 'stable-dind' tag in particular appears buggy as it is unable to
resolve DNS for Fedora repos:

- Curl error (6): Couldn't resolve host name for https://mirrors.fedoraproject.org/metalink?repo=fedora-37&arch=x86_64&countme=1 [getaddrinfo() thread failed to start]

We used the 'stable' tag previously at the recommendation of GitLab
docs, but those docs are wrong and pending a fix:

https://gitlab.com/gitlab-org/gitlab/-/issues/409430

Fixes: 5f63a67adb58478974b91f5e5c2b1222b5c7f2cc
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Camilla Conte <cconte@redhat.com>
Message-Id: <20230531140654.1141145-1-berrange@redhat.com>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>

show more ...

Add loop over docker info

Wait for docker info to return successfuly to ensure that
the docker server (daemon) started.
This is needed for jobs running on Kubernetes.
See https://wiki.qemu.org/Testi

Add loop over docker info

Wait for docker info to return successfuly to ensure that
the docker server (daemon) started.
This is needed for jobs running on Kubernetes.
See https://wiki.qemu.org/Testing/CI/KubernetesRunners.

Signed-off-by: Camilla Conte <cconte@redhat.com>
Message-Id: <20230522174153.46801-4-cconte@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>

show more ...

Use docker "stable" tag

Use the same tag in all jobs.

Signed-off-by: Camilla Conte <cconte@redhat.com>
Message-Id: <20230522174153.46801-3-cconte@redhat.com>
Reviewed-by: Daniel P. Berrangé <berran

Use docker "stable" tag

Use the same tag in all jobs.

Signed-off-by: Camilla Conte <cconte@redhat.com>
Message-Id: <20230522174153.46801-3-cconte@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>

show more ...

Revert last two patches

Unintentionally pushed.

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>

Add CI configuration for Kubernetes

Configure Gitlab CI to run on Kubernetes
according to the official documentation.
https://docs.gitlab.com/ee/ci/docker/using_docker_build.html#docker-in-docker-wi

Add CI configuration for Kubernetes

Configure Gitlab CI to run on Kubernetes
according to the official documentation.
https://docs.gitlab.com/ee/ci/docker/using_docker_build.html#docker-in-docker-with-tls-enabled-in-kubernetes

These changes are needed because of the CI jobs
using Docker-in-Docker (dind).
As soon as Docker-in-Docker is replaced with Kaniko,
these changes can be reverted.

I documented what I did to set up the Kubernetes runner on the wiki:
https://wiki.qemu.org/Testing/CI/KubernetesRunners

Signed-off-by: Camilla Conte <cconte@redhat.com>
Message-Id: <20230407145252.32955-1-cconte@redhat.com>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>

show more ...

gitlab: Use plain docker in container-template.yml

Our dockerfiles no longer reference layers from other qemu images so
we can now use 'docker build' on them.

Also reinstate the caching that was di

gitlab: Use plain docker in container-template.yml

Our dockerfiles no longer reference layers from other qemu images so
we can now use 'docker build' on them.

Also reinstate the caching that was disabled due to bad interactions
with certain runners. See commit 6ddc3dc7a8 ("tests/docker: don't use
BUILDKIT in GitLab either"). We now believe those issues to be fixed.

The COMMON_TAG needed to be fixed for the caching to work. The
docker.py script was not using the variable, but constructing the
correct URL directly.

Signed-off-by: Fabiano Rosas <farosas@suse.de>
Tested-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20230227151110.31455-2-farosas@suse.de>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20230228190653.1602033-16-alex.bennee@linaro.org>

show more ...

gitlab: convert build/container jobs to .base_job_template

This converts the main build and container jobs to use the
base job rules, defining the following new variables

- QEMU_JOB_SKIPPED - jobs

gitlab: convert build/container jobs to .base_job_template

This converts the main build and container jobs to use the
base job rules, defining the following new variables

- QEMU_JOB_SKIPPED - jobs that are known to be currently
broken and should not be run. Can still be manually
launched if desired.

- QEMU_JOB_AVOCADO - jobs that run the Avocado integration
test harness.

- QEMU_JOB_PUBLISH - jobs that publish content after the
branch is merged upstream

As build-tools-and-docs runs on master we declare the requirement of
building amd64-debian-container optional as it should already exits
once we merge.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20220526110705.59952-5-berrange@redhat.com>
[AJB: fix upstream typo, mention optional container req]
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20220527153603.887929-32-alex.bennee@linaro.org>

show more ...

Merge remote-tracking branch 'remotes/jasowang/tags/net-pull-request' into staging

# gpg: Signature made Fri 04 Jun 2021 08:26:16 BST
# gpg: using RSA key EF04965B398D6211

Merge remote-tracking branch 'remotes/jasowang/tags/net-pull-request' into staging

# gpg: Signature made Fri 04 Jun 2021 08:26:16 BST
# gpg: using RSA key EF04965B398D6211
# gpg: Good signature from "Jason Wang (Jason Wang on RedHat) <jasowang@redhat.com>" [marginal]
# gpg: WARNING: This key is not certified with sufficiently trusted signatures!
# gpg: It is not certain that the signature belongs to the owner.
# Primary key fingerprint: 215D 46F4 8246 689E C77F 3562 EF04 965B 398D 6211

* remotes/jasowang/tags/net-pull-request:
MAINTAINERS: Added eBPF maintainers information.
docs: Added eBPF documentation.
virtio-net: Added eBPF RSS to virtio-net.
ebpf: Added eBPF RSS loader.
ebpf: Added eBPF RSS program.
net: Added SetSteeringEBPF method for NetClientState.
net/tap: Added TUNSETSTEERINGEBPF code.

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

Merge remote-tracking branch 'remotes/kraxel/tags/vga-20210527-pull-request' into staging

virtio-gpu: add blob resource support.
vhost-user-gpu: security fixes.

# gpg: Signature

Merge remote-tracking branch 'remotes/kraxel/tags/vga-20210527-pull-request' into staging

virtio-gpu: add blob resource support.
vhost-user-gpu: security fixes.

# gpg: Signature made Thu 27 May 2021 15:23:25 BST
# gpg: using RSA key A0328CFFB93A17A79901FE7D4CB6D8EED3E87138
# gpg: Good signature from "Gerd Hoffmann (work) <kraxel@redhat.com>" [full]
# gpg: aka "Gerd Hoffmann <gerd@kraxel.org>" [full]
# gpg: aka "Gerd Hoffmann (private) <kraxel@gmail.com>" [full]
# Primary key fingerprint: A032 8CFF B93A 17A7 9901 FE7D 4CB6 D8EE D3E8 7138

* remotes/kraxel/tags/vga-20210527-pull-request: (23 commits)
virtio-gpu: Update cursor data using blob
virtio-gpu: Add virtio_gpu_set_scanout_blob
virtio-gpu: Factor out update scanout
virtio-gpu: Add helpers to create and destroy dmabuf objects
ui/pixman: Add qemu_pixman_to_drm_format()
virtio-gpu: Add virtio_gpu_resource_create_blob
virtio-gpu: Add initial definitions for blob resources
virtio-gpu: Refactor virtio_gpu_create_mapping_iov
virtio-gpu: Refactor virtio_gpu_set_scanout
virtio-gpu: Add virtio_gpu_find_check_resource
stubs: Add stubs for udmabuf helpers
virtio-gpu: Add udmabuf helpers
headers: Add udmabuf.h
ui: Get the fd associated with udmabuf driver
hw/display/qxl: Set pci rom address aligned with page size
vhost-user-gpu: abstract vg_cleanup_mapping_iov
vhost-user-gpu: fix OOB write in 'virgl_cmd_get_capset' (CVE-2021-3546)
vhost-user-gpu: fix memory leak in 'virgl_resource_attach_backing' (CVE-2021-3544)
vhost-user-gpu: fix memory leak in 'virgl_cmd_resource_unref' (CVE-2021-3544)
vhost-user-gpu: fix memory leak while calling 'vg_resource_unref' (CVE-2021-3544)
...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

Merge remote-tracking branch 'remotes/jasowang/tags/net-pull-request' into staging

# gpg: Signature made Thu 27 May 2021 04:06:17 BST
# gpg: using RSA key EF04965B398D6211

Merge remote-tracking branch 'remotes/jasowang/tags/net-pull-request' into staging

# gpg: Signature made Thu 27 May 2021 04:06:17 BST
# gpg: using RSA key EF04965B398D6211
# gpg: Good signature from "Jason Wang (Jason Wang on RedHat) <jasowang@redhat.com>" [marginal]
# gpg: WARNING: This key is not certified with sufficiently trusted signatures!
# gpg: It is not certain that the signature belongs to the owner.
# Primary key fingerprint: 215D 46F4 8246 689E C77F 3562 EF04 965B 398D 6211

* remotes/jasowang/tags/net-pull-request:
tap-bsd: Remove special casing for older OpenBSD releases
virtio-net: failover: add missing remove_migration_state_change_notifier()
hw/net/imx_fec: return 0xffff when accessing non-existing PHY

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

Merge remote-tracking branch 'remotes/rth-gitlab/tags/pull-tcg-20210526' into staging

Adjust types for some memory access functions.
Reduce inclusion of tcg headers.
Fix watchpoints

Merge remote-tracking branch 'remotes/rth-gitlab/tags/pull-tcg-20210526' into staging

Adjust types for some memory access functions.
Reduce inclusion of tcg headers.
Fix watchpoints vs replay.
Fix tcg/aarch64 roli expansion.
Introduce SysemuCPUOps structure.

# gpg: Signature made Thu 27 May 2021 00:43:54 BST
# gpg: using RSA key 7A481E78868B4DB6A85A05C064DF38E8AF7E215F
# gpg: issuer "richard.henderson@linaro.org"
# gpg: Good signature from "Richard Henderson <richard.henderson@linaro.org>" [full]
# Primary key fingerprint: 7A48 1E78 868B 4DB6 A85A 05C0 64DF 38E8 AF7E 215F

* remotes/rth-gitlab/tags/pull-tcg-20210526: (31 commits)
hw/core: Constify TCGCPUOps
target/mips: Fold jazz behaviour into mips_cpu_do_transaction_failed
cpu: Move CPUClass::get_paging_enabled to SysemuCPUOps
cpu: Move CPUClass::get_memory_mapping to SysemuCPUOps
cpu: Move CPUClass::get_phys_page_debug to SysemuCPUOps
cpu: Move CPUClass::asidx_from_attrs to SysemuCPUOps
cpu: Move CPUClass::write_elf* to SysemuCPUOps
cpu: Move CPUClass::get_crash_info to SysemuCPUOps
cpu: Move CPUClass::virtio_is_big_endian to SysemuCPUOps
cpu: Move CPUClass::vmsd to SysemuCPUOps
cpu: Introduce SysemuCPUOps structure
cpu: Move AVR target vmsd field from CPUClass to DeviceClass
cpu: Rename CPUClass vmsd -> legacy_vmsd
cpu: Assert DeviceClass::vmsd is NULL on user emulation
cpu: Directly use get_memory_mapping() fallback handlers in place
cpu: Directly use get_paging_enabled() fallback handlers in place
cpu: Directly use cpu_write_elf*() fallback handlers in place
cpu: Introduce cpu_virtio_is_big_endian()
cpu: Un-inline cpu_get_phys_page_debug and cpu_asidx_from_attrs
cpu: Split as cpu-common / cpu-sysemu
...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

Merge remote-tracking branch 'remotes/vivier/tags/m68k-for-6.1-pull-request' into staging

m68k pull request 20210526

implement m68k "any instruction" trace mode

# gpg: Sign

Merge remote-tracking branch 'remotes/vivier/tags/m68k-for-6.1-pull-request' into staging

m68k pull request 20210526

implement m68k "any instruction" trace mode

# gpg: Signature made Wed 26 May 2021 20:56:58 BST
# gpg: using RSA key CD2F75DDC8E3A4DC2E4F5173F30C38BD3F2FBE3C
# gpg: issuer "laurent@vivier.eu"
# gpg: Good signature from "Laurent Vivier <lvivier@redhat.com>" [full]
# gpg: aka "Laurent Vivier <laurent@vivier.eu>" [full]
# gpg: aka "Laurent Vivier (Red Hat) <lvivier@redhat.com>" [full]
# Primary key fingerprint: CD2F 75DD C8E3 A4DC 2E4F 5173 F30C 38BD 3F2F BE3C

* remotes/vivier/tags/m68k-for-6.1-pull-request:
target/m68k: implement m68k "any instruction" trace mode
target/m68k: introduce gen_singlestep_exception() function
target/m68k: call gen_raise_exception() directly if single-stepping in gen_jmp_tb()
target/m68k: introduce is_singlestepping() function

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

Merge remote-tracking branch 'remotes/philmd/tags/gitlab-ci-20210527' into staging

GitLab CI patches queue

- Explode .gitlab-ci.yml in reusable templates
- Add job to cross buil

Merge remote-tracking branch 'remotes/philmd/tags/gitlab-ci-20210527' into staging

GitLab CI patches queue

- Explode .gitlab-ci.yml in reusable templates
- Add job to cross build/test TCI on i386 host
- Remove CentOS 7 linux-user build job
- Temporarily set Avocado-based jobs in manual mode
- Increase time to hold Avocado reports to 1 week

# gpg: Signature made Thu 27 May 2021 15:40:58 BST
# gpg: using RSA key FAABE75E12917221DCFD6BB2E3E32C2CDEADC0DE
# gpg: Good signature from "Philippe Mathieu-Daudé (F4BUG) <f4bug@amsat.org>" [full]
# Primary key fingerprint: FAAB E75E 1291 7221 DCFD 6BB2 E3E3 2C2C DEAD C0DE

* remotes/philmd/tags/gitlab-ci-20210527:
gitlab: Convert check-dco/check-patch jobs to the 'rules' syntax
gitlab: Use $CI_DEFAULT_BRANCH instead of hardcoded 'master'
gitlab: Run Avocado tests manually (except mainstream CI)
gitlab: Keep Avocado reports during 1 week
gitlab: Extract cross-container jobs to container-cross.yml
gitlab: Document how forks can use different set of jobs
gitlab: Move current job set to qemu-project.yml
gitlab: Extract all default build/test jobs to buildtest.yml
gitlab: Drop linux user build job for CentOS 7
gitlab: Extract core container jobs to container-core.yml
gitlab: Extract default build/test jobs templates
gitlab: Extract build stages to stages.yml
gitlab: Extract DCO/style check jobs to static_checks.yml
gitlab: Extract crossbuild job templates to crossbuild-template.yml
gitlab: Extract container job template to container-template.yml
gitlab: Enable cross-i386 builds of TCI
gitlab: Rename ACCEL_CONFIGURE_OPTS to EXTRA_CONFIGURE_OPTS
gitlab: Replace YAML anchors by extends (acceptance_test_job)

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

gitlab: Extract container job template to container-template.yml

Extract the container job template to a new file
(container-template.yml) to be able to reuse it
without having to ru

gitlab: Extract container job template to container-template.yml

Extract the container job template to a new file
(container-template.yml) to be able to reuse it
without having to run all the jobs included, which
are mainly useful for mainstream CI.

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Message-Id: <20210519185504.2198573-4-f4bug@amsat.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>

show more ...