1Jobs on Custom Runners
2======================
3
4Besides the jobs run under the various CI systems listed before, there
5are a number additional jobs that will run before an actual merge.
6These use the same GitLab CI's service/framework already used for all
7other GitLab based CI jobs, but rely on additional systems, not the
8ones provided by GitLab as "shared runners".
9
10The architecture of GitLab's CI service allows different machines to
11be set up with GitLab's "agent", called gitlab-runner, which will take
12care of running jobs created by events such as a push to a branch.
13Here, the combination of a machine, properly configured with GitLab's
14gitlab-runner, is called a "custom runner".
15
16The GitLab CI jobs definition for the custom runners are located under::
17
18  .gitlab-ci.d/custom-runners.yml
19
20Custom runners entail custom machines.  To see a list of the machines
21currently deployed in the QEMU GitLab CI and their maintainers, please
22refer to the QEMU `wiki <https://wiki.qemu.org/AdminContacts>`__.
23
24Machine Setup Howto
25-------------------
26
27For all Linux based systems, the setup can be mostly automated by the
28execution of two Ansible playbooks.  Create an ``inventory`` file
29under ``scripts/ci/setup``, such as this::
30
31  fully.qualified.domain
32  other.machine.hostname
33
34You may need to set some variables in the inventory file itself.  One
35very common need is to tell Ansible to use a Python 3 interpreter on
36those hosts.  This would look like::
37
38  fully.qualified.domain ansible_python_interpreter=/usr/bin/python3
39  other.machine.hostname ansible_python_interpreter=/usr/bin/python3
40
41Build environment
42~~~~~~~~~~~~~~~~~
43
44The ``scripts/ci/setup/$DISTRO/build-environment.yml`` Ansible
45playbook will set up machines with the environment needed to perform
46builds and run QEMU tests. This playbook consists on the installation
47of various required packages (and a general package update while at
48it).
49
50The minimum required version of Ansible successfully tested in this
51playbook is 2.8.0 (a version check is embedded within the playbook
52itself).  To run the playbook, execute::
53
54  cd scripts/ci/setup
55  ansible-playbook -i inventory $DISTRO/build-environment.yml
56
57Please note that most of the tasks in the playbook require superuser
58privileges, such as those from the ``root`` account or those obtained
59by ``sudo``.  If necessary, please refer to ``ansible-playbook``
60options such as ``--become``, ``--become-method``, ``--become-user``
61and ``--ask-become-pass``.
62
63gitlab-runner setup and registration
64~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
65
66The gitlab-runner agent needs to be installed on each machine that
67will run jobs.  The association between a machine and a GitLab project
68happens with a registration token.  To find the registration token for
69your repository/project, navigate on GitLab's web UI to:
70
71 * Settings (the gears-like icon at the bottom of the left hand side
72   vertical toolbar), then
73 * CI/CD, then
74 * Runners, and click on the "Expand" button, then
75 * Under "Set up a specific Runner manually", look for the value under
76   "And this registration token:"
77
78Copy the ``scripts/ci/setup/vars.yml.template`` file to
79``scripts/ci/setup/vars.yml``.  Then, set the
80``gitlab_runner_registration_token`` variable to the value obtained
81earlier.
82
83To run the playbook, execute::
84
85  cd scripts/ci/setup
86  ansible-playbook -i inventory gitlab-runner.yml
87
88Following the registration, it's necessary to configure the runner tags,
89and optionally other configurations on the GitLab UI.  Navigate to:
90
91 * Settings (the gears like icon), then
92 * CI/CD, then
93 * Runners, and click on the "Expand" button, then
94 * "Runners activated for this project", then
95 * Click on the "Edit" icon (next to the "Lock" Icon)
96
97Tags are very important as they are used to route specific jobs to
98specific types of runners, so it's a good idea to double check that
99the automatically created tags are consistent with the OS and
100architecture.  For instance, an Ubuntu 20.04 aarch64 system should
101have tags set as::
102
103  ubuntu_20.04,aarch64
104
105Because the job definition at ``.gitlab-ci.d/custom-runners.yml``
106would contain::
107
108  ubuntu-20.04-aarch64-all:
109   tags:
110   - ubuntu_20.04
111   - aarch64
112
113It's also recommended to:
114
115 * increase the "Maximum job timeout" to something like ``2h``
116 * give it a better Description
117