17266ecceSAlex Bennée.. _ci_var: 27266ecceSAlex Bennée 3bcfdfae7SPaolo BonziniCustom CI/CD variables 4bcfdfae7SPaolo Bonzini====================== 5bcfdfae7SPaolo Bonzini 6bcfdfae7SPaolo BonziniQEMU CI pipelines can be tuned by setting some CI environment variables. 7bcfdfae7SPaolo Bonzini 8bcfdfae7SPaolo BonziniSet variable globally in the user's CI namespace 9bcfdfae7SPaolo Bonzini------------------------------------------------ 10bcfdfae7SPaolo Bonzini 11bcfdfae7SPaolo BonziniVariables can be set globally in the user's CI namespace setting. 12bcfdfae7SPaolo Bonzini 13bcfdfae7SPaolo BonziniFor further information about how to set these variables, please refer to:: 14bcfdfae7SPaolo Bonzini 15bcfdfae7SPaolo Bonzini https://docs.gitlab.com/ee/ci/variables/#add-a-cicd-variable-to-a-project 16bcfdfae7SPaolo Bonzini 17bcfdfae7SPaolo BonziniSet variable manually when pushing a branch or tag to the user's repository 18bcfdfae7SPaolo Bonzini--------------------------------------------------------------------------- 19bcfdfae7SPaolo Bonzini 20bcfdfae7SPaolo BonziniVariables can be set manually when pushing a branch or tag, using 21bcfdfae7SPaolo Bonzinigit-push command line arguments. 22bcfdfae7SPaolo Bonzini 23bcfdfae7SPaolo BonziniExample setting the QEMU_CI_EXAMPLE_VAR variable: 24bcfdfae7SPaolo Bonzini 25bcfdfae7SPaolo Bonzini.. code:: 26bcfdfae7SPaolo Bonzini 27bcfdfae7SPaolo Bonzini git push -o ci.variable="QEMU_CI_EXAMPLE_VAR=value" myrepo mybranch 28bcfdfae7SPaolo Bonzini 29bcfdfae7SPaolo BonziniFor further information about how to set these variables, please refer to:: 30bcfdfae7SPaolo Bonzini 31bcfdfae7SPaolo Bonzini https://docs.gitlab.com/ee/user/project/push_options.html#push-options-for-gitlab-cicd 32bcfdfae7SPaolo Bonzini 3328357dc5SDaniel P. BerrangéSetting aliases in your git config 3428357dc5SDaniel P. Berrangé---------------------------------- 3528357dc5SDaniel P. Berrangé 3628357dc5SDaniel P. BerrangéYou can use aliases to make it easier to push branches with different 3728357dc5SDaniel P. BerrangéCI configurations. For example define an alias for triggering CI: 3828357dc5SDaniel P. Berrangé 3928357dc5SDaniel P. Berrangé.. code:: 4028357dc5SDaniel P. Berrangé 4128357dc5SDaniel P. Berrangé git config --local alias.push-ci "push -o ci.variable=QEMU_CI=1" 4228357dc5SDaniel P. Berrangé git config --local alias.push-ci-now "push -o ci.variable=QEMU_CI=2" 4328357dc5SDaniel P. Berrangé 4428357dc5SDaniel P. BerrangéWhich lets you run: 4528357dc5SDaniel P. Berrangé 4628357dc5SDaniel P. Berrangé.. code:: 4728357dc5SDaniel P. Berrangé 4828357dc5SDaniel P. Berrangé git push-ci 4928357dc5SDaniel P. Berrangé 5028357dc5SDaniel P. Berrangéto create the pipeline, or: 5128357dc5SDaniel P. Berrangé 5228357dc5SDaniel P. Berrangé.. code:: 5328357dc5SDaniel P. Berrangé 5428357dc5SDaniel P. Berrangé git push-ci-now 5528357dc5SDaniel P. Berrangé 5628357dc5SDaniel P. Berrangéto create and run the pipeline 5728357dc5SDaniel P. Berrangé 5828357dc5SDaniel P. Berrangé 596a0e7ea7SDaniel P. BerrangéVariable naming and grouping 606a0e7ea7SDaniel P. Berrangé---------------------------- 616a0e7ea7SDaniel P. Berrangé 626a0e7ea7SDaniel P. BerrangéThe variables used by QEMU's CI configuration are grouped together 636a0e7ea7SDaniel P. Berrangéin a handful of namespaces 646a0e7ea7SDaniel P. Berrangé 656a0e7ea7SDaniel P. Berrangé * QEMU_JOB_nnnn - variables to be defined in individual jobs 666a0e7ea7SDaniel P. Berrangé or templates, to influence the shared rules defined in the 676a0e7ea7SDaniel P. Berrangé .base_job_template. 686a0e7ea7SDaniel P. Berrangé 696a0e7ea7SDaniel P. Berrangé * QEMU_CI_nnn - variables to be set by contributors in their 706a0e7ea7SDaniel P. Berrangé repository CI settings, or as git push variables, to influence 716a0e7ea7SDaniel P. Berrangé which jobs get run in a pipeline 726a0e7ea7SDaniel P. Berrangé 73d4c7a565SDaniel P. Berrangé * QEMU_CI_CONTAINER_TAG - the tag used to publish containers 74d4c7a565SDaniel P. Berrangé in stage 1, for use by build jobs in stage 2. Defaults to 75d4c7a565SDaniel P. Berrangé 'latest', but if running pipelines for different branches 76d4c7a565SDaniel P. Berrangé concurrently, it should be overridden per pipeline. 77d4c7a565SDaniel P. Berrangé 781ddd2ff9SDaniel P. Berrangé * QEMU_CI_UPSTREAM - gitlab namespace that is considered to be 791ddd2ff9SDaniel P. Berrangé the 'upstream'. This defaults to 'qemu-project'. Contributors 801ddd2ff9SDaniel P. Berrangé may choose to override this if they are modifying rules in 811ddd2ff9SDaniel P. Berrangé base.yml and need to validate how they will operate when in 821ddd2ff9SDaniel P. Berrangé an upstream context, as opposed to their fork context. 831ddd2ff9SDaniel P. Berrangé 846a0e7ea7SDaniel P. Berrangé * nnn - other misc variables not falling into the above 856a0e7ea7SDaniel P. Berrangé categories, or using different names for historical reasons 866a0e7ea7SDaniel P. Berrangé and not yet converted. 876a0e7ea7SDaniel P. Berrangé 886a0e7ea7SDaniel P. BerrangéMaintainer controlled job variables 896a0e7ea7SDaniel P. Berrangé----------------------------------- 906a0e7ea7SDaniel P. Berrangé 916a0e7ea7SDaniel P. BerrangéThe following variables may be set when defining a job in the 926a0e7ea7SDaniel P. BerrangéCI configuration file. 936a0e7ea7SDaniel P. Berrangé 9400125414SDaniel P. BerrangéQEMU_JOB_CIRRUS 9500125414SDaniel P. Berrangé~~~~~~~~~~~~~~~ 9600125414SDaniel P. Berrangé 9700125414SDaniel P. BerrangéThe job makes use of Cirrus CI infrastructure, requiring the 9800125414SDaniel P. Berrangéconfiguration setup for cirrus-run to be present in the repository 9900125414SDaniel P. Berrangé 10000125414SDaniel P. BerrangéQEMU_JOB_OPTIONAL 10100125414SDaniel P. Berrangé~~~~~~~~~~~~~~~~~ 10200125414SDaniel P. Berrangé 10300125414SDaniel P. BerrangéThe job is expected to be successful in general, but is not run 10400125414SDaniel P. Berrangéby default due to need to conserve limited CI resources. It is 10500125414SDaniel P. Berrangéavailable to be started manually by the contributor in the CI 10600125414SDaniel P. Berrangépipelines UI. 10700125414SDaniel P. Berrangé 10816fee101SDaniel P. BerrangéQEMU_JOB_ONLY_FORKS 10916fee101SDaniel P. Berrangé~~~~~~~~~~~~~~~~~~~ 11016fee101SDaniel P. Berrangé 11116fee101SDaniel P. BerrangéThe job results are only of interest to contributors prior to 11216fee101SDaniel P. Berrangésubmitting code. They are not required as part of the gating 11316fee101SDaniel P. BerrangéCI pipeline. 11416fee101SDaniel P. Berrangé 115e312d1fdSDaniel P. BerrangéQEMU_JOB_SKIPPED 116e312d1fdSDaniel P. Berrangé~~~~~~~~~~~~~~~~ 117e312d1fdSDaniel P. Berrangé 118e312d1fdSDaniel P. BerrangéThe job is not reliably successsful in general, so is not 119e312d1fdSDaniel P. Berrangécurrently suitable to be run by default. Ideally this should 120e312d1fdSDaniel P. Berrangébe a temporary marker until the problems can be addressed, or 121e312d1fdSDaniel P. Berrangéthe job permanently removed. 122e312d1fdSDaniel P. Berrangé 123e312d1fdSDaniel P. BerrangéQEMU_JOB_PUBLISH 124e312d1fdSDaniel P. Berrangé~~~~~~~~~~~~~~~~ 125e312d1fdSDaniel P. Berrangé 126e312d1fdSDaniel P. BerrangéThe job is for publishing content after a branch has been 127e312d1fdSDaniel P. Berrangémerged into the upstream default branch. 128e312d1fdSDaniel P. Berrangé 129e312d1fdSDaniel P. BerrangéQEMU_JOB_AVOCADO 130e312d1fdSDaniel P. Berrangé~~~~~~~~~~~~~~~~ 131e312d1fdSDaniel P. Berrangé 132e312d1fdSDaniel P. BerrangéThe job runs the Avocado integration test suite 133e312d1fdSDaniel P. Berrangé 1346a0e7ea7SDaniel P. BerrangéContributor controlled runtime variables 1356a0e7ea7SDaniel P. Berrangé---------------------------------------- 1366a0e7ea7SDaniel P. Berrangé 1376a0e7ea7SDaniel P. BerrangéThe following variables may be set by contributors to control 1386a0e7ea7SDaniel P. Berrangéjob execution 139bcfdfae7SPaolo Bonzini 14028357dc5SDaniel P. BerrangéQEMU_CI 14128357dc5SDaniel P. Berrangé~~~~~~~ 14228357dc5SDaniel P. Berrangé 14328357dc5SDaniel P. BerrangéBy default, no pipelines will be created on contributor forks 14428357dc5SDaniel P. Berrangéin order to preserve CI credits 14528357dc5SDaniel P. Berrangé 14628357dc5SDaniel P. BerrangéSet this variable to 1 to create the pipelines, but leave all 14728357dc5SDaniel P. Berrangéthe jobs to be manually started from the UI 14828357dc5SDaniel P. Berrangé 14928357dc5SDaniel P. BerrangéSet this variable to 2 to create the pipelines and run all 15028357dc5SDaniel P. Berrangéthe jobs immediately, as was historicaly behaviour 15128357dc5SDaniel P. Berrangé 152bcfdfae7SPaolo BonziniQEMU_CI_AVOCADO_TESTING 153bcfdfae7SPaolo Bonzini~~~~~~~~~~~~~~~~~~~~~~~ 154bcfdfae7SPaolo BonziniBy default, tests using the Avocado framework are not run automatically in 155bcfdfae7SPaolo Bonzinithe pipelines (because multiple artifacts have to be downloaded, and if 156bcfdfae7SPaolo Bonzinithese artifacts are not already cached, downloading them make the jobs 157bcfdfae7SPaolo Bonzinireach the timeout limit). Set this variable to have the tests using the 158bcfdfae7SPaolo BonziniAvocado framework run automatically. 159bcfdfae7SPaolo Bonzini 1606a0e7ea7SDaniel P. BerrangéOther misc variables 1616a0e7ea7SDaniel P. Berrangé-------------------- 1626a0e7ea7SDaniel P. Berrangé 1636a0e7ea7SDaniel P. BerrangéThese variables are primarily to control execution of jobs on 1646a0e7ea7SDaniel P. Berrangéprivate runners 1656a0e7ea7SDaniel P. Berrangé 166bcfdfae7SPaolo BonziniAARCH64_RUNNER_AVAILABLE 167bcfdfae7SPaolo Bonzini~~~~~~~~~~~~~~~~~~~~~~~~ 168bcfdfae7SPaolo BonziniIf you've got access to an aarch64 host that can be used as a gitlab-CI 169bcfdfae7SPaolo Bonzinirunner, you can set this variable to enable the tests that require this 170bcfdfae7SPaolo Bonzinikind of host. The runner should be tagged with "aarch64". 171bcfdfae7SPaolo Bonzini 172cc44a160SAlex BennéeAARCH32_RUNNER_AVAILABLE 173cc44a160SAlex Bennée~~~~~~~~~~~~~~~~~~~~~~~~ 174cc44a160SAlex BennéeIf you've got access to an armhf host or an arch64 host that can run 175cc44a160SAlex Bennéeaarch32 EL0 code to be used as a gitlab-CI runner, you can set this 176cc44a160SAlex Bennéevariable to enable the tests that require this kind of host. The 177cc44a160SAlex Bennéerunner should be tagged with "aarch32". 178cc44a160SAlex Bennée 179bcfdfae7SPaolo BonziniS390X_RUNNER_AVAILABLE 180bcfdfae7SPaolo Bonzini~~~~~~~~~~~~~~~~~~~~~~ 181bcfdfae7SPaolo BonziniIf you've got access to an IBM Z host that can be used as a gitlab-CI 182bcfdfae7SPaolo Bonzinirunner, you can set this variable to enable the tests that require this 183bcfdfae7SPaolo Bonzinikind of host. The runner should be tagged with "s390x". 184d7c2e2b3SCleber Rosa 185d7c2e2b3SCleber RosaCENTOS_STREAM_8_x86_64_RUNNER_AVAILABLE 186d7c2e2b3SCleber Rosa~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 187d7c2e2b3SCleber RosaIf you've got access to a CentOS Stream 8 x86_64 host that can be 188d7c2e2b3SCleber Rosaused as a gitlab-CI runner, you can set this variable to enable the 189d7c2e2b3SCleber Rosatests that require this kind of host. The runner should be tagged with 190d7c2e2b3SCleber Rosaboth "centos_stream_8" and "x86_64". 191*2f7350cdSDaniel P. Berrangé 192*2f7350cdSDaniel P. BerrangéCCACHE_DISABLE 193*2f7350cdSDaniel P. Berrangé~~~~~~~~~~~~~~ 194*2f7350cdSDaniel P. BerrangéThe jobs are configured to use "ccache" by default since this typically 195*2f7350cdSDaniel P. Berrangéreduces compilation time, at the cost of increased storage. If the 196*2f7350cdSDaniel P. Berrangéuse of "ccache" is suspected to be hurting the overall job execution 197*2f7350cdSDaniel P. Berrangétime, setting the "CCACHE_DISABLE=1" env variable to disable it. 198