1# Patchtest 2 3## Introduction 4 5Patchtest is a test framework for community patches based on the standard 6unittest python module. As input, it needs tree elements to work properly: 7a patch in mbox format (either created with `git format-patch` or fetched 8from 'patchwork'), a test suite and a target repository. 9 10The first test suite intended to be used with patchtest is found in the 11openembedded-core repository [1] targeted for patches that get into the 12openembedded-core mailing list [2]. This suite is also intended as a 13baseline for development of similar suites for other layers as needed. 14 15Patchtest can either run on a host or a guest machine, depending on which 16environment the execution needs to be done. If you plan to test your own patches 17(a good practice before these are sent to the mailing list), the easiest way is 18to install and execute on your local host; in the other hand, if automatic 19testing is intended, the guest method is strongly recommended. The guest 20method requires the use of the patchtest layer, in addition to the tools 21available in oe-core: https://git.yoctoproject.org/patchtest/ 22 23## Installation 24 25As a tool for use with the Yocto Project, the [quick start guide](https://docs.yoctoproject.org/brief-yoctoprojectqs/index.html) 26contains the necessary prerequisites for a basic project. In addition, 27patchtest relies on the following Python modules: 28 29- boto3 (for sending automated results emails only) 30- git-pw>=2.5.0 31- jinja2 32- pylint 33- pyparsing>=3.0.9 34- unidiff 35 36These can be installed by running `pip install -r 37meta/lib/patchtest/requirements.txt`. Note that git-pw is not 38automatically added to the user's PATH; by default, it is installed at 39~/.local/bin/git-pw. 40 41For git-pw (and therefore scripts such as patchtest-get--series) to work, you need 42to provide a Patchwork instance in your user's .gitconfig, like so (the project 43can be specified using the --project argument): 44 45 git config --global pw.server "https://patchwork.yoctoproject.org/api/1.2/" 46 47To work with patchtest, you should have the following repositories cloned: 48 491. https://git.openembedded.org/openembedded-core/ (or https://git.yoctoproject.org/poky/) 502. https://git.openembedded.org/bitbake/ (if not using poky) 513. https://git.yoctoproject.org/patchtest (if using guest mode) 52 53## Usage 54 55### Obtaining Patches 56 57Patch files can be obtained directly from cloned repositories using `git 58format-patch -N` (where N is the number of patches starting from HEAD to 59generate). git-pw can also be used with filters for users, patch/series IDs, 60and timeboxes if specific patches are desired. For more information, see the 61git-pw [documentation](https://patchwork.readthedocs.io/projects/git-pw/en/latest/). 62 63Alternatively, `scripts/patchtest-get-series` can be used to pull mbox files from 64the Patchwork instance configured previously in .gitconfig. It uses a log file 65called ".series_test.log" to store and compare series IDs so that the same 66versions of a patch are not tested multiple times unintentionally. By default, 67it will pull up to five patch series from the last 30 minutes using oe-core as 68the target project, but these parameters can be configured using the `--limit`, 69`--interval`, and `--project` arguments respectively. For more information, run 70`patchtest-get-series -h`. 71 72### Host Mode 73 74To run patchtest on the host, do the following: 75 761. In openembedded-core/poky, do `source oe-init-build-env` 772. Generate patch files from the target repository by doing `git-format patch -N`, 78 where N is the number of patches starting at HEAD, or by using git-pw 79 or patchtest-get-series 803. Run patchtest on a patch file by doing the following: 81 82 patchtest --patch /path/to/patch/file 83 84 or, if you have stored the patch files in a directory, do: 85 86 patchtest --directory /path/to/patch/directory 87 88 For example, to test `master-gcc-Fix--fstack-protector-issue-on-aarch64.patch` against the oe-core test suite: 89 90 patchtest --patch master-gcc-Fix--fstack-protector-issue-on-aarch64.patch 91 92 If you want to use a different test suite or target repository, you can use the --testdir and --repodir flags: 93 94 patchtest --patch /path/to/patch/file --repodir /path/to/repo --testdir /path/to/test/dir 95 96### Guest Mode 97 98Patchtest's guest mode has been refactored to more closely mirror the 99typical Yocto Project image build workflow, but there are still some key 100differences to keep in mind. The primary objective is to provide a level 101of isolation from the host when testing patches pulled automatically 102from the mailing lists. When executed this way, the test process is 103essentially running random code from the internet and could be 104catastrophic if malicious bits or even poorly-handled edge cases aren't 105protected against. In order to use this mode, the 106https://git.yoctoproject.org/patchtest/ repository must be cloned and 107the meta-patchtest layer added to bblayers.conf. 108 109The general flow of guest mode is: 110 1111. Run patchtest-setup-sharedir --directory <dirname> to create a 112 directory for mounting 1132. Collect patches via patchtest-get-series (or other manual step) into the 114 <dirname>/mboxes path 1153. Ensure that a user with ID 1200 has appropriate read/write 116 permissions to <dirname> and <dirname>/mboxes, so that the 117 "patchtest" user in the core-image-patchtest image can function 1184. Build the core-image-patchtest image 1195. Run the core-image-patchtest image with the mounted sharedir, like 120 so: 121 `runqemu kvm nographic qemuparams="-snapshot -fsdev 122 local,id=test_mount,path=/workspace/yocto/poky/build/patchtestdir,security_model=mapped 123 -device virtio-9p-pci,fsdev=test_mount,mount_tag=test_mount -smp 4 -m 124 2048"` 125 126Patchtest runs as an initscript for the core-image-patchtest image and 127shuts down after completion, so there is no input required from a user 128during operation. Unlike in host mode, the guest is designed to 129automatically generate test result files, in the same directory as the 130targeted patch files but with .testresult as an extension. These contain 131the entire output of the patchtest run for each respective pass, 132including the PASS, FAIL, and SKIP indicators for each test run. 133 134## Contributing 135 136The yocto mailing list (openembedded-core@lists.openembedded.org) is used for questions, 137comments and patch review. It is subscriber only, so please register before 138posting. 139 140When sending single patches, please use something like: 141 142 git send-email -M -1 --to=openembedded-core@lists.openembedded.org --subject-prefix=OE-core][PATCH 143 144## Maintenance 145----------- 146 147Maintainers: 148 Trevor Gamblin <tgamblin@baylibre.com> 149 150## Links 151----- 152[1] https://git.openembedded.org/openembedded-core/ 153[2] https://www.yoctoproject.org/community/mailing-lists/ 154