1.. SPDX-License-Identifier: CC-BY-SA-2.0-UK 2 3*************************** 4Testing Packages With ptest 5*************************** 6 7A Package Test (ptest) runs tests against packages built by the 8OpenEmbedded build system on the target machine. A ptest contains at 9least two items: the actual test, and a shell script (``run-ptest``) 10that starts the test. The shell script that starts the test must not 11contain the actual test --- the script only starts the test. On the other 12hand, the test can be anything from a simple shell script that runs a 13binary and checks the output to an elaborate system of test binaries and 14data files. 15 16The test generates output in the format used by Automake:: 17 18 result: testname 19 20where the result can be ``PASS``, ``FAIL``, or ``SKIP``, and 21the testname can be any identifying string. 22 23For a list of Yocto Project recipes that are already enabled with ptest, 24see the :yocto_wiki:`Ptest </Ptest>` wiki page. 25 26.. note:: 27 28 A recipe is "ptest-enabled" if it inherits the :ref:`ref-classes-ptest` 29 class. 30 31Adding ptest to Your Build 32========================== 33 34To add package testing to your build, add the :term:`DISTRO_FEATURES` and 35:term:`EXTRA_IMAGE_FEATURES` variables to your ``local.conf`` file, which 36is found in the :term:`Build Directory`:: 37 38 DISTRO_FEATURES:append = " ptest" 39 EXTRA_IMAGE_FEATURES += "ptest-pkgs" 40 41Once your build is complete, the ptest files are installed into the 42``/usr/lib/package/ptest`` directory within the image, where ``package`` 43is the name of the package. 44 45Running ptest 46============= 47 48The ``ptest-runner`` package installs a shell script that loops through 49all installed ptest test suites and runs them in sequence. Consequently, 50you might want to add this package to your image. 51 52Getting Your Package Ready 53========================== 54 55In order to enable a recipe to run installed ptests on target hardware, 56you need to prepare the recipes that build the packages you want to 57test. Here is what you have to do for each recipe: 58 59- *Be sure the recipe inherits the* :ref:`ref-classes-ptest` *class:* 60 Include the following line in each recipe:: 61 62 inherit ptest 63 64 .. note:: 65 66 Classes for common frameworks already exist in :term:`OpenEmbedded-Core 67 (OE-Core)`, such as: 68 69 - :oe_git:`go-ptest </openembedded-core/tree/meta/classes-recipe/go-ptest.bbclass>` 70 - :ref:`ref-classes-ptest-cargo` 71 - :ref:`ref-classes-ptest-gnome` 72 - :oe_git:`ptest-perl </openembedded-core/tree/meta/classes-recipe/ptest-perl.bbclass>` 73 - :oe_git:`ptest-python-pytest </openembedded-core/tree/meta/classes-recipe/ptest-python-pytest.bbclass>` 74 75 Inheriting these classes with the ``inherit`` keyword in your recipe will 76 make the next steps automatic. 77 78- *Create run-ptest:* This script starts your test. Locate the 79 script where you will refer to it using 80 :term:`SRC_URI`. Here is an 81 example that starts a test for ``dbus``:: 82 83 #!/bin/sh 84 cd test 85 make -k runtest-TESTS 86 87- *Ensure dependencies are met:* If the test adds build or runtime 88 dependencies that normally do not exist for the package (such as 89 requiring "make" to run the test suite), use the 90 :term:`DEPENDS` and 91 :term:`RDEPENDS` variables in 92 your recipe in order for the package to meet the dependencies. Here 93 is an example where the package has a runtime dependency on "make":: 94 95 RDEPENDS:${PN}-ptest += "make" 96 97- *Add a function to build the test suite:* Not many packages support 98 cross-compilation of their test suites. Consequently, you usually 99 need to add a cross-compilation function to the package. 100 101 Many packages based on Automake compile and run the test suite by 102 using a single command such as ``make check``. However, the host 103 ``make check`` builds and runs on the same computer, while 104 cross-compiling requires that the package is built on the host but 105 executed for the target architecture (though often, as in the case 106 for ptest, the execution occurs on the host). The built version of 107 Automake that ships with the Yocto Project includes a patch that 108 separates building and execution. Consequently, packages that use the 109 unaltered, patched version of ``make check`` automatically 110 cross-compiles. 111 112 Regardless, you still must add a ``do_compile_ptest`` function to 113 build the test suite. Add a function similar to the following to your 114 recipe:: 115 116 do_compile_ptest() { 117 oe_runmake buildtest-TESTS 118 } 119 120- *Ensure special configurations are set:* If the package requires 121 special configurations prior to compiling the test code, you must 122 insert a ``do_configure_ptest`` function into the recipe. 123 124- *Install the test suite:* The :ref:`ref-classes-ptest` class 125 automatically copies the file ``run-ptest`` to the target and then runs make 126 ``install-ptest`` to run the tests. If this is not enough, you need 127 to create a ``do_install_ptest`` function and make sure it gets 128 called after the "make install-ptest" completes. 129