1.. SPDX-License-Identifier: CC-BY-SA-2.0-UK 2 3******************* 4Reproducible Builds 5******************* 6 7================ 8How we define it 9================ 10 11The Yocto Project defines reproducibility as where a given input build 12configuration will give the same binary output regardless of when it is built 13(now or in 5 years time), regardless of the path on the filesystem the build is 14run in, and regardless of the distro and tools on the underlying host system the 15build is running on. 16 17============== 18Why it matters 19============== 20 21The project aligns with the `Reproducible Builds project 22<https://reproducible-builds.org/>`__, which shares information about why 23reproducibility matters. The primary focus of the project is the ability to 24detect security issues being introduced. However, from a Yocto Project 25perspective, it is also hugely important that our builds are deterministic. When 26you build a given input set of metadata, we expect you to get consistent output. 27This has always been a key focus but, :yocto_docs:`since release 3.1 ("dunfell") 28</ref-manual/migration-3.1.html#reproducible-builds-now-enabled-by-default>`, 29it is now true down to the binary level including timestamps. 30 31For example, at some point in the future life of a product, you find that you 32need to rebuild to add a security fix. If this happens, only the components that 33have been modified should change at the binary level. This would lead to much 34easier and clearer bounds on where validation is needed. 35 36This also gives an additional benefit to the project builds themselves, our 37:ref:`overview-manual/concepts:Hash Equivalence` for 38:ref:`overview-manual/concepts:Shared State` object reuse works much more 39effectively when the binary output remains the same. 40 41.. note:: 42 43 We strongly advise you to make sure your project builds reproducibly 44 before finalizing your production images. It would be too late if you 45 only address this issue when the first updates are required. 46 47=================== 48How we implement it 49=================== 50 51There are many different aspects to build reproducibility, but some particular 52things we do within the build system to ensure reproducibility include: 53 54- Adding mappings to the compiler options to ensure debug filepaths are mapped 55 to consistent target compatible paths. This is done through the 56 :term:`DEBUG_PREFIX_MAP` variable which sets the ``-fmacro-prefix-map`` and 57 ``-fdebug-prefix-map`` compiler options correctly to map to target paths. 58- Being explicit about recipe dependencies and their configuration (no floating 59 configure options or host dependencies creeping in). In particular this means 60 making sure :term:`PACKAGECONFIG` coverage covers configure options which may 61 otherwise try and auto-detect host dependencies. 62- Using recipe specific sysroots to isolate recipes so they only see their 63 dependencies. These are visible as ``recipe-sysroot`` and 64 ``recipe-sysroot-native`` directories within the :term:`WORKDIR` of a given 65 recipe and are populated only with the dependencies a recipe has. 66- Build images from a reduced package set: only packages from recipes the image 67 depends upon. 68- Filtering the tools available from the host's ``PATH`` to only a specific set 69 of tools, set using the :term:`HOSTTOOLS` variable. 70 71.. note:: 72 73 Because of an open bug in GCC, using ``DISTRO_FEATURES:append = " lto"`` or 74 adding ``-flto`` (Link Time Optimization) to :term:`CFLAGS` makes the resulting 75 binary non-reproducible, in that it depends on the full absolute build path 76 to ``recipe-sysroot-native``, so installing the Yocto Project in a different 77 directory results in a different binary. 78 79 This issue is addressed by 80 :yocto_bugs:`bug 14481 - Programs built with -flto are not reproducible</show_bug.cgi?id=14481>`. 81 82========================================= 83Can we prove the project is reproducible? 84========================================= 85 86Yes, we can prove it and we regularly test this on the Autobuilder. At the 87time of writing (release 3.3, "hardknott"), :term:`OpenEmbedded-Core (OE-Core)` 88is 100% reproducible for all its recipes (i.e. world builds) apart from the Go 89language and Ruby documentation packages. Unfortunately, the current 90implementation of the Go language has fundamental reproducibility problems as 91it always depends upon the paths it is built in. 92 93.. note:: 94 95 Only BitBake and :term:`OpenEmbedded-Core (OE-Core)`, which is the ``meta`` 96 layer in Poky, guarantee complete reproducibility. The moment you add 97 another layer, this warranty is voided, because of additional configuration 98 files, ``bbappend`` files, overridden classes, etc. 99 100To run our automated selftest, as we use in our CI on the Autobuilder, you can 101run:: 102 103 oe-selftest -r reproducible.ReproducibleTests.test_reproducible_builds 104 105This defaults to including a ``world`` build so, if other layers are added, it would 106also run the tests for recipes in the additional layers. The first build will be 107run using :ref:`Shared State <overview-manual/concepts:Shared State>` if 108available, the second build explicitly disables 109:ref:`Shared State <overview-manual/concepts:Shared State>` and builds on the 110specific host the build is running on. This means we can test reproducibility 111builds between different host distributions over time on the Autobuilder. 112 113If ``OEQA_DEBUGGING_SAVED_OUTPUT`` is set, any differing packages will be saved 114here. The test is also able to run the ``diffoscope`` command on the output to 115generate HTML files showing the differences between the packages, to aid 116debugging. On the Autobuilder, these appear under 117https://autobuilder.yocto.io/pub/repro-fail/ in the form ``oe-reproducible + 118<date> + <random ID>``, e.g. ``oe-reproducible-20200202-1lm8o1th``. 119 120The project's current reproducibility status can be seen at 121:yocto_home:`/reproducible-build-results/` 122 123You can also check the reproducibility status on supported host distributions: 124 125- CentOS: :yocto_ab:`/typhoon/#/builders/reproducible-centos` 126- Debian: :yocto_ab:`/typhoon/#/builders/reproducible-debian` 127- Fedora: :yocto_ab:`/typhoon/#/builders/reproducible-fedora` 128- Ubuntu: :yocto_ab:`/typhoon/#/builders/reproducible-ubuntu` 129 130=============================== 131Can I test my layer or recipes? 132=============================== 133 134Once again, you can run a ``world`` test using the 135:ref:`oe-selftest <ref-manual/release-process:Testing and Quality Assurance>` 136command provided above. This functionality is implemented 137in :oe_git:`meta/lib/oeqa/selftest/cases/reproducible.py 138</openembedded-core/tree/meta/lib/oeqa/selftest/cases/reproducible.py>`. 139 140You could subclass the test and change ``targets`` to a different target. 141 142You may also change ``sstate_targets`` which would allow you to "pre-cache" some 143set of recipes before the test, meaning they are excluded from reproducibility 144testing. As a practical example, you could set ``sstate_targets`` to 145``core-image-sato``, then setting ``targets`` to ``core-image-sato-sdk`` would 146run reproducibility tests only on the targets belonging only to ``core-image-sato-sdk``. 147