1.. SPDX-License-Identifier: CC-BY-SA-2.0-UK 2 3Upgrading Recipes 4***************** 5 6Over time, upstream developers publish new versions for software built 7by layer recipes. It is recommended to keep recipes up-to-date with 8upstream version releases. 9 10While there are several methods to upgrade a recipe, you might 11consider checking on the upgrade status of a recipe first. You can do so 12using the ``devtool check-upgrade-status`` command. See the 13":ref:`devtool-checking-on-the-upgrade-status-of-a-recipe`" 14section in the Yocto Project Reference Manual for more information. 15 16The remainder of this section describes three ways you can upgrade a 17recipe. You can use the Automated Upgrade Helper (AUH) to set up 18automatic version upgrades. Alternatively, you can use 19``devtool upgrade`` to set up semi-automatic version upgrades. Finally, 20you can manually upgrade a recipe by editing the recipe itself. 21 22Using the Auto Upgrade Helper (AUH) 23=================================== 24 25The AUH utility works in conjunction with the OpenEmbedded build system 26in order to automatically generate upgrades for recipes based on new 27versions being published upstream. Use AUH when you want to create a 28service that performs the upgrades automatically and optionally sends 29you an email with the results. 30 31AUH allows you to update several recipes with a single use. You can also 32optionally perform build and integration tests using images with the 33results saved to your hard drive and emails of results optionally sent 34to recipe maintainers. Finally, AUH creates Git commits with appropriate 35commit messages in the layer's tree for the changes made to recipes. 36 37.. note:: 38 39 In some conditions, you should not use AUH to upgrade recipes 40 and should instead use either ``devtool upgrade`` or upgrade your 41 recipes manually: 42 43 - When AUH cannot complete the upgrade sequence. This situation 44 usually results because custom patches carried by the recipe 45 cannot be automatically rebased to the new version. In this case, 46 ``devtool upgrade`` allows you to manually resolve conflicts. 47 48 - When for any reason you want fuller control over the upgrade 49 process. For example, when you want special arrangements for 50 testing. 51 52The following steps describe how to set up the AUH utility: 53 54#. *Be Sure the Development Host is Set Up:* You need to be sure that 55 your development host is set up to use the Yocto Project. For 56 information on how to set up your host, see the 57 ":ref:`dev-manual/start:Preparing the Build Host`" section. 58 59#. *Make Sure Git is Configured:* The AUH utility requires Git to be 60 configured because AUH uses Git to save upgrades. Thus, you must have 61 Git user and email configured. The following command shows your 62 configurations:: 63 64 $ git config --list 65 66 If you do not have the user and 67 email configured, you can use the following commands to do so:: 68 69 $ git config --global user.name some_name 70 $ git config --global user.email username@domain.com 71 72#. *Clone the AUH Repository:* To use AUH, you must clone the repository 73 onto your development host. The following command uses Git to create 74 a local copy of the repository on your system:: 75 76 $ git clone git://git.yoctoproject.org/auto-upgrade-helper 77 Cloning into 'auto-upgrade-helper'... remote: Counting objects: 768, done. 78 remote: Compressing objects: 100% (300/300), done. 79 remote: Total 768 (delta 499), reused 703 (delta 434) 80 Receiving objects: 100% (768/768), 191.47 KiB | 98.00 KiB/s, done. 81 Resolving deltas: 100% (499/499), done. 82 Checking connectivity... done. 83 84 AUH is not part of the :term:`OpenEmbedded-Core (OE-Core)` or 85 :term:`Poky` repositories. 86 87#. *Create a Dedicated Build Directory:* Run the :ref:`structure-core-script` 88 script to create a fresh :term:`Build Directory` that you use exclusively 89 for running the AUH utility:: 90 91 $ cd poky 92 $ source oe-init-build-env your_AUH_build_directory 93 94 Re-using an existing :term:`Build Directory` and its configurations is not 95 recommended as existing settings could cause AUH to fail or behave 96 undesirably. 97 98#. *Make Configurations in Your Local Configuration File:* Several 99 settings are needed in the ``local.conf`` file in the build 100 directory you just created for AUH. Make these following 101 configurations: 102 103 - If you want to enable :ref:`Build 104 History <dev-manual/build-quality:maintaining build output quality>`, 105 which is optional, you need the following lines in the 106 ``conf/local.conf`` file:: 107 108 INHERIT =+ "buildhistory" 109 BUILDHISTORY_COMMIT = "1" 110 111 With this configuration and a successful 112 upgrade, a build history "diff" file appears in the 113 ``upgrade-helper/work/recipe/buildhistory-diff.txt`` file found in 114 your :term:`Build Directory`. 115 116 - If you want to enable testing through the :ref:`ref-classes-testimage` 117 class, which is optional, you need to have the following set in 118 your ``conf/local.conf`` file:: 119 120 IMAGE_CLASSES += "testimage" 121 122 .. note:: 123 124 If your distro does not enable by default ptest, which Poky 125 does, you need the following in your ``local.conf`` file:: 126 127 DISTRO_FEATURES:append = " ptest" 128 129 130#. *Optionally Start a vncserver:* If you are running in a server 131 without an X11 session, you need to start a vncserver:: 132 133 $ vncserver :1 134 $ export DISPLAY=:1 135 136#. *Create and Edit an AUH Configuration File:* You need to have the 137 ``upgrade-helper/upgrade-helper.conf`` configuration file in your 138 :term:`Build Directory`. You can find a sample configuration file in the 139 :yocto_git:`AUH source repository </auto-upgrade-helper/tree/>`. 140 141 Read through the sample file and make configurations as needed. For 142 example, if you enabled build history in your ``local.conf`` as 143 described earlier, you must enable it in ``upgrade-helper.conf``. 144 145 Also, if you are using the default ``maintainers.inc`` file supplied 146 with Poky and located in ``meta-yocto`` and you do not set a 147 "maintainers_whitelist" or "global_maintainer_override" in the 148 ``upgrade-helper.conf`` configuration, and you specify "-e all" on 149 the AUH command-line, the utility automatically sends out emails to 150 all the default maintainers. Please avoid this. 151 152This next set of examples describes how to use the AUH: 153 154- *Upgrading a Specific Recipe:* To upgrade a specific recipe, use the 155 following form:: 156 157 $ upgrade-helper.py recipe_name 158 159 For example, this command upgrades the ``xmodmap`` recipe:: 160 161 $ upgrade-helper.py xmodmap 162 163- *Upgrading a Specific Recipe to a Particular Version:* To upgrade a 164 specific recipe to a particular version, use the following form:: 165 166 $ upgrade-helper.py recipe_name -t version 167 168 For example, this command upgrades the ``xmodmap`` recipe to version 1.2.3:: 169 170 $ upgrade-helper.py xmodmap -t 1.2.3 171 172- *Upgrading all Recipes to the Latest Versions and Suppressing Email 173 Notifications:* To upgrade all recipes to their most recent versions 174 and suppress the email notifications, use the following command:: 175 176 $ upgrade-helper.py all 177 178- *Upgrading all Recipes to the Latest Versions and Send Email 179 Notifications:* To upgrade all recipes to their most recent versions 180 and send email messages to maintainers for each attempted recipe as 181 well as a status email, use the following command:: 182 183 $ upgrade-helper.py -e all 184 185Once you have run the AUH utility, you can find the results in the AUH 186:term:`Build Directory`:: 187 188 ${BUILDDIR}/upgrade-helper/timestamp 189 190The AUH utility 191also creates recipe update commits from successful upgrade attempts in 192the layer tree. 193 194You can easily set up to run the AUH utility on a regular basis by using 195a cron job. See the 196:yocto_git:`weeklyjob.sh </auto-upgrade-helper/tree/weeklyjob.sh>` 197file distributed with the utility for an example. 198 199Using ``devtool upgrade`` 200========================= 201 202As mentioned earlier, an alternative method for upgrading recipes to 203newer versions is to use 204:doc:`devtool upgrade </ref-manual/devtool-reference>`. 205You can read about ``devtool upgrade`` in general in the 206":ref:`sdk-manual/extensible:use \`\`devtool upgrade\`\` to create a version of the recipe that supports a newer version of the software`" 207section in the Yocto Project Application Development and the Extensible 208Software Development Kit (eSDK) Manual. 209 210To see all the command-line options available with ``devtool upgrade``, 211use the following help command:: 212 213 $ devtool upgrade -h 214 215If you want to find out what version a recipe is currently at upstream 216without any attempt to upgrade your local version of the recipe, you can 217use the following command:: 218 219 $ devtool latest-version recipe_name 220 221As mentioned in the previous section describing AUH, ``devtool upgrade`` 222works in a less-automated manner than AUH. Specifically, 223``devtool upgrade`` only works on a single recipe that you name on the 224command line, cannot perform build and integration testing using images, 225and does not automatically generate commits for changes in the source 226tree. Despite all these "limitations", ``devtool upgrade`` updates the 227recipe file to the new upstream version and attempts to rebase custom 228patches contained by the recipe as needed. 229 230.. note:: 231 232 AUH uses much of ``devtool upgrade`` behind the scenes making AUH somewhat 233 of a "wrapper" application for ``devtool upgrade``. 234 235A typical scenario involves having used Git to clone an upstream 236repository that you use during build operations. Because you have built the 237recipe in the past, the layer is likely added to your 238configuration already. If for some reason, the layer is not added, you 239could add it easily using the 240":ref:`bitbake-layers <bsp-guide/bsp:creating a new bsp layer using the \`\`bitbake-layers\`\` script>`" 241script. For example, suppose you use the ``nano.bb`` recipe from the 242``meta-oe`` layer in the ``meta-openembedded`` repository. For this 243example, assume that the layer has been cloned into following area:: 244 245 /home/scottrif/meta-openembedded 246 247The following command from your :term:`Build Directory` adds the layer to 248your build configuration (i.e. ``${BUILDDIR}/conf/bblayers.conf``):: 249 250 $ bitbake-layers add-layer /home/scottrif/meta-openembedded/meta-oe 251 NOTE: Starting bitbake server... 252 Parsing recipes: 100% |##########################################| Time: 0:00:55 253 Parsing of 1431 .bb files complete (0 cached, 1431 parsed). 2040 targets, 56 skipped, 0 masked, 0 errors. 254 Removing 12 recipes from the x86_64 sysroot: 100% |##############| Time: 0:00:00 255 Removing 1 recipes from the x86_64_i586 sysroot: 100% |##########| Time: 0:00:00 256 Removing 5 recipes from the i586 sysroot: 100% |#################| Time: 0:00:00 257 Removing 5 recipes from the qemux86 sysroot: 100% |##############| Time: 0:00:00 258 259For this example, assume that the ``nano.bb`` recipe that 260is upstream has a 2.9.3 version number. However, the version in the 261local repository is 2.7.4. The following command from your build 262directory automatically upgrades the recipe for you:: 263 264 $ devtool upgrade nano -V 2.9.3 265 NOTE: Starting bitbake server... 266 NOTE: Creating workspace layer in /home/scottrif/poky/build/workspace 267 Parsing recipes: 100% |##########################################| Time: 0:00:46 268 Parsing of 1431 .bb files complete (0 cached, 1431 parsed). 2040 targets, 56 skipped, 0 masked, 0 errors. 269 NOTE: Extracting current version source... 270 NOTE: Resolving any missing task queue dependencies 271 . 272 . 273 . 274 NOTE: Executing SetScene Tasks 275 NOTE: Executing RunQueue Tasks 276 NOTE: Tasks Summary: Attempted 74 tasks of which 72 didn't need to be rerun and all succeeded. 277 Adding changed files: 100% |#####################################| Time: 0:00:00 278 NOTE: Upgraded source extracted to /home/scottrif/poky/build/workspace/sources/nano 279 NOTE: New recipe is /home/scottrif/poky/build/workspace/recipes/nano/nano_2.9.3.bb 280 281.. note:: 282 283 Using the ``-V`` option is not necessary. Omitting the version number causes 284 ``devtool upgrade`` to upgrade the recipe to the most recent version. 285 286Continuing with this example, you can use ``devtool build`` to build the 287newly upgraded recipe:: 288 289 $ devtool build nano 290 NOTE: Starting bitbake server... 291 Loading cache: 100% |################################################################################################| Time: 0:00:01 292 Loaded 2040 entries from dependency cache. 293 Parsing recipes: 100% |##############################################################################################| Time: 0:00:00 294 Parsing of 1432 .bb files complete (1431 cached, 1 parsed). 2041 targets, 56 skipped, 0 masked, 0 errors. 295 NOTE: Resolving any missing task queue dependencies 296 . 297 . 298 . 299 NOTE: Executing SetScene Tasks 300 NOTE: Executing RunQueue Tasks 301 NOTE: nano: compiling from external source tree /home/scottrif/poky/build/workspace/sources/nano 302 NOTE: Tasks Summary: Attempted 520 tasks of which 304 didn't need to be rerun and all succeeded. 303 304Within the ``devtool upgrade`` workflow, you can 305deploy and test your rebuilt software. For this example, 306however, running ``devtool finish`` cleans up the workspace once the 307source in your workspace is clean. This usually means using Git to stage 308and submit commits for the changes generated by the upgrade process. 309 310Once the tree is clean, you can clean things up in this example with the 311following command from the ``${BUILDDIR}/workspace/sources/nano`` 312directory:: 313 314 $ devtool finish nano meta-oe 315 NOTE: Starting bitbake server... 316 Loading cache: 100% |################################################################################################| Time: 0:00:00 317 Loaded 2040 entries from dependency cache. 318 Parsing recipes: 100% |##############################################################################################| Time: 0:00:01 319 Parsing of 1432 .bb files complete (1431 cached, 1 parsed). 2041 targets, 56 skipped, 0 masked, 0 errors. 320 NOTE: Adding new patch 0001-nano.bb-Stuff-I-changed-when-upgrading-nano.bb.patch 321 NOTE: Updating recipe nano_2.9.3.bb 322 NOTE: Removing file /home/scottrif/meta-openembedded/meta-oe/recipes-support/nano/nano_2.7.4.bb 323 NOTE: Moving recipe file to /home/scottrif/meta-openembedded/meta-oe/recipes-support/nano 324 NOTE: Leaving source tree /home/scottrif/poky/build/workspace/sources/nano as-is; if you no longer need it then please delete it manually 325 326 327Using the ``devtool finish`` command cleans up the workspace and creates a patch 328file based on your commits. The tool puts all patch files back into the 329source directory in a sub-directory named ``nano`` in this case. 330 331Manually Upgrading a Recipe 332=========================== 333 334If for some reason you choose not to upgrade recipes using 335:ref:`dev-manual/upgrading-recipes:Using the Auto Upgrade Helper (AUH)` or 336by :ref:`dev-manual/upgrading-recipes:Using \`\`devtool upgrade\`\``, 337you can manually edit the recipe files to upgrade the versions. 338 339.. note:: 340 341 Manually updating multiple recipes scales poorly and involves many 342 steps. The recommendation to upgrade recipe versions is through AUH 343 or ``devtool upgrade``, both of which automate some steps and provide 344 guidance for others needed for the manual process. 345 346To manually upgrade recipe versions, follow these general steps: 347 348#. *Change the Version:* Rename the recipe such that the version (i.e. 349 the :term:`PV` part of the recipe name) 350 changes appropriately. If the version is not part of the recipe name, 351 change the value as it is set for :term:`PV` within the recipe itself. 352 353#. *Update* :term:`SRCREV` *if Needed*: If the source code your recipe builds 354 is fetched from Git or some other version control system, update 355 :term:`SRCREV` to point to the 356 commit hash that matches the new version. 357 358#. *Build the Software:* Try to build the recipe using BitBake. Typical 359 build failures include the following: 360 361 - License statements were updated for the new version. For this 362 case, you need to review any changes to the license and update the 363 values of :term:`LICENSE` and 364 :term:`LIC_FILES_CHKSUM` 365 as needed. 366 367 .. note:: 368 369 License changes are often inconsequential. For example, the 370 license text's copyright year might have changed. 371 372 - Custom patches carried by the older version of the recipe might 373 fail to apply to the new version. For these cases, you need to 374 review the failures. Patches might not be necessary for the new 375 version of the software if the upgraded version has fixed those 376 issues. If a patch is necessary and failing, you need to rebase it 377 into the new version. 378 379#. *Optionally Attempt to Build for Several Architectures:* Once you 380 successfully build the new software for a given architecture, you 381 could test the build for other architectures by changing the 382 :term:`MACHINE` variable and 383 rebuilding the software. This optional step is especially important 384 if the recipe is to be released publicly. 385 386#. *Check the Upstream Change Log or Release Notes:* Checking both these 387 reveals if there are new features that could break 388 backwards-compatibility. If so, you need to take steps to mitigate or 389 eliminate that situation. 390 391#. *Optionally Create a Bootable Image and Test:* If you want, you can 392 test the new software by booting it onto actual hardware. 393 394#. *Create a Commit with the Change in the Layer Repository:* After all 395 builds work and any testing is successful, you can create commits for 396 any changes in the layer holding your upgraded recipe. 397 398