1.. SPDX-License-Identifier: CC-BY-SA-2.0-UK 2 3************************ 4Using the Extensible SDK 5************************ 6 7This chapter describes the extensible SDK and how to install it. 8Information covers the pieces of the SDK, how to install it, and 9presents a look at using the ``devtool`` functionality. The extensible 10SDK makes it easy to add new applications and libraries to an image, 11modify the source for an existing component, test changes on the target 12hardware, and ease integration into the rest of the 13:term:`OpenEmbedded Build System`. 14 15.. note:: 16 17 For a side-by-side comparison of main features supported for an 18 extensible SDK as compared to a standard SDK, see the 19 :ref:`sdk-manual/intro:introduction` section. 20 21In addition to the functionality available through ``devtool``, you can 22alternatively make use of the toolchain directly, for example from 23Makefile and Autotools. See the 24":ref:`sdk-manual/working-projects:using the sdk toolchain directly`" chapter 25for more information. 26 27Why use the Extensible SDK and What is in It? 28============================================= 29 30The extensible SDK provides a cross-development toolchain and libraries 31tailored to the contents of a specific image. You would use the 32Extensible SDK if you want a toolchain experience supplemented with the 33powerful set of ``devtool`` commands tailored for the Yocto Project 34environment. 35 36The installed extensible SDK consists of several files and directories. 37Basically, it contains an SDK environment setup script, some 38configuration files, an internal build system, and the ``devtool`` 39functionality. 40 41Installing the Extensible SDK 42============================= 43 44Two ways to install the Extensible SDK 45-------------------------------------- 46 47Extensible SDK can be installed in two different ways, and both have 48their own pros and cons: 49 50#. *Setting up the Extensible SDK environment directly in a Yocto build*. This 51avoids having to produce, test, distribute and maintain separate SDK installer 52archives, which can get very large. There is only one environment for the regular 53Yocto build and the SDK and less code paths where things can go not according to plan. 54It's easier to update the SDK: it simply means updating the Yocto layers with 55git fetch or layer management tooling. The SDK extensibility is better than in the 56second option: just run ``bitbake`` again to add more things to the sysroot, or add layers 57if even more things are required. 58 59#. *Setting up the Extensible SDK from a standalone installer*. This has the benefit of 60having a single, self-contained archive that includes all the needed binary artifacts. 61So nothing needs to be rebuilt, and there is no need to provide a well-functioning 62binary artefact cache over the network for developers with underpowered laptops. 63 64Setting up the Extensible SDK environment directly in a Yocto build 65------------------------------------------------------------------- 66 67#. Set up all the needed layers and a Yocto :term:`Build Directory`, e.g. a regular Yocto 68 build where ``bitbake`` can be executed. 69 70#. Run: 71 $ bitbake meta-ide-support 72 $ bitbake -c populate_sysroot gtk+3 73 (or any other target or native item that the application developer would need) 74 $ bitbake build-sysroots 75 76 77Setting up the Extensible SDK from a standalone installer 78--------------------------------------------------------- 79 80The first thing you need to do is install the SDK on your :term:`Build 81Host` by running the ``*.sh`` installation script. 82 83You can download a tarball installer, which includes the pre-built 84toolchain, the ``runqemu`` script, the internal build system, 85``devtool``, and support files from the appropriate 86:yocto_dl:`toolchain </releases/yocto/yocto-&DISTRO;/toolchain/>` directory within the Index of 87Releases. Toolchains are available for several 32-bit and 64-bit 88architectures with the ``x86_64`` directories, respectively. The 89toolchains the Yocto Project provides are based off the 90``core-image-sato`` and ``core-image-minimal`` images and contain 91libraries appropriate for developing against that image. 92 93The names of the tarball installer scripts are such that a string 94representing the host system appears first in the filename and then is 95immediately followed by a string representing the target architecture. 96An extensible SDK has the string "-ext" as part of the name. Following 97is the general form:: 98 99 poky-glibc-host_system-image_type-arch-toolchain-ext-release_version.sh 100 101 Where: 102 host_system is a string representing your development system: 103 104 i686 or x86_64. 105 106 image_type is the image for which the SDK was built: 107 108 core-image-sato or core-image-minimal 109 110 arch is a string representing the tuned target architecture: 111 112 aarch64, armv5e, core2-64, i586, mips32r2, mips64, ppc7400, or cortexa8hf-neon 113 114 release_version is a string representing the release number of the Yocto Project: 115 116 &DISTRO;, &DISTRO;+snapshot 117 118For example, the following SDK installer is for a 64-bit 119development host system and a i586-tuned target architecture based off 120the SDK for ``core-image-sato`` and using the current &DISTRO; snapshot:: 121 122 poky-glibc-x86_64-core-image-sato-i586-toolchain-ext-&DISTRO;.sh 123 124.. note:: 125 126 As an alternative to downloading an SDK, you can build the SDK 127 installer. For information on building the installer, see the 128 :ref:`sdk-manual/appendix-obtain:building an sdk installer` 129 section. 130 131The SDK and toolchains are self-contained and by default are installed 132into the ``poky_sdk`` folder in your home directory. You can choose to 133install the extensible SDK in any location when you run the installer. 134However, because files need to be written under that directory during 135the normal course of operation, the location you choose for installation 136must be writable for whichever users need to use the SDK. 137 138The following command shows how to run the installer given a toolchain 139tarball for a 64-bit x86 development host system and a 64-bit x86 target 140architecture. The example assumes the SDK installer is located in 141``~/Downloads/`` and has execution rights:: 142 143 $ ./Downloads/poky-glibc-x86_64-core-image-minimal-core2-64-toolchain-ext-2.5.sh 144 Poky (Yocto Project Reference Distro) Extensible SDK installer version 2.5 145 ========================================================================== 146 Enter target directory for SDK (default: poky_sdk): 147 You are about to install the SDK to "/home/scottrif/poky_sdk". Proceed [Y/n]? Y 148 Extracting SDK..............done 149 Setting it up... 150 Extracting buildtools... 151 Preparing build system... 152 Parsing recipes: 100% |##################################################################| Time: 0:00:52 153 Initialising tasks: 100% |###############################################################| Time: 0:00:00 154 Checking sstate mirror object availability: 100% |#######################################| Time: 0:00:00 155 Loading cache: 100% |####################################################################| Time: 0:00:00 156 Initialising tasks: 100% |###############################################################| Time: 0:00:00 157 done 158 SDK has been successfully set up and is ready to be used. 159 Each time you wish to use the SDK in a new shell session, you need to source the environment setup script e.g. 160 $ . /home/scottrif/poky_sdk/environment-setup-core2-64-poky-linux 161 162.. note:: 163 164 If you do not have write permissions for the directory into which you 165 are installing the SDK, the installer notifies you and exits. For 166 that case, set up the proper permissions in the directory and run the 167 installer again. 168 169Running the Extensible SDK Environment Setup Script 170=================================================== 171 172Once you have the SDK installed, you must run the SDK environment setup 173script before you can actually use the SDK. 174 175When using a SDK directly in a Yocto build, you will find the script in 176``tmp/deploy/images/qemux86-64/`` in your :term:`Build Directory`. 177 178When using a standalone SDK installer, this setup script resides in 179the directory you chose when you installed the SDK, which is either the 180default ``poky_sdk`` directory or the directory you chose during 181installation. 182 183Before running the script, be sure it is the one that matches the 184architecture for which you are developing. Environment setup scripts 185begin with the string "``environment-setup``" and include as part of 186their name the tuned target architecture. As an example, the following 187commands set the working directory to where the SDK was installed and 188then source the environment setup script. In this example, the setup 189script is for an IA-based target machine using i586 tuning:: 190 191 $ cd /home/scottrif/poky_sdk 192 $ source environment-setup-core2-64-poky-linux 193 SDK environment now set up; additionally you may now run devtool to perform development tasks. 194 Run devtool --help for further details. 195 196When using the environment script directly in a Yocto build, it can 197be run similarly: 198 199 $ source tmp/deploy/images/qemux86-64/environment-setup-core2-64-poky-linux 200 201Running the setup script defines many environment variables needed in 202order to use the SDK (e.g. ``PATH``, 203:term:`CC`, 204:term:`LD`, and so forth). If you want to 205see all the environment variables the script exports, examine the 206installation file itself. 207 208Using ``devtool`` in Your SDK Workflow 209====================================== 210 211The cornerstone of the extensible SDK is a command-line tool called 212``devtool``. This tool provides a number of features that help you 213build, test and package software within the extensible SDK, and 214optionally integrate it into an image built by the OpenEmbedded build 215system. 216 217.. note:: 218 219 The use of 220 devtool 221 is not limited to the extensible SDK. You can use 222 devtool 223 to help you easily develop any project whose build output must be 224 part of an image built using the build system. 225 226The ``devtool`` command line is organized similarly to 227:ref:`overview-manual/development-environment:git` in that it has a number of 228sub-commands for each function. You can run ``devtool --help`` to see 229all the commands. 230 231.. note:: 232 233 See the " 234 devtool 235 Quick Reference 236 " in the Yocto Project Reference Manual for a 237 devtool 238 quick reference. 239 240Three ``devtool`` subcommands provide entry-points into 241development: 242 243- *devtool add*: Assists in adding new software to be built. 244 245- *devtool modify*: Sets up an environment to enable you to modify 246 the source of an existing component. 247 248- *devtool upgrade*: Updates an existing recipe so that you can 249 build it for an updated set of source files. 250 251As with the build system, "recipes" represent software packages within 252``devtool``. When you use ``devtool add``, a recipe is automatically 253created. When you use ``devtool modify``, the specified existing recipe 254is used in order to determine where to get the source code and how to 255patch it. In both cases, an environment is set up so that when you build 256the recipe a source tree that is under your control is used in order to 257allow you to make changes to the source as desired. By default, new 258recipes and the source go into a "workspace" directory under the SDK. 259 260The remainder of this section presents the ``devtool add``, 261``devtool modify``, and ``devtool upgrade`` workflows. 262 263Use ``devtool add`` to Add an Application 264----------------------------------------- 265 266The ``devtool add`` command generates a new recipe based on existing 267source code. This command takes advantage of the 268:ref:`devtool-the-workspace-layer-structure` 269layer that many ``devtool`` commands use. The command is flexible enough 270to allow you to extract source code into both the workspace or a 271separate local Git repository and to use existing code that does not 272need to be extracted. 273 274Depending on your particular scenario, the arguments and options you use 275with ``devtool add`` form different combinations. The following diagram 276shows common development flows you would use with the ``devtool add`` 277command: 278 279.. image:: figures/sdk-devtool-add-flow.png 280 :width: 100% 281 282#. *Generating the New Recipe*: The top part of the flow shows three 283 scenarios by which you could use ``devtool add`` to generate a recipe 284 based on existing source code. 285 286 In a shared development environment, it is typical for other 287 developers to be responsible for various areas of source code. As a 288 developer, you are probably interested in using that source code as 289 part of your development within the Yocto Project. All you need is 290 access to the code, a recipe, and a controlled area in which to do 291 your work. 292 293 Within the diagram, three possible scenarios feed into the 294 ``devtool add`` workflow: 295 296 - *Left*: The left scenario in the figure represents a common 297 situation where the source code does not exist locally and needs 298 to be extracted. In this situation, the source code is extracted 299 to the default workspace --- you do not want the files in some 300 specific location outside of the workspace. Thus, everything you 301 need will be located in the workspace:: 302 303 $ devtool add recipe fetchuri 304 305 With this command, ``devtool`` extracts the upstream 306 source files into a local Git repository within the ``sources`` 307 folder. The command then creates a recipe named recipe and a 308 corresponding append file in the workspace. If you do not provide 309 recipe, the command makes an attempt to determine the recipe name. 310 311 - *Middle*: The middle scenario in the figure also represents a 312 situation where the source code does not exist locally. In this 313 case, the code is again upstream and needs to be extracted to some 314 local area --- this time outside of the default workspace. 315 316 .. note:: 317 318 If required, 319 devtool 320 always creates a Git repository locally during the extraction. 321 322 Furthermore, the first positional argument ``srctree`` in this case 323 identifies where the ``devtool add`` command will locate the 324 extracted code outside of the workspace. You need to specify an 325 empty directory:: 326 327 $ devtool add recipe srctree fetchuri 328 329 In summary, 330 the source code is pulled from fetchuri and extracted into the 331 location defined by ``srctree`` as a local Git repository. 332 333 Within workspace, ``devtool`` creates a recipe named recipe along 334 with an associated append file. 335 336 - *Right*: The right scenario in the figure represents a situation 337 where the ``srctree`` has been previously prepared outside of the 338 ``devtool`` workspace. 339 340 The following command provides a new recipe name and identifies 341 the existing source tree location:: 342 343 $ devtool add recipe srctree 344 345 The command examines the source code and creates a recipe named 346 recipe for the code and places the recipe into the workspace. 347 348 Because the extracted source code already exists, ``devtool`` does 349 not try to relocate the source code into the workspace --- only the 350 new recipe is placed in the workspace. 351 352 Aside from a recipe folder, the command also creates an associated 353 append folder and places an initial ``*.bbappend`` file within. 354 355#. *Edit the Recipe*: You can use ``devtool edit-recipe`` to open up the 356 editor as defined by the ``$EDITOR`` environment variable and modify 357 the file:: 358 359 $ devtool edit-recipe recipe 360 361 From within the editor, you 362 can make modifications to the recipe that take effect when you build 363 it later. 364 365#. *Build the Recipe or Rebuild the Image*: The next step you take 366 depends on what you are going to do with the new code. 367 368 If you need to eventually move the build output to the target 369 hardware, use the following ``devtool`` command: 370 :; 371 372 $ devtool build recipe 373 374 On the other hand, if you want an image to contain the recipe's 375 packages from the workspace for immediate deployment onto a device 376 (e.g. for testing purposes), you can use the ``devtool build-image`` 377 command:: 378 379 $ devtool build-image image 380 381#. *Deploy the Build Output*: When you use the ``devtool build`` command 382 to build out your recipe, you probably want to see if the resulting 383 build output works as expected on the target hardware. 384 385 .. note:: 386 387 This step assumes you have a previously built image that is 388 already either running in QEMU or is running on actual hardware. 389 Also, it is assumed that for deployment of the image to the 390 target, SSH is installed in the image and, if the image is running 391 on real hardware, you have network access to and from your 392 development machine. 393 394 You can deploy your build output to that target hardware by using the 395 ``devtool deploy-target`` command: $ devtool deploy-target recipe 396 target The target is a live target machine running as an SSH server. 397 398 You can, of course, also deploy the image you build to actual 399 hardware by using the ``devtool build-image`` command. However, 400 ``devtool`` does not provide a specific command that allows you to 401 deploy the image to actual hardware. 402 403#. *Finish Your Work With the Recipe*: The ``devtool finish`` command 404 creates any patches corresponding to commits in the local Git 405 repository, moves the new recipe to a more permanent layer, and then 406 resets the recipe so that the recipe is built normally rather than 407 from the workspace:: 408 409 $ devtool finish recipe layer 410 411 .. note:: 412 413 Any changes you want to turn into patches must be committed to the 414 Git repository in the source tree. 415 416 As mentioned, the ``devtool finish`` command moves the final recipe 417 to its permanent layer. 418 419 As a final process of the ``devtool finish`` command, the state of 420 the standard layers and the upstream source is restored so that you 421 can build the recipe from those areas rather than the workspace. 422 423 .. note:: 424 425 You can use the 426 devtool reset 427 command to put things back should you decide you do not want to 428 proceed with your work. If you do use this command, realize that 429 the source tree is preserved. 430 431Use ``devtool modify`` to Modify the Source of an Existing Component 432-------------------------------------------------------------------- 433 434The ``devtool modify`` command prepares the way to work on existing code 435that already has a local recipe in place that is used to build the 436software. The command is flexible enough to allow you to extract code 437from an upstream source, specify the existing recipe, and keep track of 438and gather any patch files from other developers that are associated 439with the code. 440 441Depending on your particular scenario, the arguments and options you use 442with ``devtool modify`` form different combinations. The following 443diagram shows common development flows for the ``devtool modify`` 444command: 445 446.. image:: figures/sdk-devtool-modify-flow.png 447 :width: 100% 448 449#. *Preparing to Modify the Code*: The top part of the flow shows three 450 scenarios by which you could use ``devtool modify`` to prepare to 451 work on source files. Each scenario assumes the following: 452 453 - The recipe exists locally in a layer external to the ``devtool`` 454 workspace. 455 456 - The source files exist either upstream in an un-extracted state or 457 locally in a previously extracted state. 458 459 The typical situation is where another developer has created a layer 460 for use with the Yocto Project and their recipe already resides in 461 that layer. Furthermore, their source code is readily available 462 either upstream or locally. 463 464 - *Left*: The left scenario in the figure represents a common 465 situation where the source code does not exist locally and it 466 needs to be extracted from an upstream source. In this situation, 467 the source is extracted into the default ``devtool`` workspace 468 location. The recipe, in this scenario, is in its own layer 469 outside the workspace (i.e. ``meta-``\ layername). 470 471 The following command identifies the recipe and, by default, 472 extracts the source files:: 473 474 $ devtool modify recipe 475 476 Once 477 ``devtool``\ locates the recipe, ``devtool`` uses the recipe's 478 :term:`SRC_URI` statements to 479 locate the source code and any local patch files from other 480 developers. 481 482 With this scenario, there is no ``srctree`` argument. Consequently, the 483 default behavior of the ``devtool modify`` command is to extract 484 the source files pointed to by the :term:`SRC_URI` statements into a 485 local Git structure. Furthermore, the location for the extracted 486 source is the default area within the ``devtool`` workspace. The 487 result is that the command sets up both the source code and an 488 append file within the workspace while the recipe remains in its 489 original location. 490 491 Additionally, if you have any non-patch local files (i.e. files 492 referred to with ``file://`` entries in :term:`SRC_URI` statement 493 excluding ``*.patch/`` or ``*.diff``), these files are copied to 494 an ``oe-local-files`` folder under the newly created source tree. 495 Copying the files here gives you a convenient area from which you 496 can modify the files. Any changes or additions you make to those 497 files are incorporated into the build the next time you build the 498 software just as are other changes you might have made to the 499 source. 500 501 - *Middle*: The middle scenario in the figure represents a situation 502 where the source code also does not exist locally. In this case, 503 the code is again upstream and needs to be extracted to some local 504 area as a Git repository. The recipe, in this scenario, is again 505 local and in its own layer outside the workspace. 506 507 The following command tells ``devtool`` the recipe with which to 508 work and, in this case, identifies a local area for the extracted 509 source files that exists outside of the default ``devtool`` 510 workspace:: 511 512 $ devtool modify recipe srctree 513 514 .. note:: 515 516 You cannot provide a URL for 517 srctree 518 using the 519 devtool 520 command. 521 522 As with all extractions, the command uses the recipe's :term:`SRC_URI` 523 statements to locate the source files and any associated patch 524 files. Non-patch files are copied to an ``oe-local-files`` folder 525 under the newly created source tree. 526 527 Once the files are located, the command by default extracts them 528 into ``srctree``. 529 530 Within workspace, ``devtool`` creates an append file for the 531 recipe. The recipe remains in its original location but the source 532 files are extracted to the location you provide with ``srctree``. 533 534 - *Right*: The right scenario in the figure represents a situation 535 where the source tree (``srctree``) already exists locally as a 536 previously extracted Git structure outside of the ``devtool`` 537 workspace. In this example, the recipe also exists elsewhere 538 locally in its own layer. 539 540 The following command tells ``devtool`` the recipe with which to 541 work, uses the "-n" option to indicate source does not need to be 542 extracted, and uses ``srctree`` to point to the previously extracted 543 source files:: 544 545 $ devtool modify -n recipe srctree 546 547 If an ``oe-local-files`` subdirectory happens to exist and it 548 contains non-patch files, the files are used. However, if the 549 subdirectory does not exist and you run the ``devtool finish`` 550 command, any non-patch files that might exist next to the recipe 551 are removed because it appears to ``devtool`` that you have 552 deleted those files. 553 554 Once the ``devtool modify`` command finishes, it creates only an 555 append file for the recipe in the ``devtool`` workspace. The 556 recipe and the source code remain in their original locations. 557 558#. *Edit the Source*: Once you have used the ``devtool modify`` command, 559 you are free to make changes to the source files. You can use any 560 editor you like to make and save your source code modifications. 561 562#. *Build the Recipe or Rebuild the Image*: The next step you take 563 depends on what you are going to do with the new code. 564 565 If you need to eventually move the build output to the target 566 hardware, use the following ``devtool`` command:: 567 568 $ devtool build recipe 569 570 On the other hand, if you want an image to contain the recipe's 571 packages from the workspace for immediate deployment onto a device 572 (e.g. for testing purposes), you can use the ``devtool build-image`` 573 command: $ devtool build-image image 574 575#. *Deploy the Build Output*: When you use the ``devtool build`` command 576 to build out your recipe, you probably want to see if the resulting 577 build output works as expected on target hardware. 578 579 .. note:: 580 581 This step assumes you have a previously built image that is 582 already either running in QEMU or running on actual hardware. 583 Also, it is assumed that for deployment of the image to the 584 target, SSH is installed in the image and if the image is running 585 on real hardware that you have network access to and from your 586 development machine. 587 588 You can deploy your build output to that target hardware by using the 589 ``devtool deploy-target`` command:: 590 591 $ devtool deploy-target recipe target 592 593 The target is a live target machine running as an SSH server. 594 595 You can, of course, use other methods to deploy the image you built 596 using the ``devtool build-image`` command to actual hardware. 597 ``devtool`` does not provide a specific command to deploy the image 598 to actual hardware. 599 600#. *Finish Your Work With the Recipe*: The ``devtool finish`` command 601 creates any patches corresponding to commits in the local Git 602 repository, updates the recipe to point to them (or creates a 603 ``.bbappend`` file to do so, depending on the specified destination 604 layer), and then resets the recipe so that the recipe is built 605 normally rather than from the workspace:: 606 607 $ devtool finish recipe layer 608 609 .. note:: 610 611 Any changes you want to turn into patches must be staged and 612 committed within the local Git repository before you use the 613 devtool finish 614 command. 615 616 Because there is no need to move the recipe, ``devtool finish`` 617 either updates the original recipe in the original layer or the 618 command creates a ``.bbappend`` file in a different layer as provided 619 by layer. Any work you did in the ``oe-local-files`` directory is 620 preserved in the original files next to the recipe during the 621 ``devtool finish`` command. 622 623 As a final process of the ``devtool finish`` command, the state of 624 the standard layers and the upstream source is restored so that you 625 can build the recipe from those areas rather than from the workspace. 626 627 .. note:: 628 629 You can use the 630 devtool reset 631 command to put things back should you decide you do not want to 632 proceed with your work. If you do use this command, realize that 633 the source tree is preserved. 634 635Use ``devtool upgrade`` to Create a Version of the Recipe that Supports a Newer Version of the Software 636------------------------------------------------------------------------------------------------------- 637 638The ``devtool upgrade`` command upgrades an existing recipe to that of a 639more up-to-date version found upstream. Throughout the life of software, 640recipes continually undergo version upgrades by their upstream 641publishers. You can use the ``devtool upgrade`` workflow to make sure 642your recipes you are using for builds are up-to-date with their upstream 643counterparts. 644 645.. note:: 646 647 Several methods exist by which you can upgrade recipes - 648 ``devtool upgrade`` 649 happens to be one. You can read about all the methods by which you 650 can upgrade recipes in the 651 :ref:`dev-manual/upgrading-recipes:upgrading recipes` section 652 of the Yocto Project Development Tasks Manual. 653 654The ``devtool upgrade`` command is flexible enough to allow you to 655specify source code revision and versioning schemes, extract code into 656or out of the ``devtool`` 657:ref:`devtool-the-workspace-layer-structure`, 658and work with any source file forms that the 659:ref:`bitbake:bitbake-user-manual/bitbake-user-manual-fetching:fetchers` support. 660 661The following diagram shows the common development flow used with the 662``devtool upgrade`` command: 663 664.. image:: figures/sdk-devtool-upgrade-flow.png 665 :width: 100% 666 667#. *Initiate the Upgrade*: The top part of the flow shows the typical 668 scenario by which you use the ``devtool upgrade`` command. The 669 following conditions exist: 670 671 - The recipe exists in a local layer external to the ``devtool`` 672 workspace. 673 674 - The source files for the new release exist in the same location 675 pointed to by :term:`SRC_URI` 676 in the recipe (e.g. a tarball with the new version number in the 677 name, or as a different revision in the upstream Git repository). 678 679 A common situation is where third-party software has undergone a 680 revision so that it has been upgraded. The recipe you have access to 681 is likely in your own layer. Thus, you need to upgrade the recipe to 682 use the newer version of the software:: 683 684 $ devtool upgrade -V version recipe 685 686 By default, the ``devtool upgrade`` command extracts source 687 code into the ``sources`` directory in the 688 :ref:`devtool-the-workspace-layer-structure`. 689 If you want the code extracted to any other location, you need to 690 provide the ``srctree`` positional argument with the command as follows:: 691 692 $ devtool upgrade -V version recipe srctree 693 694 .. note:: 695 696 In this example, the "-V" option specifies the new version. If you 697 don't use "-V", the command upgrades the recipe to the latest 698 version. 699 700 If the source files pointed to by the :term:`SRC_URI` statement in the 701 recipe are in a Git repository, you must provide the "-S" option and 702 specify a revision for the software. 703 704 Once ``devtool`` locates the recipe, it uses the :term:`SRC_URI` variable 705 to locate the source code and any local patch files from other 706 developers. The result is that the command sets up the source code, 707 the new version of the recipe, and an append file all within the 708 workspace. 709 710 Additionally, if you have any non-patch local files (i.e. files 711 referred to with ``file://`` entries in :term:`SRC_URI` statement 712 excluding ``*.patch/`` or ``*.diff``), these files are copied to an 713 ``oe-local-files`` folder under the newly created source tree. 714 Copying the files here gives you a convenient area from which you can 715 modify the files. Any changes or additions you make to those files 716 are incorporated into the build the next time you build the software 717 just as are other changes you might have made to the source. 718 719#. *Resolve any Conflicts created by the Upgrade*: Conflicts could happen 720 after upgrading the software to a new version. Conflicts occur 721 if your recipe specifies some patch files in :term:`SRC_URI` that 722 conflict with changes made in the new version of the software. For 723 such cases, you need to resolve the conflicts by editing the source 724 and following the normal ``git rebase`` conflict resolution process. 725 726 Before moving onto the next step, be sure to resolve any such 727 conflicts created through use of a newer or different version of the 728 software. 729 730#. *Build the Recipe or Rebuild the Image*: The next step you take 731 depends on what you are going to do with the new code. 732 733 If you need to eventually move the build output to the target 734 hardware, use the following ``devtool`` command:: 735 736 $ devtool build recipe 737 738 On the other hand, if you want an image to contain the recipe's 739 packages from the workspace for immediate deployment onto a device 740 (e.g. for testing purposes), you can use the ``devtool build-image`` 741 command:: 742 743 $ devtool build-image image 744 745#. *Deploy the Build Output*: When you use the ``devtool build`` command 746 or ``bitbake`` to build your recipe, you probably want to see if the 747 resulting build output works as expected on target hardware. 748 749 .. note:: 750 751 This step assumes you have a previously built image that is 752 already either running in QEMU or running on actual hardware. 753 Also, it is assumed that for deployment of the image to the 754 target, SSH is installed in the image and if the image is running 755 on real hardware that you have network access to and from your 756 development machine. 757 758 You can deploy your build output to that target hardware by using the 759 ``devtool deploy-target`` command: $ devtool deploy-target recipe 760 target The target is a live target machine running as an SSH server. 761 762 You can, of course, also deploy the image you build using the 763 ``devtool build-image`` command to actual hardware. However, 764 ``devtool`` does not provide a specific command that allows you to do 765 this. 766 767#. *Finish Your Work With the Recipe*: The ``devtool finish`` command 768 creates any patches corresponding to commits in the local Git 769 repository, moves the new recipe to a more permanent layer, and then 770 resets the recipe so that the recipe is built normally rather than 771 from the workspace. 772 773 Any work you did in the ``oe-local-files`` directory is preserved in 774 the original files next to the recipe during the ``devtool finish`` 775 command. 776 777 If you specify a destination layer that is the same as the original 778 source, then the old version of the recipe and associated files are 779 removed prior to adding the new version:: 780 781 $ devtool finish recipe layer 782 783 .. note:: 784 785 Any changes you want to turn into patches must be committed to the 786 Git repository in the source tree. 787 788 As a final process of the ``devtool finish`` command, the state of 789 the standard layers and the upstream source is restored so that you 790 can build the recipe from those areas rather than the workspace. 791 792 .. note:: 793 794 You can use the 795 devtool reset 796 command to put things back should you decide you do not want to 797 proceed with your work. If you do use this command, realize that 798 the source tree is preserved. 799 800A Closer Look at ``devtool add`` 801================================ 802 803The ``devtool add`` command automatically creates a recipe based on the 804source tree you provide with the command. Currently, the command has 805support for the following: 806 807- Autotools (``autoconf`` and ``automake``) 808 809- CMake 810 811- Scons 812 813- ``qmake`` 814 815- Plain ``Makefile`` 816 817- Out-of-tree kernel module 818 819- Binary package (i.e. "-b" option) 820 821- Node.js module 822 823- Python modules that use ``setuptools`` or ``distutils`` 824 825Apart from binary packages, the determination of how a source tree 826should be treated is automatic based on the files present within that 827source tree. For example, if a ``CMakeLists.txt`` file is found, then 828the source tree is assumed to be using CMake and is treated accordingly. 829 830.. note:: 831 832 In most cases, you need to edit the automatically generated recipe in 833 order to make it build properly. Typically, you would go through 834 several edit and build cycles until the recipe successfully builds. 835 Once the recipe builds, you could use possible further iterations to 836 test the recipe on the target device. 837 838The remainder of this section covers specifics regarding how parts of 839the recipe are generated. 840 841Name and Version 842---------------- 843 844If you do not specify a name and version on the command line, 845``devtool add`` uses various metadata within the source tree in an 846attempt to determine the name and version of the software being built. 847Based on what the tool determines, ``devtool`` sets the name of the 848created recipe file accordingly. 849 850If ``devtool`` cannot determine the name and version, the command prints 851an error. For such cases, you must re-run the command and provide the 852name and version, just the name, or just the version as part of the 853command line. 854 855Sometimes the name or version determined from the source tree might be 856incorrect. For such a case, you must reset the recipe:: 857 858 $ devtool reset -n recipename 859 860After running the ``devtool reset`` command, you need to 861run ``devtool add`` again and provide the name or the version. 862 863Dependency Detection and Mapping 864-------------------------------- 865 866The ``devtool add`` command attempts to detect build-time dependencies 867and map them to other recipes in the system. During this mapping, the 868command fills in the names of those recipes as part of the 869:term:`DEPENDS` variable within the 870recipe. If a dependency cannot be mapped, ``devtool`` places a comment 871in the recipe indicating such. The inability to map a dependency can 872result from naming not being recognized or because the dependency simply 873is not available. For cases where the dependency is not available, you 874must use the ``devtool add`` command to add an additional recipe that 875satisfies the dependency. Once you add that recipe, you need to update 876the :term:`DEPENDS` variable in the original recipe to include the new 877recipe. 878 879If you need to add runtime dependencies, you can do so by adding the 880following to your recipe:: 881 882 RDEPENDS:${PN} += "dependency1 dependency2 ..." 883 884.. note:: 885 886 The 887 devtool add 888 command often cannot distinguish between mandatory and optional 889 dependencies. Consequently, some of the detected dependencies might 890 in fact be optional. When in doubt, consult the documentation or the 891 configure script for the software the recipe is building for further 892 details. In some cases, you might find you can substitute the 893 dependency with an option that disables the associated functionality 894 passed to the configure script. 895 896License Detection 897----------------- 898 899The ``devtool add`` command attempts to determine if the software you 900are adding is able to be distributed under a common, open-source 901license. If so, the command sets the 902:term:`LICENSE` value accordingly. 903You should double-check the value added by the command against the 904documentation or source files for the software you are building and, if 905necessary, update that :term:`LICENSE` value. 906 907The ``devtool add`` command also sets the 908:term:`LIC_FILES_CHKSUM` 909value to point to all files that appear to be license-related. Realize 910that license statements often appear in comments at the top of source 911files or within the documentation. In such cases, the command does not 912recognize those license statements. Consequently, you might need to 913amend the :term:`LIC_FILES_CHKSUM` variable to point to one or more of those 914comments if present. Setting :term:`LIC_FILES_CHKSUM` is particularly 915important for third-party software. The mechanism attempts to ensure 916correct licensing should you upgrade the recipe to a newer upstream 917version in future. Any change in licensing is detected and you receive 918an error prompting you to check the license text again. 919 920If the ``devtool add`` command cannot determine licensing information, 921``devtool`` sets the :term:`LICENSE` value to "CLOSED" and leaves the 922:term:`LIC_FILES_CHKSUM` value unset. This behavior allows you to continue 923with development even though the settings are unlikely to be correct in 924all cases. You should check the documentation or source files for the 925software you are building to determine the actual license. 926 927Adding Makefile-Only Software 928----------------------------- 929 930The use of Make by itself is very common in both proprietary and 931open-source software. Unfortunately, Makefiles are often not written 932with cross-compilation in mind. Thus, ``devtool add`` often cannot do 933very much to ensure that these Makefiles build correctly. It is very 934common, for example, to explicitly call ``gcc`` instead of using the 935:term:`CC` variable. Usually, in a 936cross-compilation environment, ``gcc`` is the compiler for the build 937host and the cross-compiler is named something similar to 938``arm-poky-linux-gnueabi-gcc`` and might require arguments (e.g. to 939point to the associated sysroot for the target machine). 940 941When writing a recipe for Makefile-only software, keep the following in 942mind: 943 944- You probably need to patch the Makefile to use variables instead of 945 hardcoding tools within the toolchain such as ``gcc`` and ``g++``. 946 947- The environment in which Make runs is set up with various standard 948 variables for compilation (e.g. :term:`CC`, :term:`CXX`, and so forth) in a 949 similar manner to the environment set up by the SDK's environment 950 setup script. One easy way to see these variables is to run the 951 ``devtool build`` command on the recipe and then look in 952 ``oe-logs/run.do_compile``. Towards the top of this file, there is 953 a list of environment variables that are set. You can take 954 advantage of these variables within the Makefile. 955 956- If the Makefile sets a default for a variable using "=", that default 957 overrides the value set in the environment, which is usually not 958 desirable. For this case, you can either patch the Makefile so it 959 sets the default using the "?=" operator, or you can alternatively 960 force the value on the ``make`` command line. To force the value on 961 the command line, add the variable setting to 962 :term:`EXTRA_OEMAKE` or 963 :term:`PACKAGECONFIG_CONFARGS` 964 within the recipe. Here is an example using :term:`EXTRA_OEMAKE`:: 965 966 EXTRA_OEMAKE += "'CC=${CC}' 'CXX=${CXX}'" 967 968 In the above example, 969 single quotes are used around the variable settings as the values are 970 likely to contain spaces because required default options are passed 971 to the compiler. 972 973- Hardcoding paths inside Makefiles is often problematic in a 974 cross-compilation environment. This is particularly true because 975 those hardcoded paths often point to locations on the build host and 976 thus will either be read-only or will introduce contamination into 977 the cross-compilation because they are specific to the build host 978 rather than the target. Patching the Makefile to use prefix variables 979 or other path variables is usually the way to handle this situation. 980 981- Sometimes a Makefile runs target-specific commands such as 982 ``ldconfig``. For such cases, you might be able to apply patches that 983 remove these commands from the Makefile. 984 985Adding Native Tools 986------------------- 987 988Often, you need to build additional tools that run on the :term:`Build 989Host` as opposed to 990the target. You should indicate this requirement by using one of the 991following methods when you run ``devtool add``: 992 993- Specify the name of the recipe such that it ends with "-native". 994 Specifying the name like this produces a recipe that only builds for 995 the build host. 996 997- Specify the "--also-native" option with the ``devtool add`` 998 command. Specifying this option creates a recipe file that still 999 builds for the target but also creates a variant with a "-native" 1000 suffix that builds for the build host. 1001 1002.. note:: 1003 1004 If you need to add a tool that is shipped as part of a source tree 1005 that builds code for the target, you can typically accomplish this by 1006 building the native and target parts separately rather than within 1007 the same compilation process. Realize though that with the 1008 "--also-native" option, you can add the tool using just one 1009 recipe file. 1010 1011Adding Node.js Modules 1012---------------------- 1013 1014You can use the ``devtool add`` command two different ways to add 1015Node.js modules: 1) Through ``npm`` and, 2) from a repository or local 1016source. 1017 1018Use the following form to add Node.js modules through ``npm``:: 1019 1020 $ devtool add "npm://registry.npmjs.org;name=forever;version=0.15.1" 1021 1022The name and 1023version parameters are mandatory. Lockdown and shrinkwrap files are 1024generated and pointed to by the recipe in order to freeze the version 1025that is fetched for the dependencies according to the first time. This 1026also saves checksums that are verified on future fetches. Together, 1027these behaviors ensure the reproducibility and integrity of the build. 1028 1029.. note:: 1030 1031 - You must use quotes around the URL. The ``devtool add`` does not 1032 require the quotes, but the shell considers ";" as a splitter 1033 between multiple commands. Thus, without the quotes, 1034 ``devtool add`` does not receive the other parts, which results in 1035 several "command not found" errors. 1036 1037 - In order to support adding Node.js modules, a ``nodejs`` recipe 1038 must be part of your SDK. 1039 1040As mentioned earlier, you can also add Node.js modules directly from a 1041repository or local source tree. To add modules this way, use 1042``devtool add`` in the following form:: 1043 1044 $ devtool add https://github.com/diversario/node-ssdp 1045 1046In this example, ``devtool`` 1047fetches the specified Git repository, detects the code as Node.js code, 1048fetches dependencies using ``npm``, and sets 1049:term:`SRC_URI` accordingly. 1050 1051Working With Recipes 1052==================== 1053 1054When building a recipe using the ``devtool build`` command, the typical 1055build progresses as follows: 1056 1057#. Fetch the source 1058 1059#. Unpack the source 1060 1061#. Configure the source 1062 1063#. Compile the source 1064 1065#. Install the build output 1066 1067#. Package the installed output 1068 1069For recipes in the workspace, fetching and unpacking is disabled as the 1070source tree has already been prepared and is persistent. Each of these 1071build steps is defined as a function (task), usually with a "do\_" prefix 1072(e.g. :ref:`ref-tasks-fetch`, 1073:ref:`ref-tasks-unpack`, and so 1074forth). These functions are typically shell scripts but can instead be 1075written in Python. 1076 1077If you look at the contents of a recipe, you will see that the recipe 1078does not include complete instructions for building the software. 1079Instead, common functionality is encapsulated in classes inherited with 1080the ``inherit`` directive. This technique leaves the recipe to describe 1081just the things that are specific to the software being built. There is 1082a :ref:`ref-classes-base` class that is implicitly inherited by all recipes 1083and provides the functionality that most recipes typically need. 1084 1085The remainder of this section presents information useful when working 1086with recipes. 1087 1088Finding Logs and Work Files 1089--------------------------- 1090 1091After the first run of the ``devtool build`` command, recipes that were 1092previously created using the ``devtool add`` command or whose sources 1093were modified using the ``devtool modify`` command contain symbolic 1094links created within the source tree: 1095 1096- ``oe-logs``: This link points to the directory in which log files and 1097 run scripts for each build step are created. 1098 1099- ``oe-workdir``: This link points to the temporary work area for the 1100 recipe. The following locations under ``oe-workdir`` are particularly 1101 useful: 1102 1103 - ``image/``: Contains all of the files installed during the 1104 :ref:`ref-tasks-install` stage. 1105 Within a recipe, this directory is referred to by the expression 1106 ``${``\ :term:`D`\ ``}``. 1107 1108 - ``sysroot-destdir/``: Contains a subset of files installed within 1109 :ref:`ref-tasks-install` that have been put into the shared sysroot. For 1110 more information, see the 1111 ":ref:`dev-manual/new-recipe:sharing files between recipes`" section. 1112 1113 - ``packages-split/``: Contains subdirectories for each package 1114 produced by the recipe. For more information, see the 1115 ":ref:`sdk-manual/extensible:packaging`" section. 1116 1117You can use these links to get more information on what is happening at 1118each build step. 1119 1120Setting Configure Arguments 1121--------------------------- 1122 1123If the software your recipe is building uses GNU autoconf, then a fixed 1124set of arguments is passed to it to enable cross-compilation plus any 1125extras specified by 1126:term:`EXTRA_OECONF` or 1127:term:`PACKAGECONFIG_CONFARGS` 1128set within the recipe. If you wish to pass additional options, add them 1129to :term:`EXTRA_OECONF` or :term:`PACKAGECONFIG_CONFARGS`. Other supported build 1130tools have similar variables (e.g. 1131:term:`EXTRA_OECMAKE` for 1132CMake, :term:`EXTRA_OESCONS` 1133for Scons, and so forth). If you need to pass anything on the ``make`` 1134command line, you can use :term:`EXTRA_OEMAKE` or the 1135:term:`PACKAGECONFIG_CONFARGS` 1136variables to do so. 1137 1138You can use the ``devtool configure-help`` command to help you set the 1139arguments listed in the previous paragraph. The command determines the 1140exact options being passed, and shows them to you along with any custom 1141arguments specified through :term:`EXTRA_OECONF` or 1142:term:`PACKAGECONFIG_CONFARGS`. If applicable, the command also shows you 1143the output of the configure script's "--help" option as a 1144reference. 1145 1146Sharing Files Between Recipes 1147----------------------------- 1148 1149Recipes often need to use files provided by other recipes on the 1150:term:`Build Host`. For example, 1151an application linking to a common library needs access to the library 1152itself and its associated headers. The way this access is accomplished 1153within the extensible SDK is through the sysroot. There is one sysroot per 1154"machine" for which the SDK is being built. In practical terms, this 1155means there is a sysroot for the target machine, and a sysroot for 1156the build host. 1157 1158Recipes should never write files directly into the sysroot. Instead, 1159files should be installed into standard locations during the 1160:ref:`ref-tasks-install` task within 1161the ``${``\ :term:`D`\ ``}`` directory. A 1162subset of these files automatically goes into the sysroot. The reason 1163for this limitation is that almost all files that go into the sysroot 1164are cataloged in manifests in order to ensure they can be removed later 1165when a recipe is modified or removed. Thus, the sysroot is able to 1166remain free from stale files. 1167 1168Packaging 1169--------- 1170 1171Packaging is not always particularly relevant within the extensible SDK. 1172However, if you examine how build output gets into the final image on 1173the target device, it is important to understand packaging because the 1174contents of the image are expressed in terms of packages and not 1175recipes. 1176 1177During the :ref:`ref-tasks-package` 1178task, files installed during the 1179:ref:`ref-tasks-install` task are 1180split into one main package, which is almost always named the same as 1181the recipe, and into several other packages. This separation exists 1182because not all of those installed files are useful in every image. For 1183example, you probably do not need any of the documentation installed in 1184a production image. Consequently, for each recipe the documentation 1185files are separated into a ``-doc`` package. Recipes that package 1186software containing optional modules or plugins might undergo additional 1187package splitting as well. 1188 1189After building a recipe, you can see where files have gone by looking in 1190the ``oe-workdir/packages-split`` directory, which contains a 1191subdirectory for each package. Apart from some advanced cases, the 1192:term:`PACKAGES` and 1193:term:`FILES` variables controls 1194splitting. The :term:`PACKAGES` variable lists all of the packages to be 1195produced, while the :term:`FILES` variable specifies which files to include 1196in each package by using an override to specify the package. For 1197example, ``FILES:${PN}`` specifies the files to go into the main package 1198(i.e. the main package has the same name as the recipe and 1199``${``\ :term:`PN`\ ``}`` evaluates to the 1200recipe name). The order of the :term:`PACKAGES` value is significant. For 1201each installed file, the first package whose :term:`FILES` value matches the 1202file is the package into which the file goes. Both the :term:`PACKAGES` and 1203:term:`FILES` variables have default values. Consequently, you might find 1204you do not even need to set these variables in your recipe unless the 1205software the recipe is building installs files into non-standard 1206locations. 1207 1208Restoring the Target Device to its Original State 1209================================================= 1210 1211If you use the ``devtool deploy-target`` command to write a recipe's 1212build output to the target, and you are working on an existing component 1213of the system, then you might find yourself in a situation where you 1214need to restore the original files that existed prior to running the 1215``devtool deploy-target`` command. Because the ``devtool deploy-target`` 1216command backs up any files it overwrites, you can use the 1217``devtool undeploy-target`` command to restore those files and remove 1218any other files the recipe deployed. Consider the following example:: 1219 1220 $ devtool undeploy-target lighttpd root@192.168.7.2 1221 1222If you have deployed 1223multiple applications, you can remove them all using the "-a" option 1224thus restoring the target device to its original state:: 1225 1226 $ devtool undeploy-target -a root@192.168.7.2 1227 1228Information about files deployed to 1229the target as well as any backed up files are stored on the target 1230itself. This storage, of course, requires some additional space on the 1231target machine. 1232 1233.. note:: 1234 1235 The 1236 devtool deploy-target 1237 and 1238 devtool undeploy-target 1239 commands do not currently interact with any package management system 1240 on the target device (e.g. RPM or OPKG). Consequently, you should not 1241 intermingle 1242 devtool deploy-target 1243 and package manager operations on the target device. Doing so could 1244 result in a conflicting set of files. 1245 1246Installing Additional Items Into the Extensible SDK 1247=================================================== 1248 1249Out of the box the extensible SDK typically only comes with a small 1250number of tools and libraries. A minimal SDK starts mostly empty and is 1251populated on-demand. Sometimes you must explicitly install extra items 1252into the SDK. If you need these extra items, you can first search for 1253the items using the ``devtool search`` command. For example, suppose you 1254need to link to libGL but you are not sure which recipe provides libGL. 1255You can use the following command to find out:: 1256 1257 $ devtool search libGL mesa 1258 A free implementation of the OpenGL API 1259 1260Once you know the recipe 1261(i.e. ``mesa`` in this example), you can install it. 1262 1263When using the extensible SDK directly in a Yocto build 1264------------------------------------------------------- 1265 1266In this scenario, the Yocto build tooling, e.g. ``bitbake`` 1267is directly accessible to build additional items, and it 1268can simply be executed directly: 1269 1270 $ bitbake mesa 1271 $ bitbake build-sysroots 1272 1273When using a standalone installer for the Extensible SDK 1274-------------------------------------------------------- 1275 1276 $ devtool sdk-install mesa 1277 1278By default, the ``devtool sdk-install`` command assumes 1279the item is available in pre-built form from your SDK provider. If the 1280item is not available and it is acceptable to build the item from 1281source, you can add the "-s" option as follows:: 1282 1283 $ devtool sdk-install -s mesa 1284 1285It is important to remember that building the item from source 1286takes significantly longer than installing the pre-built artifact. Also, 1287if there is no recipe for the item you want to add to the SDK, you must 1288instead add the item using the ``devtool add`` command. 1289 1290Applying Updates to an Installed Extensible SDK 1291=============================================== 1292 1293If you are working with an installed extensible SDK that gets 1294occasionally updated (e.g. a third-party SDK), then you will need to 1295manually "pull down" the updates into the installed SDK. 1296 1297To update your installed SDK, use ``devtool`` as follows:: 1298 1299 $ devtool sdk-update 1300 1301The previous command assumes your SDK provider has set the 1302default update URL for you through the :term:`SDK_UPDATE_URL` 1303variable as described in the 1304":ref:`sdk-manual/appendix-customizing:Providing Updates to the Extensible SDK After Installation`" 1305section. If the SDK provider has not set that default URL, you need to 1306specify it yourself in the command as follows: $ devtool sdk-update 1307path_to_update_directory 1308 1309.. note:: 1310 1311 The URL needs to point specifically to a published SDK and not to an 1312 SDK installer that you would download and install. 1313 1314Creating a Derivative SDK With Additional Components 1315==================================================== 1316 1317You might need to produce an SDK that contains your own custom 1318libraries. A good example would be if you were a vendor with customers 1319that use your SDK to build their own platform-specific software and 1320those customers need an SDK that has custom libraries. In such a case, 1321you can produce a derivative SDK based on the currently installed SDK 1322fairly easily by following these steps: 1323 1324#. If necessary, install an extensible SDK that you want to use as a 1325 base for your derivative SDK. 1326 1327#. Source the environment script for the SDK. 1328 1329#. Add the extra libraries or other components you want by using the 1330 ``devtool add`` command. 1331 1332#. Run the ``devtool build-sdk`` command. 1333 1334The previous steps take the recipes added to the workspace and construct 1335a new SDK installer that contains those recipes and the resulting binary 1336artifacts. The recipes go into their own separate layer in the 1337constructed derivative SDK, which leaves the workspace clean and ready 1338for users to add their own recipes. 1339