1.. SPDX-License-Identifier: CC-BY-SA-2.0-UK 2 3************************ 4Using the Extensible SDK 5************************ 6 7This chapter describes the extensible SDK and how to install it. 8Information covers the pieces of the SDK, how to install it, and 9presents a look at using the ``devtool`` functionality. The extensible 10SDK makes it easy to add new applications and libraries to an image, 11modify the source for an existing component, test changes on the target 12hardware, and ease integration into the rest of the 13:term:`OpenEmbedded Build System`. 14 15.. note:: 16 17 For a side-by-side comparison of main features supported for an 18 extensible SDK as compared to a standard SDK, see the " 19 Introduction 20 " section. 21 22In addition to the functionality available through ``devtool``, you can 23alternatively make use of the toolchain directly, for example from 24Makefile and Autotools. See the 25":ref:`sdk-manual/working-projects:using the sdk toolchain directly`" chapter 26for more information. 27 28Why use the Extensible SDK and What is in It? 29============================================= 30 31The extensible SDK provides a cross-development toolchain and libraries 32tailored to the contents of a specific image. You would use the 33Extensible SDK if you want a toolchain experience supplemented with the 34powerful set of ``devtool`` commands tailored for the Yocto Project 35environment. 36 37The installed extensible SDK consists of several files and directories. 38Basically, it contains an SDK environment setup script, some 39configuration files, an internal build system, and the ``devtool`` 40functionality. 41 42Installing the Extensible SDK 43============================= 44 45The first thing you need to do is install the SDK on your :term:`Build 46Host` by running the ``*.sh`` installation script. 47 48You can download a tarball installer, which includes the pre-built 49toolchain, the ``runqemu`` script, the internal build system, 50``devtool``, and support files from the appropriate 51:yocto_dl:`toolchain </releases/yocto/yocto-&DISTRO;/toolchain/>` directory within the Index of 52Releases. Toolchains are available for several 32-bit and 64-bit 53architectures with the ``x86_64`` directories, respectively. The 54toolchains the Yocto Project provides are based off the 55``core-image-sato`` and ``core-image-minimal`` images and contain 56libraries appropriate for developing against that image. 57 58The names of the tarball installer scripts are such that a string 59representing the host system appears first in the filename and then is 60immediately followed by a string representing the target architecture. 61An extensible SDK has the string "-ext" as part of the name. Following 62is the general form:: 63 64 poky-glibc-host_system-image_type-arch-toolchain-ext-release_version.sh 65 66 Where: 67 host_system is a string representing your development system: 68 69 i686 or x86_64. 70 71 image_type is the image for which the SDK was built: 72 73 core-image-sato or core-image-minimal 74 75 arch is a string representing the tuned target architecture: 76 77 aarch64, armv5e, core2-64, i586, mips32r2, mips64, ppc7400, or cortexa8hf-neon 78 79 release_version is a string representing the release number of the Yocto Project: 80 81 &DISTRO;, &DISTRO;+snapshot 82 83For example, the following SDK installer is for a 64-bit 84development host system and a i586-tuned target architecture based off 85the SDK for ``core-image-sato`` and using the current &DISTRO; snapshot:: 86 87 poky-glibc-x86_64-core-image-sato-i586-toolchain-ext-&DISTRO;.sh 88 89.. note:: 90 91 As an alternative to downloading an SDK, you can build the SDK 92 installer. For information on building the installer, see the " 93 Building an SDK Installer 94 " section. 95 96The SDK and toolchains are self-contained and by default are installed 97into the ``poky_sdk`` folder in your home directory. You can choose to 98install the extensible SDK in any location when you run the installer. 99However, because files need to be written under that directory during 100the normal course of operation, the location you choose for installation 101must be writable for whichever users need to use the SDK. 102 103The following command shows how to run the installer given a toolchain 104tarball for a 64-bit x86 development host system and a 64-bit x86 target 105architecture. The example assumes the SDK installer is located in 106``~/Downloads/`` and has execution rights. 107 108.. note:: 109 110 If you do not have write permissions for the directory into which you 111 are installing the SDK, the installer notifies you and exits. For 112 that case, set up the proper permissions in the directory and run the 113 installer again. 114 115:: 116 117 $ ./Downloads/poky-glibc-x86_64-core-image-minimal-core2-64-toolchain-ext-2.5.sh 118 Poky (Yocto Project Reference Distro) Extensible SDK installer version 2.5 119 ========================================================================== 120 Enter target directory for SDK (default: poky_sdk): 121 You are about to install the SDK to "/home/scottrif/poky_sdk". Proceed [Y/n]? Y 122 Extracting SDK..............done 123 Setting it up... 124 Extracting buildtools... 125 Preparing build system... 126 Parsing recipes: 100% |##################################################################| Time: 0:00:52 127 Initialising tasks: 100% |###############################################################| Time: 0:00:00 128 Checking sstate mirror object availability: 100% |#######################################| Time: 0:00:00 129 Loading cache: 100% |####################################################################| Time: 0:00:00 130 Initialising tasks: 100% |###############################################################| Time: 0:00:00 131 done 132 SDK has been successfully set up and is ready to be used. 133 Each time you wish to use the SDK in a new shell session, you need to source the environment setup script e.g. 134 $ . /home/scottrif/poky_sdk/environment-setup-core2-64-poky-linux 135 136Running the Extensible SDK Environment Setup Script 137=================================================== 138 139Once you have the SDK installed, you must run the SDK environment setup 140script before you can actually use the SDK. This setup script resides in 141the directory you chose when you installed the SDK, which is either the 142default ``poky_sdk`` directory or the directory you chose during 143installation. 144 145Before running the script, be sure it is the one that matches the 146architecture for which you are developing. Environment setup scripts 147begin with the string "``environment-setup``" and include as part of 148their name the tuned target architecture. As an example, the following 149commands set the working directory to where the SDK was installed and 150then source the environment setup script. In this example, the setup 151script is for an IA-based target machine using i586 tuning:: 152 153 $ cd /home/scottrif/poky_sdk 154 $ source environment-setup-core2-64-poky-linux 155 SDK environment now set up; additionally you may now run devtool to perform development tasks. 156 Run devtool --help for further details. 157 158Running the setup script defines many environment variables needed in 159order to use the SDK (e.g. ``PATH``, 160:term:`CC`, 161:term:`LD`, and so forth). If you want to 162see all the environment variables the script exports, examine the 163installation file itself. 164 165Using ``devtool`` in Your SDK Workflow 166====================================== 167 168The cornerstone of the extensible SDK is a command-line tool called 169``devtool``. This tool provides a number of features that help you 170build, test and package software within the extensible SDK, and 171optionally integrate it into an image built by the OpenEmbedded build 172system. 173 174.. note:: 175 176 The use of 177 devtool 178 is not limited to the extensible SDK. You can use 179 devtool 180 to help you easily develop any project whose build output must be 181 part of an image built using the build system. 182 183The ``devtool`` command line is organized similarly to 184:ref:`overview-manual/development-environment:git` in that it has a number of 185sub-commands for each function. You can run ``devtool --help`` to see 186all the commands. 187 188.. note:: 189 190 See the " 191 devtool 192 Quick Reference 193 " in the Yocto Project Reference Manual for a 194 devtool 195 quick reference. 196 197Three ``devtool`` subcommands provide entry-points into 198development: 199 200- *devtool add*: Assists in adding new software to be built. 201 202- *devtool modify*: Sets up an environment to enable you to modify 203 the source of an existing component. 204 205- *devtool upgrade*: Updates an existing recipe so that you can 206 build it for an updated set of source files. 207 208As with the build system, "recipes" represent software packages within 209``devtool``. When you use ``devtool add``, a recipe is automatically 210created. When you use ``devtool modify``, the specified existing recipe 211is used in order to determine where to get the source code and how to 212patch it. In both cases, an environment is set up so that when you build 213the recipe a source tree that is under your control is used in order to 214allow you to make changes to the source as desired. By default, new 215recipes and the source go into a "workspace" directory under the SDK. 216 217The remainder of this section presents the ``devtool add``, 218``devtool modify``, and ``devtool upgrade`` workflows. 219 220Use ``devtool add`` to Add an Application 221----------------------------------------- 222 223The ``devtool add`` command generates a new recipe based on existing 224source code. This command takes advantage of the 225:ref:`devtool-the-workspace-layer-structure` 226layer that many ``devtool`` commands use. The command is flexible enough 227to allow you to extract source code into both the workspace or a 228separate local Git repository and to use existing code that does not 229need to be extracted. 230 231Depending on your particular scenario, the arguments and options you use 232with ``devtool add`` form different combinations. The following diagram 233shows common development flows you would use with the ``devtool add`` 234command: 235 236.. image:: figures/sdk-devtool-add-flow.png 237 :align: center 238 2391. *Generating the New Recipe*: The top part of the flow shows three 240 scenarios by which you could use ``devtool add`` to generate a recipe 241 based on existing source code. 242 243 In a shared development environment, it is typical for other 244 developers to be responsible for various areas of source code. As a 245 developer, you are probably interested in using that source code as 246 part of your development within the Yocto Project. All you need is 247 access to the code, a recipe, and a controlled area in which to do 248 your work. 249 250 Within the diagram, three possible scenarios feed into the 251 ``devtool add`` workflow: 252 253 - *Left*: The left scenario in the figure represents a common 254 situation where the source code does not exist locally and needs 255 to be extracted. In this situation, the source code is extracted 256 to the default workspace - you do not want the files in some 257 specific location outside of the workspace. Thus, everything you 258 need will be located in the workspace:: 259 260 $ devtool add recipe fetchuri 261 262 With this command, ``devtool`` extracts the upstream 263 source files into a local Git repository within the ``sources`` 264 folder. The command then creates a recipe named recipe and a 265 corresponding append file in the workspace. If you do not provide 266 recipe, the command makes an attempt to determine the recipe name. 267 268 - *Middle*: The middle scenario in the figure also represents a 269 situation where the source code does not exist locally. In this 270 case, the code is again upstream and needs to be extracted to some 271 local area - this time outside of the default workspace. 272 273 .. note:: 274 275 If required, 276 devtool 277 always creates a Git repository locally during the extraction. 278 279 Furthermore, the first positional argument ``srctree`` in this case 280 identifies where the ``devtool add`` command will locate the 281 extracted code outside of the workspace. You need to specify an 282 empty directory:: 283 284 $ devtool add recipe srctree fetchuri 285 286 In summary, 287 the source code is pulled from fetchuri and extracted into the 288 location defined by ``srctree`` as a local Git repository. 289 290 Within workspace, ``devtool`` creates a recipe named recipe along 291 with an associated append file. 292 293 - *Right*: The right scenario in the figure represents a situation 294 where the ``srctree`` has been previously prepared outside of the 295 ``devtool`` workspace. 296 297 The following command provides a new recipe name and identifies 298 the existing source tree location:: 299 300 $ devtool add recipe srctree 301 302 The command examines the source code and creates a recipe named 303 recipe for the code and places the recipe into the workspace. 304 305 Because the extracted source code already exists, ``devtool`` does 306 not try to relocate the source code into the workspace - only the 307 new recipe is placed in the workspace. 308 309 Aside from a recipe folder, the command also creates an associated 310 append folder and places an initial ``*.bbappend`` file within. 311 3122. *Edit the Recipe*: You can use ``devtool edit-recipe`` to open up the 313 editor as defined by the ``$EDITOR`` environment variable and modify 314 the file:: 315 316 $ devtool edit-recipe recipe 317 318 From within the editor, you 319 can make modifications to the recipe that take affect when you build 320 it later. 321 3223. *Build the Recipe or Rebuild the Image*: The next step you take 323 depends on what you are going to do with the new code. 324 325 If you need to eventually move the build output to the target 326 hardware, use the following ``devtool`` command: 327 :; 328 329 $ devtool build recipe 330 331 On the other hand, if you want an image to contain the recipe's 332 packages from the workspace for immediate deployment onto a device 333 (e.g. for testing purposes), you can use the ``devtool build-image`` 334 command:: 335 336 $ devtool build-image image 337 3384. *Deploy the Build Output*: When you use the ``devtool build`` command 339 to build out your recipe, you probably want to see if the resulting 340 build output works as expected on the target hardware. 341 342 .. note:: 343 344 This step assumes you have a previously built image that is 345 already either running in QEMU or is running on actual hardware. 346 Also, it is assumed that for deployment of the image to the 347 target, SSH is installed in the image and, if the image is running 348 on real hardware, you have network access to and from your 349 development machine. 350 351 You can deploy your build output to that target hardware by using the 352 ``devtool deploy-target`` command: $ devtool deploy-target recipe 353 target The target is a live target machine running as an SSH server. 354 355 You can, of course, also deploy the image you build to actual 356 hardware by using the ``devtool build-image`` command. However, 357 ``devtool`` does not provide a specific command that allows you to 358 deploy the image to actual hardware. 359 3605. *Finish Your Work With the Recipe*: The ``devtool finish`` command 361 creates any patches corresponding to commits in the local Git 362 repository, moves the new recipe to a more permanent layer, and then 363 resets the recipe so that the recipe is built normally rather than 364 from the workspace. 365 :: 366 367 $ devtool finish recipe layer 368 369 .. note:: 370 371 Any changes you want to turn into patches must be committed to the 372 Git repository in the source tree. 373 374 As mentioned, the ``devtool finish`` command moves the final recipe 375 to its permanent layer. 376 377 As a final process of the ``devtool finish`` command, the state of 378 the standard layers and the upstream source is restored so that you 379 can build the recipe from those areas rather than the workspace. 380 381 .. note:: 382 383 You can use the 384 devtool reset 385 command to put things back should you decide you do not want to 386 proceed with your work. If you do use this command, realize that 387 the source tree is preserved. 388 389Use ``devtool modify`` to Modify the Source of an Existing Component 390-------------------------------------------------------------------- 391 392The ``devtool modify`` command prepares the way to work on existing code 393that already has a local recipe in place that is used to build the 394software. The command is flexible enough to allow you to extract code 395from an upstream source, specify the existing recipe, and keep track of 396and gather any patch files from other developers that are associated 397with the code. 398 399Depending on your particular scenario, the arguments and options you use 400with ``devtool modify`` form different combinations. The following 401diagram shows common development flows for the ``devtool modify`` 402command: 403 404.. image:: figures/sdk-devtool-modify-flow.png 405 :align: center 406 4071. *Preparing to Modify the Code*: The top part of the flow shows three 408 scenarios by which you could use ``devtool modify`` to prepare to 409 work on source files. Each scenario assumes the following: 410 411 - The recipe exists locally in a layer external to the ``devtool`` 412 workspace. 413 414 - The source files exist either upstream in an un-extracted state or 415 locally in a previously extracted state. 416 417 The typical situation is where another developer has created a layer 418 for use with the Yocto Project and their recipe already resides in 419 that layer. Furthermore, their source code is readily available 420 either upstream or locally. 421 422 - *Left*: The left scenario in the figure represents a common 423 situation where the source code does not exist locally and it 424 needs to be extracted from an upstream source. In this situation, 425 the source is extracted into the default ``devtool`` workspace 426 location. The recipe, in this scenario, is in its own layer 427 outside the workspace (i.e. ``meta-``\ layername). 428 429 The following command identifies the recipe and, by default, 430 extracts the source files:: 431 432 $ devtool modify recipe 433 434 Once 435 ``devtool``\ locates the recipe, ``devtool`` uses the recipe's 436 :term:`SRC_URI` statements to 437 locate the source code and any local patch files from other 438 developers. 439 440 With this scenario, there is no ``srctree`` argument. Consequently, the 441 default behavior of the ``devtool modify`` command is to extract 442 the source files pointed to by the ``SRC_URI`` statements into a 443 local Git structure. Furthermore, the location for the extracted 444 source is the default area within the ``devtool`` workspace. The 445 result is that the command sets up both the source code and an 446 append file within the workspace while the recipe remains in its 447 original location. 448 449 Additionally, if you have any non-patch local files (i.e. files 450 referred to with ``file://`` entries in ``SRC_URI`` statement 451 excluding ``*.patch/`` or ``*.diff``), these files are copied to 452 an ``oe-local-files`` folder under the newly created source tree. 453 Copying the files here gives you a convenient area from which you 454 can modify the files. Any changes or additions you make to those 455 files are incorporated into the build the next time you build the 456 software just as are other changes you might have made to the 457 source. 458 459 - *Middle*: The middle scenario in the figure represents a situation 460 where the source code also does not exist locally. In this case, 461 the code is again upstream and needs to be extracted to some local 462 area as a Git repository. The recipe, in this scenario, is again 463 local and in its own layer outside the workspace. 464 465 The following command tells ``devtool`` the recipe with which to 466 work and, in this case, identifies a local area for the extracted 467 source files that exists outside of the default ``devtool`` 468 workspace:: 469 470 $ devtool modify recipe srctree 471 472 .. note:: 473 474 You cannot provide a URL for 475 srctree 476 using the 477 devtool 478 command. 479 480 As with all extractions, the command uses the recipe's ``SRC_URI`` 481 statements to locate the source files and any associated patch 482 files. Non-patch files are copied to an ``oe-local-files`` folder 483 under the newly created source tree. 484 485 Once the files are located, the command by default extracts them 486 into ``srctree``. 487 488 Within workspace, ``devtool`` creates an append file for the 489 recipe. The recipe remains in its original location but the source 490 files are extracted to the location you provide with ``srctree``. 491 492 - *Right*: The right scenario in the figure represents a situation 493 where the source tree (``srctree``) already exists locally as a 494 previously extracted Git structure outside of the ``devtool`` 495 workspace. In this example, the recipe also exists elsewhere 496 locally in its own layer. 497 498 The following command tells ``devtool`` the recipe with which to 499 work, uses the "-n" option to indicate source does not need to be 500 extracted, and uses ``srctree`` to point to the previously extracted 501 source files:: 502 503 $ devtool modify -n recipe srctree 504 505 If an ``oe-local-files`` subdirectory happens to exist and it 506 contains non-patch files, the files are used. However, if the 507 subdirectory does not exist and you run the ``devtool finish`` 508 command, any non-patch files that might exist next to the recipe 509 are removed because it appears to ``devtool`` that you have 510 deleted those files. 511 512 Once the ``devtool modify`` command finishes, it creates only an 513 append file for the recipe in the ``devtool`` workspace. The 514 recipe and the source code remain in their original locations. 515 5162. *Edit the Source*: Once you have used the ``devtool modify`` command, 517 you are free to make changes to the source files. You can use any 518 editor you like to make and save your source code modifications. 519 5203. *Build the Recipe or Rebuild the Image*: The next step you take 521 depends on what you are going to do with the new code. 522 523 If you need to eventually move the build output to the target 524 hardware, use the following ``devtool`` command:: 525 526 $ devtool build recipe 527 528 On the other hand, if you want an image to contain the recipe's 529 packages from the workspace for immediate deployment onto a device 530 (e.g. for testing purposes), you can use the ``devtool build-image`` 531 command: $ devtool build-image image 532 5334. *Deploy the Build Output*: When you use the ``devtool build`` command 534 to build out your recipe, you probably want to see if the resulting 535 build output works as expected on target hardware. 536 537 .. note:: 538 539 This step assumes you have a previously built image that is 540 already either running in QEMU or running on actual hardware. 541 Also, it is assumed that for deployment of the image to the 542 target, SSH is installed in the image and if the image is running 543 on real hardware that you have network access to and from your 544 development machine. 545 546 You can deploy your build output to that target hardware by using the 547 ``devtool deploy-target`` command:: 548 549 $ devtool deploy-target recipe target 550 551 The target is a live target machine running as an SSH server. 552 553 You can, of course, use other methods to deploy the image you built 554 using the ``devtool build-image`` command to actual hardware. 555 ``devtool`` does not provide a specific command to deploy the image 556 to actual hardware. 557 5585. *Finish Your Work With the Recipe*: The ``devtool finish`` command 559 creates any patches corresponding to commits in the local Git 560 repository, updates the recipe to point to them (or creates a 561 ``.bbappend`` file to do so, depending on the specified destination 562 layer), and then resets the recipe so that the recipe is built 563 normally rather than from the workspace. 564 :: 565 566 $ devtool finish recipe layer 567 568 .. note:: 569 570 Any changes you want to turn into patches must be staged and 571 committed within the local Git repository before you use the 572 devtool finish 573 command. 574 575 Because there is no need to move the recipe, ``devtool finish`` 576 either updates the original recipe in the original layer or the 577 command creates a ``.bbappend`` file in a different layer as provided 578 by layer. Any work you did in the ``oe-local-files`` directory is 579 preserved in the original files next to the recipe during the 580 ``devtool finish`` command. 581 582 As a final process of the ``devtool finish`` command, the state of 583 the standard layers and the upstream source is restored so that you 584 can build the recipe from those areas rather than from the workspace. 585 586 .. note:: 587 588 You can use the 589 devtool reset 590 command to put things back should you decide you do not want to 591 proceed with your work. If you do use this command, realize that 592 the source tree is preserved. 593 594Use ``devtool upgrade`` to Create a Version of the Recipe that Supports a Newer Version of the Software 595------------------------------------------------------------------------------------------------------- 596 597The ``devtool upgrade`` command upgrades an existing recipe to that of a 598more up-to-date version found upstream. Throughout the life of software, 599recipes continually undergo version upgrades by their upstream 600publishers. You can use the ``devtool upgrade`` workflow to make sure 601your recipes you are using for builds are up-to-date with their upstream 602counterparts. 603 604.. note:: 605 606 Several methods exist by which you can upgrade recipes - 607 devtool upgrade 608 happens to be one. You can read about all the methods by which you 609 can upgrade recipes in the " 610 Upgrading Recipes 611 " section of the Yocto Project Development Tasks Manual. 612 613The ``devtool upgrade`` command is flexible enough to allow you to 614specify source code revision and versioning schemes, extract code into 615or out of the ``devtool`` 616:ref:`devtool-the-workspace-layer-structure`, 617and work with any source file forms that the 618:ref:`fetchers <bitbake:bitbake-user-manual/bitbake-user-manual-fetching:fetchers>` support. 619 620The following diagram shows the common development flow used with the 621``devtool upgrade`` command: 622 623.. image:: figures/sdk-devtool-upgrade-flow.png 624 :align: center 625 6261. *Initiate the Upgrade*: The top part of the flow shows the typical 627 scenario by which you use the ``devtool upgrade`` command. The 628 following conditions exist: 629 630 - The recipe exists in a local layer external to the ``devtool`` 631 workspace. 632 633 - The source files for the new release exist in the same location 634 pointed to by :term:`SRC_URI` 635 in the recipe (e.g. a tarball with the new version number in the 636 name, or as a different revision in the upstream Git repository). 637 638 A common situation is where third-party software has undergone a 639 revision so that it has been upgraded. The recipe you have access to 640 is likely in your own layer. Thus, you need to upgrade the recipe to 641 use the newer version of the software:: 642 643 $ devtool upgrade -V version recipe 644 645 By default, the ``devtool upgrade`` command extracts source 646 code into the ``sources`` directory in the 647 :ref:`devtool-the-workspace-layer-structure`. 648 If you want the code extracted to any other location, you need to 649 provide the ``srctree`` positional argument with the command as follows:: 650 651 $ devtool upgrade -V version recipe srctree 652 653 .. note:: 654 655 In this example, the "-V" option specifies the new version. If you 656 don't use "-V", the command upgrades the recipe to the latest 657 version. 658 659 If the source files pointed to by the ``SRC_URI`` statement in the 660 recipe are in a Git repository, you must provide the "-S" option and 661 specify a revision for the software. 662 663 Once ``devtool`` locates the recipe, it uses the ``SRC_URI`` variable 664 to locate the source code and any local patch files from other 665 developers. The result is that the command sets up the source code, 666 the new version of the recipe, and an append file all within the 667 workspace. 668 669 Additionally, if you have any non-patch local files (i.e. files 670 referred to with ``file://`` entries in ``SRC_URI`` statement 671 excluding ``*.patch/`` or ``*.diff``), these files are copied to an 672 ``oe-local-files`` folder under the newly created source tree. 673 Copying the files here gives you a convenient area from which you can 674 modify the files. Any changes or additions you make to those files 675 are incorporated into the build the next time you build the software 676 just as are other changes you might have made to the source. 677 6782. *Resolve any Conflicts created by the Upgrade*: Conflicts could happen 679 after upgrading the software to a new version. Conflicts occur 680 if your recipe specifies some patch files in ``SRC_URI`` that 681 conflict with changes made in the new version of the software. For 682 such cases, you need to resolve the conflicts by editing the source 683 and following the normal ``git rebase`` conflict resolution process. 684 685 Before moving onto the next step, be sure to resolve any such 686 conflicts created through use of a newer or different version of the 687 software. 688 6893. *Build the Recipe or Rebuild the Image*: The next step you take 690 depends on what you are going to do with the new code. 691 692 If you need to eventually move the build output to the target 693 hardware, use the following ``devtool`` command:: 694 695 $ devtool build recipe 696 697 On the other hand, if you want an image to contain the recipe's 698 packages from the workspace for immediate deployment onto a device 699 (e.g. for testing purposes), you can use the ``devtool build-image`` 700 command:: 701 702 $ devtool build-image image 703 7044. *Deploy the Build Output*: When you use the ``devtool build`` command 705 or ``bitbake`` to build your recipe, you probably want to see if the 706 resulting build output works as expected on target hardware. 707 708 .. note:: 709 710 This step assumes you have a previously built image that is 711 already either running in QEMU or running on actual hardware. 712 Also, it is assumed that for deployment of the image to the 713 target, SSH is installed in the image and if the image is running 714 on real hardware that you have network access to and from your 715 development machine. 716 717 You can deploy your build output to that target hardware by using the 718 ``devtool deploy-target`` command: $ devtool deploy-target recipe 719 target The target is a live target machine running as an SSH server. 720 721 You can, of course, also deploy the image you build using the 722 ``devtool build-image`` command to actual hardware. However, 723 ``devtool`` does not provide a specific command that allows you to do 724 this. 725 7265. *Finish Your Work With the Recipe*: The ``devtool finish`` command 727 creates any patches corresponding to commits in the local Git 728 repository, moves the new recipe to a more permanent layer, and then 729 resets the recipe so that the recipe is built normally rather than 730 from the workspace. 731 732 Any work you did in the ``oe-local-files`` directory is preserved in 733 the original files next to the recipe during the ``devtool finish`` 734 command. 735 736 If you specify a destination layer that is the same as the original 737 source, then the old version of the recipe and associated files are 738 removed prior to adding the new version. 739 :: 740 741 $ devtool finish recipe layer 742 743 .. note:: 744 745 Any changes you want to turn into patches must be committed to the 746 Git repository in the source tree. 747 748 As a final process of the ``devtool finish`` command, the state of 749 the standard layers and the upstream source is restored so that you 750 can build the recipe from those areas rather than the workspace. 751 752 .. note:: 753 754 You can use the 755 devtool reset 756 command to put things back should you decide you do not want to 757 proceed with your work. If you do use this command, realize that 758 the source tree is preserved. 759 760A Closer Look at ``devtool add`` 761================================ 762 763The ``devtool add`` command automatically creates a recipe based on the 764source tree you provide with the command. Currently, the command has 765support for the following: 766 767- Autotools (``autoconf`` and ``automake``) 768 769- CMake 770 771- Scons 772 773- ``qmake`` 774 775- Plain ``Makefile`` 776 777- Out-of-tree kernel module 778 779- Binary package (i.e. "-b" option) 780 781- Node.js module 782 783- Python modules that use ``setuptools`` or ``distutils`` 784 785Apart from binary packages, the determination of how a source tree 786should be treated is automatic based on the files present within that 787source tree. For example, if a ``CMakeLists.txt`` file is found, then 788the source tree is assumed to be using CMake and is treated accordingly. 789 790.. note:: 791 792 In most cases, you need to edit the automatically generated recipe in 793 order to make it build properly. Typically, you would go through 794 several edit and build cycles until the recipe successfully builds. 795 Once the recipe builds, you could use possible further iterations to 796 test the recipe on the target device. 797 798The remainder of this section covers specifics regarding how parts of 799the recipe are generated. 800 801Name and Version 802---------------- 803 804If you do not specify a name and version on the command line, 805``devtool add`` uses various metadata within the source tree in an 806attempt to determine the name and version of the software being built. 807Based on what the tool determines, ``devtool`` sets the name of the 808created recipe file accordingly. 809 810If ``devtool`` cannot determine the name and version, the command prints 811an error. For such cases, you must re-run the command and provide the 812name and version, just the name, or just the version as part of the 813command line. 814 815Sometimes the name or version determined from the source tree might be 816incorrect. For such a case, you must reset the recipe:: 817 818 $ devtool reset -n recipename 819 820After running the ``devtool reset`` command, you need to 821run ``devtool add`` again and provide the name or the version. 822 823Dependency Detection and Mapping 824-------------------------------- 825 826The ``devtool add`` command attempts to detect build-time dependencies 827and map them to other recipes in the system. During this mapping, the 828command fills in the names of those recipes as part of the 829:term:`DEPENDS` variable within the 830recipe. If a dependency cannot be mapped, ``devtool`` places a comment 831in the recipe indicating such. The inability to map a dependency can 832result from naming not being recognized or because the dependency simply 833is not available. For cases where the dependency is not available, you 834must use the ``devtool add`` command to add an additional recipe that 835satisfies the dependency. Once you add that recipe, you need to update 836the ``DEPENDS`` variable in the original recipe to include the new 837recipe. 838 839If you need to add runtime dependencies, you can do so by adding the 840following to your recipe:: 841 842 RDEPENDS_${PN} += "dependency1 dependency2 ..." 843 844.. note:: 845 846 The 847 devtool add 848 command often cannot distinguish between mandatory and optional 849 dependencies. Consequently, some of the detected dependencies might 850 in fact be optional. When in doubt, consult the documentation or the 851 configure script for the software the recipe is building for further 852 details. In some cases, you might find you can substitute the 853 dependency with an option that disables the associated functionality 854 passed to the configure script. 855 856License Detection 857----------------- 858 859The ``devtool add`` command attempts to determine if the software you 860are adding is able to be distributed under a common, open-source 861license. If so, the command sets the 862:term:`LICENSE` value accordingly. 863You should double-check the value added by the command against the 864documentation or source files for the software you are building and, if 865necessary, update that ``LICENSE`` value. 866 867The ``devtool add`` command also sets the 868:term:`LIC_FILES_CHKSUM` 869value to point to all files that appear to be license-related. Realize 870that license statements often appear in comments at the top of source 871files or within the documentation. In such cases, the command does not 872recognize those license statements. Consequently, you might need to 873amend the ``LIC_FILES_CHKSUM`` variable to point to one or more of those 874comments if present. Setting ``LIC_FILES_CHKSUM`` is particularly 875important for third-party software. The mechanism attempts to ensure 876correct licensing should you upgrade the recipe to a newer upstream 877version in future. Any change in licensing is detected and you receive 878an error prompting you to check the license text again. 879 880If the ``devtool add`` command cannot determine licensing information, 881``devtool`` sets the ``LICENSE`` value to "CLOSED" and leaves the 882``LIC_FILES_CHKSUM`` value unset. This behavior allows you to continue 883with development even though the settings are unlikely to be correct in 884all cases. You should check the documentation or source files for the 885software you are building to determine the actual license. 886 887Adding Makefile-Only Software 888----------------------------- 889 890The use of Make by itself is very common in both proprietary and 891open-source software. Unfortunately, Makefiles are often not written 892with cross-compilation in mind. Thus, ``devtool add`` often cannot do 893very much to ensure that these Makefiles build correctly. It is very 894common, for example, to explicitly call ``gcc`` instead of using the 895:term:`CC` variable. Usually, in a 896cross-compilation environment, ``gcc`` is the compiler for the build 897host and the cross-compiler is named something similar to 898``arm-poky-linux-gnueabi-gcc`` and might require arguments (e.g. to 899point to the associated sysroot for the target machine). 900 901When writing a recipe for Makefile-only software, keep the following in 902mind: 903 904- You probably need to patch the Makefile to use variables instead of 905 hardcoding tools within the toolchain such as ``gcc`` and ``g++``. 906 907- The environment in which Make runs is set up with various standard 908 variables for compilation (e.g. ``CC``, ``CXX``, and so forth) in a 909 similar manner to the environment set up by the SDK's environment 910 setup script. One easy way to see these variables is to run the 911 ``devtool build`` command on the recipe and then look in 912 ``oe-logs/run.do_compile``. Towards the top of this file, there is 913 a list of environment variables that are set. You can take 914 advantage of these variables within the Makefile. 915 916- If the Makefile sets a default for a variable using "=", that default 917 overrides the value set in the environment, which is usually not 918 desirable. For this case, you can either patch the Makefile so it 919 sets the default using the "?=" operator, or you can alternatively 920 force the value on the ``make`` command line. To force the value on 921 the command line, add the variable setting to 922 :term:`EXTRA_OEMAKE` or 923 :term:`PACKAGECONFIG_CONFARGS` 924 within the recipe. Here is an example using ``EXTRA_OEMAKE``:: 925 926 EXTRA_OEMAKE += "'CC=${CC}' 'CXX=${CXX}'" 927 928 In the above example, 929 single quotes are used around the variable settings as the values are 930 likely to contain spaces because required default options are passed 931 to the compiler. 932 933- Hardcoding paths inside Makefiles is often problematic in a 934 cross-compilation environment. This is particularly true because 935 those hardcoded paths often point to locations on the build host and 936 thus will either be read-only or will introduce contamination into 937 the cross-compilation because they are specific to the build host 938 rather than the target. Patching the Makefile to use prefix variables 939 or other path variables is usually the way to handle this situation. 940 941- Sometimes a Makefile runs target-specific commands such as 942 ``ldconfig``. For such cases, you might be able to apply patches that 943 remove these commands from the Makefile. 944 945Adding Native Tools 946------------------- 947 948Often, you need to build additional tools that run on the :term:`Build 949Host` as opposed to 950the target. You should indicate this requirement by using one of the 951following methods when you run ``devtool add``: 952 953- Specify the name of the recipe such that it ends with "-native". 954 Specifying the name like this produces a recipe that only builds for 955 the build host. 956 957- Specify the "--also-native" option with the ``devtool add`` 958 command. Specifying this option creates a recipe file that still 959 builds for the target but also creates a variant with a "-native" 960 suffix that builds for the build host. 961 962.. note:: 963 964 If you need to add a tool that is shipped as part of a source tree 965 that builds code for the target, you can typically accomplish this by 966 building the native and target parts separately rather than within 967 the same compilation process. Realize though that with the 968 "--also-native" option, you can add the tool using just one 969 recipe file. 970 971Adding Node.js Modules 972---------------------- 973 974You can use the ``devtool add`` command two different ways to add 975Node.js modules: 1) Through ``npm`` and, 2) from a repository or local 976source. 977 978Use the following form to add Node.js modules through ``npm``:: 979 980 $ devtool add "npm://registry.npmjs.org;name=forever;version=0.15.1" 981 982The name and 983version parameters are mandatory. Lockdown and shrinkwrap files are 984generated and pointed to by the recipe in order to freeze the version 985that is fetched for the dependencies according to the first time. This 986also saves checksums that are verified on future fetches. Together, 987these behaviors ensure the reproducibility and integrity of the build. 988 989.. note:: 990 991 - You must use quotes around the URL. The ``devtool add`` does not 992 require the quotes, but the shell considers ";" as a splitter 993 between multiple commands. Thus, without the quotes, 994 ``devtool add`` does not receive the other parts, which results in 995 several "command not found" errors. 996 997 - In order to support adding Node.js modules, a ``nodejs`` recipe 998 must be part of your SDK. 999 1000As mentioned earlier, you can also add Node.js modules directly from a 1001repository or local source tree. To add modules this way, use 1002``devtool add`` in the following form:: 1003 1004 $ devtool add https://github.com/diversario/node-ssdp 1005 1006In this example, ``devtool`` 1007fetches the specified Git repository, detects the code as Node.js code, 1008fetches dependencies using ``npm``, and sets 1009:term:`SRC_URI` accordingly. 1010 1011Working With Recipes 1012==================== 1013 1014When building a recipe using the ``devtool build`` command, the typical 1015build progresses as follows: 1016 10171. Fetch the source 1018 10192. Unpack the source 1020 10213. Configure the source 1022 10234. Compile the source 1024 10255. Install the build output 1026 10276. Package the installed output 1028 1029For recipes in the workspace, fetching and unpacking is disabled as the 1030source tree has already been prepared and is persistent. Each of these 1031build steps is defined as a function (task), usually with a "do\_" prefix 1032(e.g. :ref:`ref-tasks-fetch`, 1033:ref:`ref-tasks-unpack`, and so 1034forth). These functions are typically shell scripts but can instead be 1035written in Python. 1036 1037If you look at the contents of a recipe, you will see that the recipe 1038does not include complete instructions for building the software. 1039Instead, common functionality is encapsulated in classes inherited with 1040the ``inherit`` directive. This technique leaves the recipe to describe 1041just the things that are specific to the software being built. There is 1042a :ref:`base <ref-classes-base>` class that 1043is implicitly inherited by all recipes and provides the functionality 1044that most recipes typically need. 1045 1046The remainder of this section presents information useful when working 1047with recipes. 1048 1049Finding Logs and Work Files 1050--------------------------- 1051 1052After the first run of the ``devtool build`` command, recipes that were 1053previously created using the ``devtool add`` command or whose sources 1054were modified using the ``devtool modify`` command contain symbolic 1055links created within the source tree: 1056 1057- ``oe-logs``: This link points to the directory in which log files and 1058 run scripts for each build step are created. 1059 1060- ``oe-workdir``: This link points to the temporary work area for the 1061 recipe. The following locations under ``oe-workdir`` are particularly 1062 useful: 1063 1064 - ``image/``: Contains all of the files installed during the 1065 :ref:`ref-tasks-install` stage. 1066 Within a recipe, this directory is referred to by the expression 1067 ``${``\ :term:`D`\ ``}``. 1068 1069 - ``sysroot-destdir/``: Contains a subset of files installed within 1070 ``do_install`` that have been put into the shared sysroot. For 1071 more information, see the 1072 ":ref:`dev-manual/common-tasks:sharing files between recipes`" section. 1073 1074 - ``packages-split/``: Contains subdirectories for each package 1075 produced by the recipe. For more information, see the 1076 ":ref:`sdk-manual/extensible:packaging`" section. 1077 1078You can use these links to get more information on what is happening at 1079each build step. 1080 1081Setting Configure Arguments 1082--------------------------- 1083 1084If the software your recipe is building uses GNU autoconf, then a fixed 1085set of arguments is passed to it to enable cross-compilation plus any 1086extras specified by 1087:term:`EXTRA_OECONF` or 1088:term:`PACKAGECONFIG_CONFARGS` 1089set within the recipe. If you wish to pass additional options, add them 1090to ``EXTRA_OECONF`` or ``PACKAGECONFIG_CONFARGS``. Other supported build 1091tools have similar variables (e.g. 1092:term:`EXTRA_OECMAKE` for 1093CMake, :term:`EXTRA_OESCONS` 1094for Scons, and so forth). If you need to pass anything on the ``make`` 1095command line, you can use ``EXTRA_OEMAKE`` or the 1096:term:`PACKAGECONFIG_CONFARGS` 1097variables to do so. 1098 1099You can use the ``devtool configure-help`` command to help you set the 1100arguments listed in the previous paragraph. The command determines the 1101exact options being passed, and shows them to you along with any custom 1102arguments specified through ``EXTRA_OECONF`` or 1103``PACKAGECONFIG_CONFARGS``. If applicable, the command also shows you 1104the output of the configure script's "--help" option as a 1105reference. 1106 1107Sharing Files Between Recipes 1108----------------------------- 1109 1110Recipes often need to use files provided by other recipes on the 1111:term:`Build Host`. For example, 1112an application linking to a common library needs access to the library 1113itself and its associated headers. The way this access is accomplished 1114within the extensible SDK is through the sysroot. There is one sysroot per 1115"machine" for which the SDK is being built. In practical terms, this 1116means there is a sysroot for the target machine, and a sysroot for 1117the build host. 1118 1119Recipes should never write files directly into the sysroot. Instead, 1120files should be installed into standard locations during the 1121:ref:`ref-tasks-install` task within 1122the ``${``\ :term:`D`\ ``}`` directory. A 1123subset of these files automatically goes into the sysroot. The reason 1124for this limitation is that almost all files that go into the sysroot 1125are cataloged in manifests in order to ensure they can be removed later 1126when a recipe is modified or removed. Thus, the sysroot is able to 1127remain free from stale files. 1128 1129Packaging 1130--------- 1131 1132Packaging is not always particularly relevant within the extensible SDK. 1133However, if you examine how build output gets into the final image on 1134the target device, it is important to understand packaging because the 1135contents of the image are expressed in terms of packages and not 1136recipes. 1137 1138During the :ref:`ref-tasks-package` 1139task, files installed during the 1140:ref:`ref-tasks-install` task are 1141split into one main package, which is almost always named the same as 1142the recipe, and into several other packages. This separation exists 1143because not all of those installed files are useful in every image. For 1144example, you probably do not need any of the documentation installed in 1145a production image. Consequently, for each recipe the documentation 1146files are separated into a ``-doc`` package. Recipes that package 1147software containing optional modules or plugins might undergo additional 1148package splitting as well. 1149 1150After building a recipe, you can see where files have gone by looking in 1151the ``oe-workdir/packages-split`` directory, which contains a 1152subdirectory for each package. Apart from some advanced cases, the 1153:term:`PACKAGES` and 1154:term:`FILES` variables controls 1155splitting. The ``PACKAGES`` variable lists all of the packages to be 1156produced, while the ``FILES`` variable specifies which files to include 1157in each package by using an override to specify the package. For 1158example, ``FILES_${PN}`` specifies the files to go into the main package 1159(i.e. the main package has the same name as the recipe and 1160``${``\ :term:`PN`\ ``}`` evaluates to the 1161recipe name). The order of the ``PACKAGES`` value is significant. For 1162each installed file, the first package whose ``FILES`` value matches the 1163file is the package into which the file goes. Both the ``PACKAGES`` and 1164``FILES`` variables have default values. Consequently, you might find 1165you do not even need to set these variables in your recipe unless the 1166software the recipe is building installs files into non-standard 1167locations. 1168 1169Restoring the Target Device to its Original State 1170================================================= 1171 1172If you use the ``devtool deploy-target`` command to write a recipe's 1173build output to the target, and you are working on an existing component 1174of the system, then you might find yourself in a situation where you 1175need to restore the original files that existed prior to running the 1176``devtool deploy-target`` command. Because the ``devtool deploy-target`` 1177command backs up any files it overwrites, you can use the 1178``devtool undeploy-target`` command to restore those files and remove 1179any other files the recipe deployed. Consider the following example:: 1180 1181 $ devtool undeploy-target lighttpd root@192.168.7.2 1182 1183If you have deployed 1184multiple applications, you can remove them all using the "-a" option 1185thus restoring the target device to its original state:: 1186 1187 $ devtool undeploy-target -a root@192.168.7.2 1188 1189Information about files deployed to 1190the target as well as any backed up files are stored on the target 1191itself. This storage, of course, requires some additional space on the 1192target machine. 1193 1194.. note:: 1195 1196 The 1197 devtool deploy-target 1198 and 1199 devtool undeploy-target 1200 commands do not currently interact with any package management system 1201 on the target device (e.g. RPM or OPKG). Consequently, you should not 1202 intermingle 1203 devtool deploy-target 1204 and package manager operations on the target device. Doing so could 1205 result in a conflicting set of files. 1206 1207Installing Additional Items Into the Extensible SDK 1208=================================================== 1209 1210Out of the box the extensible SDK typically only comes with a small 1211number of tools and libraries. A minimal SDK starts mostly empty and is 1212populated on-demand. Sometimes you must explicitly install extra items 1213into the SDK. If you need these extra items, you can first search for 1214the items using the ``devtool search`` command. For example, suppose you 1215need to link to libGL but you are not sure which recipe provides libGL. 1216You can use the following command to find out:: 1217 1218 $ devtool search libGL mesa 1219 1220A free implementation of the OpenGL API Once you know the recipe 1221(i.e. ``mesa`` in this example), you can install it:: 1222 1223 $ devtool sdk-install mesa 1224 1225By default, the ``devtool sdk-install`` command assumes 1226the item is available in pre-built form from your SDK provider. If the 1227item is not available and it is acceptable to build the item from 1228source, you can add the "-s" option as follows:: 1229 1230 $ devtool sdk-install -s mesa 1231 1232It is important to remember that building the item from source 1233takes significantly longer than installing the pre-built artifact. Also, 1234if there is no recipe for the item you want to add to the SDK, you must 1235instead add the item using the ``devtool add`` command. 1236 1237Applying Updates to an Installed Extensible SDK 1238=============================================== 1239 1240If you are working with an installed extensible SDK that gets 1241occasionally updated (e.g. a third-party SDK), then you will need to 1242manually "pull down" the updates into the installed SDK. 1243 1244To update your installed SDK, use ``devtool`` as follows:: 1245 1246 $ devtool sdk-update 1247 1248The previous command assumes your SDK provider has set the 1249default update URL for you through the :term:`SDK_UPDATE_URL` 1250variable as described in the 1251":ref:`sdk-manual/appendix-customizing:Providing Updates to the Extensible SDK After Installation`" 1252section. If the SDK provider has not set that default URL, you need to 1253specify it yourself in the command as follows: $ devtool sdk-update 1254path_to_update_directory 1255 1256.. note:: 1257 1258 The URL needs to point specifically to a published SDK and not to an 1259 SDK installer that you would download and install. 1260 1261Creating a Derivative SDK With Additional Components 1262==================================================== 1263 1264You might need to produce an SDK that contains your own custom 1265libraries. A good example would be if you were a vendor with customers 1266that use your SDK to build their own platform-specific software and 1267those customers need an SDK that has custom libraries. In such a case, 1268you can produce a derivative SDK based on the currently installed SDK 1269fairly easily by following these steps: 1270 12711. If necessary, install an extensible SDK that you want to use as a 1272 base for your derivative SDK. 1273 12742. Source the environment script for the SDK. 1275 12763. Add the extra libraries or other components you want by using the 1277 ``devtool add`` command. 1278 12794. Run the ``devtool build-sdk`` command. 1280 1281The previous steps take the recipes added to the workspace and construct 1282a new SDK installer that contains those recipes and the resulting binary 1283artifacts. The recipes go into their own separate layer in the 1284constructed derivative SDK, which leaves the workspace clean and ready 1285for users to add their own recipes. 1286