1.. SPDX-License-Identifier: CC-BY-SA-2.0-UK 2 3************************ 4Using the Extensible SDK 5************************ 6 7This chapter describes the extensible SDK and how to install it. 8Information covers the pieces of the SDK, how to install it, and 9presents a look at using the ``devtool`` functionality. The extensible 10SDK makes it easy to add new applications and libraries to an image, 11modify the source for an existing component, test changes on the target 12hardware, and ease integration into the rest of the 13:term:`OpenEmbedded Build System`. 14 15.. note:: 16 17 For a side-by-side comparison of main features supported for an 18 extensible SDK as compared to a standard SDK, see the 19 :ref:`sdk-manual/intro:introduction` section. 20 21In addition to the functionality available through ``devtool``, you can 22alternatively make use of the toolchain directly, for example from 23Makefile and Autotools. See the 24":ref:`sdk-manual/working-projects:using the sdk toolchain directly`" chapter 25for more information. 26 27Why use the Extensible SDK and What is in It? 28============================================= 29 30The extensible SDK provides a cross-development toolchain and libraries 31tailored to the contents of a specific image. You would use the 32Extensible SDK if you want a toolchain experience supplemented with the 33powerful set of ``devtool`` commands tailored for the Yocto Project 34environment. 35 36The installed extensible SDK consists of several files and directories. 37Basically, it contains an SDK environment setup script, some 38configuration files, an internal build system, and the ``devtool`` 39functionality. 40 41Installing the Extensible SDK 42============================= 43 44The first thing you need to do is install the SDK on your :term:`Build 45Host` by running the ``*.sh`` installation script. 46 47You can download a tarball installer, which includes the pre-built 48toolchain, the ``runqemu`` script, the internal build system, 49``devtool``, and support files from the appropriate 50:yocto_dl:`toolchain </releases/yocto/yocto-&DISTRO;/toolchain/>` directory within the Index of 51Releases. Toolchains are available for several 32-bit and 64-bit 52architectures with the ``x86_64`` directories, respectively. The 53toolchains the Yocto Project provides are based off the 54``core-image-sato`` and ``core-image-minimal`` images and contain 55libraries appropriate for developing against that image. 56 57The names of the tarball installer scripts are such that a string 58representing the host system appears first in the filename and then is 59immediately followed by a string representing the target architecture. 60An extensible SDK has the string "-ext" as part of the name. Following 61is the general form:: 62 63 poky-glibc-host_system-image_type-arch-toolchain-ext-release_version.sh 64 65 Where: 66 host_system is a string representing your development system: 67 68 i686 or x86_64. 69 70 image_type is the image for which the SDK was built: 71 72 core-image-sato or core-image-minimal 73 74 arch is a string representing the tuned target architecture: 75 76 aarch64, armv5e, core2-64, i586, mips32r2, mips64, ppc7400, or cortexa8hf-neon 77 78 release_version is a string representing the release number of the Yocto Project: 79 80 &DISTRO;, &DISTRO;+snapshot 81 82For example, the following SDK installer is for a 64-bit 83development host system and a i586-tuned target architecture based off 84the SDK for ``core-image-sato`` and using the current &DISTRO; snapshot:: 85 86 poky-glibc-x86_64-core-image-sato-i586-toolchain-ext-&DISTRO;.sh 87 88.. note:: 89 90 As an alternative to downloading an SDK, you can build the SDK 91 installer. For information on building the installer, see the 92 :ref:`sdk-manual/appendix-obtain:building an sdk installer` 93 section. 94 95The SDK and toolchains are self-contained and by default are installed 96into the ``poky_sdk`` folder in your home directory. You can choose to 97install the extensible SDK in any location when you run the installer. 98However, because files need to be written under that directory during 99the normal course of operation, the location you choose for installation 100must be writable for whichever users need to use the SDK. 101 102The following command shows how to run the installer given a toolchain 103tarball for a 64-bit x86 development host system and a 64-bit x86 target 104architecture. The example assumes the SDK installer is located in 105``~/Downloads/`` and has execution rights. 106 107.. note:: 108 109 If you do not have write permissions for the directory into which you 110 are installing the SDK, the installer notifies you and exits. For 111 that case, set up the proper permissions in the directory and run the 112 installer again. 113 114:: 115 116 $ ./Downloads/poky-glibc-x86_64-core-image-minimal-core2-64-toolchain-ext-2.5.sh 117 Poky (Yocto Project Reference Distro) Extensible SDK installer version 2.5 118 ========================================================================== 119 Enter target directory for SDK (default: poky_sdk): 120 You are about to install the SDK to "/home/scottrif/poky_sdk". Proceed [Y/n]? Y 121 Extracting SDK..............done 122 Setting it up... 123 Extracting buildtools... 124 Preparing build system... 125 Parsing recipes: 100% |##################################################################| Time: 0:00:52 126 Initialising tasks: 100% |###############################################################| Time: 0:00:00 127 Checking sstate mirror object availability: 100% |#######################################| Time: 0:00:00 128 Loading cache: 100% |####################################################################| Time: 0:00:00 129 Initialising tasks: 100% |###############################################################| Time: 0:00:00 130 done 131 SDK has been successfully set up and is ready to be used. 132 Each time you wish to use the SDK in a new shell session, you need to source the environment setup script e.g. 133 $ . /home/scottrif/poky_sdk/environment-setup-core2-64-poky-linux 134 135Running the Extensible SDK Environment Setup Script 136=================================================== 137 138Once you have the SDK installed, you must run the SDK environment setup 139script before you can actually use the SDK. This setup script resides in 140the directory you chose when you installed the SDK, which is either the 141default ``poky_sdk`` directory or the directory you chose during 142installation. 143 144Before running the script, be sure it is the one that matches the 145architecture for which you are developing. Environment setup scripts 146begin with the string "``environment-setup``" and include as part of 147their name the tuned target architecture. As an example, the following 148commands set the working directory to where the SDK was installed and 149then source the environment setup script. In this example, the setup 150script is for an IA-based target machine using i586 tuning:: 151 152 $ cd /home/scottrif/poky_sdk 153 $ source environment-setup-core2-64-poky-linux 154 SDK environment now set up; additionally you may now run devtool to perform development tasks. 155 Run devtool --help for further details. 156 157Running the setup script defines many environment variables needed in 158order to use the SDK (e.g. ``PATH``, 159:term:`CC`, 160:term:`LD`, and so forth). If you want to 161see all the environment variables the script exports, examine the 162installation file itself. 163 164Using ``devtool`` in Your SDK Workflow 165====================================== 166 167The cornerstone of the extensible SDK is a command-line tool called 168``devtool``. This tool provides a number of features that help you 169build, test and package software within the extensible SDK, and 170optionally integrate it into an image built by the OpenEmbedded build 171system. 172 173.. note:: 174 175 The use of 176 devtool 177 is not limited to the extensible SDK. You can use 178 devtool 179 to help you easily develop any project whose build output must be 180 part of an image built using the build system. 181 182The ``devtool`` command line is organized similarly to 183:ref:`overview-manual/development-environment:git` in that it has a number of 184sub-commands for each function. You can run ``devtool --help`` to see 185all the commands. 186 187.. note:: 188 189 See the " 190 devtool 191 Quick Reference 192 " in the Yocto Project Reference Manual for a 193 devtool 194 quick reference. 195 196Three ``devtool`` subcommands provide entry-points into 197development: 198 199- *devtool add*: Assists in adding new software to be built. 200 201- *devtool modify*: Sets up an environment to enable you to modify 202 the source of an existing component. 203 204- *devtool upgrade*: Updates an existing recipe so that you can 205 build it for an updated set of source files. 206 207As with the build system, "recipes" represent software packages within 208``devtool``. When you use ``devtool add``, a recipe is automatically 209created. When you use ``devtool modify``, the specified existing recipe 210is used in order to determine where to get the source code and how to 211patch it. In both cases, an environment is set up so that when you build 212the recipe a source tree that is under your control is used in order to 213allow you to make changes to the source as desired. By default, new 214recipes and the source go into a "workspace" directory under the SDK. 215 216The remainder of this section presents the ``devtool add``, 217``devtool modify``, and ``devtool upgrade`` workflows. 218 219Use ``devtool add`` to Add an Application 220----------------------------------------- 221 222The ``devtool add`` command generates a new recipe based on existing 223source code. This command takes advantage of the 224:ref:`devtool-the-workspace-layer-structure` 225layer that many ``devtool`` commands use. The command is flexible enough 226to allow you to extract source code into both the workspace or a 227separate local Git repository and to use existing code that does not 228need to be extracted. 229 230Depending on your particular scenario, the arguments and options you use 231with ``devtool add`` form different combinations. The following diagram 232shows common development flows you would use with the ``devtool add`` 233command: 234 235.. image:: figures/sdk-devtool-add-flow.png 236 :align: center 237 2381. *Generating the New Recipe*: The top part of the flow shows three 239 scenarios by which you could use ``devtool add`` to generate a recipe 240 based on existing source code. 241 242 In a shared development environment, it is typical for other 243 developers to be responsible for various areas of source code. As a 244 developer, you are probably interested in using that source code as 245 part of your development within the Yocto Project. All you need is 246 access to the code, a recipe, and a controlled area in which to do 247 your work. 248 249 Within the diagram, three possible scenarios feed into the 250 ``devtool add`` workflow: 251 252 - *Left*: The left scenario in the figure represents a common 253 situation where the source code does not exist locally and needs 254 to be extracted. In this situation, the source code is extracted 255 to the default workspace - you do not want the files in some 256 specific location outside of the workspace. Thus, everything you 257 need will be located in the workspace:: 258 259 $ devtool add recipe fetchuri 260 261 With this command, ``devtool`` extracts the upstream 262 source files into a local Git repository within the ``sources`` 263 folder. The command then creates a recipe named recipe and a 264 corresponding append file in the workspace. If you do not provide 265 recipe, the command makes an attempt to determine the recipe name. 266 267 - *Middle*: The middle scenario in the figure also represents a 268 situation where the source code does not exist locally. In this 269 case, the code is again upstream and needs to be extracted to some 270 local area - this time outside of the default workspace. 271 272 .. note:: 273 274 If required, 275 devtool 276 always creates a Git repository locally during the extraction. 277 278 Furthermore, the first positional argument ``srctree`` in this case 279 identifies where the ``devtool add`` command will locate the 280 extracted code outside of the workspace. You need to specify an 281 empty directory:: 282 283 $ devtool add recipe srctree fetchuri 284 285 In summary, 286 the source code is pulled from fetchuri and extracted into the 287 location defined by ``srctree`` as a local Git repository. 288 289 Within workspace, ``devtool`` creates a recipe named recipe along 290 with an associated append file. 291 292 - *Right*: The right scenario in the figure represents a situation 293 where the ``srctree`` has been previously prepared outside of the 294 ``devtool`` workspace. 295 296 The following command provides a new recipe name and identifies 297 the existing source tree location:: 298 299 $ devtool add recipe srctree 300 301 The command examines the source code and creates a recipe named 302 recipe for the code and places the recipe into the workspace. 303 304 Because the extracted source code already exists, ``devtool`` does 305 not try to relocate the source code into the workspace - only the 306 new recipe is placed in the workspace. 307 308 Aside from a recipe folder, the command also creates an associated 309 append folder and places an initial ``*.bbappend`` file within. 310 3112. *Edit the Recipe*: You can use ``devtool edit-recipe`` to open up the 312 editor as defined by the ``$EDITOR`` environment variable and modify 313 the file:: 314 315 $ devtool edit-recipe recipe 316 317 From within the editor, you 318 can make modifications to the recipe that take effect when you build 319 it later. 320 3213. *Build the Recipe or Rebuild the Image*: The next step you take 322 depends on what you are going to do with the new code. 323 324 If you need to eventually move the build output to the target 325 hardware, use the following ``devtool`` command: 326 :; 327 328 $ devtool build recipe 329 330 On the other hand, if you want an image to contain the recipe's 331 packages from the workspace for immediate deployment onto a device 332 (e.g. for testing purposes), you can use the ``devtool build-image`` 333 command:: 334 335 $ devtool build-image image 336 3374. *Deploy the Build Output*: When you use the ``devtool build`` command 338 to build out your recipe, you probably want to see if the resulting 339 build output works as expected on the target hardware. 340 341 .. note:: 342 343 This step assumes you have a previously built image that is 344 already either running in QEMU or is running on actual hardware. 345 Also, it is assumed that for deployment of the image to the 346 target, SSH is installed in the image and, if the image is running 347 on real hardware, you have network access to and from your 348 development machine. 349 350 You can deploy your build output to that target hardware by using the 351 ``devtool deploy-target`` command: $ devtool deploy-target recipe 352 target The target is a live target machine running as an SSH server. 353 354 You can, of course, also deploy the image you build to actual 355 hardware by using the ``devtool build-image`` command. However, 356 ``devtool`` does not provide a specific command that allows you to 357 deploy the image to actual hardware. 358 3595. *Finish Your Work With the Recipe*: The ``devtool finish`` command 360 creates any patches corresponding to commits in the local Git 361 repository, moves the new recipe to a more permanent layer, and then 362 resets the recipe so that the recipe is built normally rather than 363 from the workspace. 364 :: 365 366 $ devtool finish recipe layer 367 368 .. note:: 369 370 Any changes you want to turn into patches must be committed to the 371 Git repository in the source tree. 372 373 As mentioned, the ``devtool finish`` command moves the final recipe 374 to its permanent layer. 375 376 As a final process of the ``devtool finish`` command, the state of 377 the standard layers and the upstream source is restored so that you 378 can build the recipe from those areas rather than the workspace. 379 380 .. note:: 381 382 You can use the 383 devtool reset 384 command to put things back should you decide you do not want to 385 proceed with your work. If you do use this command, realize that 386 the source tree is preserved. 387 388Use ``devtool modify`` to Modify the Source of an Existing Component 389-------------------------------------------------------------------- 390 391The ``devtool modify`` command prepares the way to work on existing code 392that already has a local recipe in place that is used to build the 393software. The command is flexible enough to allow you to extract code 394from an upstream source, specify the existing recipe, and keep track of 395and gather any patch files from other developers that are associated 396with the code. 397 398Depending on your particular scenario, the arguments and options you use 399with ``devtool modify`` form different combinations. The following 400diagram shows common development flows for the ``devtool modify`` 401command: 402 403.. image:: figures/sdk-devtool-modify-flow.png 404 :align: center 405 4061. *Preparing to Modify the Code*: The top part of the flow shows three 407 scenarios by which you could use ``devtool modify`` to prepare to 408 work on source files. Each scenario assumes the following: 409 410 - The recipe exists locally in a layer external to the ``devtool`` 411 workspace. 412 413 - The source files exist either upstream in an un-extracted state or 414 locally in a previously extracted state. 415 416 The typical situation is where another developer has created a layer 417 for use with the Yocto Project and their recipe already resides in 418 that layer. Furthermore, their source code is readily available 419 either upstream or locally. 420 421 - *Left*: The left scenario in the figure represents a common 422 situation where the source code does not exist locally and it 423 needs to be extracted from an upstream source. In this situation, 424 the source is extracted into the default ``devtool`` workspace 425 location. The recipe, in this scenario, is in its own layer 426 outside the workspace (i.e. ``meta-``\ layername). 427 428 The following command identifies the recipe and, by default, 429 extracts the source files:: 430 431 $ devtool modify recipe 432 433 Once 434 ``devtool``\ locates the recipe, ``devtool`` uses the recipe's 435 :term:`SRC_URI` statements to 436 locate the source code and any local patch files from other 437 developers. 438 439 With this scenario, there is no ``srctree`` argument. Consequently, the 440 default behavior of the ``devtool modify`` command is to extract 441 the source files pointed to by the :term:`SRC_URI` statements into a 442 local Git structure. Furthermore, the location for the extracted 443 source is the default area within the ``devtool`` workspace. The 444 result is that the command sets up both the source code and an 445 append file within the workspace while the recipe remains in its 446 original location. 447 448 Additionally, if you have any non-patch local files (i.e. files 449 referred to with ``file://`` entries in :term:`SRC_URI` statement 450 excluding ``*.patch/`` or ``*.diff``), these files are copied to 451 an ``oe-local-files`` folder under the newly created source tree. 452 Copying the files here gives you a convenient area from which you 453 can modify the files. Any changes or additions you make to those 454 files are incorporated into the build the next time you build the 455 software just as are other changes you might have made to the 456 source. 457 458 - *Middle*: The middle scenario in the figure represents a situation 459 where the source code also does not exist locally. In this case, 460 the code is again upstream and needs to be extracted to some local 461 area as a Git repository. The recipe, in this scenario, is again 462 local and in its own layer outside the workspace. 463 464 The following command tells ``devtool`` the recipe with which to 465 work and, in this case, identifies a local area for the extracted 466 source files that exists outside of the default ``devtool`` 467 workspace:: 468 469 $ devtool modify recipe srctree 470 471 .. note:: 472 473 You cannot provide a URL for 474 srctree 475 using the 476 devtool 477 command. 478 479 As with all extractions, the command uses the recipe's :term:`SRC_URI` 480 statements to locate the source files and any associated patch 481 files. Non-patch files are copied to an ``oe-local-files`` folder 482 under the newly created source tree. 483 484 Once the files are located, the command by default extracts them 485 into ``srctree``. 486 487 Within workspace, ``devtool`` creates an append file for the 488 recipe. The recipe remains in its original location but the source 489 files are extracted to the location you provide with ``srctree``. 490 491 - *Right*: The right scenario in the figure represents a situation 492 where the source tree (``srctree``) already exists locally as a 493 previously extracted Git structure outside of the ``devtool`` 494 workspace. In this example, the recipe also exists elsewhere 495 locally in its own layer. 496 497 The following command tells ``devtool`` the recipe with which to 498 work, uses the "-n" option to indicate source does not need to be 499 extracted, and uses ``srctree`` to point to the previously extracted 500 source files:: 501 502 $ devtool modify -n recipe srctree 503 504 If an ``oe-local-files`` subdirectory happens to exist and it 505 contains non-patch files, the files are used. However, if the 506 subdirectory does not exist and you run the ``devtool finish`` 507 command, any non-patch files that might exist next to the recipe 508 are removed because it appears to ``devtool`` that you have 509 deleted those files. 510 511 Once the ``devtool modify`` command finishes, it creates only an 512 append file for the recipe in the ``devtool`` workspace. The 513 recipe and the source code remain in their original locations. 514 5152. *Edit the Source*: Once you have used the ``devtool modify`` command, 516 you are free to make changes to the source files. You can use any 517 editor you like to make and save your source code modifications. 518 5193. *Build the Recipe or Rebuild the Image*: The next step you take 520 depends on what you are going to do with the new code. 521 522 If you need to eventually move the build output to the target 523 hardware, use the following ``devtool`` command:: 524 525 $ devtool build recipe 526 527 On the other hand, if you want an image to contain the recipe's 528 packages from the workspace for immediate deployment onto a device 529 (e.g. for testing purposes), you can use the ``devtool build-image`` 530 command: $ devtool build-image image 531 5324. *Deploy the Build Output*: When you use the ``devtool build`` command 533 to build out your recipe, you probably want to see if the resulting 534 build output works as expected on target hardware. 535 536 .. note:: 537 538 This step assumes you have a previously built image that is 539 already either running in QEMU or running on actual hardware. 540 Also, it is assumed that for deployment of the image to the 541 target, SSH is installed in the image and if the image is running 542 on real hardware that you have network access to and from your 543 development machine. 544 545 You can deploy your build output to that target hardware by using the 546 ``devtool deploy-target`` command:: 547 548 $ devtool deploy-target recipe target 549 550 The target is a live target machine running as an SSH server. 551 552 You can, of course, use other methods to deploy the image you built 553 using the ``devtool build-image`` command to actual hardware. 554 ``devtool`` does not provide a specific command to deploy the image 555 to actual hardware. 556 5575. *Finish Your Work With the Recipe*: The ``devtool finish`` command 558 creates any patches corresponding to commits in the local Git 559 repository, updates the recipe to point to them (or creates a 560 ``.bbappend`` file to do so, depending on the specified destination 561 layer), and then resets the recipe so that the recipe is built 562 normally rather than from the workspace. 563 :: 564 565 $ devtool finish recipe layer 566 567 .. note:: 568 569 Any changes you want to turn into patches must be staged and 570 committed within the local Git repository before you use the 571 devtool finish 572 command. 573 574 Because there is no need to move the recipe, ``devtool finish`` 575 either updates the original recipe in the original layer or the 576 command creates a ``.bbappend`` file in a different layer as provided 577 by layer. Any work you did in the ``oe-local-files`` directory is 578 preserved in the original files next to the recipe during the 579 ``devtool finish`` command. 580 581 As a final process of the ``devtool finish`` command, the state of 582 the standard layers and the upstream source is restored so that you 583 can build the recipe from those areas rather than from the workspace. 584 585 .. note:: 586 587 You can use the 588 devtool reset 589 command to put things back should you decide you do not want to 590 proceed with your work. If you do use this command, realize that 591 the source tree is preserved. 592 593Use ``devtool upgrade`` to Create a Version of the Recipe that Supports a Newer Version of the Software 594------------------------------------------------------------------------------------------------------- 595 596The ``devtool upgrade`` command upgrades an existing recipe to that of a 597more up-to-date version found upstream. Throughout the life of software, 598recipes continually undergo version upgrades by their upstream 599publishers. You can use the ``devtool upgrade`` workflow to make sure 600your recipes you are using for builds are up-to-date with their upstream 601counterparts. 602 603.. note:: 604 605 Several methods exist by which you can upgrade recipes - 606 ``devtool upgrade`` 607 happens to be one. You can read about all the methods by which you 608 can upgrade recipes in the 609 :ref:`dev-manual/common-tasks:upgrading recipes` section 610 of the Yocto Project Development Tasks Manual. 611 612The ``devtool upgrade`` command is flexible enough to allow you to 613specify source code revision and versioning schemes, extract code into 614or out of the ``devtool`` 615:ref:`devtool-the-workspace-layer-structure`, 616and work with any source file forms that the 617:ref:`bitbake:bitbake-user-manual/bitbake-user-manual-fetching:fetchers` support. 618 619The following diagram shows the common development flow used with the 620``devtool upgrade`` command: 621 622.. image:: figures/sdk-devtool-upgrade-flow.png 623 :align: center 624 6251. *Initiate the Upgrade*: The top part of the flow shows the typical 626 scenario by which you use the ``devtool upgrade`` command. The 627 following conditions exist: 628 629 - The recipe exists in a local layer external to the ``devtool`` 630 workspace. 631 632 - The source files for the new release exist in the same location 633 pointed to by :term:`SRC_URI` 634 in the recipe (e.g. a tarball with the new version number in the 635 name, or as a different revision in the upstream Git repository). 636 637 A common situation is where third-party software has undergone a 638 revision so that it has been upgraded. The recipe you have access to 639 is likely in your own layer. Thus, you need to upgrade the recipe to 640 use the newer version of the software:: 641 642 $ devtool upgrade -V version recipe 643 644 By default, the ``devtool upgrade`` command extracts source 645 code into the ``sources`` directory in the 646 :ref:`devtool-the-workspace-layer-structure`. 647 If you want the code extracted to any other location, you need to 648 provide the ``srctree`` positional argument with the command as follows:: 649 650 $ devtool upgrade -V version recipe srctree 651 652 .. note:: 653 654 In this example, the "-V" option specifies the new version. If you 655 don't use "-V", the command upgrades the recipe to the latest 656 version. 657 658 If the source files pointed to by the :term:`SRC_URI` statement in the 659 recipe are in a Git repository, you must provide the "-S" option and 660 specify a revision for the software. 661 662 Once ``devtool`` locates the recipe, it uses the :term:`SRC_URI` variable 663 to locate the source code and any local patch files from other 664 developers. The result is that the command sets up the source code, 665 the new version of the recipe, and an append file all within the 666 workspace. 667 668 Additionally, if you have any non-patch local files (i.e. files 669 referred to with ``file://`` entries in :term:`SRC_URI` statement 670 excluding ``*.patch/`` or ``*.diff``), these files are copied to an 671 ``oe-local-files`` folder under the newly created source tree. 672 Copying the files here gives you a convenient area from which you can 673 modify the files. Any changes or additions you make to those files 674 are incorporated into the build the next time you build the software 675 just as are other changes you might have made to the source. 676 6772. *Resolve any Conflicts created by the Upgrade*: Conflicts could happen 678 after upgrading the software to a new version. Conflicts occur 679 if your recipe specifies some patch files in :term:`SRC_URI` that 680 conflict with changes made in the new version of the software. For 681 such cases, you need to resolve the conflicts by editing the source 682 and following the normal ``git rebase`` conflict resolution process. 683 684 Before moving onto the next step, be sure to resolve any such 685 conflicts created through use of a newer or different version of the 686 software. 687 6883. *Build the Recipe or Rebuild the Image*: The next step you take 689 depends on what you are going to do with the new code. 690 691 If you need to eventually move the build output to the target 692 hardware, use the following ``devtool`` command:: 693 694 $ devtool build recipe 695 696 On the other hand, if you want an image to contain the recipe's 697 packages from the workspace for immediate deployment onto a device 698 (e.g. for testing purposes), you can use the ``devtool build-image`` 699 command:: 700 701 $ devtool build-image image 702 7034. *Deploy the Build Output*: When you use the ``devtool build`` command 704 or ``bitbake`` to build your recipe, you probably want to see if the 705 resulting build output works as expected on target hardware. 706 707 .. note:: 708 709 This step assumes you have a previously built image that is 710 already either running in QEMU or running on actual hardware. 711 Also, it is assumed that for deployment of the image to the 712 target, SSH is installed in the image and if the image is running 713 on real hardware that you have network access to and from your 714 development machine. 715 716 You can deploy your build output to that target hardware by using the 717 ``devtool deploy-target`` command: $ devtool deploy-target recipe 718 target The target is a live target machine running as an SSH server. 719 720 You can, of course, also deploy the image you build using the 721 ``devtool build-image`` command to actual hardware. However, 722 ``devtool`` does not provide a specific command that allows you to do 723 this. 724 7255. *Finish Your Work With the Recipe*: The ``devtool finish`` command 726 creates any patches corresponding to commits in the local Git 727 repository, moves the new recipe to a more permanent layer, and then 728 resets the recipe so that the recipe is built normally rather than 729 from the workspace. 730 731 Any work you did in the ``oe-local-files`` directory is preserved in 732 the original files next to the recipe during the ``devtool finish`` 733 command. 734 735 If you specify a destination layer that is the same as the original 736 source, then the old version of the recipe and associated files are 737 removed prior to adding the new version. 738 :: 739 740 $ devtool finish recipe layer 741 742 .. note:: 743 744 Any changes you want to turn into patches must be committed to the 745 Git repository in the source tree. 746 747 As a final process of the ``devtool finish`` command, the state of 748 the standard layers and the upstream source is restored so that you 749 can build the recipe from those areas rather than the workspace. 750 751 .. note:: 752 753 You can use the 754 devtool reset 755 command to put things back should you decide you do not want to 756 proceed with your work. If you do use this command, realize that 757 the source tree is preserved. 758 759A Closer Look at ``devtool add`` 760================================ 761 762The ``devtool add`` command automatically creates a recipe based on the 763source tree you provide with the command. Currently, the command has 764support for the following: 765 766- Autotools (``autoconf`` and ``automake``) 767 768- CMake 769 770- Scons 771 772- ``qmake`` 773 774- Plain ``Makefile`` 775 776- Out-of-tree kernel module 777 778- Binary package (i.e. "-b" option) 779 780- Node.js module 781 782- Python modules that use ``setuptools`` or ``distutils`` 783 784Apart from binary packages, the determination of how a source tree 785should be treated is automatic based on the files present within that 786source tree. For example, if a ``CMakeLists.txt`` file is found, then 787the source tree is assumed to be using CMake and is treated accordingly. 788 789.. note:: 790 791 In most cases, you need to edit the automatically generated recipe in 792 order to make it build properly. Typically, you would go through 793 several edit and build cycles until the recipe successfully builds. 794 Once the recipe builds, you could use possible further iterations to 795 test the recipe on the target device. 796 797The remainder of this section covers specifics regarding how parts of 798the recipe are generated. 799 800Name and Version 801---------------- 802 803If you do not specify a name and version on the command line, 804``devtool add`` uses various metadata within the source tree in an 805attempt to determine the name and version of the software being built. 806Based on what the tool determines, ``devtool`` sets the name of the 807created recipe file accordingly. 808 809If ``devtool`` cannot determine the name and version, the command prints 810an error. For such cases, you must re-run the command and provide the 811name and version, just the name, or just the version as part of the 812command line. 813 814Sometimes the name or version determined from the source tree might be 815incorrect. For such a case, you must reset the recipe:: 816 817 $ devtool reset -n recipename 818 819After running the ``devtool reset`` command, you need to 820run ``devtool add`` again and provide the name or the version. 821 822Dependency Detection and Mapping 823-------------------------------- 824 825The ``devtool add`` command attempts to detect build-time dependencies 826and map them to other recipes in the system. During this mapping, the 827command fills in the names of those recipes as part of the 828:term:`DEPENDS` variable within the 829recipe. If a dependency cannot be mapped, ``devtool`` places a comment 830in the recipe indicating such. The inability to map a dependency can 831result from naming not being recognized or because the dependency simply 832is not available. For cases where the dependency is not available, you 833must use the ``devtool add`` command to add an additional recipe that 834satisfies the dependency. Once you add that recipe, you need to update 835the :term:`DEPENDS` variable in the original recipe to include the new 836recipe. 837 838If you need to add runtime dependencies, you can do so by adding the 839following to your recipe:: 840 841 RDEPENDS:${PN} += "dependency1 dependency2 ..." 842 843.. note:: 844 845 The 846 devtool add 847 command often cannot distinguish between mandatory and optional 848 dependencies. Consequently, some of the detected dependencies might 849 in fact be optional. When in doubt, consult the documentation or the 850 configure script for the software the recipe is building for further 851 details. In some cases, you might find you can substitute the 852 dependency with an option that disables the associated functionality 853 passed to the configure script. 854 855License Detection 856----------------- 857 858The ``devtool add`` command attempts to determine if the software you 859are adding is able to be distributed under a common, open-source 860license. If so, the command sets the 861:term:`LICENSE` value accordingly. 862You should double-check the value added by the command against the 863documentation or source files for the software you are building and, if 864necessary, update that :term:`LICENSE` value. 865 866The ``devtool add`` command also sets the 867:term:`LIC_FILES_CHKSUM` 868value to point to all files that appear to be license-related. Realize 869that license statements often appear in comments at the top of source 870files or within the documentation. In such cases, the command does not 871recognize those license statements. Consequently, you might need to 872amend the :term:`LIC_FILES_CHKSUM` variable to point to one or more of those 873comments if present. Setting :term:`LIC_FILES_CHKSUM` is particularly 874important for third-party software. The mechanism attempts to ensure 875correct licensing should you upgrade the recipe to a newer upstream 876version in future. Any change in licensing is detected and you receive 877an error prompting you to check the license text again. 878 879If the ``devtool add`` command cannot determine licensing information, 880``devtool`` sets the :term:`LICENSE` value to "CLOSED" and leaves the 881:term:`LIC_FILES_CHKSUM` value unset. This behavior allows you to continue 882with development even though the settings are unlikely to be correct in 883all cases. You should check the documentation or source files for the 884software you are building to determine the actual license. 885 886Adding Makefile-Only Software 887----------------------------- 888 889The use of Make by itself is very common in both proprietary and 890open-source software. Unfortunately, Makefiles are often not written 891with cross-compilation in mind. Thus, ``devtool add`` often cannot do 892very much to ensure that these Makefiles build correctly. It is very 893common, for example, to explicitly call ``gcc`` instead of using the 894:term:`CC` variable. Usually, in a 895cross-compilation environment, ``gcc`` is the compiler for the build 896host and the cross-compiler is named something similar to 897``arm-poky-linux-gnueabi-gcc`` and might require arguments (e.g. to 898point to the associated sysroot for the target machine). 899 900When writing a recipe for Makefile-only software, keep the following in 901mind: 902 903- You probably need to patch the Makefile to use variables instead of 904 hardcoding tools within the toolchain such as ``gcc`` and ``g++``. 905 906- The environment in which Make runs is set up with various standard 907 variables for compilation (e.g. :term:`CC`, :term:`CXX`, and so forth) in a 908 similar manner to the environment set up by the SDK's environment 909 setup script. One easy way to see these variables is to run the 910 ``devtool build`` command on the recipe and then look in 911 ``oe-logs/run.do_compile``. Towards the top of this file, there is 912 a list of environment variables that are set. You can take 913 advantage of these variables within the Makefile. 914 915- If the Makefile sets a default for a variable using "=", that default 916 overrides the value set in the environment, which is usually not 917 desirable. For this case, you can either patch the Makefile so it 918 sets the default using the "?=" operator, or you can alternatively 919 force the value on the ``make`` command line. To force the value on 920 the command line, add the variable setting to 921 :term:`EXTRA_OEMAKE` or 922 :term:`PACKAGECONFIG_CONFARGS` 923 within the recipe. Here is an example using :term:`EXTRA_OEMAKE`:: 924 925 EXTRA_OEMAKE += "'CC=${CC}' 'CXX=${CXX}'" 926 927 In the above example, 928 single quotes are used around the variable settings as the values are 929 likely to contain spaces because required default options are passed 930 to the compiler. 931 932- Hardcoding paths inside Makefiles is often problematic in a 933 cross-compilation environment. This is particularly true because 934 those hardcoded paths often point to locations on the build host and 935 thus will either be read-only or will introduce contamination into 936 the cross-compilation because they are specific to the build host 937 rather than the target. Patching the Makefile to use prefix variables 938 or other path variables is usually the way to handle this situation. 939 940- Sometimes a Makefile runs target-specific commands such as 941 ``ldconfig``. For such cases, you might be able to apply patches that 942 remove these commands from the Makefile. 943 944Adding Native Tools 945------------------- 946 947Often, you need to build additional tools that run on the :term:`Build 948Host` as opposed to 949the target. You should indicate this requirement by using one of the 950following methods when you run ``devtool add``: 951 952- Specify the name of the recipe such that it ends with "-native". 953 Specifying the name like this produces a recipe that only builds for 954 the build host. 955 956- Specify the "--also-native" option with the ``devtool add`` 957 command. Specifying this option creates a recipe file that still 958 builds for the target but also creates a variant with a "-native" 959 suffix that builds for the build host. 960 961.. note:: 962 963 If you need to add a tool that is shipped as part of a source tree 964 that builds code for the target, you can typically accomplish this by 965 building the native and target parts separately rather than within 966 the same compilation process. Realize though that with the 967 "--also-native" option, you can add the tool using just one 968 recipe file. 969 970Adding Node.js Modules 971---------------------- 972 973You can use the ``devtool add`` command two different ways to add 974Node.js modules: 1) Through ``npm`` and, 2) from a repository or local 975source. 976 977Use the following form to add Node.js modules through ``npm``:: 978 979 $ devtool add "npm://registry.npmjs.org;name=forever;version=0.15.1" 980 981The name and 982version parameters are mandatory. Lockdown and shrinkwrap files are 983generated and pointed to by the recipe in order to freeze the version 984that is fetched for the dependencies according to the first time. This 985also saves checksums that are verified on future fetches. Together, 986these behaviors ensure the reproducibility and integrity of the build. 987 988.. note:: 989 990 - You must use quotes around the URL. The ``devtool add`` does not 991 require the quotes, but the shell considers ";" as a splitter 992 between multiple commands. Thus, without the quotes, 993 ``devtool add`` does not receive the other parts, which results in 994 several "command not found" errors. 995 996 - In order to support adding Node.js modules, a ``nodejs`` recipe 997 must be part of your SDK. 998 999As mentioned earlier, you can also add Node.js modules directly from a 1000repository or local source tree. To add modules this way, use 1001``devtool add`` in the following form:: 1002 1003 $ devtool add https://github.com/diversario/node-ssdp 1004 1005In this example, ``devtool`` 1006fetches the specified Git repository, detects the code as Node.js code, 1007fetches dependencies using ``npm``, and sets 1008:term:`SRC_URI` accordingly. 1009 1010Working With Recipes 1011==================== 1012 1013When building a recipe using the ``devtool build`` command, the typical 1014build progresses as follows: 1015 10161. Fetch the source 1017 10182. Unpack the source 1019 10203. Configure the source 1021 10224. Compile the source 1023 10245. Install the build output 1025 10266. Package the installed output 1027 1028For recipes in the workspace, fetching and unpacking is disabled as the 1029source tree has already been prepared and is persistent. Each of these 1030build steps is defined as a function (task), usually with a "do\_" prefix 1031(e.g. :ref:`ref-tasks-fetch`, 1032:ref:`ref-tasks-unpack`, and so 1033forth). These functions are typically shell scripts but can instead be 1034written in Python. 1035 1036If you look at the contents of a recipe, you will see that the recipe 1037does not include complete instructions for building the software. 1038Instead, common functionality is encapsulated in classes inherited with 1039the ``inherit`` directive. This technique leaves the recipe to describe 1040just the things that are specific to the software being built. There is 1041a :ref:`base <ref-classes-base>` class that 1042is implicitly inherited by all recipes and provides the functionality 1043that most recipes typically need. 1044 1045The remainder of this section presents information useful when working 1046with recipes. 1047 1048Finding Logs and Work Files 1049--------------------------- 1050 1051After the first run of the ``devtool build`` command, recipes that were 1052previously created using the ``devtool add`` command or whose sources 1053were modified using the ``devtool modify`` command contain symbolic 1054links created within the source tree: 1055 1056- ``oe-logs``: This link points to the directory in which log files and 1057 run scripts for each build step are created. 1058 1059- ``oe-workdir``: This link points to the temporary work area for the 1060 recipe. The following locations under ``oe-workdir`` are particularly 1061 useful: 1062 1063 - ``image/``: Contains all of the files installed during the 1064 :ref:`ref-tasks-install` stage. 1065 Within a recipe, this directory is referred to by the expression 1066 ``${``\ :term:`D`\ ``}``. 1067 1068 - ``sysroot-destdir/``: Contains a subset of files installed within 1069 ``do_install`` that have been put into the shared sysroot. For 1070 more information, see the 1071 ":ref:`dev-manual/common-tasks:sharing files between recipes`" section. 1072 1073 - ``packages-split/``: Contains subdirectories for each package 1074 produced by the recipe. For more information, see the 1075 ":ref:`sdk-manual/extensible:packaging`" section. 1076 1077You can use these links to get more information on what is happening at 1078each build step. 1079 1080Setting Configure Arguments 1081--------------------------- 1082 1083If the software your recipe is building uses GNU autoconf, then a fixed 1084set of arguments is passed to it to enable cross-compilation plus any 1085extras specified by 1086:term:`EXTRA_OECONF` or 1087:term:`PACKAGECONFIG_CONFARGS` 1088set within the recipe. If you wish to pass additional options, add them 1089to :term:`EXTRA_OECONF` or :term:`PACKAGECONFIG_CONFARGS`. Other supported build 1090tools have similar variables (e.g. 1091:term:`EXTRA_OECMAKE` for 1092CMake, :term:`EXTRA_OESCONS` 1093for Scons, and so forth). If you need to pass anything on the ``make`` 1094command line, you can use :term:`EXTRA_OEMAKE` or the 1095:term:`PACKAGECONFIG_CONFARGS` 1096variables to do so. 1097 1098You can use the ``devtool configure-help`` command to help you set the 1099arguments listed in the previous paragraph. The command determines the 1100exact options being passed, and shows them to you along with any custom 1101arguments specified through :term:`EXTRA_OECONF` or 1102:term:`PACKAGECONFIG_CONFARGS`. If applicable, the command also shows you 1103the output of the configure script's "--help" option as a 1104reference. 1105 1106Sharing Files Between Recipes 1107----------------------------- 1108 1109Recipes often need to use files provided by other recipes on the 1110:term:`Build Host`. For example, 1111an application linking to a common library needs access to the library 1112itself and its associated headers. The way this access is accomplished 1113within the extensible SDK is through the sysroot. There is one sysroot per 1114"machine" for which the SDK is being built. In practical terms, this 1115means there is a sysroot for the target machine, and a sysroot for 1116the build host. 1117 1118Recipes should never write files directly into the sysroot. Instead, 1119files should be installed into standard locations during the 1120:ref:`ref-tasks-install` task within 1121the ``${``\ :term:`D`\ ``}`` directory. A 1122subset of these files automatically goes into the sysroot. The reason 1123for this limitation is that almost all files that go into the sysroot 1124are cataloged in manifests in order to ensure they can be removed later 1125when a recipe is modified or removed. Thus, the sysroot is able to 1126remain free from stale files. 1127 1128Packaging 1129--------- 1130 1131Packaging is not always particularly relevant within the extensible SDK. 1132However, if you examine how build output gets into the final image on 1133the target device, it is important to understand packaging because the 1134contents of the image are expressed in terms of packages and not 1135recipes. 1136 1137During the :ref:`ref-tasks-package` 1138task, files installed during the 1139:ref:`ref-tasks-install` task are 1140split into one main package, which is almost always named the same as 1141the recipe, and into several other packages. This separation exists 1142because not all of those installed files are useful in every image. For 1143example, you probably do not need any of the documentation installed in 1144a production image. Consequently, for each recipe the documentation 1145files are separated into a ``-doc`` package. Recipes that package 1146software containing optional modules or plugins might undergo additional 1147package splitting as well. 1148 1149After building a recipe, you can see where files have gone by looking in 1150the ``oe-workdir/packages-split`` directory, which contains a 1151subdirectory for each package. Apart from some advanced cases, the 1152:term:`PACKAGES` and 1153:term:`FILES` variables controls 1154splitting. The :term:`PACKAGES` variable lists all of the packages to be 1155produced, while the :term:`FILES` variable specifies which files to include 1156in each package by using an override to specify the package. For 1157example, ``FILES:${PN}`` specifies the files to go into the main package 1158(i.e. the main package has the same name as the recipe and 1159``${``\ :term:`PN`\ ``}`` evaluates to the 1160recipe name). The order of the :term:`PACKAGES` value is significant. For 1161each installed file, the first package whose :term:`FILES` value matches the 1162file is the package into which the file goes. Both the :term:`PACKAGES` and 1163:term:`FILES` variables have default values. Consequently, you might find 1164you do not even need to set these variables in your recipe unless the 1165software the recipe is building installs files into non-standard 1166locations. 1167 1168Restoring the Target Device to its Original State 1169================================================= 1170 1171If you use the ``devtool deploy-target`` command to write a recipe's 1172build output to the target, and you are working on an existing component 1173of the system, then you might find yourself in a situation where you 1174need to restore the original files that existed prior to running the 1175``devtool deploy-target`` command. Because the ``devtool deploy-target`` 1176command backs up any files it overwrites, you can use the 1177``devtool undeploy-target`` command to restore those files and remove 1178any other files the recipe deployed. Consider the following example:: 1179 1180 $ devtool undeploy-target lighttpd root@192.168.7.2 1181 1182If you have deployed 1183multiple applications, you can remove them all using the "-a" option 1184thus restoring the target device to its original state:: 1185 1186 $ devtool undeploy-target -a root@192.168.7.2 1187 1188Information about files deployed to 1189the target as well as any backed up files are stored on the target 1190itself. This storage, of course, requires some additional space on the 1191target machine. 1192 1193.. note:: 1194 1195 The 1196 devtool deploy-target 1197 and 1198 devtool undeploy-target 1199 commands do not currently interact with any package management system 1200 on the target device (e.g. RPM or OPKG). Consequently, you should not 1201 intermingle 1202 devtool deploy-target 1203 and package manager operations on the target device. Doing so could 1204 result in a conflicting set of files. 1205 1206Installing Additional Items Into the Extensible SDK 1207=================================================== 1208 1209Out of the box the extensible SDK typically only comes with a small 1210number of tools and libraries. A minimal SDK starts mostly empty and is 1211populated on-demand. Sometimes you must explicitly install extra items 1212into the SDK. If you need these extra items, you can first search for 1213the items using the ``devtool search`` command. For example, suppose you 1214need to link to libGL but you are not sure which recipe provides libGL. 1215You can use the following command to find out:: 1216 1217 $ devtool search libGL mesa 1218 1219A free implementation of the OpenGL API Once you know the recipe 1220(i.e. ``mesa`` in this example), you can install it:: 1221 1222 $ devtool sdk-install mesa 1223 1224By default, the ``devtool sdk-install`` command assumes 1225the item is available in pre-built form from your SDK provider. If the 1226item is not available and it is acceptable to build the item from 1227source, you can add the "-s" option as follows:: 1228 1229 $ devtool sdk-install -s mesa 1230 1231It is important to remember that building the item from source 1232takes significantly longer than installing the pre-built artifact. Also, 1233if there is no recipe for the item you want to add to the SDK, you must 1234instead add the item using the ``devtool add`` command. 1235 1236Applying Updates to an Installed Extensible SDK 1237=============================================== 1238 1239If you are working with an installed extensible SDK that gets 1240occasionally updated (e.g. a third-party SDK), then you will need to 1241manually "pull down" the updates into the installed SDK. 1242 1243To update your installed SDK, use ``devtool`` as follows:: 1244 1245 $ devtool sdk-update 1246 1247The previous command assumes your SDK provider has set the 1248default update URL for you through the :term:`SDK_UPDATE_URL` 1249variable as described in the 1250":ref:`sdk-manual/appendix-customizing:Providing Updates to the Extensible SDK After Installation`" 1251section. If the SDK provider has not set that default URL, you need to 1252specify it yourself in the command as follows: $ devtool sdk-update 1253path_to_update_directory 1254 1255.. note:: 1256 1257 The URL needs to point specifically to a published SDK and not to an 1258 SDK installer that you would download and install. 1259 1260Creating a Derivative SDK With Additional Components 1261==================================================== 1262 1263You might need to produce an SDK that contains your own custom 1264libraries. A good example would be if you were a vendor with customers 1265that use your SDK to build their own platform-specific software and 1266those customers need an SDK that has custom libraries. In such a case, 1267you can produce a derivative SDK based on the currently installed SDK 1268fairly easily by following these steps: 1269 12701. If necessary, install an extensible SDK that you want to use as a 1271 base for your derivative SDK. 1272 12732. Source the environment script for the SDK. 1274 12753. Add the extra libraries or other components you want by using the 1276 ``devtool add`` command. 1277 12784. Run the ``devtool build-sdk`` command. 1279 1280The previous steps take the recipes added to the workspace and construct 1281a new SDK installer that contains those recipes and the resulting binary 1282artifacts. The recipes go into their own separate layer in the 1283constructed derivative SDK, which leaves the workspace clean and ready 1284for users to add their own recipes. 1285