1.. SPDX-License-Identifier: CC-BY-SA-2.0-UK 2 3Building 4******** 5 6This section describes various build procedures, such as the steps 7needed for a simple build, building a target for multiple configurations, 8generating an image for more than one machine, and so forth. 9 10Building a Simple Image 11======================= 12 13In the development environment, you need to build an image whenever you 14change hardware support, add or change system libraries, or add or 15change services that have dependencies. There are several methods that allow 16you to build an image within the Yocto Project. This section presents 17the basic steps you need to build a simple image using BitBake from a 18build host running Linux. 19 20.. note:: 21 22 - For information on how to build an image using 23 :term:`Toaster`, see the 24 :doc:`/toaster-manual/index`. 25 26 - For information on how to use ``devtool`` to build images, see the 27 ":ref:`sdk-manual/extensible:using \`\`devtool\`\` in your sdk workflow`" 28 section in the Yocto Project Application Development and the 29 Extensible Software Development Kit (eSDK) manual. 30 31 - For a quick example on how to build an image using the 32 OpenEmbedded build system, see the 33 :doc:`/brief-yoctoprojectqs/index` document. 34 35The build process creates an entire Linux distribution from source and 36places it in your :term:`Build Directory` under ``tmp/deploy/images``. For 37detailed information on the build process using BitBake, see the 38":ref:`overview-manual/concepts:images`" section in the Yocto Project Overview 39and Concepts Manual. 40 41The following figure and list overviews the build process: 42 43.. image:: figures/bitbake-build-flow.png 44 :width: 100% 45 46#. *Set up Your Host Development System to Support Development Using the 47 Yocto Project*: See the ":doc:`start`" section for options on how to get a 48 build host ready to use the Yocto Project. 49 50#. *Initialize the Build Environment:* Initialize the build environment 51 by sourcing the build environment script (i.e. 52 :ref:`structure-core-script`):: 53 54 $ source oe-init-build-env [build_dir] 55 56 When you use the initialization script, the OpenEmbedded build system 57 uses ``build`` as the default :term:`Build Directory` in your current work 58 directory. You can use a `build_dir` argument with the script to 59 specify a different :term:`Build Directory`. 60 61 .. note:: 62 63 A common practice is to use a different :term:`Build Directory` for 64 different targets; for example, ``~/build/x86`` for a ``qemux86`` 65 target, and ``~/build/arm`` for a ``qemuarm`` target. In any 66 event, it's typically cleaner to locate the :term:`Build Directory` 67 somewhere outside of your source directory. 68 69#. *Make Sure Your* ``local.conf`` *File is Correct*: Ensure the 70 ``conf/local.conf`` configuration file, which is found in the 71 :term:`Build Directory`, is set up how you want it. This file defines many 72 aspects of the build environment including the target machine architecture 73 through the :term:`MACHINE` variable, the packaging format used during 74 the build (:term:`PACKAGE_CLASSES`), and a centralized tarball download 75 directory through the :term:`DL_DIR` variable. 76 77#. *Build the Image:* Build the image using the ``bitbake`` command:: 78 79 $ bitbake target 80 81 .. note:: 82 83 For information on BitBake, see the :doc:`bitbake:index`. 84 85 The target is the name of the recipe you want to build. Common 86 targets are the images in ``meta/recipes-core/images``, 87 ``meta/recipes-sato/images``, and so forth all found in the 88 :term:`Source Directory`. Alternatively, the target 89 can be the name of a recipe for a specific piece of software such as 90 BusyBox. For more details about the images the OpenEmbedded build 91 system supports, see the 92 ":ref:`ref-manual/images:Images`" chapter in the Yocto 93 Project Reference Manual. 94 95 As an example, the following command builds the 96 ``core-image-minimal`` image:: 97 98 $ bitbake core-image-minimal 99 100 Once an 101 image has been built, it often needs to be installed. The images and 102 kernels built by the OpenEmbedded build system are placed in the 103 :term:`Build Directory` in ``tmp/deploy/images``. For information on how to 104 run pre-built images such as ``qemux86`` and ``qemuarm``, see the 105 :doc:`/sdk-manual/index` manual. For 106 information about how to install these images, see the documentation 107 for your particular board or machine. 108 109Building Images for Multiple Targets Using Multiple Configurations 110================================================================== 111 112You can use a single ``bitbake`` command to build multiple images or 113packages for different targets where each image or package requires a 114different configuration (multiple configuration builds). The builds, in 115this scenario, are sometimes referred to as "multiconfigs", and this 116section uses that term throughout. 117 118This section describes how to set up for multiple configuration builds 119and how to account for cross-build dependencies between the 120multiconfigs. 121 122Setting Up and Running a Multiple Configuration Build 123----------------------------------------------------- 124 125To accomplish a multiple configuration build, you must define each 126target's configuration separately using a parallel configuration file in 127the :term:`Build Directory` or configuration directory within a layer, and you 128must follow a required file hierarchy. Additionally, you must enable the 129multiple configuration builds in your ``local.conf`` file. 130 131Follow these steps to set up and execute multiple configuration builds: 132 133- *Create Separate Configuration Files*: You need to create a single 134 configuration file for each build target (each multiconfig). 135 The configuration definitions are implementation dependent but often 136 each configuration file will define the machine and the 137 temporary directory BitBake uses for the build. Whether the same 138 temporary directory (:term:`TMPDIR`) can be shared will depend on what is 139 similar and what is different between the configurations. Multiple MACHINE 140 targets can share the same (:term:`TMPDIR`) as long as the rest of the 141 configuration is the same, multiple :term:`DISTRO` settings would need separate 142 (:term:`TMPDIR`) directories. 143 144 For example, consider a scenario with two different multiconfigs for the same 145 :term:`MACHINE`: "qemux86" built 146 for two distributions such as "poky" and "poky-lsb". In this case, 147 you would need to use the different :term:`TMPDIR`. 148 149 Here is an example showing the minimal statements needed in a 150 configuration file for a "qemux86" target whose temporary build 151 directory is ``tmpmultix86``:: 152 153 MACHINE = "qemux86" 154 TMPDIR = "${TOPDIR}/tmpmultix86" 155 156 The location for these multiconfig configuration files is specific. 157 They must reside in the current :term:`Build Directory` in a sub-directory of 158 ``conf`` named ``multiconfig`` or within a layer's ``conf`` directory 159 under a directory named ``multiconfig``. Following is an example that defines 160 two configuration files for the "x86" and "arm" multiconfigs: 161 162 .. image:: figures/multiconfig_files.png 163 :align: center 164 :width: 50% 165 166 The usual :term:`BBPATH` search path is used to locate multiconfig files in 167 a similar way to other conf files. 168 169- *Add the BitBake Multi-configuration Variable to the Local 170 Configuration File*: Use the 171 :term:`BBMULTICONFIG` 172 variable in your ``conf/local.conf`` configuration file to specify 173 each multiconfig. Continuing with the example from the previous 174 figure, the :term:`BBMULTICONFIG` variable needs to enable two 175 multiconfigs: "x86" and "arm" by specifying each configuration file:: 176 177 BBMULTICONFIG = "x86 arm" 178 179 .. note:: 180 181 A "default" configuration already exists by definition. This 182 configuration is named: "" (i.e. empty string) and is defined by 183 the variables coming from your ``local.conf`` 184 file. Consequently, the previous example actually adds two 185 additional configurations to your build: "arm" and "x86" along 186 with "". 187 188- *Launch BitBake*: Use the following BitBake command form to launch 189 the multiple configuration build:: 190 191 $ bitbake [mc:multiconfigname:]target [[[mc:multiconfigname:]target] ... ] 192 193 For the example in this section, the following command applies:: 194 195 $ bitbake mc:x86:core-image-minimal mc:arm:core-image-sato mc::core-image-base 196 197 The previous BitBake command builds a ``core-image-minimal`` image 198 that is configured through the ``x86.conf`` configuration file, a 199 ``core-image-sato`` image that is configured through the ``arm.conf`` 200 configuration file and a ``core-image-base`` that is configured 201 through your ``local.conf`` configuration file. 202 203.. note:: 204 205 Support for multiple configuration builds in the Yocto Project &DISTRO; 206 (&DISTRO_NAME;) Release does not include Shared State (sstate) 207 optimizations. Consequently, if a build uses the same object twice 208 in, for example, two different :term:`TMPDIR` 209 directories, the build either loads from an existing sstate cache for 210 that build at the start or builds the object fresh. 211 212Enabling Multiple Configuration Build Dependencies 213-------------------------------------------------- 214 215Sometimes dependencies can exist between targets (multiconfigs) in a 216multiple configuration build. For example, suppose that in order to 217build a ``core-image-sato`` image for an "x86" multiconfig, the root 218filesystem of an "arm" multiconfig must exist. This dependency is 219essentially that the 220:ref:`ref-tasks-image` task in the 221``core-image-sato`` recipe depends on the completion of the 222:ref:`ref-tasks-rootfs` task of the 223``core-image-minimal`` recipe. 224 225To enable dependencies in a multiple configuration build, you must 226declare the dependencies in the recipe using the following statement 227form:: 228 229 task_or_package[mcdepends] = "mc:from_multiconfig:to_multiconfig:recipe_name:task_on_which_to_depend" 230 231To better show how to use this statement, consider the example scenario 232from the first paragraph of this section. The following statement needs 233to be added to the recipe that builds the ``core-image-sato`` image:: 234 235 do_image[mcdepends] = "mc:x86:arm:core-image-minimal:do_rootfs" 236 237In this example, the `from_multiconfig` is "x86". The `to_multiconfig` is "arm". The 238task on which the :ref:`ref-tasks-image` task in the recipe depends is the 239:ref:`ref-tasks-rootfs` task from the ``core-image-minimal`` recipe associated 240with the "arm" multiconfig. 241 242Once you set up this dependency, you can build the "x86" multiconfig 243using a BitBake command as follows:: 244 245 $ bitbake mc:x86:core-image-sato 246 247This command executes all the tasks needed to create the 248``core-image-sato`` image for the "x86" multiconfig. Because of the 249dependency, BitBake also executes through the :ref:`ref-tasks-rootfs` task for the 250"arm" multiconfig build. 251 252Having a recipe depend on the root filesystem of another build might not 253seem that useful. Consider this change to the statement in the 254``core-image-sato`` recipe:: 255 256 do_image[mcdepends] = "mc:x86:arm:core-image-minimal:do_image" 257 258In this case, BitBake must 259create the ``core-image-minimal`` image for the "arm" build since the 260"x86" build depends on it. 261 262Because "x86" and "arm" are enabled for multiple configuration builds 263and have separate configuration files, BitBake places the artifacts for 264each build in the respective temporary build directories (i.e. 265:term:`TMPDIR`). 266 267Building an Initial RAM Filesystem (Initramfs) Image 268==================================================== 269 270An initial RAM filesystem (:term:`Initramfs`) image provides a temporary root 271filesystem used for early system initialization, typically providing tools and 272loading modules needed to locate and mount the final root filesystem. 273 274Follow these steps to create an :term:`Initramfs` image: 275 276#. *Create the :term:`Initramfs` Image Recipe:* You can reference the 277 ``core-image-minimal-initramfs.bb`` recipe found in the 278 ``meta/recipes-core`` directory of the :term:`Source Directory` 279 as an example from which to work. 280 281#. *Decide if You Need to Bundle the :term:`Initramfs` Image Into the Kernel 282 Image:* If you want the :term:`Initramfs` image that is built to be bundled 283 in with the kernel image, set the :term:`INITRAMFS_IMAGE_BUNDLE` 284 variable to ``"1"`` in your ``local.conf`` configuration file and set the 285 :term:`INITRAMFS_IMAGE` variable in the recipe that builds the kernel image. 286 287 Setting the :term:`INITRAMFS_IMAGE_BUNDLE` flag causes the :term:`Initramfs` 288 image to be unpacked into the ``${B}/usr/`` directory. The unpacked 289 :term:`Initramfs` image is then passed to the kernel's ``Makefile`` using the 290 :term:`CONFIG_INITRAMFS_SOURCE` variable, allowing the :term:`Initramfs` 291 image to be built into the kernel normally. 292 293#. *Optionally Add Items to the Initramfs Image Through the Initramfs 294 Image Recipe:* If you add items to the :term:`Initramfs` image by way of its 295 recipe, you should use :term:`PACKAGE_INSTALL` rather than 296 :term:`IMAGE_INSTALL`. :term:`PACKAGE_INSTALL` gives more direct control of 297 what is added to the image as compared to the defaults you might not 298 necessarily want that are set by the :ref:`ref-classes-image` 299 or :ref:`ref-classes-core-image` classes. 300 301#. *Build the Kernel Image and the Initramfs Image:* Build your kernel 302 image using BitBake. Because the :term:`Initramfs` image recipe is a 303 dependency of the kernel image, the :term:`Initramfs` image is built as well 304 and bundled with the kernel image if you used the 305 :term:`INITRAMFS_IMAGE_BUNDLE` variable described earlier. 306 307Bundling an Initramfs Image From a Separate Multiconfig 308------------------------------------------------------- 309 310There may be a case where we want to build an :term:`Initramfs` image which does not 311inherit the same distro policy as our main image, for example, we may want 312our main image to use ``TCLIBC="glibc"``, but to use ``TCLIBC="musl"`` in our :term:`Initramfs` 313image to keep a smaller footprint. However, by performing the steps mentioned 314above the :term:`Initramfs` image will inherit ``TCLIBC="glibc"`` without allowing us 315to override it. 316 317To achieve this, you need to perform some additional steps: 318 319#. *Create a multiconfig for your Initramfs image:* You can perform the steps 320 on ":ref:`dev-manual/building:building images for multiple targets using multiple configurations`" to create a separate multiconfig. 321 For the sake of simplicity let's assume such multiconfig is called: ``initramfscfg.conf`` and 322 contains the variables:: 323 324 TMPDIR="${TOPDIR}/tmp-initramfscfg" 325 TCLIBC="musl" 326 327#. *Set additional Initramfs variables on your main configuration:* 328 Additionally, on your main configuration (``local.conf``) you need to set the 329 variables:: 330 331 INITRAMFS_MULTICONFIG = "initramfscfg" 332 INITRAMFS_DEPLOY_DIR_IMAGE = "${TOPDIR}/tmp-initramfscfg/deploy/images/${MACHINE}" 333 334 The variables :term:`INITRAMFS_MULTICONFIG` and :term:`INITRAMFS_DEPLOY_DIR_IMAGE` 335 are used to create a multiconfig dependency from the kernel to the :term:`INITRAMFS_IMAGE` 336 to be built coming from the ``initramfscfg`` multiconfig, and to let the 337 buildsystem know where the :term:`INITRAMFS_IMAGE` will be located. 338 339 Building a system with such configuration will build the kernel using the 340 main configuration but the :ref:`ref-tasks-bundle_initramfs` task will grab the 341 selected :term:`INITRAMFS_IMAGE` from :term:`INITRAMFS_DEPLOY_DIR_IMAGE` 342 instead, resulting in a musl based :term:`Initramfs` image bundled in the kernel 343 but a glibc based main image. 344 345 The same is applicable to avoid inheriting :term:`DISTRO_FEATURES` on :term:`INITRAMFS_IMAGE` 346 or to build a different :term:`DISTRO` for it such as ``poky-tiny``. 347 348 349Building a Tiny System 350====================== 351 352Very small distributions have some significant advantages such as 353requiring less on-die or in-package memory (cheaper), better performance 354through efficient cache usage, lower power requirements due to less 355memory, faster boot times, and reduced development overhead. Some 356real-world examples where a very small distribution gives you distinct 357advantages are digital cameras, medical devices, and small headless 358systems. 359 360This section presents information that shows you how you can trim your 361distribution to even smaller sizes than the ``poky-tiny`` distribution, 362which is around 5 Mbytes, that can be built out-of-the-box using the 363Yocto Project. 364 365Tiny System Overview 366-------------------- 367 368The following list presents the overall steps you need to consider and 369perform to create distributions with smaller root filesystems, achieve 370faster boot times, maintain your critical functionality, and avoid 371initial RAM disks: 372 373- :ref:`Determine your goals and guiding principles 374 <dev-manual/building:goals and guiding principles>` 375 376- :ref:`dev-manual/building:understand what contributes to your image size` 377 378- :ref:`Reduce the size of the root filesystem 379 <dev-manual/building:trim the root filesystem>` 380 381- :ref:`Reduce the size of the kernel <dev-manual/building:trim the kernel>` 382 383- :ref:`dev-manual/building:remove package management requirements` 384 385- :ref:`dev-manual/building:look for other ways to minimize size` 386 387- :ref:`dev-manual/building:iterate on the process` 388 389Goals and Guiding Principles 390---------------------------- 391 392Before you can reach your destination, you need to know where you are 393going. Here is an example list that you can use as a guide when creating 394very small distributions: 395 396- Determine how much space you need (e.g. a kernel that is 1 Mbyte or 397 less and a root filesystem that is 3 Mbytes or less). 398 399- Find the areas that are currently taking 90% of the space and 400 concentrate on reducing those areas. 401 402- Do not create any difficult "hacks" to achieve your goals. 403 404- Leverage the device-specific options. 405 406- Work in a separate layer so that you keep changes isolated. For 407 information on how to create layers, see the 408 ":ref:`dev-manual/layers:understanding and creating layers`" section. 409 410Understand What Contributes to Your Image Size 411---------------------------------------------- 412 413It is easiest to have something to start with when creating your own 414distribution. You can use the Yocto Project out-of-the-box to create the 415``poky-tiny`` distribution. Ultimately, you will want to make changes in 416your own distribution that are likely modeled after ``poky-tiny``. 417 418.. note:: 419 420 To use ``poky-tiny`` in your build, set the :term:`DISTRO` variable in your 421 ``local.conf`` file to "poky-tiny" as described in the 422 ":ref:`dev-manual/custom-distribution:creating your own distribution`" 423 section. 424 425Understanding some memory concepts will help you reduce the system size. 426Memory consists of static, dynamic, and temporary memory. Static memory 427is the TEXT (code), DATA (initialized data in the code), and BSS 428(uninitialized data) sections. Dynamic memory represents memory that is 429allocated at runtime: stacks, hash tables, and so forth. Temporary 430memory is recovered after the boot process. This memory consists of 431memory used for decompressing the kernel and for the ``__init__`` 432functions. 433 434To help you see where you currently are with kernel and root filesystem 435sizes, you can use two tools found in the :term:`Source Directory` 436in the 437``scripts/tiny/`` directory: 438 439- ``ksize.py``: Reports component sizes for the kernel build objects. 440 441- ``dirsize.py``: Reports component sizes for the root filesystem. 442 443This next tool and command help you organize configuration fragments and 444view file dependencies in a human-readable form: 445 446- ``merge_config.sh``: Helps you manage configuration files and 447 fragments within the kernel. With this tool, you can merge individual 448 configuration fragments together. The tool allows you to make 449 overrides and warns you of any missing configuration options. The 450 tool is ideal for allowing you to iterate on configurations, create 451 minimal configurations, and create configuration files for different 452 machines without having to duplicate your process. 453 454 The ``merge_config.sh`` script is part of the Linux Yocto kernel Git 455 repositories (i.e. ``linux-yocto-3.14``, ``linux-yocto-3.10``, 456 ``linux-yocto-3.8``, and so forth) in the ``scripts/kconfig`` 457 directory. 458 459 For more information on configuration fragments, see the 460 ":ref:`kernel-dev/common:creating configuration fragments`" 461 section in the Yocto Project Linux Kernel Development Manual. 462 463- ``bitbake -u taskexp -g bitbake_target``: Using the BitBake command 464 with these options brings up a Dependency Explorer from which you can 465 view file dependencies. Understanding these dependencies allows you 466 to make informed decisions when cutting out various pieces of the 467 kernel and root filesystem. 468 469Trim the Root Filesystem 470------------------------ 471 472The root filesystem is made up of packages for booting, libraries, and 473applications. To change things, you can configure how the packaging 474happens, which changes the way you build them. You can also modify the 475filesystem itself or select a different filesystem. 476 477First, find out what is hogging your root filesystem by running the 478``dirsize.py`` script from your root directory:: 479 480 $ cd root-directory-of-image 481 $ dirsize.py 100000 > dirsize-100k.log 482 $ cat dirsize-100k.log 483 484You can apply a filter to the script to ignore files 485under a certain size. The previous example filters out any files below 486100 Kbytes. The sizes reported by the tool are uncompressed, and thus 487will be smaller by a relatively constant factor in a compressed root 488filesystem. When you examine your log file, you can focus on areas of 489the root filesystem that take up large amounts of memory. 490 491You need to be sure that what you eliminate does not cripple the 492functionality you need. One way to see how packages relate to each other 493is by using the Dependency Explorer UI with the BitBake command:: 494 495 $ cd image-directory 496 $ bitbake -u taskexp -g image 497 498Use the interface to 499select potential packages you wish to eliminate and see their dependency 500relationships. 501 502When deciding how to reduce the size, get rid of packages that result in 503minimal impact on the feature set. For example, you might not need a VGA 504display. Or, you might be able to get by with ``devtmpfs`` and ``mdev`` 505instead of ``udev``. 506 507Use your ``local.conf`` file to make changes. For example, to eliminate 508``udev`` and ``glib``, set the following in the local configuration 509file:: 510 511 VIRTUAL-RUNTIME_dev_manager = "" 512 513Finally, you should consider exactly the type of root filesystem you 514need to meet your needs while also reducing its size. For example, 515consider ``cramfs``, ``squashfs``, ``ubifs``, ``ext2``, or an 516:term:`Initramfs` using ``initramfs``. Be aware that ``ext3`` requires a 1 517Mbyte journal. If you are okay with running read-only, you do not need 518this journal. 519 520.. note:: 521 522 After each round of elimination, you need to rebuild your system and 523 then use the tools to see the effects of your reductions. 524 525Trim the Kernel 526--------------- 527 528The kernel is built by including policies for hardware-independent 529aspects. What subsystems do you enable? For what architecture are you 530building? Which drivers do you build by default? 531 532.. note:: 533 534 You can modify the kernel source if you want to help with boot time. 535 536Run the ``ksize.py`` script from the top-level Linux build directory to 537get an idea of what is making up the kernel:: 538 539 $ cd top-level-linux-build-directory 540 $ ksize.py > ksize.log 541 $ cat ksize.log 542 543When you examine the log, you will see how much space is taken up with 544the built-in ``.o`` files for drivers, networking, core kernel files, 545filesystem, sound, and so forth. The sizes reported by the tool are 546uncompressed, and thus will be smaller by a relatively constant factor 547in a compressed kernel image. Look to reduce the areas that are large 548and taking up around the "90% rule." 549 550To examine, or drill down, into any particular area, use the ``-d`` 551option with the script:: 552 553 $ ksize.py -d > ksize.log 554 555Using this option 556breaks out the individual file information for each area of the kernel 557(e.g. drivers, networking, and so forth). 558 559Use your log file to see what you can eliminate from the kernel based on 560features you can let go. For example, if you are not going to need 561sound, you do not need any drivers that support sound. 562 563After figuring out what to eliminate, you need to reconfigure the kernel 564to reflect those changes during the next build. You could run 565``menuconfig`` and make all your changes at once. However, that makes it 566difficult to see the effects of your individual eliminations and also 567makes it difficult to replicate the changes for perhaps another target 568device. A better method is to start with no configurations using 569``allnoconfig``, create configuration fragments for individual changes, 570and then manage the fragments into a single configuration file using 571``merge_config.sh``. The tool makes it easy for you to iterate using the 572configuration change and build cycle. 573 574Each time you make configuration changes, you need to rebuild the kernel 575and check to see what impact your changes had on the overall size. 576 577Remove Package Management Requirements 578-------------------------------------- 579 580Packaging requirements add size to the image. One way to reduce the size 581of the image is to remove all the packaging requirements from the image. 582This reduction includes both removing the package manager and its unique 583dependencies as well as removing the package management data itself. 584 585To eliminate all the packaging requirements for an image, be sure that 586"package-management" is not part of your 587:term:`IMAGE_FEATURES` 588statement for the image. When you remove this feature, you are removing 589the package manager as well as its dependencies from the root 590filesystem. 591 592Look for Other Ways to Minimize Size 593------------------------------------ 594 595Depending on your particular circumstances, other areas that you can 596trim likely exist. The key to finding these areas is through tools and 597methods described here combined with experimentation and iteration. Here 598are a couple of areas to experiment with: 599 600- ``glibc``: In general, follow this process: 601 602 #. Remove ``glibc`` features from 603 :term:`DISTRO_FEATURES` 604 that you think you do not need. 605 606 #. Build your distribution. 607 608 #. If the build fails due to missing symbols in a package, determine 609 if you can reconfigure the package to not need those features. For 610 example, change the configuration to not support wide character 611 support as is done for ``ncurses``. Or, if support for those 612 characters is needed, determine what ``glibc`` features provide 613 the support and restore the configuration. 614 615 4. Rebuild and repeat the process. 616 617- ``busybox``: For BusyBox, use a process similar as described for 618 ``glibc``. A difference is you will need to boot the resulting system 619 to see if you are able to do everything you expect from the running 620 system. You need to be sure to integrate configuration fragments into 621 Busybox because BusyBox handles its own core features and then allows 622 you to add configuration fragments on top. 623 624Iterate on the Process 625---------------------- 626 627If you have not reached your goals on system size, you need to iterate 628on the process. The process is the same. Use the tools and see just what 629is taking up 90% of the root filesystem and the kernel. Decide what you 630can eliminate without limiting your device beyond what you need. 631 632Depending on your system, a good place to look might be Busybox, which 633provides a stripped down version of Unix tools in a single, executable 634file. You might be able to drop virtual terminal services or perhaps 635ipv6. 636 637Building Images for More than One Machine 638========================================= 639 640A common scenario developers face is creating images for several 641different machines that use the same software environment. In this 642situation, it is tempting to set the tunings and optimization flags for 643each build specifically for the targeted hardware (i.e. "maxing out" the 644tunings). Doing so can considerably add to build times and package feed 645maintenance collectively for the machines. For example, selecting tunes 646that are extremely specific to a CPU core used in a system might enable 647some micro optimizations in GCC for that particular system but would 648otherwise not gain you much of a performance difference across the other 649systems as compared to using a more general tuning across all the builds 650(e.g. setting :term:`DEFAULTTUNE` 651specifically for each machine's build). Rather than "max out" each 652build's tunings, you can take steps that cause the OpenEmbedded build 653system to reuse software across the various machines where it makes 654sense. 655 656If build speed and package feed maintenance are considerations, you 657should consider the points in this section that can help you optimize 658your tunings to best consider build times and package feed maintenance. 659 660- *Share the :term:`Build Directory`:* If at all possible, share the 661 :term:`TMPDIR` across builds. The Yocto Project supports switching between 662 different :term:`MACHINE` values in the same :term:`TMPDIR`. This practice 663 is well supported and regularly used by developers when building for 664 multiple machines. When you use the same :term:`TMPDIR` for multiple 665 machine builds, the OpenEmbedded build system can reuse the existing native 666 and often cross-recipes for multiple machines. Thus, build time decreases. 667 668 .. note:: 669 670 If :term:`DISTRO` settings change or fundamental configuration settings 671 such as the filesystem layout, you need to work with a clean :term:`TMPDIR`. 672 Sharing :term:`TMPDIR` under these circumstances might work but since it is 673 not guaranteed, you should use a clean :term:`TMPDIR`. 674 675- *Enable the Appropriate Package Architecture:* By default, the 676 OpenEmbedded build system enables three levels of package 677 architectures: "all", "tune" or "package", and "machine". Any given 678 recipe usually selects one of these package architectures (types) for 679 its output. Depending for what a given recipe creates packages, 680 making sure you enable the appropriate package architecture can 681 directly impact the build time. 682 683 A recipe that just generates scripts can enable "all" architecture 684 because there are no binaries to build. To specifically enable "all" 685 architecture, be sure your recipe inherits the 686 :ref:`ref-classes-allarch` class. 687 This class is useful for "all" architectures because it configures 688 many variables so packages can be used across multiple architectures. 689 690 If your recipe needs to generate packages that are machine-specific 691 or when one of the build or runtime dependencies is already 692 machine-architecture dependent, which makes your recipe also 693 machine-architecture dependent, make sure your recipe enables the 694 "machine" package architecture through the 695 :term:`MACHINE_ARCH` 696 variable:: 697 698 PACKAGE_ARCH = "${MACHINE_ARCH}" 699 700 When you do not 701 specifically enable a package architecture through the 702 :term:`PACKAGE_ARCH`, The 703 OpenEmbedded build system defaults to the 704 :term:`TUNE_PKGARCH` setting:: 705 706 PACKAGE_ARCH = "${TUNE_PKGARCH}" 707 708- *Choose a Generic Tuning File if Possible:* Some tunes are more 709 generic and can run on multiple targets (e.g. an ``armv5`` set of 710 packages could run on ``armv6`` and ``armv7`` processors in most 711 cases). Similarly, ``i486`` binaries could work on ``i586`` and 712 higher processors. You should realize, however, that advances on 713 newer processor versions would not be used. 714 715 If you select the same tune for several different machines, the 716 OpenEmbedded build system reuses software previously built, thus 717 speeding up the overall build time. Realize that even though a new 718 sysroot for each machine is generated, the software is not recompiled 719 and only one package feed exists. 720 721- *Manage Granular Level Packaging:* Sometimes there are cases where 722 injecting another level of package architecture beyond the three 723 higher levels noted earlier can be useful. For example, consider how 724 NXP (formerly Freescale) allows for the easy reuse of binary packages 725 in their layer 726 :yocto_git:`meta-freescale </meta-freescale/>`. 727 In this example, the 728 :yocto_git:`fsl-dynamic-packagearch </meta-freescale/tree/classes/fsl-dynamic-packagearch.bbclass>` 729 class shares GPU packages for i.MX53 boards because all boards share 730 the AMD GPU. The i.MX6-based boards can do the same because all 731 boards share the Vivante GPU. This class inspects the BitBake 732 datastore to identify if the package provides or depends on one of 733 the sub-architecture values. If so, the class sets the 734 :term:`PACKAGE_ARCH` value 735 based on the ``MACHINE_SUBARCH`` value. If the package does not 736 provide or depend on one of the sub-architecture values but it 737 matches a value in the machine-specific filter, it sets 738 :term:`MACHINE_ARCH`. This 739 behavior reduces the number of packages built and saves build time by 740 reusing binaries. 741 742- *Use Tools to Debug Issues:* Sometimes you can run into situations 743 where software is being rebuilt when you think it should not be. For 744 example, the OpenEmbedded build system might not be using shared 745 state between machines when you think it should be. These types of 746 situations are usually due to references to machine-specific 747 variables such as :term:`MACHINE`, 748 :term:`SERIAL_CONSOLES`, 749 :term:`XSERVER`, 750 :term:`MACHINE_FEATURES`, 751 and so forth in code that is supposed to only be tune-specific or 752 when the recipe depends 753 (:term:`DEPENDS`, 754 :term:`RDEPENDS`, 755 :term:`RRECOMMENDS`, 756 :term:`RSUGGESTS`, and so forth) 757 on some other recipe that already has 758 :term:`PACKAGE_ARCH` defined 759 as "${MACHINE_ARCH}". 760 761 .. note:: 762 763 Patches to fix any issues identified are most welcome as these 764 issues occasionally do occur. 765 766 For such cases, you can use some tools to help you sort out the 767 situation: 768 769 - ``state-diff-machines.sh``*:* You can find this tool in the 770 ``scripts`` directory of the Source Repositories. See the comments 771 in the script for information on how to use the tool. 772 773 - *BitBake's "-S printdiff" Option:* Using this option causes 774 BitBake to try to establish the closest signature match it can 775 (e.g. in the shared state cache) and then run ``bitbake-diffsigs`` 776 over the matches to determine the stamps and delta where these two 777 stamp trees diverge. 778 779Building Software from an External Source 780========================================= 781 782By default, the OpenEmbedded build system uses the :term:`Build Directory` 783when building source code. The build process involves fetching the source 784files, unpacking them, and then patching them if necessary before the build 785takes place. 786 787There are situations where you might want to build software from source 788files that are external to and thus outside of the OpenEmbedded build 789system. For example, suppose you have a project that includes a new BSP 790with a heavily customized kernel. And, you want to minimize exposing the 791build system to the development team so that they can focus on their 792project and maintain everyone's workflow as much as possible. In this 793case, you want a kernel source directory on the development machine 794where the development occurs. You want the recipe's 795:term:`SRC_URI` variable to point to 796the external directory and use it as is, not copy it. 797 798To build from software that comes from an external source, all you need to do 799is inherit the :ref:`ref-classes-externalsrc` class and then set 800the :term:`EXTERNALSRC` variable to point to your external source code. Here 801are the statements to put in your ``local.conf`` file:: 802 803 INHERIT += "externalsrc" 804 EXTERNALSRC:pn-myrecipe = "path-to-your-source-tree" 805 806This next example shows how to accomplish the same thing by setting 807:term:`EXTERNALSRC` in the recipe itself or in the recipe's append file:: 808 809 EXTERNALSRC = "path" 810 EXTERNALSRC_BUILD = "path" 811 812.. note:: 813 814 In order for these settings to take effect, you must globally or 815 locally inherit the :ref:`ref-classes-externalsrc` class. 816 817By default, :ref:`ref-classes-externalsrc` builds the source code in a 818directory separate from the external source directory as specified by 819:term:`EXTERNALSRC`. If you need 820to have the source built in the same directory in which it resides, or 821some other nominated directory, you can set 822:term:`EXTERNALSRC_BUILD` 823to point to that directory:: 824 825 EXTERNALSRC_BUILD:pn-myrecipe = "path-to-your-source-tree" 826 827Replicating a Build Offline 828=========================== 829 830It can be useful to take a "snapshot" of upstream sources used in a 831build and then use that "snapshot" later to replicate the build offline. 832To do so, you need to first prepare and populate your downloads 833directory your "snapshot" of files. Once your downloads directory is 834ready, you can use it at any time and from any machine to replicate your 835build. 836 837Follow these steps to populate your Downloads directory: 838 839#. *Create a Clean Downloads Directory:* Start with an empty downloads 840 directory (:term:`DL_DIR`). You 841 start with an empty downloads directory by either removing the files 842 in the existing directory or by setting :term:`DL_DIR` to point to either 843 an empty location or one that does not yet exist. 844 845#. *Generate Tarballs of the Source Git Repositories:* Edit your 846 ``local.conf`` configuration file as follows:: 847 848 DL_DIR = "/home/your-download-dir/" 849 BB_GENERATE_MIRROR_TARBALLS = "1" 850 851 During 852 the fetch process in the next step, BitBake gathers the source files 853 and creates tarballs in the directory pointed to by :term:`DL_DIR`. See 854 the 855 :term:`BB_GENERATE_MIRROR_TARBALLS` 856 variable for more information. 857 858#. *Populate Your Downloads Directory Without Building:* Use BitBake to 859 fetch your sources but inhibit the build:: 860 861 $ bitbake target --runonly=fetch 862 863 The downloads directory (i.e. ``${DL_DIR}``) now has 864 a "snapshot" of the source files in the form of tarballs, which can 865 be used for the build. 866 867#. *Optionally Remove Any Git or other SCM Subdirectories From the 868 Downloads Directory:* If you want, you can clean up your downloads 869 directory by removing any Git or other Source Control Management 870 (SCM) subdirectories such as ``${DL_DIR}/git2/*``. The tarballs 871 already contain these subdirectories. 872 873Once your downloads directory has everything it needs regarding source 874files, you can create your "own-mirror" and build your target. 875Understand that you can use the files to build the target offline from 876any machine and at any time. 877 878Follow these steps to build your target using the files in the downloads 879directory: 880 881#. *Using Local Files Only:* Inside your ``local.conf`` file, add the 882 :term:`SOURCE_MIRROR_URL` variable, inherit the 883 :ref:`ref-classes-own-mirrors` class, and use the 884 :term:`BB_NO_NETWORK` variable to your ``local.conf``:: 885 886 SOURCE_MIRROR_URL ?= "file:///home/your-download-dir/" 887 INHERIT += "own-mirrors" 888 BB_NO_NETWORK = "1" 889 890 The :term:`SOURCE_MIRROR_URL` and :ref:`ref-classes-own-mirrors` 891 class set up the system to use the downloads directory as your "own 892 mirror". Using the :term:`BB_NO_NETWORK` variable makes sure that 893 BitBake's fetching process in step 3 stays local, which means files 894 from your "own-mirror" are used. 895 896#. *Start With a Clean Build:* You can start with a clean build by 897 removing the ``${``\ :term:`TMPDIR`\ ``}`` directory or using a new 898 :term:`Build Directory`. 899 900#. *Build Your Target:* Use BitBake to build your target:: 901 902 $ bitbake target 903 904 The build completes using the known local "snapshot" of source 905 files from your mirror. The resulting tarballs for your "snapshot" of 906 source files are in the downloads directory. 907 908 .. note:: 909 910 The offline build does not work if recipes attempt to find the 911 latest version of software by setting 912 :term:`SRCREV` to 913 ``${``\ :term:`AUTOREV`\ ``}``:: 914 915 SRCREV = "${AUTOREV}" 916 917 When a recipe sets :term:`SRCREV` to 918 ``${``\ :term:`AUTOREV`\ ``}``, the build system accesses the network in an 919 attempt to determine the latest version of software from the SCM. 920 Typically, recipes that use :term:`AUTOREV` are custom or modified 921 recipes. Recipes that reside in public repositories usually do not 922 use :term:`AUTOREV`. 923 924 If you do have recipes that use :term:`AUTOREV`, you can take steps to 925 still use the recipes in an offline build. Do the following: 926 927 #. Use a configuration generated by enabling :ref:`build 928 history <dev-manual/build-quality:maintaining build output quality>`. 929 930 #. Use the ``buildhistory-collect-srcrevs`` command to collect the 931 stored :term:`SRCREV` values from the build's history. For more 932 information on collecting these values, see the 933 ":ref:`dev-manual/build-quality:build history package information`" 934 section. 935 936 #. Once you have the correct source revisions, you can modify 937 those recipes to set :term:`SRCREV` to specific versions of the 938 software. 939 940