1# Copyright (c) 2013 The Chromium OS Authors. 2# 3# SPDX-License-Identifier: GPL-2.0+ 4# 5 6(Please read 'How to change from MAKEALL' if you are used to that tool) 7 8Quick-start 9=========== 10 11If you just want to quickly set up buildman so you can build something (for 12example Raspberry Pi 2): 13 14 cd /path/to/u-boot 15 PATH=$PATH:`pwd`/tools/buildman 16 buildman --fetch-arch arm 17 buildman -k rpi_2 18 ls ../current/rpi_2 19 # u-boot.bin is the output image 20 21 22What is this? 23============= 24 25This tool handles building U-Boot to check that you have not broken it 26with your patch series. It can build each individual commit and report 27which boards fail on which commits, and which errors come up. It aims 28to make full use of multi-processor machines. 29 30A key feature of buildman is its output summary, which allows warnings, 31errors or image size increases in a particular commit or board to be 32quickly identified and the offending commit pinpointed. This can be a big 33help for anyone working with >10 patches at a time. 34 35 36Caveats 37======= 38 39Buildman can be stopped and restarted, in which case it will continue 40where it left off. This should happen cleanly and without side-effects. 41If not, it is a bug, for which a patch would be welcome. 42 43Buildman gets so tied up in its work that it can ignore the outside world. 44You may need to press Ctrl-C several times to quit it. Also it will print 45out various exceptions when stopped. You may have to kill it since the 46Ctrl-C handling is somewhat broken. 47 48 49Theory of Operation 50=================== 51 52(please read this section in full twice or you will be perpetually confused) 53 54Buildman is a builder. It is not make, although it runs make. It does not 55produce any useful output on the terminal while building, except for 56progress information (except with -v, see below). All the output (errors, 57warnings and binaries if you ask for them) is stored in output 58directories, which you can look at while the build is progressing, or when 59it is finished. 60 61Buildman is designed to build entire git branches, i.e. muliple commits. It 62can be run repeatedly on the same branch. In this case it will automatically 63rebuild commits which have changed (and remove its old results for that 64commit). It is possible to build a branch for one board, then later build it 65for another board. If you want buildman to re-build a commit it has already 66built (e.g. because of a toolchain update), use the -f flag. 67 68Buildman produces a concise summary of which boards succeeded and failed. 69It shows which commit introduced which board failure using a simple 70red/green colour coding. Full error information can be requested, in which 71case it is de-duped and displayed against the commit that introduced the 72error. An example workflow is below. 73 74Buildman stores image size information and can report changes in image size 75from commit to commit. An example of this is below. 76 77Buildman starts multiple threads, and each thread builds for one board at 78a time. A thread starts at the first commit, configures the source for your 79board and builds it. Then it checks out the next commit and does an 80incremental build. Eventually the thread reaches the last commit and stops. 81If errors or warnings are found along the way, the thread will reconfigure 82after every commit, and your build will be very slow. This is because a 83file that produces just a warning would not normally be rebuilt in an 84incremental build. 85 86Buildman works in an entirely separate place from your U-Boot repository. 87It creates a separate working directory for each thread, and puts the 88output files in the working directory, organised by commit name and board 89name, in a two-level hierarchy. 90 91Buildman is invoked in your U-Boot directory, the one with the .git 92directory. It clones this repository into a copy for each thread, and the 93threads do not affect the state of your git repository. Any checkouts done 94by the thread affect only the working directory for that thread. 95 96Buildman automatically selects the correct tool chain for each board. You 97must supply suitable tool chains, but buildman takes care of selecting the 98right one. 99 100Buildman generally builds a branch (with the -b flag), and in this case 101builds the upstream commit as well, for comparison. It cannot build 102individual commits at present, unless (maybe) you point it at an empty 103branch. Put all your commits in a branch, set the branch's upstream to a 104valid value, and all will be well. Otherwise buildman will perform random 105actions. Use -n to check what the random actions might be. 106 107If you just want to build the current source tree, leave off the -b flag 108and add -e. This will display results and errors as they happen. You can 109still look at them later using -se. Note that buildman will assume that the 110source has changed, and will build all specified boards in this case. 111 112Buildman is optimised for building many commits at once, for many boards. 113On multi-core machines, Buildman is fast because it uses most of the 114available CPU power. When it gets to the end, or if you are building just 115a few commits or boards, it will be pretty slow. As a tip, if you don't 116plan to use your machine for anything else, you can use -T to increase the 117number of threads beyond the default. 118 119Buildman lets you build all boards, or a subset. Specify the subset by passing 120command-line arguments that list the desired board name, architecture name, 121SOC name, or anything else in the boards.cfg file. Multiple arguments are 122allowed. Each argument will be interpreted as a regular expression, so 123behaviour is a superset of exact or substring matching. Examples are: 124 125* 'tegra20' All boards with a Tegra20 SoC 126* 'tegra' All boards with any Tegra Soc (Tegra20, Tegra30, Tegra114...) 127* '^tegra[23]0$' All boards with either Tegra20 or Tegra30 SoC 128* 'powerpc' All PowerPC boards 129 130While the default is to OR the terms together, you can also make use of 131the '&' operator to limit the selection: 132 133* 'freescale & arm sandbox' All Freescale boards with ARM architecture, 134 plus sandbox 135 136You can also use -x to specifically exclude some boards. For example: 137 138 buildmand arm -x nvidia,freescale,.*ball$ 139 140means to build all arm boards except nvidia, freescale and anything ending 141with 'ball'. 142 143It is convenient to use the -n option to see what will be built based on 144the subset given. Use -v as well to get an actual list of boards. 145 146Buildman does not store intermediate object files. It optionally copies 147the binary output into a directory when a build is successful. Size 148information is always recorded. It needs a fair bit of disk space to work, 149typically 250MB per thread. 150 151 152Setting up 153========== 154 1551. Get the U-Boot source. You probably already have it, but if not these 156steps should get you started with a repo and some commits for testing. 157 158$ cd /path/to/u-boot 159$ git clone git://git.denx.de/u-boot.git . 160$ git checkout -b my-branch origin/master 161$ # Add some commits to the branch, reading for testing 162 1632. Create ~/.buildman to tell buildman where to find tool chains (see 'The 164.buildman file' later for details). As an example: 165 166# Buildman settings file 167 168[toolchain] 169root: / 170rest: /toolchains/* 171eldk: /opt/eldk-4.2 172arm: /opt/linaro/gcc-linaro-arm-linux-gnueabihf-4.8-2013.08_linux 173aarch64: /opt/linaro/gcc-linaro-aarch64-none-elf-4.8-2013.10_linux 174 175[toolchain-alias] 176x86: i386 177blackfin: bfin 178nds32: nds32le 179openrisc: or1k 180 181 182This selects the available toolchain paths. Add the base directory for 183each of your toolchains here. Buildman will search inside these directories 184and also in any '/usr' and '/usr/bin' subdirectories. 185 186Make sure the tags (here root: rest: and eldk:) are unique. 187 188The toolchain-alias section indicates that the i386 toolchain should be used 189to build x86 commits. 190 191Note that you can also specific exactly toolchain prefixes if you like: 192 193[toolchain-prefix] 194arm: /opt/arm-eabi-4.6/bin/arm-eabi- 195 196or even: 197 198[toolchain-prefix] 199arm: /opt/arm-eabi-4.6/bin/arm-eabi-gcc 200 201This tells buildman that you want to use this exact toolchain for the arm 202architecture. This will override any toolchains found by searching using the 203[toolchain] settings. 204 205Since the toolchain prefix is an explicit request, buildman will report an 206error if a toolchain is not found with that prefix. The current PATH will be 207searched, so it is possible to use: 208 209[toolchain-prefix] 210arm: arm-none-eabi- 211 212and buildman will find arm-none-eabi-gcc in /usr/bin if you have it installed. 213 214[toolchain-wrapper] 215wrapper: ccache 216 217This tells buildman to use a compiler wrapper in front of CROSS_COMPILE. In 218this example, ccache. It doesn't affect the toolchain scan. The wrapper is 219added when CROSS_COMPILE environtal variable is set. The name in this 220section is ignored. If more than one line is provided, only the last one 221is taken. 222 2233. Make sure you have the require Python pre-requisites 224 225Buildman uses multiprocessing, Queue, shutil, StringIO, ConfigParser and 226urllib2. These should normally be available, but if you get an error like 227this then you will need to obtain those modules: 228 229 ImportError: No module named multiprocessing 230 231 2324. Check the available toolchains 233 234Run this check to make sure that you have a toolchain for every architecture. 235 236$ ./tools/buildman/buildman --list-tool-chains 237Scanning for tool chains 238 - scanning prefix '/opt/gcc-4.6.3-nolibc/x86_64-linux/bin/x86_64-linux-' 239Tool chain test: OK, arch='x86', priority 1 240 - scanning prefix '/opt/arm-eabi-4.6/bin/arm-eabi-' 241Tool chain test: OK, arch='arm', priority 1 242 - scanning path '/toolchains/gcc-4.9.0-nolibc/i386-linux' 243 - looking in '/toolchains/gcc-4.9.0-nolibc/i386-linux/.' 244 - looking in '/toolchains/gcc-4.9.0-nolibc/i386-linux/bin' 245 - found '/toolchains/gcc-4.9.0-nolibc/i386-linux/bin/i386-linux-gcc' 246 - looking in '/toolchains/gcc-4.9.0-nolibc/i386-linux/usr/bin' 247Tool chain test: OK, arch='i386', priority 4 248 - scanning path '/toolchains/gcc-4.9.0-nolibc/aarch64-linux' 249 - looking in '/toolchains/gcc-4.9.0-nolibc/aarch64-linux/.' 250 - looking in '/toolchains/gcc-4.9.0-nolibc/aarch64-linux/bin' 251 - found '/toolchains/gcc-4.9.0-nolibc/aarch64-linux/bin/aarch64-linux-gcc' 252 - looking in '/toolchains/gcc-4.9.0-nolibc/aarch64-linux/usr/bin' 253Tool chain test: OK, arch='aarch64', priority 4 254 - scanning path '/toolchains/gcc-4.9.0-nolibc/microblaze-linux' 255 - looking in '/toolchains/gcc-4.9.0-nolibc/microblaze-linux/.' 256 - looking in '/toolchains/gcc-4.9.0-nolibc/microblaze-linux/bin' 257 - found '/toolchains/gcc-4.9.0-nolibc/microblaze-linux/bin/microblaze-linux-gcc' 258 - looking in '/toolchains/gcc-4.9.0-nolibc/microblaze-linux/usr/bin' 259Tool chain test: OK, arch='microblaze', priority 4 260 - scanning path '/toolchains/gcc-4.9.0-nolibc/mips64-linux' 261 - looking in '/toolchains/gcc-4.9.0-nolibc/mips64-linux/.' 262 - looking in '/toolchains/gcc-4.9.0-nolibc/mips64-linux/bin' 263 - found '/toolchains/gcc-4.9.0-nolibc/mips64-linux/bin/mips64-linux-gcc' 264 - looking in '/toolchains/gcc-4.9.0-nolibc/mips64-linux/usr/bin' 265Tool chain test: OK, arch='mips64', priority 4 266 - scanning path '/toolchains/gcc-4.9.0-nolibc/sparc64-linux' 267 - looking in '/toolchains/gcc-4.9.0-nolibc/sparc64-linux/.' 268 - looking in '/toolchains/gcc-4.9.0-nolibc/sparc64-linux/bin' 269 - found '/toolchains/gcc-4.9.0-nolibc/sparc64-linux/bin/sparc64-linux-gcc' 270 - looking in '/toolchains/gcc-4.9.0-nolibc/sparc64-linux/usr/bin' 271Tool chain test: OK, arch='sparc64', priority 4 272 - scanning path '/toolchains/gcc-4.9.0-nolibc/arm-unknown-linux-gnueabi' 273 - looking in '/toolchains/gcc-4.9.0-nolibc/arm-unknown-linux-gnueabi/.' 274 - looking in '/toolchains/gcc-4.9.0-nolibc/arm-unknown-linux-gnueabi/bin' 275 - found '/toolchains/gcc-4.9.0-nolibc/arm-unknown-linux-gnueabi/bin/arm-unknown-linux-gnueabi-gcc' 276 - looking in '/toolchains/gcc-4.9.0-nolibc/arm-unknown-linux-gnueabi/usr/bin' 277Tool chain test: OK, arch='arm', priority 3 278Toolchain '/toolchains/gcc-4.9.0-nolibc/arm-unknown-linux-gnueabi/bin/arm-unknown-linux-gnueabi-gcc' at priority 3 will be ignored because another toolchain for arch 'arm' has priority 1 279 - scanning path '/toolchains/gcc-4.9.0-nolibc/sparc-linux' 280 - looking in '/toolchains/gcc-4.9.0-nolibc/sparc-linux/.' 281 - looking in '/toolchains/gcc-4.9.0-nolibc/sparc-linux/bin' 282 - found '/toolchains/gcc-4.9.0-nolibc/sparc-linux/bin/sparc-linux-gcc' 283 - looking in '/toolchains/gcc-4.9.0-nolibc/sparc-linux/usr/bin' 284Tool chain test: OK, arch='sparc', priority 4 285 - scanning path '/toolchains/gcc-4.9.0-nolibc/mips-linux' 286 - looking in '/toolchains/gcc-4.9.0-nolibc/mips-linux/.' 287 - looking in '/toolchains/gcc-4.9.0-nolibc/mips-linux/bin' 288 - found '/toolchains/gcc-4.9.0-nolibc/mips-linux/bin/mips-linux-gcc' 289 - looking in '/toolchains/gcc-4.9.0-nolibc/mips-linux/usr/bin' 290Tool chain test: OK, arch='mips', priority 4 291 - scanning path '/toolchains/gcc-4.9.0-nolibc/x86_64-linux' 292 - looking in '/toolchains/gcc-4.9.0-nolibc/x86_64-linux/.' 293 - looking in '/toolchains/gcc-4.9.0-nolibc/x86_64-linux/bin' 294 - found '/toolchains/gcc-4.9.0-nolibc/x86_64-linux/bin/x86_64-linux-gcc' 295 - found '/toolchains/gcc-4.9.0-nolibc/x86_64-linux/bin/x86_64-linux-x86_64-linux-gcc' 296 - looking in '/toolchains/gcc-4.9.0-nolibc/x86_64-linux/usr/bin' 297Tool chain test: OK, arch='x86_64', priority 4 298Tool chain test: OK, arch='x86_64', priority 4 299Toolchain '/toolchains/gcc-4.9.0-nolibc/x86_64-linux/bin/x86_64-linux-x86_64-linux-gcc' at priority 4 will be ignored because another toolchain for arch 'x86_64' has priority 4 300 - scanning path '/toolchains/gcc-4.9.0-nolibc/m68k-linux' 301 - looking in '/toolchains/gcc-4.9.0-nolibc/m68k-linux/.' 302 - looking in '/toolchains/gcc-4.9.0-nolibc/m68k-linux/bin' 303 - found '/toolchains/gcc-4.9.0-nolibc/m68k-linux/bin/m68k-linux-gcc' 304 - looking in '/toolchains/gcc-4.9.0-nolibc/m68k-linux/usr/bin' 305Tool chain test: OK, arch='m68k', priority 4 306 - scanning path '/toolchains/gcc-4.9.0-nolibc/powerpc-linux' 307 - looking in '/toolchains/gcc-4.9.0-nolibc/powerpc-linux/.' 308 - looking in '/toolchains/gcc-4.9.0-nolibc/powerpc-linux/bin' 309 - found '/toolchains/gcc-4.9.0-nolibc/powerpc-linux/bin/powerpc-linux-gcc' 310 - looking in '/toolchains/gcc-4.9.0-nolibc/powerpc-linux/usr/bin' 311Tool chain test: OK, arch='powerpc', priority 4 312 - scanning path '/toolchains/gcc-4.6.3-nolibc/bfin-uclinux' 313 - looking in '/toolchains/gcc-4.6.3-nolibc/bfin-uclinux/.' 314 - looking in '/toolchains/gcc-4.6.3-nolibc/bfin-uclinux/bin' 315 - found '/toolchains/gcc-4.6.3-nolibc/bfin-uclinux/bin/bfin-uclinux-gcc' 316 - looking in '/toolchains/gcc-4.6.3-nolibc/bfin-uclinux/usr/bin' 317Tool chain test: OK, arch='bfin', priority 6 318 - scanning path '/toolchains/gcc-4.6.3-nolibc/sparc-linux' 319 - looking in '/toolchains/gcc-4.6.3-nolibc/sparc-linux/.' 320 - looking in '/toolchains/gcc-4.6.3-nolibc/sparc-linux/bin' 321 - found '/toolchains/gcc-4.6.3-nolibc/sparc-linux/bin/sparc-linux-gcc' 322 - looking in '/toolchains/gcc-4.6.3-nolibc/sparc-linux/usr/bin' 323Tool chain test: OK, arch='sparc', priority 4 324Toolchain '/toolchains/gcc-4.6.3-nolibc/sparc-linux/bin/sparc-linux-gcc' at priority 4 will be ignored because another toolchain for arch 'sparc' has priority 4 325 - scanning path '/toolchains/gcc-4.6.3-nolibc/mips-linux' 326 - looking in '/toolchains/gcc-4.6.3-nolibc/mips-linux/.' 327 - looking in '/toolchains/gcc-4.6.3-nolibc/mips-linux/bin' 328 - found '/toolchains/gcc-4.6.3-nolibc/mips-linux/bin/mips-linux-gcc' 329 - looking in '/toolchains/gcc-4.6.3-nolibc/mips-linux/usr/bin' 330Tool chain test: OK, arch='mips', priority 4 331Toolchain '/toolchains/gcc-4.6.3-nolibc/mips-linux/bin/mips-linux-gcc' at priority 4 will be ignored because another toolchain for arch 'mips' has priority 4 332 - scanning path '/toolchains/gcc-4.6.3-nolibc/m68k-linux' 333 - looking in '/toolchains/gcc-4.6.3-nolibc/m68k-linux/.' 334 - looking in '/toolchains/gcc-4.6.3-nolibc/m68k-linux/bin' 335 - found '/toolchains/gcc-4.6.3-nolibc/m68k-linux/bin/m68k-linux-gcc' 336 - looking in '/toolchains/gcc-4.6.3-nolibc/m68k-linux/usr/bin' 337Tool chain test: OK, arch='m68k', priority 4 338Toolchain '/toolchains/gcc-4.6.3-nolibc/m68k-linux/bin/m68k-linux-gcc' at priority 4 will be ignored because another toolchain for arch 'm68k' has priority 4 339 - scanning path '/toolchains/gcc-4.6.3-nolibc/powerpc-linux' 340 - looking in '/toolchains/gcc-4.6.3-nolibc/powerpc-linux/.' 341 - looking in '/toolchains/gcc-4.6.3-nolibc/powerpc-linux/bin' 342 - found '/toolchains/gcc-4.6.3-nolibc/powerpc-linux/bin/powerpc-linux-gcc' 343 - looking in '/toolchains/gcc-4.6.3-nolibc/powerpc-linux/usr/bin' 344Tool chain test: OK, arch='powerpc', priority 4 345Tool chain test: OK, arch='or32', priority 4 346 - scanning path '/toolchains/gcc-4.2.4-nolibc/avr32-linux' 347 - looking in '/toolchains/gcc-4.2.4-nolibc/avr32-linux/.' 348 - looking in '/toolchains/gcc-4.2.4-nolibc/avr32-linux/bin' 349 - found '/toolchains/gcc-4.2.4-nolibc/avr32-linux/bin/avr32-linux-gcc' 350 - looking in '/toolchains/gcc-4.2.4-nolibc/avr32-linux/usr/bin' 351Tool chain test: OK, arch='avr32', priority 4 352 - scanning path '/' 353 - looking in '/.' 354 - looking in '/bin' 355 - looking in '/usr/bin' 356 - found '/usr/bin/i586-mingw32msvc-gcc' 357 - found '/usr/bin/c89-gcc' 358 - found '/usr/bin/x86_64-linux-gnu-gcc' 359 - found '/usr/bin/gcc' 360 - found '/usr/bin/c99-gcc' 361 - found '/usr/bin/arm-linux-gnueabi-gcc' 362 - found '/usr/bin/aarch64-linux-gnu-gcc' 363 - found '/usr/bin/winegcc' 364 - found '/usr/bin/arm-linux-gnueabihf-gcc' 365Tool chain test: OK, arch='i586', priority 11 366Tool chain test: OK, arch='c89', priority 11 367Tool chain test: OK, arch='x86_64', priority 4 368Toolchain '/usr/bin/x86_64-linux-gnu-gcc' at priority 4 will be ignored because another toolchain for arch 'x86_64' has priority 4 369Tool chain test: OK, arch='sandbox', priority 11 370Tool chain test: OK, arch='c99', priority 11 371Tool chain test: OK, arch='arm', priority 4 372Toolchain '/usr/bin/arm-linux-gnueabi-gcc' at priority 4 will be ignored because another toolchain for arch 'arm' has priority 1 373Tool chain test: OK, arch='aarch64', priority 4 374Toolchain '/usr/bin/aarch64-linux-gnu-gcc' at priority 4 will be ignored because another toolchain for arch 'aarch64' has priority 4 375Tool chain test: OK, arch='sandbox', priority 11 376Toolchain '/usr/bin/winegcc' at priority 11 will be ignored because another toolchain for arch 'sandbox' has priority 11 377Tool chain test: OK, arch='arm', priority 4 378Toolchain '/usr/bin/arm-linux-gnueabihf-gcc' at priority 4 will be ignored because another toolchain for arch 'arm' has priority 1 379List of available toolchains (34): 380aarch64 : /toolchains/gcc-4.9.0-nolibc/aarch64-linux/bin/aarch64-linux-gcc 381alpha : /toolchains/gcc-4.9.0-nolibc/alpha-linux/bin/alpha-linux-gcc 382am33_2.0 : /toolchains/gcc-4.9.0-nolibc/am33_2.0-linux/bin/am33_2.0-linux-gcc 383arm : /opt/arm-eabi-4.6/bin/arm-eabi-gcc 384avr32 : /toolchains/gcc-4.2.4-nolibc/avr32-linux/bin/avr32-linux-gcc 385bfin : /toolchains/gcc-4.6.3-nolibc/bfin-uclinux/bin/bfin-uclinux-gcc 386c89 : /usr/bin/c89-gcc 387c99 : /usr/bin/c99-gcc 388frv : /toolchains/gcc-4.9.0-nolibc/frv-linux/bin/frv-linux-gcc 389h8300 : /toolchains/gcc-4.9.0-nolibc/h8300-elf/bin/h8300-elf-gcc 390hppa : /toolchains/gcc-4.9.0-nolibc/hppa-linux/bin/hppa-linux-gcc 391hppa64 : /toolchains/gcc-4.9.0-nolibc/hppa64-linux/bin/hppa64-linux-gcc 392i386 : /toolchains/gcc-4.9.0-nolibc/i386-linux/bin/i386-linux-gcc 393i586 : /usr/bin/i586-mingw32msvc-gcc 394ia64 : /toolchains/gcc-4.9.0-nolibc/ia64-linux/bin/ia64-linux-gcc 395m32r : /toolchains/gcc-4.9.0-nolibc/m32r-linux/bin/m32r-linux-gcc 396m68k : /toolchains/gcc-4.9.0-nolibc/m68k-linux/bin/m68k-linux-gcc 397microblaze: /toolchains/gcc-4.9.0-nolibc/microblaze-linux/bin/microblaze-linux-gcc 398mips : /toolchains/gcc-4.9.0-nolibc/mips-linux/bin/mips-linux-gcc 399mips64 : /toolchains/gcc-4.9.0-nolibc/mips64-linux/bin/mips64-linux-gcc 400or32 : /toolchains/gcc-4.5.1-nolibc/or32-linux/bin/or32-linux-gcc 401powerpc : /toolchains/gcc-4.9.0-nolibc/powerpc-linux/bin/powerpc-linux-gcc 402powerpc64 : /toolchains/gcc-4.9.0-nolibc/powerpc64-linux/bin/powerpc64-linux-gcc 403ppc64le : /toolchains/gcc-4.9.0-nolibc/ppc64le-linux/bin/ppc64le-linux-gcc 404s390x : /toolchains/gcc-4.9.0-nolibc/s390x-linux/bin/s390x-linux-gcc 405sandbox : /usr/bin/gcc 406sh4 : /toolchains/gcc-4.6.3-nolibc/sh4-linux/bin/sh4-linux-gcc 407sparc : /toolchains/gcc-4.9.0-nolibc/sparc-linux/bin/sparc-linux-gcc 408sparc64 : /toolchains/gcc-4.9.0-nolibc/sparc64-linux/bin/sparc64-linux-gcc 409tilegx : /toolchains/gcc-4.6.2-nolibc/tilegx-linux/bin/tilegx-linux-gcc 410x86 : /opt/gcc-4.6.3-nolibc/x86_64-linux/bin/x86_64-linux-gcc 411x86_64 : /toolchains/gcc-4.9.0-nolibc/x86_64-linux/bin/x86_64-linux-gcc 412 413 414You can see that everything is covered, even some strange ones that won't 415be used (c88 and c99). This is a feature. 416 417 4185. Install new toolchains if needed 419 420You can download toolchains and update the [toolchain] section of the 421settings file to find them. 422 423To make this easier, buildman can automatically download and install 424toolchains from kernel.org. First list the available architectures: 425 426$ ./tools/buildman/buildman --fetch-arch list 427Checking: https://www.kernel.org/pub/tools/crosstool/files/bin/x86_64/4.6.3/ 428Checking: https://www.kernel.org/pub/tools/crosstool/files/bin/x86_64/4.6.2/ 429Checking: https://www.kernel.org/pub/tools/crosstool/files/bin/x86_64/4.5.1/ 430Checking: https://www.kernel.org/pub/tools/crosstool/files/bin/x86_64/4.2.4/ 431Available architectures: alpha am33_2.0 arm avr32 bfin cris crisv32 frv h8300 432hppa hppa64 i386 ia64 m32r m68k mips mips64 or32 powerpc powerpc64 s390x sh4 433sparc sparc64 tilegx x86_64 xtensa 434 435Then pick one and download it: 436 437$ ./tools/buildman/buildman --fetch-arch or32 438Checking: https://www.kernel.org/pub/tools/crosstool/files/bin/x86_64/4.6.3/ 439Checking: https://www.kernel.org/pub/tools/crosstool/files/bin/x86_64/4.6.2/ 440Checking: https://www.kernel.org/pub/tools/crosstool/files/bin/x86_64/4.5.1/ 441Downloading: https://www.kernel.org/pub/tools/crosstool/files/bin/x86_64/4.5.1//x86_64-gcc-4.5.1-nolibc_or32-linux.tar.xz 442Unpacking to: /home/sjg/.buildman-toolchains 443Testing 444 - looking in '/home/sjg/.buildman-toolchains/gcc-4.5.1-nolibc/or32-linux/.' 445 - looking in '/home/sjg/.buildman-toolchains/gcc-4.5.1-nolibc/or32-linux/bin' 446 - found '/home/sjg/.buildman-toolchains/gcc-4.5.1-nolibc/or32-linux/bin/or32-linux-gcc' 447Tool chain test: OK 448 449Or download them all from kernel.org and move them to /toolchains directory, 450 451$ ./tools/buildman/buildman --fetch-arch all 452$ sudo mkdir -p /toolchains 453$ sudo mv ~/.buildman-toolchains/*/* /toolchains/ 454 455For those not available from kernel.org, download from the following links. 456 457arc: https://github.com/foss-for-synopsys-dwc-arc-processors/toolchain/releases/ 458 download/arc-2016.09-release/arc_gnu_2016.09_prebuilt_uclibc_le_archs_linux_install.tar.gz 459blackfin: http://sourceforge.net/projects/adi-toolchain/files/ 460 blackfin-toolchain-elf-gcc-4.5-2014R1_45-RC2.x86_64.tar.bz2 461nds32: http://osdk.andestech.com/packages/ 462 nds32le-linux-glibc-v1.tgz 463nios2: http://sourcery.mentor.com/public/gnu_toolchain/nios2-linux-gnu/ 464 sourceryg++-2015.11-27-nios2-linux-gnu-i686-pc-linux-gnu.tar.bz2 465sh: http://sourcery.mentor.com/public/gnu_toolchain/sh-linux-gnu/ 466 renesas-4.4-200-sh-linux-gnu-i686-pc-linux-gnu.tar.bz2 467 468Note openrisc kernel.org toolchain is out of date. Download the latest one from 469http://opencores.org/or1k/OpenRISC_GNU_tool_chain#Prebuilt_versions - eg: 470ftp://ocuser:ocuser@openrisc.opencores.org/toolchain/gcc-or1k-elf-4.8.1-x86.tar.bz2. 471 472Buildman should now be set up to use your new toolchain. 473 474At the time of writing, U-Boot has these architectures: 475 476 arc, arm, avr32, blackfin, m68k, microblaze, mips, nds32, nios2, openrisc 477 powerpc, sandbox, sh, sparc, x86 478 479Of these, only arc and nds32 are not available at kernel.org.. 480 481 482How to run it 483============= 484 485First do a dry run using the -n flag: (replace <branch> with a real, local 486branch with a valid upstream) 487 488$ ./tools/buildman/buildman -b <branch> -n 489 490If it can't detect the upstream branch, try checking out the branch, and 491doing something like 'git branch --set-upstream-to upstream/master' 492or something similar. Buildman will try to guess a suitable upstream branch 493if it can't find one (you will see a message like" Guessing upstream as ...). 494 495As an example: 496 497Dry run, so not doing much. But I would do this: 498 499Building 18 commits for 1059 boards (4 threads, 1 job per thread) 500Build directory: ../lcd9b 501 5bb3505 Merge branch 'master' of git://git.denx.de/u-boot-arm 502 c18f1b4 tegra: Use const for pinmux_config_pingroup/table() 503 2f043ae tegra: Add display support to funcmux 504 e349900 tegra: fdt: Add pwm binding and node 505 424a5f0 tegra: fdt: Add LCD definitions for Tegra 506 0636ccf tegra: Add support for PWM 507 a994fe7 tegra: Add SOC support for display/lcd 508 fcd7350 tegra: Add LCD driver 509 4d46e9d tegra: Add LCD support to Nvidia boards 510 991bd48 arm: Add control over cachability of memory regions 511 54e8019 lcd: Add CONFIG_LCD_ALIGNMENT to select frame buffer alignment 512 d92aff7 lcd: Add support for flushing LCD fb from dcache after update 513 dbd0677 tegra: Align LCD frame buffer to section boundary 514 0cff9b8 tegra: Support control of cache settings for LCD 515 9c56900 tegra: fdt: Add LCD definitions for Seaboard 516 5cc29db lcd: Add CONFIG_CONSOLE_SCROLL_LINES option to speed console 517 cac5a23 tegra: Enable display/lcd support on Seaboard 518 49ff541 wip 519 520Total boards to build for each commit: 1059 521 522This shows that it will build all 1059 boards, using 4 threads (because 523we have a 4-core CPU). Each thread will run with -j1, meaning that each 524make job will use a single CPU. The list of commits to be built helps you 525confirm that things look about right. Notice that buildman has chosen a 526'base' directory for you, immediately above your source tree. 527 528Buildman works entirely inside the base directory, here ../lcd9b, 529creating a working directory for each thread, and creating output 530directories for each commit and board. 531 532 533Suggested Workflow 534================== 535 536To run the build for real, take off the -n: 537 538$ ./tools/buildman/buildman -b <branch> 539 540Buildman will set up some working directories, and get started. After a 541minute or so it will settle down to a steady pace, with a display like this: 542 543Building 18 commits for 1059 boards (4 threads, 1 job per thread) 544 528 36 124 /19062 1:13:30 : SIMPC8313_SP 545 546This means that it is building 19062 board/commit combinations. So far it 547has managed to successfully build 528. Another 36 have built with warnings, 548and 124 more didn't build at all. Buildman expects to complete the process 549in around an hour and a quarter. Use this time to buy a faster computer. 550 551 552To find out how the build went, ask for a summary with -s. You can do this 553either before the build completes (presumably in another terminal) or 554afterwards. Let's work through an example of how this is used: 555 556$ ./tools/buildman/buildman -b lcd9b -s 557... 55801: Merge branch 'master' of git://git.denx.de/u-boot-arm 559 powerpc: + galaxy5200_LOWBOOT 56002: tegra: Use const for pinmux_config_pingroup/table() 56103: tegra: Add display support to funcmux 56204: tegra: fdt: Add pwm binding and node 56305: tegra: fdt: Add LCD definitions for Tegra 56406: tegra: Add support for PWM 56507: tegra: Add SOC support for display/lcd 56608: tegra: Add LCD driver 56709: tegra: Add LCD support to Nvidia boards 56810: arm: Add control over cachability of memory regions 56911: lcd: Add CONFIG_LCD_ALIGNMENT to select frame buffer alignment 57012: lcd: Add support for flushing LCD fb from dcache after update 571 arm: + lubbock 57213: tegra: Align LCD frame buffer to section boundary 57314: tegra: Support control of cache settings for LCD 57415: tegra: fdt: Add LCD definitions for Seaboard 57516: lcd: Add CONFIG_CONSOLE_SCROLL_LINES option to speed console 57617: tegra: Enable display/lcd support on Seaboard 57718: wip 578 579This shows which commits have succeeded and which have failed. In this case 580the build is still in progress so many boards are not built yet (use -u to 581see which ones). But still we can see a few failures. The galaxy5200_LOWBOOT 582never builds correctly. This could be a problem with our toolchain, or it 583could be a bug in the upstream. The good news is that we probably don't need 584to blame our commits. The bad news is that our commits are not tested on that 585board. 586 587Commit 12 broke lubbock. That's what the '+ lubbock' means. The failure 588is never fixed by a later commit, or you would see lubbock again, in green, 589without the +. 590 591To see the actual error: 592 593$ ./tools/buildman/buildman -b <branch> -se lubbock 594... 59512: lcd: Add support for flushing LCD fb from dcache after update 596 arm: + lubbock 597+common/libcommon.o: In function `lcd_sync': 598+/u-boot/lcd9b/.bm-work/00/common/lcd.c:120: undefined reference to `flush_dcache_range' 599+arm-none-linux-gnueabi-ld: BFD (Sourcery G++ Lite 2010q1-202) 2.19.51.20090709 assertion fail /scratch/julian/2010q1-release-linux-lite/obj/binutils-src-2010q1-202-arm-none-linux-gnueabi-i686-pc-linux-gnu/bfd/elf32-arm.c:12572 600+make: *** [/u-boot/lcd9b/.bm-work/00/build/u-boot] Error 139 60113: tegra: Align LCD frame buffer to section boundary 60214: tegra: Support control of cache settings for LCD 60315: tegra: fdt: Add LCD definitions for Seaboard 60416: lcd: Add CONFIG_CONSOLE_SCROLL_LINES option to speed console 605-/u-boot/lcd9b/.bm-work/00/common/lcd.c:120: undefined reference to `flush_dcache_range' 606+/u-boot/lcd9b/.bm-work/00/common/lcd.c:125: undefined reference to `flush_dcache_range' 60717: tegra: Enable display/lcd support on Seaboard 60818: wip 609 610So the problem is in lcd.c, due to missing cache operations. This information 611should be enough to work out what that commit is doing to break these 612boards. (In this case pxa did not have cache operations defined). 613 614If you see error lines marked with '-', that means that the errors were fixed 615by that commit. Sometimes commits can be in the wrong order, so that a 616breakage is introduced for a few commits and fixed by later commits. This 617shows up clearly with buildman. You can then reorder the commits and try 618again. 619 620At commit 16, the error moves: you can see that the old error at line 120 621is fixed, but there is a new one at line 126. This is probably only because 622we added some code and moved the broken line further down the file. 623 624If many boards have the same error, then -e will display the error only 625once. This makes the output as concise as possible. To see which boards have 626each error, use -l. So it is safe to omit the board name - you will not get 627lots of repeated output for every board. 628 629Buildman tries to distinguish warnings from errors, and shows warning lines 630separately with a 'w' prefix. 631 632The full build output in this case is available in: 633 634../lcd9b/12_of_18_gd92aff7_lcd--Add-support-for/lubbock/ 635 636 done: Indicates the build was done, and holds the return code from make. 637 This is 0 for a good build, typically 2 for a failure. 638 639 err: Output from stderr, if any. Errors and warnings appear here. 640 641 log: Output from stdout. Normally there isn't any since buildman runs 642 in silent mode. Use -V to force a verbose build (this passes V=1 643 to 'make') 644 645 toolchain: Shows information about the toolchain used for the build. 646 647 sizes: Shows image size information. 648 649It is possible to get the build binary output there also. Use the -k option 650for this. In that case you will also see some output files, like: 651 652 System.map toolchain u-boot u-boot.bin u-boot.map autoconf.mk 653 (also SPL versions u-boot-spl and u-boot-spl.bin if available) 654 655 656Checking Image Sizes 657==================== 658 659A key requirement for U-Boot is that you keep code/data size to a minimum. 660Where a new feature increases this noticeably it should normally be put 661behind a CONFIG flag so that boards can leave it disabled and keep the image 662size more or less the same with each new release. 663 664To check the impact of your commits on image size, use -S. For example: 665 666$ ./tools/buildman/buildman -b us-x86 -sS 667Summary of 10 commits for 1066 boards (4 threads, 1 job per thread) 66801: MAKEALL: add support for per architecture toolchains 66902: x86: Add function to get top of usable ram 670 x86: (for 1/3 boards) text -272.0 rodata +41.0 67103: x86: Add basic cache operations 67204: x86: Permit bootstage and timer data to be used prior to relocation 673 x86: (for 1/3 boards) data +16.0 67405: x86: Add an __end symbol to signal the end of the U-Boot binary 675 x86: (for 1/3 boards) text +76.0 67606: x86: Rearrange the output input to remove BSS 677 x86: (for 1/3 boards) bss -2140.0 67807: x86: Support relocation of FDT on start-up 679 x86: + coreboot-x86 68008: x86: Add error checking to x86 relocation code 68109: x86: Adjust link device tree include file 68210: x86: Enable CONFIG_OF_CONTROL on coreboot 683 684 685You can see that image size only changed on x86, which is good because this 686series is not supposed to change any other board. From commit 7 onwards the 687build fails so we don't get code size numbers. The numbers are fractional 688because they are an average of all boards for that architecture. The 689intention is to allow you to quickly find image size problems introduced by 690your commits. 691 692Note that the 'text' region and 'rodata' are split out. You should add the 693two together to get the total read-only size (reported as the first column 694in the output from binutil's 'size' utility). 695 696A useful option is --step which lets you skip some commits. For example 697--step 2 will show the image sizes for only every 2nd commit (so it will 698compare the image sizes of the 1st, 3rd, 5th... commits). You can also use 699--step 0 which will compare only the first and last commits. This is useful 700for an overview of how your entire series affects code size. It will build 701only the upstream commit and your final branch commit. 702 703You can also use -d to see a detailed size breakdown for each board. This 704list is sorted in order from largest growth to largest reduction. 705 706It is even possible to go a little further with the -B option (--bloat). This 707shows where U-Boot has bloated, breaking the size change down to the function 708level. Example output is below: 709 710$ ./tools/buildman/buildman -b us-mem4 -sSdB 711... 71219: Roll crc32 into hash infrastructure 713 arm: (for 10/10 boards) all -143.4 bss +1.2 data -4.8 rodata -48.2 text -91.6 714 paz00 : all +23 bss -4 rodata -29 text +56 715 u-boot: add: 1/0, grow: 3/-2 bytes: 168/-104 (64) 716 function old new delta 717 hash_command 80 160 +80 718 crc32_wd_buf - 56 +56 719 ext4fs_read_file 540 568 +28 720 insert_var_value_sub 688 692 +4 721 run_list_real 1996 1992 -4 722 do_mem_crc 168 68 -100 723 trimslice : all -9 bss +16 rodata -29 text +4 724 u-boot: add: 1/0, grow: 1/-3 bytes: 136/-124 (12) 725 function old new delta 726 hash_command 80 160 +80 727 crc32_wd_buf - 56 +56 728 ext4fs_iterate_dir 672 668 -4 729 ext4fs_read_file 568 548 -20 730 do_mem_crc 168 68 -100 731 whistler : all -9 bss +16 rodata -29 text +4 732 u-boot: add: 1/0, grow: 1/-3 bytes: 136/-124 (12) 733 function old new delta 734 hash_command 80 160 +80 735 crc32_wd_buf - 56 +56 736 ext4fs_iterate_dir 672 668 -4 737 ext4fs_read_file 568 548 -20 738 do_mem_crc 168 68 -100 739 seaboard : all -9 bss -28 rodata -29 text +48 740 u-boot: add: 1/0, grow: 3/-2 bytes: 160/-104 (56) 741 function old new delta 742 hash_command 80 160 +80 743 crc32_wd_buf - 56 +56 744 ext4fs_read_file 548 568 +20 745 run_list_real 1996 2000 +4 746 do_nandboot 760 756 -4 747 do_mem_crc 168 68 -100 748 colibri_t20 : all -9 rodata -29 text +20 749 u-boot: add: 1/0, grow: 2/-3 bytes: 140/-112 (28) 750 function old new delta 751 hash_command 80 160 +80 752 crc32_wd_buf - 56 +56 753 read_abs_bbt 204 208 +4 754 do_nandboot 760 756 -4 755 ext4fs_read_file 576 568 -8 756 do_mem_crc 168 68 -100 757 ventana : all -37 bss -12 rodata -29 text +4 758 u-boot: add: 1/0, grow: 1/-3 bytes: 136/-124 (12) 759 function old new delta 760 hash_command 80 160 +80 761 crc32_wd_buf - 56 +56 762 ext4fs_iterate_dir 672 668 -4 763 ext4fs_read_file 568 548 -20 764 do_mem_crc 168 68 -100 765 harmony : all -37 bss -16 rodata -29 text +8 766 u-boot: add: 1/0, grow: 2/-3 bytes: 140/-124 (16) 767 function old new delta 768 hash_command 80 160 +80 769 crc32_wd_buf - 56 +56 770 nand_write_oob_syndrome 428 432 +4 771 ext4fs_iterate_dir 672 668 -4 772 ext4fs_read_file 568 548 -20 773 do_mem_crc 168 68 -100 774 medcom-wide : all -417 bss +28 data -16 rodata -93 text -336 775 u-boot: add: 1/-1, grow: 1/-2 bytes: 88/-376 (-288) 776 function old new delta 777 crc32_wd_buf - 56 +56 778 do_fat_read_at 2872 2904 +32 779 hash_algo 16 - -16 780 do_mem_crc 168 68 -100 781 hash_command 420 160 -260 782 tec : all -449 bss -4 data -16 rodata -93 text -336 783 u-boot: add: 1/-1, grow: 1/-2 bytes: 88/-376 (-288) 784 function old new delta 785 crc32_wd_buf - 56 +56 786 do_fat_read_at 2872 2904 +32 787 hash_algo 16 - -16 788 do_mem_crc 168 68 -100 789 hash_command 420 160 -260 790 plutux : all -481 bss +16 data -16 rodata -93 text -388 791 u-boot: add: 1/-1, grow: 1/-3 bytes: 68/-408 (-340) 792 function old new delta 793 crc32_wd_buf - 56 +56 794 do_load_serial_bin 1688 1700 +12 795 hash_algo 16 - -16 796 do_fat_read_at 2904 2872 -32 797 do_mem_crc 168 68 -100 798 hash_command 420 160 -260 799 powerpc: (for 5/5 boards) all +37.4 data -3.2 rodata -41.8 text +82.4 800 MPC8610HPCD : all +55 rodata -29 text +84 801 u-boot: add: 1/0, grow: 0/-1 bytes: 176/-96 (80) 802 function old new delta 803 hash_command - 176 +176 804 do_mem_crc 184 88 -96 805 MPC8641HPCN : all +55 rodata -29 text +84 806 u-boot: add: 1/0, grow: 0/-1 bytes: 176/-96 (80) 807 function old new delta 808 hash_command - 176 +176 809 do_mem_crc 184 88 -96 810 MPC8641HPCN_36BIT: all +55 rodata -29 text +84 811 u-boot: add: 1/0, grow: 0/-1 bytes: 176/-96 (80) 812 function old new delta 813 hash_command - 176 +176 814 do_mem_crc 184 88 -96 815 sbc8641d : all +55 rodata -29 text +84 816 u-boot: add: 1/0, grow: 0/-1 bytes: 176/-96 (80) 817 function old new delta 818 hash_command - 176 +176 819 do_mem_crc 184 88 -96 820 xpedite517x : all -33 data -16 rodata -93 text +76 821 u-boot: add: 1/-1, grow: 0/-1 bytes: 176/-112 (64) 822 function old new delta 823 hash_command - 176 +176 824 hash_algo 16 - -16 825 do_mem_crc 184 88 -96 826... 827 828 829This shows that commit 19 has reduced codesize for arm slightly and increased 830it for powerpc. This increase was offset in by reductions in rodata and 831data/bss. 832 833Shown below the summary lines are the sizes for each board. Below each board 834are the sizes for each function. This information starts with: 835 836 add - number of functions added / removed 837 grow - number of functions which grew / shrunk 838 bytes - number of bytes of code added to / removed from all functions, 839 plus the total byte change in brackets 840 841The change seems to be that hash_command() has increased by more than the 842do_mem_crc() function has decreased. The function sizes typically add up to 843roughly the text area size, but note that every read-only section except 844rodata is included in 'text', so the function total does not exactly 845correspond. 846 847It is common when refactoring code for the rodata to decrease as the text size 848increases, and vice versa. 849 850 851The .buildman file 852================== 853 854The .buildman file provides information about the available toolchains and 855also allows build flags to be passed to 'make'. It consists of several 856sections, with the section name in square brackets. Within each section are 857a set of (tag, value) pairs. 858 859'[toolchain]' section 860 861 This lists the available toolchains. The tag here doesn't matter, but 862 make sure it is unique. The value is the path to the toolchain. Buildman 863 will look in that path for a file ending in 'gcc'. It will then execute 864 it to check that it is a C compiler, passing only the --version flag to 865 it. If the return code is 0, buildman assumes that it is a valid C 866 compiler. It uses the first part of the name as the architecture and 867 strips off the last part when setting the CROSS_COMPILE environment 868 variable (parts are delimited with a hyphen). 869 870 For example powerpc-linux-gcc will be noted as a toolchain for 'powerpc' 871 and CROSS_COMPILE will be set to powerpc-linux- when using it. 872 873'[toolchain-alias]' section 874 875 This converts toolchain architecture names to U-Boot names. For example, 876 if an x86 toolchains is called i386-linux-gcc it will not normally be 877 used for architecture 'x86'. Adding 'x86: i386 x86_64' to this section 878 will tell buildman that the i386 and x86_64 toolchains can be used for 879 the x86 architecture. 880 881'[make-flags]' section 882 883 U-Boot's build system supports a few flags (such as BUILD_TAG) which 884 affect the build product. These flags can be specified in the buildman 885 settings file. They can also be useful when building U-Boot against other 886 open source software. 887 888 [make-flags] 889 at91-boards=ENABLE_AT91_TEST=1 890 snapper9260=${at91-boards} BUILD_TAG=442 891 snapper9g45=${at91-boards} BUILD_TAG=443 892 893 This will use 'make ENABLE_AT91_TEST=1 BUILD_TAG=442' for snapper9260 894 and 'make ENABLE_AT91_TEST=1 BUILD_TAG=443' for snapper9g45. A special 895 variable ${target} is available to access the target name (snapper9260 896 and snapper9g20 in this case). Variables are resolved recursively. Note 897 that variables can only contain the characters A-Z, a-z, 0-9, hyphen (-) 898 and underscore (_). 899 900 It is expected that any variables added are dealt with in U-Boot's 901 config.mk file and documented in the README. 902 903 Note that you can pass ad-hoc options to the build using environment 904 variables, for example: 905 906 SOME_OPTION=1234 ./tools/buildman/buildman my_board 907 908 909Quick Sanity Check 910================== 911 912If you have made changes and want to do a quick sanity check of the 913currently checked-out source, run buildman without the -b flag. This will 914build the selected boards and display build status as it runs (i.e. -v is 915enabled automatically). Use -e to see errors/warnings as well. 916 917 918Building Ranges 919=============== 920 921You can build a range of commits by specifying a range instead of a branch 922when using the -b flag. For example: 923 924 upstream/master..us-buildman 925 926will build commits in us-buildman that are not in upstream/master. 927 928 929Building Faster 930=============== 931 932By default, buildman executes 'make mrproper' prior to building the first 933commit for each board. This causes everything to be built from scratch. If you 934trust the build system's incremental build capabilities, you can pass the -I 935flag to skip the 'make mproper' invocation, which will reduce the amount of 936work 'make' does, and hence speed up the build. This flag will speed up any 937buildman invocation, since it reduces the amount of work done on any build. 938 939One possible application of buildman is as part of a continual edit, build, 940edit, build, ... cycle; repeatedly applying buildman to the same change or 941series of changes while making small incremental modifications to the source 942each time. This provides quick feedback regarding the correctness of recent 943modifications. In this scenario, buildman's default choice of build directory 944causes more build work to be performed than strictly necessary. 945 946By default, each buildman thread uses a single directory for all builds. When a 947thread builds multiple boards, the configuration built in this directory will 948cycle through various different configurations, one per board built by the 949thread. Variations in the configuration will force a rebuild of affected source 950files when a thread switches between boards. Ideally, such buildman-induced 951rebuilds would not happen, thus allowing the build to operate as efficiently as 952the build system and source changes allow. buildman's -P flag may be used to 953enable this; -P causes each board to be built in a separate (board-specific) 954directory, thus avoiding any buildman-induced configuration changes in any 955build directory. 956 957U-Boot's build system embeds information such as a build timestamp into the 958final binary. This information varies each time U-Boot is built. This causes 959various files to be rebuilt even if no source changes are made, which in turn 960requires that the final U-Boot binary be re-linked. This unnecessary work can 961be avoided by turning off the timestamp feature. This can be achieved by 962setting the SOURCE_DATE_EPOCH environment variable to 0. 963 964Combining all of these options together yields the command-line shown below. 965This will provide the quickest possible feedback regarding the current content 966of the source tree, thus allowing rapid tested evolution of the code. 967 968 SOURCE_DATE_EPOCH=0 ./tools/buildman/buildman -I -P tegra 969 970 971Checking configuration 972====================== 973 974A common requirement when converting CONFIG options to Kconfig is to check 975that the effective configuration has not changed due to the conversion. 976Buildman supports this with the -K option, used after a build. This shows 977differences in effective configuration between one commit and the next. 978 979For example: 980 981 $ buildman -b kc4 -sK 982 ... 983 43: Convert CONFIG_SPL_USBETH_SUPPORT to Kconfig 984 arm: 985 + u-boot.cfg: CONFIG_SPL_ENV_SUPPORT=1 CONFIG_SPL_NET_SUPPORT=1 986 + u-boot-spl.cfg: CONFIG_SPL_MMC_SUPPORT=1 CONFIG_SPL_NAND_SUPPORT=1 987 + all: CONFIG_SPL_ENV_SUPPORT=1 CONFIG_SPL_MMC_SUPPORT=1 CONFIG_SPL_NAND_SUPPORT=1 CONFIG_SPL_NET_SUPPORT=1 988 am335x_evm_usbspl : 989 + u-boot.cfg: CONFIG_SPL_ENV_SUPPORT=1 CONFIG_SPL_NET_SUPPORT=1 990 + u-boot-spl.cfg: CONFIG_SPL_MMC_SUPPORT=1 CONFIG_SPL_NAND_SUPPORT=1 991 + all: CONFIG_SPL_ENV_SUPPORT=1 CONFIG_SPL_MMC_SUPPORT=1 CONFIG_SPL_NAND_SUPPORT=1 CONFIG_SPL_NET_SUPPORT=1 992 44: Convert CONFIG_SPL_USB_HOST_SUPPORT to Kconfig 993 ... 994 995This shows that commit 44 enabled three new options for the board 996am335x_evm_usbspl which were not enabled in commit 43. There is also a 997summary for 'arm' showing all the changes detected for that architecture. 998In this case there is only one board with changes, so 'arm' output is the 999same as 'am335x_evm_usbspl'/ 1000 1001The -K option uses the u-boot.cfg, spl/u-boot-spl.cfg and tpl/u-boot-tpl.cfg 1002files which are produced by a build. If all you want is to check the 1003configuration you can in fact avoid doing a full build, using -D. This tells 1004buildman to configuration U-Boot and create the .cfg files, but not actually 1005build the source. This is 5-10 times faster than doing a full build. 1006 1007By default buildman considers the follow two configuration methods 1008equivalent: 1009 1010 #define CONFIG_SOME_OPTION 1011 1012 CONFIG_SOME_OPTION=y 1013 1014The former would appear in a header filer and the latter in a defconfig 1015file. The achieve this, buildman considers 'y' to be '1' in configuration 1016variables. This avoids lots of useless output when converting a CONFIG 1017option to Kconfig. To disable this behaviour, use --squash-config-y. 1018 1019 1020Other options 1021============= 1022 1023Buildman has various other command line options. Try --help to see them. 1024 1025When doing builds, Buildman's return code will reflect the overall result: 1026 1027 0 (success) No errors or warnings found 1028 128 Errors found 1029 129 Warnings found 1030 1031 1032How to change from MAKEALL 1033========================== 1034 1035Buildman includes most of the features of MAKEALL and is generally faster 1036and easier to use. In particular it builds entire branches: if a particular 1037commit introduces an error in a particular board, buildman can easily show 1038you this, even if a later commit fixes that error. 1039 1040The reasons to deprecate MAKEALL are: 1041- We don't want to maintain two build systems 1042- Buildman is typically faster 1043- Buildman has a lot more features 1044 1045But still, many people will be sad to lose MAKEALL. If you are used to 1046MAKEALL, here are a few pointers. 1047 1048First you need to set up your tool chains - see the 'Setting up' section 1049for details. Once you have your required toolchain(s) detected then you are 1050ready to go. 1051 1052To build the current source tree, run buildman without a -b flag: 1053 1054 ./tools/buildman/buildman <list of things to build> 1055 1056This will build the current source tree for the given boards and display 1057the results and errors. 1058 1059However buildman usually works on entire branches, and for that you must 1060specify a board flag: 1061 1062 ./tools/buildman/buildman -b <branch_name> <list of things to build> 1063 1064followed by (afterwards, or perhaps concurrently in another terminal): 1065 1066 ./tools/buildman/buildman -b <branch_name> -s <list of things to build> 1067 1068to see the results of the build. Rather than showing you all the output, 1069buildman just shows a summary, with red indicating that a commit introduced 1070an error and green indicating that a commit fixed an error. Use the -e 1071flag to see the full errors and -l to see which boards caused which errors. 1072 1073If you really want to see build results as they happen, use -v when doing a 1074build (and -e to see the errors/warnings too). 1075 1076You don't need to stick around on that branch while buildman is running. It 1077checks out its own copy of the source code, so you can change branches, 1078add commits, etc. without affecting the build in progress. 1079 1080The <list of things to build> can include board names, architectures or the 1081like. There are no flags to disambiguate since ambiguities are rare. Using 1082the examples from MAKEALL: 1083 1084Examples: 1085 - build all Power Architecture boards: 1086 MAKEALL -a powerpc 1087 MAKEALL --arch powerpc 1088 MAKEALL powerpc 1089 ** buildman -b <branch> powerpc 1090 - build all PowerPC boards manufactured by vendor "esd": 1091 MAKEALL -a powerpc -v esd 1092 ** buildman -b <branch> esd 1093 - build all PowerPC boards manufactured either by "keymile" or "siemens": 1094 MAKEALL -a powerpc -v keymile -v siemens 1095 ** buildman -b <branch> keymile siemens 1096 - build all Freescale boards with MPC83xx CPUs, plus all 4xx boards: 1097 MAKEALL -c mpc83xx -v freescale 4xx 1098 ** buildman -b <branch> mpc83xx freescale 4xx 1099 1100Buildman automatically tries to use all the CPUs in your machine. If you 1101are building a lot of boards it will use one thread for every CPU core 1102it detects in your machine. This is like MAKEALL's BUILD_NBUILDS option. 1103You can use the -T flag to change the number of threads. If you are only 1104building a few boards, buildman will automatically run make with the -j 1105flag to increase the number of concurrent make tasks. It isn't normally 1106that helpful to fiddle with this option, but if you use the BUILD_NCPUS 1107option in MAKEALL then -j is the equivalent in buildman. 1108 1109Buildman puts its output in ../<branch_name> by default but you can change 1110this with the -o option. Buildman normally does out-of-tree builds: use -i 1111to disable that if you really want to. But be careful that once you have 1112used -i you pollute buildman's copies of the source tree, and you will need 1113to remove the build directory (normally ../<branch_name>) to run buildman 1114in normal mode (without -i). 1115 1116Buildman doesn't keep the output result normally, but use the -k option to 1117do this. 1118 1119Please read 'Theory of Operation' a few times as it will make a lot of 1120things clearer. 1121 1122Some options you might like are: 1123 1124 -B shows which functions are growing/shrinking in which commit - great 1125 for finding code bloat. 1126 -S shows image sizes for each commit (just an overall summary) 1127 -u shows boards that you haven't built yet 1128 --step 0 will build just the upstream commit and the last commit of your 1129 branch. This is often a quick sanity check that your branch doesn't 1130 break anything. But note this does not check bisectability! 1131 1132 1133TODO 1134==== 1135 1136This has mostly be written in my spare time as a response to my difficulties 1137in testing large series of patches. Apart from tidying up there is quite a 1138bit of scope for improvement. Things like better error diffs and easier 1139access to log files. Also it would be nice if buildman could 'hunt' for 1140problems, perhaps by building a few boards for each arch, or checking 1141commits for changed files and building only boards which use those files. 1142 1143A specific problem to fix is that Ctrl-C does not exit buildman cleanly when 1144multiple builder threads are active. 1145 1146Credits 1147======= 1148 1149Thanks to Grant Grundler <grundler@chromium.org> for his ideas for improving 1150the build speed by building all commits for a board instead of the other 1151way around. 1152 1153 1154Simon Glass 1155sjg@chromium.org 1156Halloween 2012 1157Updated 12-12-12 1158Updated 23-02-13 1159