1.. SPDX-License-Identifier: CC-BY-SA-2.0-UK 2.. Set default pygment highlighting to 'shell' for this document 3.. highlight:: shell 4 5**************************** 6Setting Up and Using Toaster 7**************************** 8 9Starting Toaster for Local Development 10====================================== 11 12Once you have set up the Yocto Project and installed the Toaster system 13dependencies as described in the ":ref:`toaster-manual/start:Preparing to Use 14Toaster`" chapter, you are ready to start 15Toaster. 16 17Navigate to the root of your 18:term:`Source Directory` (e.g. ``poky``):: 19 20 $ cd poky 21 22Once in that directory, source the build environment script:: 23 24 $ source oe-init-build-env 25 26Next, from the build directory (e.g. 27``poky/build``), start Toaster using this command:: 28 29 $ source toaster start 30 31You can now run your builds from the command line, or with Toaster 32as explained in section 33":ref:`toaster-manual/setup-and-use:using the toaster web interface`". 34 35To access the Toaster web interface, open your favorite browser and 36enter the following:: 37 38 http://127.0.0.1:8000 39 40Setting a Different Port 41======================== 42 43By default, Toaster starts on port 8000. You can use the ``WEBPORT`` 44parameter to set a different port. For example, the following command 45sets the port to "8400":: 46 47 $ source toaster start webport=8400 48 49Setting Up Toaster Without a Web Server 50======================================= 51 52You can start a Toaster environment without starting its web server. 53This is useful for the following: 54 55- Capturing a command-line build's statistics into the Toaster database 56 for examination later. 57 58- Capturing a command-line build's statistics when the Toaster server 59 is already running. 60 61- Having one instance of the Toaster web server track and capture 62 multiple command-line builds, where each build is started in its own 63 "noweb" Toaster environment. 64 65The following commands show how to start a Toaster environment without 66starting its web server, perform BitBake operations, and then shut down 67the Toaster environment. Once the build is complete, you can close the 68Toaster environment. Before closing the environment, however, you should 69allow a few minutes to ensure the complete transfer of its BitBake build 70statistics to the Toaster database. If you have a separate Toaster web 71server instance running, you can watch this command-line build's 72progress and examine the results as soon as they are posted:: 73 74 $ source toaster start noweb 75 $ bitbake target 76 $ source toaster stop 77 78Setting Up Toaster Without a Build Server 79========================================= 80 81You can start a Toaster environment with the "New Projects" feature 82disabled. Doing so is useful for the following: 83 84- Sharing your build results over the web server while blocking others 85 from starting builds on your host. 86 87- Allowing only local command-line builds to be captured into the 88 Toaster database. 89 90Use the following command to set up Toaster without a build server:: 91 92 $ source toaster start nobuild webport=port 93 94Setting up External Access 95========================== 96 97By default, Toaster binds to the loop back address (i.e. ``localhost``), 98which does not allow access from external hosts. To allow external 99access, use the ``WEBPORT`` parameter to open an address that connects 100to the network, specifically the IP address that your NIC uses to 101connect to the network. You can also bind to all IP addresses the 102computer supports by using the shortcut "0.0.0.0:port". 103 104The following example binds to all IP addresses on the host:: 105 106 $ source toaster start webport=0.0.0.0:8400 107 108This example binds to a specific IP address on the host's NIC:: 109 110 $ source toaster start webport=192.168.1.1:8400 111 112The Directory for Cloning Layers 113================================ 114 115Toaster creates a ``_toaster_clones`` directory inside your Source 116Directory (i.e. ``poky``) to clone any layers needed for your builds. 117 118Alternatively, if you would like all of your Toaster related files and 119directories to be in a particular location other than the default, you 120can set the ``TOASTER_DIR`` environment variable, which takes precedence 121over your current working directory. Setting this environment variable 122causes Toaster to create and use ``$TOASTER_DIR./_toaster_clones``. 123 124The Build Directory 125=================== 126 127Toaster creates a build directory within your Source Directory (e.g. 128``poky``) to execute the builds. 129 130Alternatively, if you would like all of your Toaster related files and 131directories to be in a particular location, you can set the 132``TOASTER_DIR`` environment variable, which takes precedence over your 133current working directory. Setting this environment variable causes 134Toaster to use ``$TOASTER_DIR/build`` as the build directory. 135 136Creating a Django Superuser 137=========================== 138 139Toaster is built on the `Django 140framework <https://www.djangoproject.com/>`__. Django provides an 141administration interface you can use to edit Toaster configuration 142parameters. 143 144To access the Django administration interface, you must create a 145superuser by following these steps: 146 147#. If you used ``pip3``, which is recommended, to set up the Toaster 148 system dependencies, you need be sure the local user path is in your 149 ``PATH`` list. To append the pip3 local user path, use the following 150 command:: 151 152 $ export PATH=$PATH:$HOME/.local/bin 153 154#. From the directory containing the Toaster database, which by default 155 is the :term:`Build Directory`, 156 invoke the ``createsuperuser`` command from ``manage.py``:: 157 158 $ cd poky/build 159 $ ../bitbake/lib/toaster/manage.py createsuperuser 160 161#. Django prompts you for the username, which you need to provide. 162 163#. Django prompts you for an email address, which is optional. 164 165#. Django prompts you for a password, which you must provide. 166 167#. Django prompts you to re-enter your password for verification. 168 169After completing these steps, the following confirmation message 170appears:: 171 172 Superuser created successfully. 173 174Creating a superuser allows you to access the Django administration 175interface through a browser. The URL for this interface is the same as 176the URL used for the Toaster instance with "/admin" on the end. For 177example, if you are running Toaster locally, use the following URL:: 178 179 http://127.0.0.1:8000/admin 180 181You can use the Django administration interface to set Toaster configuration 182parameters such as the build directory, layer sources, default variable 183values, and BitBake versions. 184 185Setting Up a Production Instance of Toaster 186=========================================== 187 188You can use a production instance of Toaster to share the Toaster 189instance with remote users, multiple users, or both. The production 190instance is also the setup that can handle heavier loads on the web 191service. Use the instructions in the following sections to set up 192Toaster to run builds through the Toaster web interface. 193 194Requirements 195------------ 196 197Be sure you meet the following requirements: 198 199.. note:: 200 201 You must comply with all Apache, ``mod-wsgi``, and Mysql requirements. 202 203- Have all the build requirements as described in the ":ref:`toaster-manual/start:Preparing to 204 Use Toaster`" chapter. 205 206- Have an Apache webserver. 207 208- Have ``mod-wsgi`` for the Apache webserver. 209 210- Use the Mysql database server. 211 212- If you are using Ubuntu, run the following:: 213 214 $ sudo apt install apache2 libapache2-mod-wsgi-py3 mysql-server python3-pip libmysqlclient-dev 215 216- If you are using Fedora or a RedHat distribution, run the 217 following:: 218 219 $ sudo dnf install httpd python3-mod_wsgi python3-pip mariadb-server mariadb-devel python3-devel 220 221- If you are using openSUSE, run the following:: 222 223 $ sudo zypper install apache2 apache2-mod_wsgi-python3 python3-pip mariadb mariadb-client python3-devel 224 225Installation 226------------ 227 228Perform the following steps to install Toaster: 229 230#. Create toaster user and set its home directory to 231 ``/var/www/toaster``:: 232 233 $ sudo /usr/sbin/useradd toaster -md /var/www/toaster -s /bin/false 234 $ sudo su - toaster -s /bin/bash 235 236#. Checkout a copy of ``poky`` into the web server directory. You will 237 be using ``/var/www/toaster``:: 238 239 $ git clone git://git.yoctoproject.org/poky 240 $ git checkout &DISTRO_NAME_NO_CAP; 241 242#. Install Toaster dependencies using the ``--user`` flag which keeps the 243 Python packages isolated from your system-provided packages:: 244 245 $ cd /var/www/toaster/ 246 $ pip3 install --user -r ./poky/bitbake/toaster-requirements.txt 247 $ pip3 install --user mysqlclient 248 249 .. note:: 250 251 Isolating these packages is not required but is recommended. 252 Alternatively, you can use your operating system's package 253 manager to install the packages. 254 255#. Configure Toaster by editing 256 ``/var/www/toaster/poky/bitbake/lib/toaster/toastermain/settings.py`` 257 as follows: 258 259 - Edit the 260 `DATABASES <https://docs.djangoproject.com/en/2.2/ref/settings/#databases>`__ 261 settings: 262 263 .. code-block:: python 264 265 DATABASES = { 266 'default': { 267 'ENGINE': 'django.db.backends.mysql', 268 'NAME': 'toaster_data', 269 'USER': 'toaster', 270 'PASSWORD': 'yourpasswordhere', 271 'HOST': 'localhost', 272 'PORT': '3306', 273 } 274 } 275 276 - Edit the 277 `SECRET_KEY <https://docs.djangoproject.com/en/2.2/ref/settings/#std:setting-SECRET_KEY>`__: 278 279 .. code-block:: python 280 281 SECRET_KEY = 'your_secret_key' 282 283 - Edit the 284 `STATIC_ROOT <https://docs.djangoproject.com/en/2.2/ref/settings/#std:setting-STATIC_ROOT>`__: 285 286 .. code-block:: python 287 288 STATIC_ROOT = '/var/www/toaster/static_files/' 289 290#. Add the database and user to the ``mysql`` server defined earlier:: 291 292 $ mysql -u root -p 293 mysql> CREATE DATABASE toaster_data; 294 mysql> CREATE USER 'toaster'@'localhost' identified by 'yourpasswordhere'; 295 mysql> GRANT all on toaster_data.\* to 'toaster'@'localhost'; 296 mysql> quit 297 298#. Get Toaster to create the database schema, default data, and gather 299 the statically-served files:: 300 301 $ cd /var/www/toaster/poky/ 302 $ ./bitbake/lib/toaster/manage.py migrate 303 $ TOASTER_DIR=`pwd\` TEMPLATECONF='poky' \ 304 ./bitbake/lib/toaster/manage.py checksettings 305 $ ./bitbake/lib/toaster/manage.py collectstatic 306 307 308 In the previous 309 example, from the ``poky`` directory, the ``migrate`` command 310 ensures the database schema changes have propagated correctly (i.e. 311 migrations). The next line sets the Toaster root directory 312 ``TOASTER_DIR`` and the location of the Toaster configuration file 313 ``TOASTER_CONF``, which is relative to ``TOASTER_DIR``. The 314 ``TEMPLATECONF`` value reflects the contents of 315 ``poky/.templateconf``, and by default, should include the string 316 "poky". For more information on the Toaster configuration file, see 317 the ":ref:`toaster-manual/reference:Configuring Toaster`" section. 318 319 This line also runs the ``checksettings`` command, which configures 320 the location of the Toaster :term:`Build Directory`. 321 The Toaster 322 root directory ``TOASTER_DIR`` determines where the Toaster build 323 directory is created on the file system. In the example above, 324 ``TOASTER_DIR`` is set as follows:: 325 326 /var/www/toaster/poky 327 328 329 This setting causes the Toaster build directory to be:: 330 331 /var/www/toaster/poky/build 332 333 Finally, the ``collectstatic`` command is a Django framework command 334 that collects all the statically served files into a designated 335 directory to be served up by the Apache web server as defined by 336 ``STATIC_ROOT``. 337 338#. Test and/or use the Mysql integration with Toaster's Django web 339 server. At this point, you can start up the normal Toaster Django 340 web server with the Toaster database in Mysql. You can use this web 341 server to confirm that the database migration and data population 342 from the Layer Index is complete. 343 344 To start the default Toaster Django web server with the Toaster 345 database now in Mysql, use the standard start commands:: 346 347 $ source oe-init-build-env 348 $ source toaster start 349 350 Additionally, if Django is sufficient for your requirements, you can use 351 it for your release system and migrate later to Apache as your 352 requirements change. 353 354#. Add an Apache configuration file for Toaster to your Apache web 355 server's configuration directory. If you are using Ubuntu or Debian, 356 put the file here:: 357 358 /etc/apache2/conf-available/toaster.conf 359 360 361 If you are using Fedora or RedHat, put it here:: 362 363 /etc/httpd/conf.d/toaster.conf 364 365 If you are using openSUSE, put it here:: 366 367 /etc/apache2/conf.d/toaster.conf 368 369 Following is a sample Apache configuration for Toaster you can follow: 370 371 .. code-block:: apache 372 373 Alias /static /var/www/toaster/static_files 374 <Directory /var/www/toaster/static_files> 375 <IfModule mod_access_compat.c> 376 Order allow,deny 377 Allow from all 378 </IfModule> 379 <IfModule !mod_access_compat.c> 380 Require all granted 381 </IfModule> 382 </Directory> 383 384 <Directory /var/www/toaster/poky/bitbake/lib/toaster/toastermain> 385 <Files "wsgi.py"> 386 Require all granted 387 </Files> 388 </Directory> 389 390 WSGIDaemonProcess toaster_wsgi python-path=/var/www/toaster/poky/bitbake/lib/toaster:/var/www/toaster/.local/lib/python3.4/site-packages 391 WSGIScriptAlias / "/var/www/toaster/poky/bitbake/lib/toaster/toastermain/wsgi.py" 392 <Location /> 393 WSGIProcessGroup toaster_wsgi 394 </Location> 395 396 397 If you are using Ubuntu or Debian, you will need to enable the config and 398 module for Apache:: 399 400 $ sudo a2enmod wsgi 401 $ sudo a2enconf toaster 402 $ chmod +x bitbake/lib/toaster/toastermain/wsgi.py 403 404 Finally, restart Apache to make sure all new configuration is loaded. For Ubuntu, 405 Debian, and openSUSE use:: 406 407 $ sudo service apache2 restart 408 409 For Fedora and RedHat use:: 410 411 $ sudo service httpd restart 412 413#. Prepare the systemd service to run Toaster builds. Here is a sample 414 configuration file for the service: 415 416 .. code-block:: ini 417 418 [Unit] 419 Description=Toaster runbuilds 420 421 [Service] 422 Type=forking User=toaster 423 ExecStart=/usr/bin/screen -d -m -S runbuilds /var/www/toaster/poky/bitbake/lib/toaster/runbuilds-service.sh start 424 ExecStop=/usr/bin/screen -S runbuilds -X quit 425 WorkingDirectory=/var/www/toaster/poky 426 427 [Install] 428 WantedBy=multi-user.target 429 430 431 Prepare the ``runbuilds-service.sh`` script that you need to place in the 432 ``/var/www/toaster/poky/bitbake/lib/toaster/`` directory by setting 433 up executable permissions:: 434 435 #!/bin/bash 436 437 #export http_proxy=http://proxy.host.com:8080 438 #export https_proxy=http://proxy.host.com:8080 439 #export GIT_PROXY_COMMAND=$HOME/bin/gitproxy 440 cd poky/ 441 source ./oe-init-build-env build 442 source ../bitbake/bin/toaster $1 noweb 443 [ "$1" == 'start' ] && /bin/bash 444 445#. Run the service:: 446 447 $ sudo service runbuilds start 448 449 Since the service is running in a detached screen session, you can 450 attach to it using this command:: 451 452 $ sudo su - toaster 453 $ screen -rS runbuilds 454 455 You can detach from the service again using "Ctrl-a" followed by "d" key 456 combination. 457 458You can now open up a browser and start using Toaster. 459 460Using the Toaster Web Interface 461=============================== 462 463The Toaster web interface allows you to do the following: 464 465- Browse published layers in the :oe_layerindex:`OpenEmbedded Layer Index <>` 466 that are available for your selected version of the build system. 467 468- Import your own layers for building. 469 470- Add and remove layers from your configuration. 471 472- Set configuration variables. 473 474- Select a target or multiple targets to build. 475 476- Start your builds. 477 478- See what was built (recipes and packages) and what packages were 479 installed into your final image. 480 481- Browse the directory structure of your image. 482 483- See the value of all variables in your build configuration, and which 484 files set each value. 485 486- Examine error, warning and trace messages to aid in debugging. 487 488- See information about the BitBake tasks executed and reused during 489 your build, including those that used shared state. 490 491- See dependency relationships between recipes, packages and tasks. 492 493- See performance information such as build time, task time, CPU usage, 494 and disk I/O. 495 496Toaster Web Interface Videos 497---------------------------- 498 499Following are several videos that show how to use the Toaster GUI: 500 501- *Build Configuration:* This 502 `video <https://www.youtube.com/watch?v=qYgDZ8YzV6w>`__ overviews and 503 demonstrates build configuration for Toaster. 504 505- *Build Custom Layers:* This 506 `video <https://www.youtube.com/watch?v=QJzaE_XjX5c>`__ shows you how 507 to build custom layers that are used with Toaster. 508 509- *Toaster Homepage and Table Controls:* This 510 `video <https://www.youtube.com/watch?v=QEARDnrR1Xw>`__ goes over the 511 Toaster entry page, and provides an overview of the data manipulation 512 capabilities of Toaster, which include search, sorting and filtering 513 by different criteria. 514 515- *Build Dashboard:* This 516 `video <https://www.youtube.com/watch?v=KKqHYcnp2gE>`__ shows you the 517 build dashboard, a page providing an overview of the information 518 available for a selected build. 519 520- *Image Information:* This 521 `video <https://www.youtube.com/watch?v=XqYGFsmA0Rw>`__ walks through 522 the information Toaster provides about images: packages installed and 523 root file system. 524 525- *Configuration:* This 526 `video <https://www.youtube.com/watch?v=UW-j-T2TzIg>`__ provides 527 Toaster build configuration information. 528 529- *Tasks:* This `video <https://www.youtube.com/watch?v=D4-9vGSxQtw>`__ 530 shows the information Toaster provides about the tasks run by the 531 build system. 532 533- *Recipes and Packages Built:* This 534 `video <https://www.youtube.com/watch?v=x-6dx4huNnw>`__ shows the 535 information Toaster provides about recipes and packages built. 536 537- *Performance Data:* This 538 `video <https://www.youtube.com/watch?v=qWGMrJoqusQ>`__ shows the 539 build performance data provided by Toaster. 540 541Additional Information About the Local Yocto Project Release 542------------------------------------------------------------ 543 544This section only applies if you have set up Toaster for local 545development, as explained in the 546":ref:`toaster-manual/setup-and-use:starting toaster for local development`" 547section. 548 549When you create a project in Toaster, you will be asked to provide a 550name and to select a Yocto Project release. One of the release options 551you will find is called "Local Yocto Project". 552 553.. image:: figures/new-project.png 554 :align: center 555 :scale: 75% 556 557When you select the "Local Yocto Project" release, Toaster will run your 558builds using the local Yocto Project clone you have in your computer: 559the same clone you are using to run Toaster. Unless you manually update 560this clone, your builds will always use the same Git revision. 561 562If you select any of the other release options, Toaster will fetch the 563tip of your selected release from the upstream :yocto_git:`Yocto Project 564repository <>` every time you run a build. 565Fetching this tip effectively means that if your selected release is 566updated upstream, the Git revision you are using for your builds will 567change. If you are doing development locally, you might not want this 568change to happen. In that case, the "Local Yocto Project" release might 569be the right choice. 570 571However, the "Local Yocto Project" release will not provide you with any 572compatible layers, other than the three core layers that come with the 573Yocto Project: 574 575- :oe_layer:`openembedded-core </openembedded-core>` 576 577- :oe_layer:`meta-poky </meta-poky>` 578 579- :oe_layer:`meta-yocto-bsp </meta-yocto-bsp>` 580 581.. image:: figures/compatible-layers.png 582 :align: center 583 :scale: 75% 584 585If you want to build any other layers, you will need to manually import 586them into your Toaster project, using the "Import layer" page. 587 588.. image:: figures/import-layer.png 589 :align: center 590 :scale: 75% 591 592Building a Specific Recipe Given Multiple Versions 593-------------------------------------------------- 594 595Occasionally, a layer might provide more than one version of the same 596recipe. For example, the ``openembedded-core`` layer provides two 597versions of the ``bash`` recipe (i.e. 3.2.48 and 4.3.30-r0) and two 598versions of the ``which`` recipe (i.e. 2.21 and 2.18). The following 599figure shows this exact scenario: 600 601.. image:: figures/bash-oecore.png 602 :align: center 603 :scale: 75% 604 605By default, the OpenEmbedded build system builds one of the two recipes. 606For the ``bash`` case, version 4.3.30-r0 is built by default. 607Unfortunately, Toaster as it exists, is not able to override the default 608recipe version. If you would like to build bash 3.2.48, you need to set 609the 610:term:`PREFERRED_VERSION` 611variable. You can do so from Toaster, using the "Add variable" form, 612which is available in the "BitBake variables" page of the project 613configuration section as shown in the following screen: 614 615.. image:: figures/add-variable.png 616 :align: center 617 :scale: 75% 618 619To specify ``bash`` 3.2.48 as the version to build, enter 620"PREFERRED_VERSION_bash" in the "Variable" field, and "3.2.48" in the 621"Value" field. Next, click the "Add variable" button: 622 623.. image:: figures/set-variable.png 624 :align: center 625 :scale: 75% 626 627After clicking the "Add variable" button, the settings for 628:term:`PREFERRED_VERSION` are added to the bottom of the BitBake variables 629list. With these settings, the OpenEmbedded build system builds the 630desired version of the recipe rather than the default version: 631 632.. image:: figures/variable-added.png 633 :align: center 634 :scale: 75% 635