xref: /openbmc/qemu/docs/devel/build-system.rst (revision ce799a04)
1==================================
2The QEMU build system architecture
3==================================
4
5This document aims to help developers understand the architecture of the
6QEMU build system. As with projects using GNU autotools, the QEMU build
7system has two stages; first the developer runs the "configure" script
8to determine the local build environment characteristics, then they run
9"make" to build the project.  This is about where the similarities with
10GNU autotools end, so try to forget what you know about them.
11
12The two general ways to perform a build are as follows:
13
14 - build artifacts outside of QEMU source tree entirely::
15
16     cd ../
17     mkdir build
18     cd build
19     ../qemu/configure
20     make
21
22 - build artifacts in a subdir of QEMU source tree::
23
24     mkdir build
25     cd build
26     ../configure
27     make
28
29Most of the actual build process uses Meson under the hood, therefore
30build artifacts cannot be placed in the source tree itself.
31
32
33Stage 1: configure
34==================
35
36The configure script has five tasks:
37
38 - detect the host architecture
39
40 - list the targets for which to build emulators; the list of
41   targets also affects which firmware binaries and tests to build
42
43 - find the compilers (native and cross) used to build executables,
44   firmware and tests.  The results are written as either Makefile
45   fragments (``config-host.mak``) or a Meson machine file
46   (``config-meson.cross``)
47
48 - create a virtual environment in which all Python code runs during
49   the build, and possibly install packages into it from PyPI
50
51 - invoke Meson in the virtual environment, to perform the actual
52   configuration step for the emulator build
53
54The configure script automatically recognizes command line options for
55which a same-named Meson option exists; dashes in the command line are
56replaced with underscores.
57
58Almost all QEMU developers that need to modify the build system will
59only be concerned with Meson, and therefore can skip the rest of this
60section.
61
62
63Modifying ``configure``
64-----------------------
65
66``configure`` is a shell script; it uses ``#!/bin/sh`` and therefore
67should be compatible with any POSIX shell. It is important to avoid
68using bash-isms to avoid breaking development platforms where bash is
69the primary host.
70
71The configure script provides a variety of functions to help writing
72portable shell code and providing consistent behavior across architectures
73and operating systems:
74
75``error_exit $MESSAGE $MORE...``
76   Print $MESSAGE to stderr, followed by $MORE... and then exit from the
77   configure script with non-zero status.
78
79``has $COMMAND``
80   Determine if $COMMAND exists in the current environment, either as a
81   shell builtin, or executable binary, returning 0 on success.  The
82   replacement in Meson is ``find_program()``.
83
84``probe_target_compiler $TARGET``
85  Detect a cross compiler and cross tools for the QEMU target $TARGET (e.g.,
86  ``$CPU-softmmu``, ``$CPU-linux-user``, ``$CPU-bsd-user``).  If a working
87  compiler is present, return success and set variables ``$target_cc``,
88  ``$target_ar``, etc. to non-empty values.
89
90``write_target_makefile``
91  Write a Makefile fragment to stdout, exposing the result of the most
92  ``probe_target_compiler`` call as the usual Make variables (``CC``,
93  ``AR``, ``LD``, etc.).
94
95
96Configure does not generally perform tests for compiler options beyond
97basic checks to detect the host platform and ensure the compiler is
98functioning.  These are performed using a few more helper functions:
99
100``compile_object $CFLAGS``
101   Attempt to compile a test program with the system C compiler using
102   $CFLAGS. The test program must have been previously written to a file
103   called $TMPC.
104
105``compile_prog $CFLAGS $LDFLAGS``
106   Attempt to compile a test program with the system C compiler using
107   $CFLAGS and link it with the system linker using $LDFLAGS. The test
108   program must have been previously written to a file called $TMPC.
109
110``check_define $NAME``
111   Determine if the macro $NAME is defined by the system C compiler.
112
113``do_compiler $CC $ARGS...``
114   Attempt to run the C compiler $CC, passing it $ARGS...  This function
115   does not use flags passed via options such as ``--extra-cflags``, and
116   therefore can be used to check for cross compilers.  However, most
117   such checks are done at ``make`` time instead (see for example the
118   ``cc-option`` macro in ``pc-bios/option-rom/Makefile``).
119
120``write_c_skeleton``
121   Write a minimal C program main() function to the temporary file
122   indicated by $TMPC.
123
124
125Python virtual environments and the QEMU build system
126-----------------------------------------------------
127
128TBD
129
130Stage 2: Meson
131==============
132
133The Meson build system describes the build and install process for:
134
1351) executables, which include:
136
137   - Tools - ``qemu-img``, ``qemu-nbd``, ``qemu-ga`` (guest agent), etc
138
139   - System emulators - ``qemu-system-$ARCH``
140
141   - Userspace emulators - ``qemu-$ARCH``
142
143   - Unit tests
144
1452) documentation
146
1473) ROMs, whether provided as binary blobs in the QEMU distributions
148   or cross compiled under the direction of the configure script
149
1504) other data files, such as icons or desktop files
151
152All executables are built by default, except for some ``contrib/``
153binaries that are known to fail to build on some platforms (for example
15432-bit or big-endian platforms).  Tests are also built by default,
155though that might change in the future.
156
157The source code is highly modularized, split across many files to
158facilitate building of all of these components with as little duplicated
159compilation as possible. Using the Meson "sourceset" functionality,
160``meson.build`` files group the source files in rules that are
161enabled according to the available system libraries and to various
162configuration symbols.  Sourcesets belong to one of four groups:
163
164Subsystem sourcesets:
165  Various subsystems that are common to both tools and emulators have
166  their own sourceset, for example ``block_ss`` for the block device subsystem,
167  ``chardev_ss`` for the character device subsystem, etc.  These sourcesets
168  are then turned into static libraries as follows::
169
170    libchardev = static_library('chardev', chardev_ss.sources(),
171                                name_suffix: 'fa',
172                                build_by_default: false)
173
174    chardev = declare_dependency(link_whole: libchardev)
175
176  As of Meson 0.55.1, the special ``.fa`` suffix should be used for everything
177  that is used with ``link_whole``, to ensure that the link flags are placed
178  correctly in the command line.
179
180Target-independent emulator sourcesets:
181  Various general purpose helper code is compiled only once and
182  the .o files are linked into all output binaries that need it.
183  This includes error handling infrastructure, standard data structures,
184  platform portability wrapper functions, etc.
185
186  Target-independent code lives in the ``common_ss``, ``softmmu_ss`` and
187  ``user_ss`` sourcesets.  ``common_ss`` is linked into all emulators,
188  ``softmmu_ss`` only in system emulators, ``user_ss`` only in user-mode
189  emulators.
190
191  Target-independent sourcesets must exercise particular care when using
192  ``if_false`` rules.  The ``if_false`` rule will be used correctly when linking
193  emulator binaries; however, when *compiling* target-independent files
194  into .o files, Meson may need to pick *both* the ``if_true`` and
195  ``if_false`` sides to cater for targets that want either side.  To
196  achieve that, you can add a special rule using the ``CONFIG_ALL``
197  symbol::
198
199    # Some targets have CONFIG_ACPI, some don't, so this is not enough
200    softmmu_ss.add(when: 'CONFIG_ACPI', if_true: files('acpi.c'),
201                                        if_false: files('acpi-stub.c'))
202
203    # This is required as well:
204    softmmu_ss.add(when: 'CONFIG_ALL', if_true: files('acpi-stub.c'))
205
206Target-dependent emulator sourcesets:
207  In the target-dependent set lives CPU emulation, some device emulation and
208  much glue code. This sometimes also has to be compiled multiple times,
209  once for each target being built.  Target-dependent files are included
210  in the ``specific_ss`` sourceset.
211
212  Each emulator also includes sources for files in the ``hw/`` and ``target/``
213  subdirectories.  The subdirectory used for each emulator comes
214  from the target's definition of ``TARGET_BASE_ARCH`` or (if missing)
215  ``TARGET_ARCH``, as found in ``default-configs/targets/*.mak``.
216
217  Each subdirectory in ``hw/`` adds one sourceset to the ``hw_arch`` dictionary,
218  for example::
219
220    arm_ss = ss.source_set()
221    arm_ss.add(files('boot.c'), fdt)
222    ...
223    hw_arch += {'arm': arm_ss}
224
225  The sourceset is only used for system emulators.
226
227  Each subdirectory in ``target/`` instead should add one sourceset to each
228  of the ``target_arch`` and ``target_softmmu_arch``, which are used respectively
229  for all emulators and for system emulators only.  For example::
230
231    arm_ss = ss.source_set()
232    arm_softmmu_ss = ss.source_set()
233    ...
234    target_arch += {'arm': arm_ss}
235    target_softmmu_arch += {'arm': arm_softmmu_ss}
236
237Module sourcesets:
238  There are two dictionaries for modules: ``modules`` is used for
239  target-independent modules and ``target_modules`` is used for
240  target-dependent modules.  When modules are disabled the ``module``
241  source sets are added to ``softmmu_ss`` and the ``target_modules``
242  source sets are added to ``specific_ss``.
243
244  Both dictionaries are nested.  One dictionary is created per
245  subdirectory, and these per-subdirectory dictionaries are added to
246  the toplevel dictionaries.  For example::
247
248    hw_display_modules = {}
249    qxl_ss = ss.source_set()
250    ...
251    hw_display_modules += { 'qxl': qxl_ss }
252    modules += { 'hw-display': hw_display_modules }
253
254Utility sourcesets:
255  All binaries link with a static library ``libqemuutil.a``.  This library
256  is built from several sourcesets; most of them however host generated
257  code, and the only two of general interest are ``util_ss`` and ``stub_ss``.
258
259  The separation between these two is purely for documentation purposes.
260  ``util_ss`` contains generic utility files.  Even though this code is only
261  linked in some binaries, sometimes it requires hooks only in some of
262  these and depend on other functions that are not fully implemented by
263  all QEMU binaries.  ``stub_ss`` links dummy stubs that will only be linked
264  into the binary if the real implementation is not present.  In a way,
265  the stubs can be thought of as a portable implementation of the weak
266  symbols concept.
267
268
269The following files concur in the definition of which files are linked
270into each emulator:
271
272``default-configs/devices/*.mak``
273  The files under ``default-configs/devices/`` control the boards and devices
274  that are built into each QEMU system emulation targets. They merely contain
275  a list of config variable definitions such as::
276
277    include arm-softmmu.mak
278    CONFIG_XLNX_ZYNQMP_ARM=y
279    CONFIG_XLNX_VERSAL=y
280
281``*/Kconfig``
282  These files are processed together with ``default-configs/devices/*.mak`` and
283  describe the dependencies between various features, subsystems and
284  device models.  They are described in :ref:`kconfig`
285
286``default-configs/targets/*.mak``
287  These files mostly define symbols that appear in the ``*-config-target.h``
288  file for each emulator [#cfgtarget]_.  However, the ``TARGET_ARCH``
289  and ``TARGET_BASE_ARCH`` will also be used to select the ``hw/`` and
290  ``target/`` subdirectories that are compiled into each target.
291
292.. [#cfgtarget] This header is included by ``qemu/osdep.h`` when
293                compiling files from the target-specific sourcesets.
294
295These files rarely need changing unless you are adding a completely
296new target, or enabling new devices or hardware for a particular
297system/userspace emulation target
298
299
300Adding checks
301-------------
302
303Compiler checks can be as simple as the following::
304
305  config_host_data.set('HAVE_BTRFS_H', cc.has_header('linux/btrfs.h'))
306
307A more complex task such as adding a new dependency usually
308comprises the following tasks:
309
310 - Add a Meson build option to meson_options.txt.
311
312 - Add code to perform the actual feature check.
313
314 - Add code to include the feature status in ``config-host.h``
315
316 - Add code to print out the feature status in the configure summary
317   upon completion.
318
319Taking the probe for SDL2_Image as an example, we have the following
320in ``meson_options.txt``::
321
322  option('sdl_image', type : 'feature', value : 'auto',
323         description: 'SDL Image support for icons')
324
325Unless the option was given a non-``auto`` value (on the configure
326command line), the detection code must be performed only if the
327dependency will be used::
328
329  sdl_image = not_found
330  if not get_option('sdl_image').auto() or have_system
331    sdl_image = dependency('SDL2_image', required: get_option('sdl_image'),
332                           method: 'pkg-config')
333  endif
334
335This avoids warnings on static builds of user-mode emulators, for example.
336Most of the libraries used by system-mode emulators are not available for
337static linking.
338
339The other supporting code is generally simple::
340
341  # Create config-host.h (if applicable)
342  config_host_data.set('CONFIG_SDL_IMAGE', sdl_image.found())
343
344  # Summary
345  summary_info += {'SDL image support': sdl_image.found()}
346
347For the configure script to parse the new option, the
348``scripts/meson-buildoptions.sh`` file must be up-to-date; ``make
349update-buildoptions`` (or just ``make``) will take care of updating it.
350
351
352Support scripts
353---------------
354
355Meson has a special convention for invoking Python scripts: if their
356first line is ``#! /usr/bin/env python3`` and the file is *not* executable,
357find_program() arranges to invoke the script under the same Python
358interpreter that was used to invoke Meson.  This is the most common
359and preferred way to invoke support scripts from Meson build files,
360because it automatically uses the value of configure's --python= option.
361
362In case the script is not written in Python, use a ``#! /usr/bin/env ...``
363line and make the script executable.
364
365Scripts written in Python, where it is desirable to make the script
366executable (for example for test scripts that developers may want to
367invoke from the command line, such as tests/qapi-schema/test-qapi.py),
368should be invoked through the ``python`` variable in meson.build. For
369example::
370
371  test('QAPI schema regression tests', python,
372       args: files('test-qapi.py'),
373       env: test_env, suite: ['qapi-schema', 'qapi-frontend'])
374
375This is needed to obey the --python= option passed to the configure
376script, which may point to something other than the first python3
377binary on the path.
378
379
380Stage 3: Make
381=============
382
383The next step in building QEMU is to invoke make.  GNU Make is required
384to build QEMU, and may be installed as ``gmake`` on some hosts.
385
386The output of Meson is a ``build.ninja`` file, which is used with the
387Ninja build tool.  However, QEMU's build comprises other components than
388just the emulators (namely firmware and the tests in ``tests/tcg``) which
389need different cross compilers.  The QEMU Makefile wraps both Ninja and
390the smaller build systems for firmware and tests; it also takes care of
391running ``configure`` again when the script changes.  Apart from invoking
392these sub-Makefiles, the resulting build is largely non-recursive.
393
394Tests, whether defined in ``meson.build`` or not, are also ran by the
395Makefile with the traditional ``make check`` phony target, while benchmarks
396are run with ``make bench``.  Meson test suites such as ``unit`` can be ran
397with ``make check-unit``, and ``make check-tcg`` builds and runs "non-Meson"
398tests for all targets.
399
400If desired, it is also possible to use ``ninja`` and ``meson test``,
401respectively to build emulators and run tests defined in meson.build.
402The main difference is that ``make`` needs the ``-jN`` flag in order to
403enable parallel builds or tests.
404
405Useful make targets
406-------------------
407
408``help``
409  Print a help message for the most common build targets.
410
411``print-VAR``
412  Print the value of the variable VAR. Useful for debugging the build
413  system.
414
415
416Important files for the build system
417====================================
418
419Statically defined files
420------------------------
421
422The following key files are statically defined in the source tree, with
423the rules needed to build QEMU. Their behaviour is influenced by a
424number of dynamically created files listed later.
425
426``Makefile``
427  The main entry point used when invoking make to build all the components
428  of QEMU. The default 'all' target will naturally result in the build of
429  every component.
430
431``*/meson.build``
432  The meson.build file in the root directory is the main entry point for the
433  Meson build system, and it coordinates the configuration and build of all
434  executables.  Build rules for various subdirectories are included in
435  other meson.build files spread throughout the QEMU source tree.
436
437``tests/Makefile.include``
438  Rules for external test harnesses. These include the TCG tests
439  and the Avocado-based integration tests.
440
441``tests/docker/Makefile.include``
442  Rules for Docker tests. Like ``tests/Makefile.include``, this file is
443  included directly by the top level Makefile, anything defined in this
444  file will influence the entire build system.
445
446``tests/vm/Makefile.include``
447  Rules for VM-based tests. Like ``tests/Makefile.include``, this file is
448  included directly by the top level Makefile, anything defined in this
449  file will influence the entire build system.
450
451Dynamically created files
452-------------------------
453
454The following files are generated at run-time in order to control the
455behaviour of the Makefiles. This avoids the need for QEMU makefiles to
456go through any pre-processing as seen with autotools, where configure
457generates ``Makefile`` from ``Makefile.in``.
458
459Built by configure:
460
461``config-host.mak``
462  When configure has determined the characteristics of the build host it
463  will write them to this file for use in ``Makefile`` and to a smaller
464  extent ``meson.build``. These include the paths to various tools and a
465  variety of ``CONFIG_*`` variables related to optionally enabled features.
466
467  ``config-host.mak`` is also used as a dependency checking mechanism. If make
468  sees that the modification timestamp on configure is newer than that on
469  ``config-host.mak``, then configure will be re-run.
470
471  The variables defined here apply to all QEMU
472  build outputs.
473
474``config-meson.cross``
475
476  A Meson "cross file" (or native file) used to communicate the paths to
477  the toolchain and other configuration options.
478
479``config.status``
480
481  A small shell script that will invoke configure again with the same
482  environment variables that were set during the first run.  It's used to
483  rerun configure after changes to the source code, but it can also be
484  inspected manually to check the contents of the environment.
485
486``Makefile.prereqs``
487
488  A set of Makefile dependencies that order the build and execution of
489  firmware and tests after the container images and emulators that they
490  need.
491
492``pc-bios/*/config.mak``, ``tests/tcg/config-host.mak``, ``tests/tcg/*/config-target.mak``
493
494  Configuration variables used to build the firmware and TCG tests,
495  including paths to cross compilation toolchains.
496
497``pyvenv``
498
499  A Python virtual environment that is used for all Python code running
500  during the build.  Using a virtual environment ensures that even code
501  that is run via ``sphinx-build``, ``meson`` etc. uses the same interpreter
502  and packages.
503
504Built by Meson:
505
506``config-host.h``
507  Used by C code to determine the properties of the build environment
508  and the set of enabled features for the entire build.
509
510``${TARGET-NAME}-config-devices.mak``
511  TARGET-NAME is the name of a system emulator. The file is
512  generated by Meson using files under ``configs/devices`` as input.
513
514``${TARGET-NAME}-config-target.mak``
515  TARGET-NAME is the name of a system or usermode emulator. The file is
516  generated by Meson using files under ``configs/targets`` as input.
517
518``$TARGET_NAME-config-target.h``, ``$TARGET_NAME-config-devices.h``
519  Used by C code to determine the properties and enabled
520  features for each target.  enabled.  They are generated from
521  the contents of the corresponding ``*.mak`` files using Meson's
522  ``configure_file()`` function; each target can include them using
523  the ``CONFIG_TARGET`` and ``CONFIG_DEVICES`` macro respectively.
524
525``build.ninja``
526  The build rules.
527
528
529Built by Makefile:
530
531``Makefile.ninja``
532  A Makefile include that bridges to ninja for the actual build.  The
533  Makefile is mostly a list of targets that Meson included in build.ninja.
534
535``Makefile.mtest``
536  The Makefile definitions that let "make check" run tests defined in
537  meson.build.  The rules are produced from Meson's JSON description of
538  tests (obtained with "meson introspect --tests") through the script
539  scripts/mtest2make.py.
540