devel/testing/fuzzing.rst

5 This document describes the virtual-device fuzzing infrastructure in QEMU and
9 ------
16 is an *in-process* fuzzer. For the developer, this means that it is their
17 responsibility to ensure that state is reset between fuzzing-runs.
20 --------------------
24 Here, enable-asan and enable-ubsan are optional but they allow us to reliably
25 detect bugs such as out-of-bounds accesses, uses-after-free, double-frees
28     CC=clang-8 CXX=clang++-8 /path/to/configure \
29         --enable-fuzzing --enable-asan --enable-ubsan
33     make qemu-fuzz-i386
35 This builds ``./qemu-fuzz-i386``
37 The first option to this command is: ``--fuzz-target=FUZZ_NAME``
38 To list all of the available fuzzers run ``qemu-fuzz-i386`` with no arguments.
42     ./qemu-fuzz-i386 --fuzz-target=virtio-scsi-fuzz
44 Internally, libfuzzer parses all arguments that do not begin with ``"--"``.
45 Information about these is available by passing ``-help=1``
51 ----------------------
53 As mentioned above, libFuzzer accepts some arguments. Passing ``-help=1`` will
63 * ``-max_len=4096`` : specify the maximum byte-length of the inputs libFuzzer
66 * ``-close_fd_mask={1,2,3}`` : close, stderr, or both. Useful for targets that
69 * ``-jobs=4 -workers=4`` : These arguments configure libFuzzer to run 4 fuzzers in
71   ``-jobs=N``, libFuzzer automatically spawns a number of workers less than or equal
76 * ``-use_value_profile=1`` : For each comparison operation, libFuzzer computes
83 * ``-shrink=1`` : Tries to make elements of the corpus "smaller". Might lead to
90 ---------------------------
94 unique blocks/edges covered. To examine coverage on a line-by-line basis we
101     --enable-fuzzing \
102     --extra-cflags="-fprofile-instr-generate -fcoverage-mapping"
104  3. Re-run the fuzzer. Specify $CORPUS_DIR/* as an argument, telling libfuzzer
107  4. Execute these commands to generate a detailed HTML coverage-report::
109       llvm-profdata merge -output=default.profdata default.profraw
110       llvm-cov show ./path/to/qemu-fuzz-i386 -instr-profile=default.profdata \
111       --format html -output-dir=/path/to/output/report
114 -------------------
122 1. Create a new source file. For example ``tests/qtest/fuzz/foo-device-fuzz.c``.
129 Fuzzers can be more-or-less thought of as special qtest programs which can
132 fuzzer loops over the byte-array interpreting it as a list of qtest commands,
136 ------------------
139 not be built-out within libqos). Many devices can be fuzzed to some degree,
140 without any device-specific code, using the generic-fuzz target.
142 The generic-fuzz target is capable of fuzzing devices over their PIO, MMIO,
143 and DMA input-spaces. To apply the generic-fuzz to a device, we need to define
144 two env-variables, at minimum:
147   the device attached. For example, if we want to fuzz the virtio-net device
148   attached to a pc-i440fx machine, we can specify::
150     QEMU_FUZZ_ARGS="-M pc -nodefaults -netdev user,id=user0 \
151     -device virtio-net,netdev=user0"
153 * ``QEMU_FUZZ_OBJECTS=`` is a set of space-delimited strings used to identify
157   virtio-net example, we could use one of ::
159     QEMU_FUZZ_OBJECTS='virtio-net'
164 The ``"info mtree"`` and ``"info qom-tree"`` monitor commands can be especially
168 As a generic rule-of-thumb, the more ``MemoryRegions``/Devices we match, the
169 greater the input-space, and the smaller the probability of finding crashing
175     ./qemu-fuzz-i386 --fuzz-target=generic-fuzz -runs=0
179 OSS-Fuzz
180 --------
181 QEMU is continuously fuzzed on `OSS-Fuzz
182 <https://github.com/google/oss-fuzz>`_.  By default, the OSS-Fuzz build
183 will try to fuzz every fuzz-target. Since the generic-fuzz target
185 pre-define some generic-fuzz configs in
188 - ``.name``: To identify the fuzzer config
190 - ``.args`` OR ``.argfunc``: A string or pointer to a function returning a
195 - ``.objects``: A string that specifies the ``QEMU_FUZZ_OBJECTS`` environment
198 To fuzz additional devices/device configuration on OSS-Fuzz, send patches for
199 either a new device-specific fuzzer or a new generic-fuzz config.
203 - The Dockerfile that sets up the environment for building QEMU's
204   fuzzers on OSS-Fuzz can be fund in the OSS-Fuzz repository
205   __(https://github.com/google/oss-fuzz/blob/master/projects/qemu/Dockerfile)
207 - The script responsible for building the fuzzers can be found in the
208   QEMU source tree at ``scripts/oss-fuzz/build.sh``
211 -----------------------------------------
213 can be used on a non-fuzzer build of QEMU. This filters out any potential
214 false-positives, and improves the debugging experience for developers.
216 generic-fuzz target.
218 - Ensure the crash reproduces::
220     qemu-fuzz-i386 --fuzz-target... ./crash-...
222 - Gather the QTest output for the crash::
225     qemu-fuzz-i386 --fuzz-target... ./crash-... &> /tmp/trace
227 - Reorder and clean-up the resulting trace::
229     scripts/oss-fuzz/reorder_fuzzer_qtest_trace.py /tmp/trace > /tmp/reproducer
231 - Get the arguments needed to start qemu, and provide a path to qemu::
234     export QEMU_ARGS="-machine ..."
235     export QEMU_PATH="path/to/qemu-system"
237 - Ensure the crash reproduces in qemu-system::
239     $QEMU_PATH $QEMU_ARGS -qtest stdio < /tmp/reproducer
241 - From the crash output, obtain some string that identifies the crash. This
242   can be a line in the stack-trace, for example::
244     export CRASH_TOKEN="hw/usb/hcd-xhci.c:1865"
246 - Minimize the reproducer::
248     scripts/oss-fuzz/minimize_qtest_trace.py -M1 -M2 \
249       /tmp/reproducer /tmp/reproducer-minimized
251 - Confirm that the minimized reproducer still crashes::
253     $QEMU_PATH $QEMU_ARGS -qtest stdio < /tmp/reproducer-minimized
255 - Create a one-liner reproducer that can be sent over email::
257     ./scripts/oss-fuzz/output_reproducer.py -bash /tmp/reproducer-minimized
259 - Output the C source code for a test case that will reproduce the bug::
261     ./scripts/oss-fuzz/output_reproducer.py -owner "John Smith <john@smith.com>"\
262       -name "test_function_name" /tmp/reproducer-minimized
264 - Report the bug and send a patch with the C reproducer upstream
267 -----------------------------------------
287 target-specific hooks that can be called before and after main, for
294 Since the same process is reused for many fuzzing runs, QEMU state needs to
295 be reset at the end of each run. For example, this can be done by rebooting the
298   - *Pros*: Straightforward and fast for simple fuzz targets.
300   - *Cons*: Depending on the device, does not reset all device state. If the
302     for QOS-based targets), this initialization needs to be done after each
305   - *Example target*: ``i440fx-qtest-reboot-fuzz``