xref: /openbmc/qemu/docs/devel/testing/blkdebug.rst (revision 2e1cacfb)
1Block I/O error injection using ``blkdebug``
2============================================
3
4..
5   Copyright (C) 2014-2015 Red Hat Inc
6
7   This work is licensed under the terms of the GNU GPL, version 2 or later.  See
8   the COPYING file in the top-level directory.
9
10The ``blkdebug`` block driver is a rule-based error injection engine.  It can be
11used to exercise error code paths in block drivers including ``ENOSPC`` (out of
12space) and ``EIO``.
13
14This document gives an overview of the features available in ``blkdebug``.
15
16Background
17----------
18Block drivers have many error code paths that handle I/O errors.  Image formats
19are especially complex since metadata I/O errors during cluster allocation or
20while updating tables happen halfway through request processing and require
21discipline to keep image files consistent.
22
23Error injection allows test cases to trigger I/O errors at specific points.
24This way, all error paths can be tested to make sure they are correct.
25
26Rules
27-----
28The ``blkdebug`` block driver takes a list of "rules" that tell the error injection
29engine when to fail an I/O request.
30
31Each I/O request is evaluated against the rules.  If a rule matches the request
32then its "action" is executed.
33
34Rules can be placed in a configuration file; the configuration file
35follows the same .ini-like format used by QEMU's ``-readconfig`` option, and
36each section of the file represents a rule.
37
38The following configuration file defines a single rule::
39
40  $ cat blkdebug.conf
41  [inject-error]
42  event = "read_aio"
43  errno = "28"
44
45This rule fails all aio read requests with ``ENOSPC`` (28).  Note that the errno
46value depends on the host.  On Linux, see
47``/usr/include/asm-generic/errno-base.h`` for errno values.
48
49Invoke QEMU as follows::
50
51  $ qemu-system-x86_64
52        -drive if=none,cache=none,file=blkdebug:blkdebug.conf:test.img,id=drive0 \
53        -device virtio-blk-pci,drive=drive0,id=virtio-blk-pci0
54
55Rules support the following attributes:
56
57``event``
58  which type of operation to match (e.g. ``read_aio``, ``write_aio``,
59  ``flush_to_os``, ``flush_to_disk``).  See `Events`_ for
60  information on events.
61
62``state``
63  (optional) the engine must be in this state number in order for this
64  rule to match.  See `State transitions`_ for information
65  on states.
66
67``errno``
68  the numeric errno value to return when a request matches this rule.
69  The errno values depend on the host since the numeric values are not
70  standardized in the POSIX specification.
71
72``sector``
73  (optional) a sector number that the request must overlap in order to
74  match this rule
75
76``once``
77  (optional, default ``off``) only execute this action on the first
78  matching request
79
80``immediately``
81  (optional, default ``off``) return a NULL ``BlockAIOCB``
82  pointer and fail without an errno instead.  This
83  exercises the code path where ``BlockAIOCB`` fails and the
84  caller's ``BlockCompletionFunc`` is not invoked.
85
86Events
87------
88Block drivers provide information about the type of I/O request they are about
89to make so rules can match specific types of requests.  For example, the ``qcow2``
90block driver tells ``blkdebug`` when it accesses the L1 table so rules can match
91only L1 table accesses and not other metadata or guest data requests.
92
93The core events are:
94
95``read_aio``
96  guest data read
97
98``write_aio``
99  guest data write
100
101``flush_to_os``
102  write out unwritten block driver state (e.g. cached metadata)
103
104``flush_to_disk``
105  flush the host block device's disk cache
106
107See ``qapi/block-core.json:BlkdebugEvent`` for the full list of events.
108You may need to grep block driver source code to understand the
109meaning of specific events.
110
111State transitions
112-----------------
113There are cases where more power is needed to match a particular I/O request in
114a longer sequence of requests.  For example::
115
116  write_aio
117  flush_to_disk
118  write_aio
119
120How do we match the 2nd ``write_aio`` but not the first?  This is where state
121transitions come in.
122
123The error injection engine has an integer called the "state" that always starts
124initialized to 1.  The state integer is internal to ``blkdebug`` and cannot be
125observed from outside but rules can interact with it for powerful matching
126behavior.
127
128Rules can be conditional on the current state and they can transition to a new
129state.
130
131When a rule's "state" attribute is non-zero then the current state must equal
132the attribute in order for the rule to match.
133
134For example, to match the 2nd write_aio::
135
136  [set-state]
137  event = "write_aio"
138  state = "1"
139  new_state = "2"
140
141  [inject-error]
142  event = "write_aio"
143  state = "2"
144  errno = "5"
145
146The first ``write_aio`` request matches the ``set-state`` rule and transitions from
147state 1 to state 2.  Once state 2 has been entered, the ``set-state`` rule no
148longer matches since it requires state 1.  But the ``inject-error`` rule now
149matches the next ``write_aio`` request and injects ``EIO`` (5).
150
151State transition rules support the following attributes:
152
153``event``
154  which type of operation to match (e.g. ``read_aio``, ``write_aio``,
155  ``flush_to_os`, ``flush_to_disk``).  See `Events`_ for
156  information on events.
157
158``state``
159  (optional) the engine must be in this state number in order for this
160  rule to match
161
162``new_state``
163  transition to this state number
164
165Suspend and resume
166------------------
167Exercising code paths in block drivers may require specific ordering amongst
168concurrent requests.  The "breakpoint" feature allows requests to be halted on
169a ``blkdebug`` event and resumed later.  This makes it possible to achieve
170deterministic ordering when multiple requests are in flight.
171
172Breakpoints on ``blkdebug`` events are associated with a user-defined ``tag`` string.
173This tag serves as an identifier by which the request can be resumed at a later
174point.
175
176See the ``qemu-io(1)`` ``break``, ``resume``, ``remove_break``, and ``wait_break``
177commands for details.
178