atomics.rst (e33d61cc9aef14f21fbf16c0e3cf01d2e2965717) atomics.rst (d73415a315471ac0b127ed3fad45c8ec5d711de1)
1=========================
2Atomic operations in QEMU
3=========================
4
5CPUs perform independent memory operations effectively in random order.
6but this can be a problem for CPU-CPU interaction (including interactions
7between QEMU and the guest). Multi-threaded programs use various tools
8to instruct the compiler and the CPU to restrict the order to something

--- 9 unchanged lines hidden (view full) ---

18by the C11 memory model.
19
20QEMU provides a header, ``qemu/atomic.h``, which wraps C11 atomics to
21provide better portability and a less verbose syntax. ``qemu/atomic.h``
22provides macros that fall in three camps:
23
24- compiler barriers: ``barrier()``;
25
1=========================
2Atomic operations in QEMU
3=========================
4
5CPUs perform independent memory operations effectively in random order.
6but this can be a problem for CPU-CPU interaction (including interactions
7between QEMU and the guest). Multi-threaded programs use various tools
8to instruct the compiler and the CPU to restrict the order to something

--- 9 unchanged lines hidden (view full) ---

18by the C11 memory model.
19
20QEMU provides a header, ``qemu/atomic.h``, which wraps C11 atomics to
21provide better portability and a less verbose syntax. ``qemu/atomic.h``
22provides macros that fall in three camps:
23
24- compiler barriers: ``barrier()``;
25
26- weak atomic access and manual memory barriers: ``atomic_read()``,
27 ``atomic_set()``, ``smp_rmb()``, ``smp_wmb()``, ``smp_mb()``, ``smp_mb_acquire()``,
28 ``smp_mb_release()``, ``smp_read_barrier_depends()``;
26- weak atomic access and manual memory barriers: ``qatomic_read()``,
27 ``qatomic_set()``, ``smp_rmb()``, ``smp_wmb()``, ``smp_mb()``,
28 ``smp_mb_acquire()``, ``smp_mb_release()``, ``smp_read_barrier_depends()``;
29
30- sequentially consistent atomic access: everything else.
31
32In general, use of ``qemu/atomic.h`` should be wrapped with more easily
33used data structures (e.g. the lock-free singly-linked list operations
34``QSLIST_INSERT_HEAD_ATOMIC`` and ``QSLIST_MOVE_ATOMIC``) or synchronization
35primitives (such as RCU, ``QemuEvent`` or ``QemuLockCnt``). Bare use of
36atomic operations and memory barriers should be limited to inter-thread

--- 25 unchanged lines hidden (view full) ---

62consistency*, where "the result of any execution is the same as if the
63operations of all the processors were executed in some sequential order,
64and the operations of each individual processor appear in this sequence
65in the order specified by its program".
66
67``qemu/atomic.h`` provides the following set of atomic read-modify-write
68operations::
69
29
30- sequentially consistent atomic access: everything else.
31
32In general, use of ``qemu/atomic.h`` should be wrapped with more easily
33used data structures (e.g. the lock-free singly-linked list operations
34``QSLIST_INSERT_HEAD_ATOMIC`` and ``QSLIST_MOVE_ATOMIC``) or synchronization
35primitives (such as RCU, ``QemuEvent`` or ``QemuLockCnt``). Bare use of
36atomic operations and memory barriers should be limited to inter-thread

--- 25 unchanged lines hidden (view full) ---

62consistency*, where "the result of any execution is the same as if the
63operations of all the processors were executed in some sequential order,
64and the operations of each individual processor appear in this sequence
65in the order specified by its program".
66
67``qemu/atomic.h`` provides the following set of atomic read-modify-write
68operations::
69
70 void atomic_inc(ptr)
71 void atomic_dec(ptr)
72 void atomic_add(ptr, val)
73 void atomic_sub(ptr, val)
74 void atomic_and(ptr, val)
75 void atomic_or(ptr, val)
70 void qatomic_inc(ptr)
71 void qatomic_dec(ptr)
72 void qatomic_add(ptr, val)
73 void qatomic_sub(ptr, val)
74 void qatomic_and(ptr, val)
75 void qatomic_or(ptr, val)
76
76
77 typeof(*ptr) atomic_fetch_inc(ptr)
78 typeof(*ptr) atomic_fetch_dec(ptr)
79 typeof(*ptr) atomic_fetch_add(ptr, val)
80 typeof(*ptr) atomic_fetch_sub(ptr, val)
81 typeof(*ptr) atomic_fetch_and(ptr, val)
82 typeof(*ptr) atomic_fetch_or(ptr, val)
83 typeof(*ptr) atomic_fetch_xor(ptr, val)
84 typeof(*ptr) atomic_fetch_inc_nonzero(ptr)
85 typeof(*ptr) atomic_xchg(ptr, val)
86 typeof(*ptr) atomic_cmpxchg(ptr, old, new)
77 typeof(*ptr) qatomic_fetch_inc(ptr)
78 typeof(*ptr) qatomic_fetch_dec(ptr)
79 typeof(*ptr) qatomic_fetch_add(ptr, val)
80 typeof(*ptr) qatomic_fetch_sub(ptr, val)
81 typeof(*ptr) qatomic_fetch_and(ptr, val)
82 typeof(*ptr) qatomic_fetch_or(ptr, val)
83 typeof(*ptr) qatomic_fetch_xor(ptr, val)
84 typeof(*ptr) qatomic_fetch_inc_nonzero(ptr)
85 typeof(*ptr) qatomic_xchg(ptr, val)
86 typeof(*ptr) qatomic_cmpxchg(ptr, old, new)
87
88all of which return the old value of ``*ptr``. These operations are
89polymorphic; they operate on any type that is as wide as a pointer or
90smaller.
91
92Similar operations return the new value of ``*ptr``::
93
87
88all of which return the old value of ``*ptr``. These operations are
89polymorphic; they operate on any type that is as wide as a pointer or
90smaller.
91
92Similar operations return the new value of ``*ptr``::
93
94 typeof(*ptr) atomic_inc_fetch(ptr)
95 typeof(*ptr) atomic_dec_fetch(ptr)
96 typeof(*ptr) atomic_add_fetch(ptr, val)
97 typeof(*ptr) atomic_sub_fetch(ptr, val)
98 typeof(*ptr) atomic_and_fetch(ptr, val)
99 typeof(*ptr) atomic_or_fetch(ptr, val)
100 typeof(*ptr) atomic_xor_fetch(ptr, val)
94 typeof(*ptr) qatomic_inc_fetch(ptr)
95 typeof(*ptr) qatomic_dec_fetch(ptr)
96 typeof(*ptr) qatomic_add_fetch(ptr, val)
97 typeof(*ptr) qatomic_sub_fetch(ptr, val)
98 typeof(*ptr) qatomic_and_fetch(ptr, val)
99 typeof(*ptr) qatomic_or_fetch(ptr, val)
100 typeof(*ptr) qatomic_xor_fetch(ptr, val)
101
102``qemu/atomic.h`` also provides loads and stores that cannot be reordered
103with each other::
104
101
102``qemu/atomic.h`` also provides loads and stores that cannot be reordered
103with each other::
104
105 typeof(*ptr) atomic_mb_read(ptr)
106 void atomic_mb_set(ptr, val)
105 typeof(*ptr) qatomic_mb_read(ptr)
106 void qatomic_mb_set(ptr, val)
107
108However these do not provide sequential consistency and, in particular,
109they do not participate in the total ordering enforced by
110sequentially-consistent operations. For this reason they are deprecated.
111They should instead be replaced with any of the following (ordered from
112easiest to hardest):
113
114- accesses inside a mutex or spinlock
115
116- lightweight synchronization primitives such as ``QemuEvent``
117
107
108However these do not provide sequential consistency and, in particular,
109they do not participate in the total ordering enforced by
110sequentially-consistent operations. For this reason they are deprecated.
111They should instead be replaced with any of the following (ordered from
112easiest to hardest):
113
114- accesses inside a mutex or spinlock
115
116- lightweight synchronization primitives such as ``QemuEvent``
117
118- RCU operations (``atomic_rcu_read``, ``atomic_rcu_set``) when publishing
118- RCU operations (``qatomic_rcu_read``, ``qatomic_rcu_set``) when publishing
119 or accessing a new version of a data structure
120
119 or accessing a new version of a data structure
120
121- other atomic accesses: ``atomic_read`` and ``atomic_load_acquire`` for
122 loads, ``atomic_set`` and ``atomic_store_release`` for stores, ``smp_mb``
121- other atomic accesses: ``qatomic_read`` and ``qatomic_load_acquire`` for
122 loads, ``qatomic_set`` and ``qatomic_store_release`` for stores, ``smp_mb``
123 to forbid reordering subsequent loads before a store.
124
125
126Weak atomic access and manual memory barriers
127=============================================
128
129Compared to sequentially consistent atomic access, programming with
130weaker consistency models can be considerably more complicated.

--- 13 unchanged lines hidden (view full) ---

144 components of the system, after all the LOAD or STORE operations
145 specified before.
146
147- release operations will *synchronize with* acquire operations;
148 see :ref:`acqrel` for a detailed explanation.
149
150When using this model, variables are accessed with:
151
123 to forbid reordering subsequent loads before a store.
124
125
126Weak atomic access and manual memory barriers
127=============================================
128
129Compared to sequentially consistent atomic access, programming with
130weaker consistency models can be considerably more complicated.

--- 13 unchanged lines hidden (view full) ---

144 components of the system, after all the LOAD or STORE operations
145 specified before.
146
147- release operations will *synchronize with* acquire operations;
148 see :ref:`acqrel` for a detailed explanation.
149
150When using this model, variables are accessed with:
151
152- ``atomic_read()`` and ``atomic_set()``; these prevent the compiler from
152- ``qatomic_read()`` and ``qatomic_set()``; these prevent the compiler from
153 optimizing accesses out of existence and creating unsolicited
154 accesses, but do not otherwise impose any ordering on loads and
155 stores: both the compiler and the processor are free to reorder
156 them.
157
153 optimizing accesses out of existence and creating unsolicited
154 accesses, but do not otherwise impose any ordering on loads and
155 stores: both the compiler and the processor are free to reorder
156 them.
157
158- ``atomic_load_acquire()``, which guarantees the LOAD to appear to
158- ``qatomic_load_acquire()``, which guarantees the LOAD to appear to
159 happen, with respect to the other components of the system,
160 before all the LOAD or STORE operations specified afterwards.
159 happen, with respect to the other components of the system,
160 before all the LOAD or STORE operations specified afterwards.
161 Operations coming before ``atomic_load_acquire()`` can still be
161 Operations coming before ``qatomic_load_acquire()`` can still be
162 reordered after it.
163
162 reordered after it.
163
164- ``atomic_store_release()``, which guarantees the STORE to appear to
164- ``qatomic_store_release()``, which guarantees the STORE to appear to
165 happen, with respect to the other components of the system,
166 after all the LOAD or STORE operations specified before.
165 happen, with respect to the other components of the system,
166 after all the LOAD or STORE operations specified before.
167 Operations coming after ``atomic_store_release()`` can still be
167 Operations coming after ``qatomic_store_release()`` can still be
168 reordered before it.
169
170Restrictions to the ordering of accesses can also be specified
171using the memory barrier macros: ``smp_rmb()``, ``smp_wmb()``, ``smp_mb()``,
172``smp_mb_acquire()``, ``smp_mb_release()``, ``smp_read_barrier_depends()``.
173
174Memory barriers control the order of references to shared memory.
175They come in six kinds:

--- 48 unchanged lines hidden (view full) ---

224
225 Note that the first load really has to have a _data_ dependency and not
226 a control dependency. If the address for the second load is dependent
227 on the first load, but the dependency is through a conditional rather
228 than actually loading the address itself, then it's a _control_
229 dependency and a full read barrier or better is required.
230
231
168 reordered before it.
169
170Restrictions to the ordering of accesses can also be specified
171using the memory barrier macros: ``smp_rmb()``, ``smp_wmb()``, ``smp_mb()``,
172``smp_mb_acquire()``, ``smp_mb_release()``, ``smp_read_barrier_depends()``.
173
174Memory barriers control the order of references to shared memory.
175They come in six kinds:

--- 48 unchanged lines hidden (view full) ---

224
225 Note that the first load really has to have a _data_ dependency and not
226 a control dependency. If the address for the second load is dependent
227 on the first load, but the dependency is through a conditional rather
228 than actually loading the address itself, then it's a _control_
229 dependency and a full read barrier or better is required.
230
231
232Memory barriers and ``atomic_load_acquire``/``atomic_store_release`` are
232Memory barriers and ``qatomic_load_acquire``/``qatomic_store_release`` are
233mostly used when a data structure has one thread that is always a writer
234and one thread that is always a reader:
235
236 +----------------------------------+----------------------------------+
237 | thread 1 | thread 2 |
238 +==================================+==================================+
239 | :: | :: |
240 | | |
233mostly used when a data structure has one thread that is always a writer
234and one thread that is always a reader:
235
236 +----------------------------------+----------------------------------+
237 | thread 1 | thread 2 |
238 +==================================+==================================+
239 | :: | :: |
240 | | |
241 | atomic_store_release(&a, x); | y = atomic_load_acquire(&b); |
242 | atomic_store_release(&b, y); | x = atomic_load_acquire(&a); |
241 | qatomic_store_release(&a, x); | y = qatomic_load_acquire(&b); |
242 | qatomic_store_release(&b, y); | x = qatomic_load_acquire(&a); |
243 +----------------------------------+----------------------------------+
244
245In this case, correctness is easy to check for using the "pairing"
246trick that is explained below.
247
248Sometimes, a thread is accessing many variables that are otherwise
249unrelated to each other (for example because, apart from the current
250thread, exactly one other thread will read or write each of these
251variables). In this case, it is possible to "hoist" the barriers
252outside a loop. For example:
253
254 +------------------------------------------+----------------------------------+
255 | before | after |
256 +==========================================+==================================+
257 | :: | :: |
258 | | |
259 | n = 0; | n = 0; |
260 | for (i = 0; i < 10; i++) | for (i = 0; i < 10; i++) |
243 +----------------------------------+----------------------------------+
244
245In this case, correctness is easy to check for using the "pairing"
246trick that is explained below.
247
248Sometimes, a thread is accessing many variables that are otherwise
249unrelated to each other (for example because, apart from the current
250thread, exactly one other thread will read or write each of these
251variables). In this case, it is possible to "hoist" the barriers
252outside a loop. For example:
253
254 +------------------------------------------+----------------------------------+
255 | before | after |
256 +==========================================+==================================+
257 | :: | :: |
258 | | |
259 | n = 0; | n = 0; |
260 | for (i = 0; i < 10; i++) | for (i = 0; i < 10; i++) |
261 | n += atomic_load_acquire(&a[i]); | n += atomic_read(&a[i]); |
261 | n += qatomic_load_acquire(&a[i]); | n += qatomic_read(&a[i]); |
262 | | smp_mb_acquire(); |
263 +------------------------------------------+----------------------------------+
264 | :: | :: |
265 | | |
266 | | smp_mb_release(); |
267 | for (i = 0; i < 10; i++) | for (i = 0; i < 10; i++) |
262 | | smp_mb_acquire(); |
263 +------------------------------------------+----------------------------------+
264 | :: | :: |
265 | | |
266 | | smp_mb_release(); |
267 | for (i = 0; i < 10; i++) | for (i = 0; i < 10; i++) |
268 | atomic_store_release(&a[i], false); | atomic_set(&a[i], false); |
268 | qatomic_store_release(&a[i], false); | qatomic_set(&a[i], false); |
269 +------------------------------------------+----------------------------------+
270
271Splitting a loop can also be useful to reduce the number of barriers:
272
273 +------------------------------------------+----------------------------------+
274 | before | after |
275 +==========================================+==================================+
276 | :: | :: |
277 | | |
278 | n = 0; | smp_mb_release(); |
279 | for (i = 0; i < 10; i++) { | for (i = 0; i < 10; i++) |
269 +------------------------------------------+----------------------------------+
270
271Splitting a loop can also be useful to reduce the number of barriers:
272
273 +------------------------------------------+----------------------------------+
274 | before | after |
275 +==========================================+==================================+
276 | :: | :: |
277 | | |
278 | n = 0; | smp_mb_release(); |
279 | for (i = 0; i < 10; i++) { | for (i = 0; i < 10; i++) |
280 | atomic_store_release(&a[i], false); | atomic_set(&a[i], false); |
280 | qatomic_store_release(&a[i], false); | qatomic_set(&a[i], false); |
281 | smp_mb(); | smb_mb(); |
281 | smp_mb(); | smb_mb(); |
282 | n += atomic_read(&b[i]); | n = 0; |
282 | n += qatomic_read(&b[i]); | n = 0; |
283 | } | for (i = 0; i < 10; i++) |
283 | } | for (i = 0; i < 10; i++) |
284 | | n += atomic_read(&b[i]); |
284 | | n += qatomic_read(&b[i]); |
285 +------------------------------------------+----------------------------------+
286
287In this case, a ``smp_mb_release()`` is also replaced with a (possibly cheaper, and clearer
288as well) ``smp_wmb()``:
289
290 +------------------------------------------+----------------------------------+
291 | before | after |
292 +==========================================+==================================+
293 | :: | :: |
294 | | |
295 | | smp_mb_release(); |
296 | for (i = 0; i < 10; i++) { | for (i = 0; i < 10; i++) |
285 +------------------------------------------+----------------------------------+
286
287In this case, a ``smp_mb_release()`` is also replaced with a (possibly cheaper, and clearer
288as well) ``smp_wmb()``:
289
290 +------------------------------------------+----------------------------------+
291 | before | after |
292 +==========================================+==================================+
293 | :: | :: |
294 | | |
295 | | smp_mb_release(); |
296 | for (i = 0; i < 10; i++) { | for (i = 0; i < 10; i++) |
297 | atomic_store_release(&a[i], false); | atomic_set(&a[i], false); |
298 | atomic_store_release(&b[i], false); | smb_wmb(); |
297 | qatomic_store_release(&a[i], false); | qatomic_set(&a[i], false); |
298 | qatomic_store_release(&b[i], false); | smb_wmb(); |
299 | } | for (i = 0; i < 10; i++) |
299 | } | for (i = 0; i < 10; i++) |
300 | | atomic_set(&b[i], false); |
300 | | qatomic_set(&b[i], false); |
301 +------------------------------------------+----------------------------------+
302
303
304.. _acqrel:
305
306Acquire/release pairing and the *synchronizes-with* relation
307------------------------------------------------------------
308
301 +------------------------------------------+----------------------------------+
302
303
304.. _acqrel:
305
306Acquire/release pairing and the *synchronizes-with* relation
307------------------------------------------------------------
308
309Atomic operations other than ``atomic_set()`` and ``atomic_read()`` have
309Atomic operations other than ``qatomic_set()`` and ``qatomic_read()`` have
310either *acquire* or *release* semantics [#rmw]_. This has two effects:
311
312.. [#rmw] Read-modify-write operations can have both---acquire applies to the
313 read part, and release to the write.
314
315- within a thread, they are ordered either before subsequent operations
316 (for acquire) or after previous operations (for release).
317

--- 34 unchanged lines hidden (view full) ---

352 | | |
353 | *a = 1; | |
354 | pthread_exit(a); | pthread_join(thread1, &a); |
355 | | x = *a; |
356 +----------------------+-------------------------------+
357
358Synchronization between threads basically descends from this pairing of
359a release operation and an acquire operation. Therefore, atomic operations
310either *acquire* or *release* semantics [#rmw]_. This has two effects:
311
312.. [#rmw] Read-modify-write operations can have both---acquire applies to the
313 read part, and release to the write.
314
315- within a thread, they are ordered either before subsequent operations
316 (for acquire) or after previous operations (for release).
317

--- 34 unchanged lines hidden (view full) ---

352 | | |
353 | *a = 1; | |
354 | pthread_exit(a); | pthread_join(thread1, &a); |
355 | | x = *a; |
356 +----------------------+-------------------------------+
357
358Synchronization between threads basically descends from this pairing of
359a release operation and an acquire operation. Therefore, atomic operations
360other than ``atomic_set()`` and ``atomic_read()`` will almost always be
360other than ``qatomic_set()`` and ``qatomic_read()`` will almost always be
361paired with another operation of the opposite kind: an acquire operation
362will pair with a release operation and vice versa. This rule of thumb is
363extremely useful; in the case of QEMU, however, note that the other
364operation may actually be in a driver that runs in the guest!
365
366``smp_read_barrier_depends()``, ``smp_rmb()``, ``smp_mb_acquire()``,
361paired with another operation of the opposite kind: an acquire operation
362will pair with a release operation and vice versa. This rule of thumb is
363extremely useful; in the case of QEMU, however, note that the other
364operation may actually be in a driver that runs in the guest!
365
366``smp_read_barrier_depends()``, ``smp_rmb()``, ``smp_mb_acquire()``,
367``atomic_load_acquire()`` and ``atomic_rcu_read()`` all count
367``qatomic_load_acquire()`` and ``qatomic_rcu_read()`` all count
368as acquire operations. ``smp_wmb()``, ``smp_mb_release()``,
368as acquire operations. ``smp_wmb()``, ``smp_mb_release()``,
369``atomic_store_release()`` and ``atomic_rcu_set()`` all count as release
369``qatomic_store_release()`` and ``qatomic_rcu_set()`` all count as release
370operations. ``smp_mb()`` counts as both acquire and release, therefore
371it can pair with any other atomic operation. Here is an example:
372
373 +----------------------+------------------------------+
374 | thread 1 | thread 2 |
375 +======================+==============================+
376 | :: | :: |
377 | | |
370operations. ``smp_mb()`` counts as both acquire and release, therefore
371it can pair with any other atomic operation. Here is an example:
372
373 +----------------------+------------------------------+
374 | thread 1 | thread 2 |
375 +======================+==============================+
376 | :: | :: |
377 | | |
378 | atomic_set(&a, 1); | |
378 | qatomic_set(&a, 1);| |
379 | smp_wmb(); | |
379 | smp_wmb(); | |
380 | atomic_set(&b, 2); | x = atomic_read(&b); |
380 | qatomic_set(&b, 2);| x = qatomic_read(&b); |
381 | | smp_rmb(); |
381 | | smp_rmb(); |
382 | | y = atomic_read(&a); |
382 | | y = qatomic_read(&a); |
383 +----------------------+------------------------------+
384
385Note that a load-store pair only counts if the two operations access the
386same variable: that is, a store-release on a variable ``x`` *synchronizes
387with* a load-acquire on a variable ``x``, while a release barrier
388synchronizes with any acquire operation. The following example shows
389correct synchronization:
390
391 +--------------------------------+--------------------------------+
392 | thread 1 | thread 2 |
393 +================================+================================+
394 | :: | :: |
395 | | |
383 +----------------------+------------------------------+
384
385Note that a load-store pair only counts if the two operations access the
386same variable: that is, a store-release on a variable ``x`` *synchronizes
387with* a load-acquire on a variable ``x``, while a release barrier
388synchronizes with any acquire operation. The following example shows
389correct synchronization:
390
391 +--------------------------------+--------------------------------+
392 | thread 1 | thread 2 |
393 +================================+================================+
394 | :: | :: |
395 | | |
396 | atomic_set(&a, 1); | |
397 | atomic_store_release(&b, 2); | x = atomic_load_acquire(&b); |
398 | | y = atomic_read(&a); |
396 | qatomic_set(&a, 1); | |
397 | qatomic_store_release(&b, 2);| x = qatomic_load_acquire(&b);|
398 | | y = qatomic_read(&a); |
399 +--------------------------------+--------------------------------+
400
401Acquire and release semantics of higher-level primitives can also be
402relied upon for the purpose of establishing the *synchronizes with*
403relation.
404
405Note that the "writing" thread is accessing the variables in the
406opposite order as the "reading" thread. This is expected: stores

--- 9 unchanged lines hidden (view full) ---

416 | thread 1 | thread 2 |
417 +======================+==============================+
418 | :: | :: |
419 | | |
420 | b[2] = 1; | |
421 | smp_wmb(); | |
422 | x->i = 2; | |
423 | smp_wmb(); | |
399 +--------------------------------+--------------------------------+
400
401Acquire and release semantics of higher-level primitives can also be
402relied upon for the purpose of establishing the *synchronizes with*
403relation.
404
405Note that the "writing" thread is accessing the variables in the
406opposite order as the "reading" thread. This is expected: stores

--- 9 unchanged lines hidden (view full) ---

416 | thread 1 | thread 2 |
417 +======================+==============================+
418 | :: | :: |
419 | | |
420 | b[2] = 1; | |
421 | smp_wmb(); | |
422 | x->i = 2; | |
423 | smp_wmb(); | |
424 | atomic_set(&a, x); | x = atomic_read(&a); |
424 | qatomic_set(&a, x);| x = qatomic_read(&a); |
425 | | smp_read_barrier_depends(); |
426 | | y = x->i; |
427 | | smp_read_barrier_depends(); |
428 | | z = b[y]; |
429 +----------------------+------------------------------+
430
431Comparison with Linux kernel primitives
432=======================================

--- 4 unchanged lines hidden (view full) ---

437- atomic operations in Linux are always on a 32-bit int type and
438 use a boxed ``atomic_t`` type; atomic operations in QEMU are polymorphic
439 and use normal C types.
440
441- Originally, ``atomic_read`` and ``atomic_set`` in Linux gave no guarantee
442 at all. Linux 4.1 updated them to implement volatile
443 semantics via ``ACCESS_ONCE`` (or the more recent ``READ``/``WRITE_ONCE``).
444
425 | | smp_read_barrier_depends(); |
426 | | y = x->i; |
427 | | smp_read_barrier_depends(); |
428 | | z = b[y]; |
429 +----------------------+------------------------------+
430
431Comparison with Linux kernel primitives
432=======================================

--- 4 unchanged lines hidden (view full) ---

437- atomic operations in Linux are always on a 32-bit int type and
438 use a boxed ``atomic_t`` type; atomic operations in QEMU are polymorphic
439 and use normal C types.
440
441- Originally, ``atomic_read`` and ``atomic_set`` in Linux gave no guarantee
442 at all. Linux 4.1 updated them to implement volatile
443 semantics via ``ACCESS_ONCE`` (or the more recent ``READ``/``WRITE_ONCE``).
444
445 QEMU's ``atomic_read`` and ``atomic_set`` implement C11 atomic relaxed
445 QEMU's ``qatomic_read`` and ``qatomic_set`` implement C11 atomic relaxed
446 semantics if the compiler supports it, and volatile semantics otherwise.
447 Both semantics prevent the compiler from doing certain transformations;
448 the difference is that atomic accesses are guaranteed to be atomic,
449 while volatile accesses aren't. Thus, in the volatile case we just cross
450 our fingers hoping that the compiler will generate atomic accesses,
451 since we assume the variables passed are machine-word sized and
452 properly aligned.
453
446 semantics if the compiler supports it, and volatile semantics otherwise.
447 Both semantics prevent the compiler from doing certain transformations;
448 the difference is that atomic accesses are guaranteed to be atomic,
449 while volatile accesses aren't. Thus, in the volatile case we just cross
450 our fingers hoping that the compiler will generate atomic accesses,
451 since we assume the variables passed are machine-word sized and
452 properly aligned.
453
454 No barriers are implied by ``atomic_read`` and ``atomic_set`` in either Linux
455 or QEMU.
454 No barriers are implied by ``qatomic_read`` and ``qatomic_set`` in either
455 Linux or QEMU.
456
457- atomic read-modify-write operations in Linux are of three kinds:
458
459 ===================== =========================================
460 ``atomic_OP`` returns void
461 ``atomic_OP_return`` returns new value of the variable
462 ``atomic_fetch_OP`` returns the old value of the variable
463 ``atomic_cmpxchg`` returns the old value of the variable
464 ===================== =========================================
465
466 In QEMU, the second kind is named ``atomic_OP_fetch``.
467
468- different atomic read-modify-write operations in Linux imply
469 a different set of memory barriers; in QEMU, all of them enforce
470 sequential consistency.
471
456
457- atomic read-modify-write operations in Linux are of three kinds:
458
459 ===================== =========================================
460 ``atomic_OP`` returns void
461 ``atomic_OP_return`` returns new value of the variable
462 ``atomic_fetch_OP`` returns the old value of the variable
463 ``atomic_cmpxchg`` returns the old value of the variable
464 ===================== =========================================
465
466 In QEMU, the second kind is named ``atomic_OP_fetch``.
467
468- different atomic read-modify-write operations in Linux imply
469 a different set of memory barriers; in QEMU, all of them enforce
470 sequential consistency.
471
472- in QEMU, ``atomic_read()`` and ``atomic_set()`` do not participate in
472- in QEMU, ``qatomic_read()`` and ``qatomic_set()`` do not participate in
473 the total ordering enforced by sequentially-consistent operations.
474 This is because QEMU uses the C11 memory model. The following example
475 is correct in Linux but not in QEMU:
476
477 +----------------------------------+--------------------------------+
478 | Linux (correct) | QEMU (incorrect) |
479 +==================================+================================+
480 | :: | :: |
481 | | |
473 the total ordering enforced by sequentially-consistent operations.
474 This is because QEMU uses the C11 memory model. The following example
475 is correct in Linux but not in QEMU:
476
477 +----------------------------------+--------------------------------+
478 | Linux (correct) | QEMU (incorrect) |
479 +==================================+================================+
480 | :: | :: |
481 | | |
482 | a = atomic_fetch_add(&x, 2); | a = atomic_fetch_add(&x, 2); |
483 | b = READ_ONCE(&y); | b = atomic_read(&y); |
482 | a = atomic_fetch_add(&x, 2); | a = qatomic_fetch_add(&x, 2);|
483 | b = READ_ONCE(&y); | b = qatomic_read(&y); |
484 +----------------------------------+--------------------------------+
485
486 because the read of ``y`` can be moved (by either the processor or the
487 compiler) before the write of ``x``.
488
489 Fixing this requires an ``smp_mb()`` memory barrier between the write
490 of ``x`` and the read of ``y``. In the common case where only one thread
491 writes ``x``, it is also possible to write it like this:
492
493 +--------------------------------+
494 | QEMU (correct) |
495 +================================+
496 | :: |
497 | |
484 +----------------------------------+--------------------------------+
485
486 because the read of ``y`` can be moved (by either the processor or the
487 compiler) before the write of ``x``.
488
489 Fixing this requires an ``smp_mb()`` memory barrier between the write
490 of ``x`` and the read of ``y``. In the common case where only one thread
491 writes ``x``, it is also possible to write it like this:
492
493 +--------------------------------+
494 | QEMU (correct) |
495 +================================+
496 | :: |
497 | |
498 | a = atomic_read(&x); |
499 | atomic_set(&x, a + 2); |
498 | a = qatomic_read(&x); |
499 | qatomic_set(&x, a + 2); |
500 | smp_mb(); |
500 | smp_mb(); |
501 | b = atomic_read(&y); |
501 | b = qatomic_read(&y); |
502 +--------------------------------+
503
504Sources
505=======
506
507- ``Documentation/memory-barriers.txt`` from the Linux kernel
502 +--------------------------------+
503
504Sources
505=======
506
507- ``Documentation/memory-barriers.txt`` from the Linux kernel