linux/Documentation/memory-barriers.txt

2 			 LINUX KERNEL MEMORY BARRIERS
16 meant as a guide to using the various memory barriers provided by Linux, but
31  (2) to provide a guide as to how to use the barriers that are available.
51  (*) What are memory barriers?
54      - What may not be assumed about memory barriers?
55      - Address-dependency barriers (historical).
59      - Read memory barriers vs load speculation.
62  (*) Explicit kernel barriers.
65      - CPU memory barriers.
67  (*) Implicit kernel memory barriers.
78  (*) Where are memory barriers needed?
356 WHAT ARE MEMORY BARRIERS?
364 Memory barriers are such interventions.  They impose a perceived partial
370 branch prediction and various types of caching.  Memory barriers are used to
378 Memory barriers come in four basic varieties:
380  (1) Write (or store) memory barriers.
394      [!] Note that write barriers should normally be paired with read or
395      address-dependency barriers; see the "SMP barrier pairing" subsection.
398  (2) Address-dependency barriers (historical).
430      [!] Note that address-dependency barriers should normally be paired with
431      write barriers; see the "SMP barrier pairing" subsection.
434      dependency barriers.  Nowadays, APIs for marking loads from shared
436      address-dependency barriers.
438  (3) Read (or load) memory barriers.
448      Read memory barriers imply address-dependency barriers, and so can
451      [!] Note that read barriers should normally be paired with write barriers;
455  (4) General memory barriers.
464      General memory barriers imply both read and write memory barriers, and so
514 Memory barriers are only required where there's a possibility of interaction
517 memory barriers are unnecessary in that piece of code.
525 WHAT MAY NOT BE ASSUMED ABOUT MEMORY BARRIERS?
528 There are certain things that the Linux kernel memory barriers do not guarantee:
557 ADDRESS-DEPENDENCY BARRIERS (HISTORICAL)
565 address-dependency barriers.
568 load-to-store relations, address-dependency barriers are not necessary
571 The requirement of address-dependency barriers is a little subtle, and
711 Control dependencies pair normally with other types of barriers.
761 memory barriers, for example, smp_store_release():
772 In contrast, without explicit memory barriers, two-legged-if control
923   (*) Control dependencies pair normally with other types of barriers.
938 General barriers pair with each other, though they also pair with most
939 other types of barriers, albeit without multicopy atomicity.  An acquire
941 barriers, including of course general barriers.  A write barrier pairs
998 Firstly, write barriers act as partial orderings on store operations.
1036 Secondly, address-dependency barriers act as partial orderings on address-
1266 READ MEMORY BARRIERS VS LOAD SPECULATION
1388 CPU 3's load from Y.  In addition, the memory barriers guarantee that
1406 However, dependencies, read barriers, and write barriers are not always
1426 writes.  General barriers are therefore required to ensure that all CPUs
1429 General barriers can compensate not only for non-multicopy atomicity,
1510 use general barriers throughout.
1514 EXPLICIT KERNEL BARRIERS
1517 The Linux kernel has a variety of different barriers that act at different
1522   (*) CPU memory barriers.
1824 Please note that these compiler barriers have no direct effect on the CPU,
1828 CPU MEMORY BARRIERS
1831 The Linux kernel has seven basic CPU memory barriers:
1841 All memory barriers except the address-dependency barriers imply a compiler
1854 SMP memory barriers are reduced to compiler barriers on uniprocessor compiled
1859 [!] Note that SMP memory barriers _must_ be used to control the ordering of
1863 Mandatory barriers should not be used to control SMP effects, since mandatory
1864 barriers impose unnecessary overhead on both SMP and UP systems. They may,
1866 windows.  These barriers are required even on non-SMP systems as they affect
1884      barriers, but where the code needs a memory barrier. Examples for atomic
1947      Note that the dma_*() barriers do not provide any ordering guarantees for
1963      For load from persistent memory, existing read memory barriers are sufficient
1975 IMPLICIT KERNEL MEMORY BARRIERS
1978 Some of the other functions in the linux kernel imply memory barriers, amongst
1998 for each construct.  These operations all imply certain barriers:
2034 one-way barriers is that the effects of instructions outside of a critical
2136 (RELEASE equivalent) will act as compiler barriers only.  So if memory or I/O
2137 barriers are required in such a situation, they must be provided from some
2149 barriers.
2257 [!] Note that the memory barriers implied by the sleeper and the waker do _not_
2276 code on both sides must interpolate its own memory barriers between the
2296 Other functions that imply barriers:
2298  (*) schedule() and similar imply full memory barriers.
2340 WHERE ARE MEMORY BARRIERS NEEDED?
2462 operations are noted specially as some of them imply full memory barriers and
2485 use of memory barriers unnecessary, if the accessor functions are used to refer
2487 memory barriers are required to enforce ordering.
2528 explicit barriers are used.
2532 registers that form implicit I/O barriers.
2695 barriers for the most part act at the interface between the CPU and its cache
2696 (memory barriers logically act on the dotted line in the following diagram):
2733 What memory barriers are concerned with is controlling the order in which
2738 [!] Memory barriers are _not_ needed within a given CPU, as CPUs always see
2920 barriers for this use-case would be possible but is often suboptimal.
2930 MMIO effects, use mandatory barriers.
2940 Memory barriers can be used to implement circular buffering without the need