1Buffer Sharing and Synchronization 2================================== 3 4The dma-buf subsystem provides the framework for sharing buffers for 5hardware (DMA) access across multiple device drivers and subsystems, and 6for synchronizing asynchronous hardware access. 7 8This is used, for example, by drm "prime" multi-GPU support, but is of 9course not limited to GPU use cases. 10 11The three main components of this are: (1) dma-buf, representing a 12sg_table and exposed to userspace as a file descriptor to allow passing 13between devices, (2) fence, which provides a mechanism to signal when 14one device has finished access, and (3) reservation, which manages the 15shared or exclusive fence(s) associated with the buffer. 16 17Shared DMA Buffers 18------------------ 19 20This document serves as a guide to device-driver writers on what is the dma-buf 21buffer sharing API, how to use it for exporting and using shared buffers. 22 23Any device driver which wishes to be a part of DMA buffer sharing, can do so as 24either the 'exporter' of buffers, or the 'user' or 'importer' of buffers. 25 26Say a driver A wants to use buffers created by driver B, then we call B as the 27exporter, and A as buffer-user/importer. 28 29The exporter 30 31 - implements and manages operations in :c:type:`struct dma_buf_ops 32 <dma_buf_ops>` for the buffer, 33 - allows other users to share the buffer by using dma_buf sharing APIs, 34 - manages the details of buffer allocation, wrapped in a :c:type:`struct 35 dma_buf <dma_buf>`, 36 - decides about the actual backing storage where this allocation happens, 37 - and takes care of any migration of scatterlist - for all (shared) users of 38 this buffer. 39 40The buffer-user 41 42 - is one of (many) sharing users of the buffer. 43 - doesn't need to worry about how the buffer is allocated, or where. 44 - and needs a mechanism to get access to the scatterlist that makes up this 45 buffer in memory, mapped into its own address space, so it can access the 46 same area of memory. This interface is provided by :c:type:`struct 47 dma_buf_attachment <dma_buf_attachment>`. 48 49Any exporters or users of the dma-buf buffer sharing framework must have a 50'select DMA_SHARED_BUFFER' in their respective Kconfigs. 51 52Userspace Interface Notes 53~~~~~~~~~~~~~~~~~~~~~~~~~ 54 55Mostly a DMA buffer file descriptor is simply an opaque object for userspace, 56and hence the generic interface exposed is very minimal. There's a few things to 57consider though: 58 59- Since kernel 3.12 the dma-buf FD supports the llseek system call, but only 60 with offset=0 and whence=SEEK_END|SEEK_SET. SEEK_SET is supported to allow 61 the usual size discover pattern size = SEEK_END(0); SEEK_SET(0). Every other 62 llseek operation will report -EINVAL. 63 64 If llseek on dma-buf FDs isn't support the kernel will report -ESPIPE for all 65 cases. Userspace can use this to detect support for discovering the dma-buf 66 size using llseek. 67 68- In order to avoid fd leaks on exec, the FD_CLOEXEC flag must be set 69 on the file descriptor. This is not just a resource leak, but a 70 potential security hole. It could give the newly exec'd application 71 access to buffers, via the leaked fd, to which it should otherwise 72 not be permitted access. 73 74 The problem with doing this via a separate fcntl() call, versus doing it 75 atomically when the fd is created, is that this is inherently racy in a 76 multi-threaded app[3]. The issue is made worse when it is library code 77 opening/creating the file descriptor, as the application may not even be 78 aware of the fd's. 79 80 To avoid this problem, userspace must have a way to request O_CLOEXEC 81 flag be set when the dma-buf fd is created. So any API provided by 82 the exporting driver to create a dmabuf fd must provide a way to let 83 userspace control setting of O_CLOEXEC flag passed in to dma_buf_fd(). 84 85- Memory mapping the contents of the DMA buffer is also supported. See the 86 discussion below on `CPU Access to DMA Buffer Objects`_ for the full details. 87 88- The DMA buffer FD is also pollable, see `Implicit Fence Poll Support`_ below for 89 details. 90 91Basic Operation and Device DMA Access 92~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 93 94.. kernel-doc:: drivers/dma-buf/dma-buf.c 95 :doc: dma buf device access 96 97CPU Access to DMA Buffer Objects 98~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 99 100.. kernel-doc:: drivers/dma-buf/dma-buf.c 101 :doc: cpu access 102 103Implicit Fence Poll Support 104~~~~~~~~~~~~~~~~~~~~~~~~~~~ 105 106.. kernel-doc:: drivers/dma-buf/dma-buf.c 107 :doc: implicit fence polling 108 109DMA-BUF statistics 110~~~~~~~~~~~~~~~~~~ 111.. kernel-doc:: drivers/dma-buf/dma-buf-sysfs-stats.c 112 :doc: overview 113 114Kernel Functions and Structures Reference 115~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 116 117.. kernel-doc:: drivers/dma-buf/dma-buf.c 118 :export: 119 120.. kernel-doc:: include/linux/dma-buf.h 121 :internal: 122 123Buffer Mapping Helpers 124~~~~~~~~~~~~~~~~~~~~~~ 125 126.. kernel-doc:: include/linux/dma-buf-map.h 127 :doc: overview 128 129.. kernel-doc:: include/linux/dma-buf-map.h 130 :internal: 131 132Reservation Objects 133------------------- 134 135.. kernel-doc:: drivers/dma-buf/dma-resv.c 136 :doc: Reservation Object Overview 137 138.. kernel-doc:: drivers/dma-buf/dma-resv.c 139 :export: 140 141.. kernel-doc:: include/linux/dma-resv.h 142 :internal: 143 144DMA Fences 145---------- 146 147.. kernel-doc:: drivers/dma-buf/dma-fence.c 148 :doc: DMA fences overview 149 150DMA Fence Cross-Driver Contract 151~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 152 153.. kernel-doc:: drivers/dma-buf/dma-fence.c 154 :doc: fence cross-driver contract 155 156DMA Fence Signalling Annotations 157~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 158 159.. kernel-doc:: drivers/dma-buf/dma-fence.c 160 :doc: fence signalling annotation 161 162DMA Fences Functions Reference 163~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 164 165.. kernel-doc:: drivers/dma-buf/dma-fence.c 166 :export: 167 168.. kernel-doc:: include/linux/dma-fence.h 169 :internal: 170 171Seqno Hardware Fences 172~~~~~~~~~~~~~~~~~~~~~ 173 174.. kernel-doc:: include/linux/seqno-fence.h 175 :internal: 176 177DMA Fence Array 178~~~~~~~~~~~~~~~ 179 180.. kernel-doc:: drivers/dma-buf/dma-fence-array.c 181 :export: 182 183.. kernel-doc:: include/linux/dma-fence-array.h 184 :internal: 185 186DMA Fence Chain 187~~~~~~~~~~~~~~~ 188 189.. kernel-doc:: drivers/dma-buf/dma-fence-chain.c 190 :export: 191 192.. kernel-doc:: include/linux/dma-fence-chain.h 193 :internal: 194 195DMA Fence uABI/Sync File 196~~~~~~~~~~~~~~~~~~~~~~~~ 197 198.. kernel-doc:: drivers/dma-buf/sync_file.c 199 :export: 200 201.. kernel-doc:: include/linux/sync_file.h 202 :internal: 203 204Indefinite DMA Fences 205~~~~~~~~~~~~~~~~~~~~~ 206 207At various times struct dma_fence with an indefinite time until dma_fence_wait() 208finishes have been proposed. Examples include: 209 210* Future fences, used in HWC1 to signal when a buffer isn't used by the display 211 any longer, and created with the screen update that makes the buffer visible. 212 The time this fence completes is entirely under userspace's control. 213 214* Proxy fences, proposed to handle &drm_syncobj for which the fence has not yet 215 been set. Used to asynchronously delay command submission. 216 217* Userspace fences or gpu futexes, fine-grained locking within a command buffer 218 that userspace uses for synchronization across engines or with the CPU, which 219 are then imported as a DMA fence for integration into existing winsys 220 protocols. 221 222* Long-running compute command buffers, while still using traditional end of 223 batch DMA fences for memory management instead of context preemption DMA 224 fences which get reattached when the compute job is rescheduled. 225 226Common to all these schemes is that userspace controls the dependencies of these 227fences and controls when they fire. Mixing indefinite fences with normal 228in-kernel DMA fences does not work, even when a fallback timeout is included to 229protect against malicious userspace: 230 231* Only the kernel knows about all DMA fence dependencies, userspace is not aware 232 of dependencies injected due to memory management or scheduler decisions. 233 234* Only userspace knows about all dependencies in indefinite fences and when 235 exactly they will complete, the kernel has no visibility. 236 237Furthermore the kernel has to be able to hold up userspace command submission 238for memory management needs, which means we must support indefinite fences being 239dependent upon DMA fences. If the kernel also support indefinite fences in the 240kernel like a DMA fence, like any of the above proposal would, there is the 241potential for deadlocks. 242 243.. kernel-render:: DOT 244 :alt: Indefinite Fencing Dependency Cycle 245 :caption: Indefinite Fencing Dependency Cycle 246 247 digraph "Fencing Cycle" { 248 node [shape=box bgcolor=grey style=filled] 249 kernel [label="Kernel DMA Fences"] 250 userspace [label="userspace controlled fences"] 251 kernel -> userspace [label="memory management"] 252 userspace -> kernel [label="Future fence, fence proxy, ..."] 253 254 { rank=same; kernel userspace } 255 } 256 257This means that the kernel might accidentally create deadlocks 258through memory management dependencies which userspace is unaware of, which 259randomly hangs workloads until the timeout kicks in. Workloads, which from 260userspace's perspective, do not contain a deadlock. In such a mixed fencing 261architecture there is no single entity with knowledge of all dependencies. 262Thefore preventing such deadlocks from within the kernel is not possible. 263 264The only solution to avoid dependencies loops is by not allowing indefinite 265fences in the kernel. This means: 266 267* No future fences, proxy fences or userspace fences imported as DMA fences, 268 with or without a timeout. 269 270* No DMA fences that signal end of batchbuffer for command submission where 271 userspace is allowed to use userspace fencing or long running compute 272 workloads. This also means no implicit fencing for shared buffers in these 273 cases. 274 275Recoverable Hardware Page Faults Implications 276~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 277 278Modern hardware supports recoverable page faults, which has a lot of 279implications for DMA fences. 280 281First, a pending page fault obviously holds up the work that's running on the 282accelerator and a memory allocation is usually required to resolve the fault. 283But memory allocations are not allowed to gate completion of DMA fences, which 284means any workload using recoverable page faults cannot use DMA fences for 285synchronization. Synchronization fences controlled by userspace must be used 286instead. 287 288On GPUs this poses a problem, because current desktop compositor protocols on 289Linux rely on DMA fences, which means without an entirely new userspace stack 290built on top of userspace fences, they cannot benefit from recoverable page 291faults. Specifically this means implicit synchronization will not be possible. 292The exception is when page faults are only used as migration hints and never to 293on-demand fill a memory request. For now this means recoverable page 294faults on GPUs are limited to pure compute workloads. 295 296Furthermore GPUs usually have shared resources between the 3D rendering and 297compute side, like compute units or command submission engines. If both a 3D 298job with a DMA fence and a compute workload using recoverable page faults are 299pending they could deadlock: 300 301- The 3D workload might need to wait for the compute job to finish and release 302 hardware resources first. 303 304- The compute workload might be stuck in a page fault, because the memory 305 allocation is waiting for the DMA fence of the 3D workload to complete. 306 307There are a few options to prevent this problem, one of which drivers need to 308ensure: 309 310- Compute workloads can always be preempted, even when a page fault is pending 311 and not yet repaired. Not all hardware supports this. 312 313- DMA fence workloads and workloads which need page fault handling have 314 independent hardware resources to guarantee forward progress. This could be 315 achieved through e.g. through dedicated engines and minimal compute unit 316 reservations for DMA fence workloads. 317 318- The reservation approach could be further refined by only reserving the 319 hardware resources for DMA fence workloads when they are in-flight. This must 320 cover the time from when the DMA fence is visible to other threads up to 321 moment when fence is completed through dma_fence_signal(). 322 323- As a last resort, if the hardware provides no useful reservation mechanics, 324 all workloads must be flushed from the GPU when switching between jobs 325 requiring DMA fences or jobs requiring page fault handling: This means all DMA 326 fences must complete before a compute job with page fault handling can be 327 inserted into the scheduler queue. And vice versa, before a DMA fence can be 328 made visible anywhere in the system, all compute workloads must be preempted 329 to guarantee all pending GPU page faults are flushed. 330 331- Only a fairly theoretical option would be to untangle these dependencies when 332 allocating memory to repair hardware page faults, either through separate 333 memory blocks or runtime tracking of the full dependency graph of all DMA 334 fences. This results very wide impact on the kernel, since resolving the page 335 on the CPU side can itself involve a page fault. It is much more feasible and 336 robust to limit the impact of handling hardware page faults to the specific 337 driver. 338 339Note that workloads that run on independent hardware like copy engines or other 340GPUs do not have any impact. This allows us to keep using DMA fences internally 341in the kernel even for resolving hardware page faults, e.g. by using copy 342engines to clear or copy memory needed to resolve the page fault. 343 344In some ways this page fault problem is a special case of the `Infinite DMA 345Fences` discussions: Infinite fences from compute workloads are allowed to 346depend on DMA fences, but not the other way around. And not even the page fault 347problem is new, because some other CPU thread in userspace might 348hit a page fault which holds up a userspace fence - supporting page faults on 349GPUs doesn't anything fundamentally new. 350